• 沒有找到結果。

Multi-threaded Algorithm 2

N/A
N/A
Protected

Academic year: 2022

Share "Multi-threaded Algorithm 2"

Copied!
27
0
0
顯示更多 ( 頁)

全文

(1)

Algorithm 2

Michael Tsai 2011/1/6

(2)

Scheduling

 Scheduler 的工作 : 把 strand 指定給 processo r 執行 .

 On-line: scheduler 事先並不知道什麼時候 st rand 會 spawn, 或者 spawn 出來的什麼時候會 完成 .

 Centralized scheduler: 單一的 scheduler 知道整體狀況並作 scheduling ( 比較容易分 析 )

 Distributed scheduler: 每個 thread 互相溝 通合作 , 找出最好的 scheduling 方法

(3)

Greedy scheduler

每一個 time step, 把越多 strand 分給越多 process or 執行越好

如果某個 time step 的時候 , 至少有 P 個 strand 是 準備好被執行的 , 則稱為 complete step

反之則稱為 incomplete step

Lower bounds: 最好的狀況也需要這些時間 :

work law:

span law:

接下來一連串證明 , 會說 greedy scheduler 其實是 蠻好的 scheduler.

 

(4)

 Theorem: On an ideal parallel computer wi th P processors, a greedy scheduler execu tes a multithreaded computation with work and span in time .

 Proof:

 我們想證明 complete steps 最多為個 .

 用反證法 : 假設 complete steps 超過個 .

 則所有 complete steps 所完成的工作有 :

 

矛盾 , 因為全部的工作也才 .

 

因此 complete steps 最多為個 .

 

(5)

現在考慮 incomplete step 的個數 :

最長路徑一定從 in-degree=0 的 vertex 開始

greedy scheduler 的一個 incomplete step 一定把所有 G’ 裡面 in-degre e=0 的 vertex 都執行下去 (G’’ 裡面不會有 in-degree=0 的 )

G’’ 裡面的最長路徑一定比 G’ 中的最長路徑長度少 1

意思就是說每個 incomplete step 會使”表示還沒執行 strand 的 dag” 裡 面的最長路徑少 1

因此最多所有 incomplete step 個數應為 span: .

最後 , 所以的 step 一定是 complete 或 incomplete.

因此

 

 

 

  ′ ′

… …

(6)

 Corollary: The running time of any multi threaded computation scheduled by a greed y scheduler on an ideal parallel computer with P processors is within a factor of 2 of optimal.

 Proof:

 假設為最佳 scheduler 在 P 個 processor 的機 器上所需執行的時間 .

 和分別為 work & span

 .

 

(7)

 Corollary: Let be the running time of a multithreaded computation produced by a g reedy scheduler on an ideal parallel comp uter with P processors, and let and be the work and span of the computation, res pectively. Then, if , we have , or equiva lently, a speedup of approximately P.

 Proof:

 If , then .

 So .

 (Work law: )

 Or, the speedup is .

 

When is

<< ? When the slackness (.

 

(8)
(9)
(10)

Back to P-FIB

 Parallelism:

 

P-FIB(n) if n<=1 return n else

x=spawn P-FIB(n-1) y=P-FIB(n-2)

sync

return x+y

即使對大平行電腦 , 一個普

通的 n 都可以使我們的程式 達到 near perfect linear speedup. (Parallelism 比 P 大很多 slackness 很大 )

(11)

Parallel Loops

 Parallel loops: 把 loop 的 iterations 平行 地執行 .

 在 for 關鍵字前面加上 “ parallel”.

 也可以用 spawn 和 sync, 不過這樣的語法比較 方便 .

(12)

矩陣與向量相乘

 A: 大小為 n x n 的矩陣

 x: 大小為 n 向量

 需計算出 y=Ax.

 .

 

=

y A x

i  

MAT-VEC(A,x) j n=A.rows

let y be a new vector of length n parallel for i=1 to n

parallel for i=1 to n for j=1 to n

return y  

(13)

MAT-VEC-MAIN-LOOP(A,x,y,n,i,i’) if i==i’

for j=1 to n else

mid=

spawn MAT-VEC-MAIN-LOOP(A,x,y,n,i,mid) MAT-VEC-MAIN-LOOP(A,x,y,n,mid+1,i’)

sync

 

i: 從第幾個 row 開始算 n: A 和 x 的大

i’: 從算道地幾個 row

(14)

=

y A x

i  

j

1,4 5,8

(15)

Analyze MAT-VEC

 Work: 像是把 parallel 的部分拿掉計算一般 的執行時間 .

 ( 主要是雙重迴圈的部分 )

 但是要注意 ,

spawn/parallel loop 的部分會造成額外

的 overhead!

 

MAT-VEC(A,x) n=A.rows

let y be a new vector of length n parallel for i=1 to n

parallel for i=1 to n for j=1 to n

return y  

(16)

n-1 個 internal nodes

n 個 leaves

每個 internal node 都會 spawn 一次 . (spawn overhead 為 constant)

leave 比 internal node 多 , 因 此 spawn 的 cost 可以平均分攤 給 leave

因此 spawn overhead 並不會使的 order 上升 . ( 但 constant 變大 )

 

(17)

 現在我們要計算 span.

 此時需要考慮 spawn overhead 了 .

 spawn 的次數 =log(n), 因為每次都分開成兩個 . 每次 spawn 的 overhead 為 constant.

 代表第 i 個 iteration 的 span.

 因此一般來說 , parallel loops 的 span 為 :

 Parallelism=

 

MAT-VEC(A,x) n=A.rows

let y be a new vector of length n parallel for i=1 to n

parallel for i=1 to n for j=1 to n

return y

 

(18)

Race Conditions

 一個 multithreaded algorithm 應該要是 determinist ic 的每次執行的結果都一樣

 不管怎麼去把程式的不同部分排程給不同的 processor /core 去執行

 通常”失敗”的時候 , 都是因為有” determinacy ra ce”.

 Determinacy Race: 當有兩個邏輯上平行的指令存取 相同的記憶體位置而且至少有一個指令是寫入 .

(19)

Determinacy Race Examples

Therac-25: 放射線治療機組 . 1985-1987 年間造成至少 6 個 病人受到原本設定劑量 100 倍 的輻射 , 造成死亡或嚴重輻射灼 傷 .

延伸閱讀 : 殺人的軟體 .

http://hiraman-sharma.blogspot.com/2010/07/killer- softwares.html

(20)

Determinacy Race Examples

2003 Northeast Blackout: Affected 10M people in Ontario & 45M people in 8 U.S. states.

Cause: A software bug known existed in General Electric Energy's Unix-based XA/21 energy

management system

(21)

Race-Example

RACE-EXAMPLE() x=0

parallel for i=1 to 2 x=x+1

print x

注意 : 大部分的執行順序都會得到正確的 結果 , 只有少部分的順序才會造成錯誤 !

 要找出這些錯誤非常困難 !

(22)

如何避免 race condition?

 有很多方法 (Mutex 等 , OS 裡面會教 )

 簡單的方法 : 只將平行運算運用在獨立的運算 上 , 也就是說互相之間沒有關聯性 .

 spawn 出來的 child 跟 parent, 還有其他 spaw n 出來的 child 都互相之間沒有關係 .

(23)

Socrates chess-playing program

 

1 =2048

 

1 =1024

�=32

 

32

= 2048

32 + 1=65

 

32

= 1024

32 + 8=40

 

�=512

 

512

= 2048

512 +1=5

 

512

= 1024

512 + 8=10

  Original

version

“Optimized version”

(24)

Multithreaded matrix multiplica tion

P-SQUARE-MATRIX-MULTIPLY(A,B) n=A.rows

let C be a new n x n matrix

parallel for i=1 to n

parallel for j=1 to n

for k=1 to n return C

 

  1

(�)=Θ (

3

)

 

( ) = Θ ( log n ) + Θ ( log ) + Θ ( ) =Θ(�)

1

(

)

(

)

=

Θ

(

3

)

Θ

(

)

(

2

)

 

(25)

Algorithm for Matrix Multiplicati on ( 勒勒長 )

 

C T

(26)

n=A.rows if n==1

else let T be a new n x n matrix

partition A,B,C, and T into n/2 x n/2 submatrices ( spawn P-MATRIX-MULTIPLY-RECURSIVE(

spawn P-MATRIX-MULTIPLY-RECURSIVE(

spawn P-MATRIX-MULTIPLY-RECURSIVE(

spawn P-MATRIX-MULTIPLY-RECURSIVE(

spawn P-MATRIX-MULTIPLY-RECURSIVE(

spawn P-MATRIX-MULTIPLY-RECURSIVE(

spawn P-MATRIX-MULTIPLY-RECURSIVE(

P-MATRIX-MULTIPLY-RECURSIVE(

sync

parallel for i=1 to n parallel for j=1 to n

1()=8 1

(

2

)

(

2

)

(

3

)

 

()=

(

2

)

+Θ(log)+Θ(log)

(

log2

)

 

1

(

�)

(

�)

=

Θ

(

3

)

Θ

( log

2

) ( log

32

)

 

(27)

How about Strassen’s method?

 Reading assignment: p.795-796.

 Parallelism: , slightly less than the ori ginal recursive version!

 

參考文獻

相關文件

Let, make, have, and help* are followed by an object and the bare infinitive form of a verb.. They persuaded us to join

Create an edge in T2 between two vertices if their corr- esponding faces in G share an edge in G that is not in T1 6.. Let e be the lone edge of G in the face corresponding to v

In attempting to generalize this function, we recall that interchanging two rows of a matrix changes the sign of its determinant.. This suggests the

The execution of a comparison-based algorithm can be described by a comparison tree, and the tree depth is the greatest number of comparisons, i.e., the worst-case

• An algorithm for such a problem whose running time is a polynomial of the input length and the value (not length) of the largest integer parameter is a..

• By definition, a pseudo-polynomial-time algorithm becomes polynomial-time if each integer parameter is limited to having a value polynomial in the input length.. • Corollary 42

Let us emancipate the student, and give him time and opportunity for the cultivation of his mind, so that in his pupilage he shall not be a puppet in the hands of others, but rather

a single instruction.. Thus, the operand can be modified before it can be modified before it is used. Useful for fast multipliation and dealing p g with lists, table and other

Rather than requiring a physical press of the reset button before an upload, the Arduino Uno is designed in a way that allows it to be reset by software running on a

shall be noted. In principle, documents attached by the employer shall be affixed with the seals of application unit and owner. The application and list shall be affixed with

In Section 3, the shift and scale argument from [2] is applied to show how each quantitative Landis theorem follows from the corresponding order-of-vanishing estimate.. A number

The proof is based on Hida’s ideas in [Hid04a], where Hida provided a general strategy to study the problem of the non-vanishing of Hecke L-values modulo p via a study on the

 develop a better understanding of the design and the features of the English Language curriculum with an emphasis on the senior secondary level;..  gain an insight into the

7.7 Representation of Functions by Power Series 7.8 Taylor and Maclaurin Series... Thm 7.4 (Absolute Value Thoerem) Let {a n } be a sequence of

Let T ⇤ be the temperature at which the GWs are produced from the cosmological phase transition. Without significant reheating, this temperature can be approximated by the

These are quite light states with masses in the 10 GeV to 20 GeV range and they have very small Yukawa couplings (implying that higgs to higgs pair chain decays are probable)..

Given a connected graph G together with a coloring f from the edge set of G to a set of colors, where adjacent edges may be colored the same, a u-v path P in G is said to be a

Given a sample space  and an event  in the  sample space  , let 

Let and be constants, let be a function, and let be defined on the nonnegative integers by the recu rrence. where we interpret to mean either

y A stochastic process is a collection of &#34;similar&#34; random variables ordered over time.. variables ordered

When an algorithm contains a recursive call to itself, we can often describe its running time by a recurrenceequation or recurrence, which describes the overall running time on

• Tangent-Chord Theorem (Alternate Seg- ment Theorem): Let the triangle ABC be inscribed in a circle and a tangent drawn at point A.. Let D be another point on the tan- gent line

A parallel route building algorithm for the vehicle routing and scheduling problem with time windows, European Journal of Operational Research, vol. A tabu search