Input Responsiveness: Using Canary Inputs to Dynamically Steer Approximation

(1)

Input Responsiveness: Using Canary Inputs to Dynamically Steer Approximation

Michael A. Laurenzano, Parker Hill, Mehrzad Samadi, Scott Mahlke, Jason Mars, Lingjia Tang

ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI)

June 15, 2016

(2)

Approximate Computing

(3)

Approximate Computing

(4)

Approximate Computing

• Trade small losses in accuracy for big

gains in performance/energy

(5)

Approximate Computing

• Trade small losses in accuracy for big gains in performance/energy

• Why approximation?

• Machine learning, data mining,

image, video and sound processing, statistics, graphics

• Loose constraints on output quality,

growing computational demands

(6)

Approximation Runtime Systems

(7)

Approximation Runtime Systems

• Central question — how to approximate?

(8)

Approximation Runtime Systems

• Central question — how to approximate?

• How to parameterize approximation

(9)

Approximation Runtime Systems

• Central question — how to approximate?

• How to parameterize approximation

• Where in the application to approximate

(10)

The Importance of Input

(11)

The Importance of Input

• Insight — input is a key

component in answering

this question

(12)

The Importance of Input

• Insight — input is a key component in answering this question

• Example — gamma

correction

(13)

The Importance of Input

• Insight — input is a key component in answering this question

• Example — gamma correction

input

(14)

The Importance of Input

• Insight — input is a key component in answering this question

• Example — gamma correction

input

exact

output

(15)

The Importance of Input

• Insight — input is a key component in answering this question

• Example — gamma correction

input

exact output

approximate

output

(8x8 tiling)

(16)

Why Input Responsiveness?

(17)

Why Input Responsiveness?

• Example: gamma correction, 2 tiling approximations, 800 inputs

(18)

Why Input Responsiveness?

• Example: gamma correction, 2 tiling approximations, 800 inputs

Output Quality

100% 95% 90% 85% 80% 75% 70%

0%

5%

10%

15%

20%

25%

30%

35% Missed Opportunity

Fast + High Quality TOQ = 90%

TOQ Violation

(a) 4×2 tiling approximation (5.9× speedup)

Proportion of Inputs

Output Quality

100% 95% 90% 85% 80% 75% 70%

0%

2%

4%

6%

8%

10%

12%

14%

TOQ Violation

(b) 8×8 tiling (22× speedup)

Output Quality

100% 95% 90% 85% 80% 75% 70%

0%

2%

4%

6%

8%

10%

12%

14%

Missed Opportunity Fast + High Quality TOQ = 90%

TOQ Violation

(c) 16×16 tiling (83× speedup)

Figure 2. Histograms of the accuracy of three tiling approximations applied to the same 800 images; some mix of missed opportunities and unacceptably low accuracy are present in each approximation

100% 95% 90% 85% 80% 75% 70%

0%

10%

20%

30%

40%

TOQ Violation

(a) Output quality achieved Output Quality

0%

2%

4%

6%

8%

10%

0× 100× 200× 300× 400×

(b) Speedup achieved (average is 61×) Speedup

0%

5%

10%

15%

(c) Tiling method chosen

Proportion of Inputs 16×16 8×16 16×32 8×32 32×32 4×16 8×8 16×64 32×16 32×64 64×128 4×8 4×32 16×128 32×128 4×4 8×64 8×4 24 others

Figure 3. A dynamic oracle approximation system using the most effective tiling approximation method (fastest without violating TOQ) achieves an average speedup of 61× and uses 42 different approximation options

results in an average speedup of 10.2× across our 13 test applications with small accuracy losses in program outputs.

IRA outperforms oracle versions of two classes of prior work while delivering the same level of accuracy, achieving more than 4× the speedup of oracle calibration techniques and 2× the speedup of oracle static profiling.

2. The Case for Input Driven Dynamism

The ability of approximate computation to produce high- quality results is one of the keys to making approximation broadly deployable in real systems. Many techniques for preserving result accuracy when choosing how to approximate focus on the worst case, resulting in overly conser- vative approximation for other cases. Here we discuss the opportunity available in the presence of a technique that dynamically controls approximation for individual inputs.

2.1 Input Matters for Output Quality

Input is an important part of the accuracy of an approximate computation. To illustrate this, we detail the output quality produced by three different tiling approximations [50] of an image processing application called gamma correction [42]

applied to 800 input images. Tiling is based on the assumption that, in many application domains such as image and video processing, elements nearby one another (e.g., pixels in an image) are likely to have similar values. Instead of computing each element of the output, a tiling approxi-

mation computes a single output element and projects that output onto the surrounding elements to form a tile. Tiling can be tuned to trade off lower accuracy for better performance by increasing the size of the tile.

Figure 2 presents histograms of the output quality for 800 different images across three tile sizes. For the purposes of illustration, we assume that the target output quality¹ (TOQ) of the approximation is 90%. As shown in the figure, for all three tile sizes, different inputs can result in very different output qualities. For example, across these inputs 8×8 tiling (Figure 2(b)) results in output qualities ranging from 78%-99% because the assumption made by the approximation technique (that nearby pixels are similar to one another) holds true to a different extent depending on the compo- sition of the input. Furthermore, we have observed that a wide range in output quality across inputs is not unique to tiling approximation and gamma correction, occurring across many applications and approximation techniques.

2.2 Limitations of Conventional Approaches

A common approach used to choose how to approximate is to select a single approximation option for some program and apply that approximation to multiple inputs. This approach to approximation suffers from a form of the prob- lem of aggregation, in which aggregate behavior (average or

1Target output quality (TOQ) is the minimum acceptable result accuracy [49], supplied by the user of the application.

163

Output Quality

100% 95% 90% 85% 80% 75% 70%

0%

5%

10%

15%

20%

25%

30%

TOQ Violation

(a) 4×2 tiling approximation (5.9× speedup)

Output Quality

100% 95% 90% 85% 80% 75% 70%

0%

2%

4%

6%

8%

10%

12%

14%

TOQ Violation

(b) 8×8 tiling (22× speedup)

Output Quality

100% 95% 90% 85% 80% 75% 70%

0%

2%

4%

6%

8%

10%

12%

14%

TOQ Violation

(c) 16×16 tiling (83× speedup)

Figure 2. Histograms of the accuracy of three tiling approximations applied to the same 800 images; some mix of missed opportunities and unacceptably low accuracy are present in each approximation

100% 95% 90% 85% 80% 75% 70%

0%

10%

20%

30%

40%

TOQ Violation

(a) Output quality achieved

Output Quality

0%

2%

4%

6%

8%

10%

0× 100× 200× 300× 400×

(b) Speedup achieved (average is 61×)

Speedup

0%

5%

10%

15%

(c) Tiling method chosen

Figure 3. A dynamic oracle approximation system using the most effective tiling approximation method (fastest without violating TOQ) achieves an average speedup of 61 × and uses 42 different approximation options

results in an average speedup of 10.2 × across our 13 test applications with small accuracy losses in program outputs.

IRA outperforms oracle versions of two classes of prior work while delivering the same level of accuracy, achieving more than 4 × the speedup of oracle calibration techniques and 2 × the speedup of oracle static profiling.

2. The Case for Input Driven Dynamism

The ability of approximate computation to produce high- quality results is one of the keys to making approximation broadly deployable in real systems. Many techniques for preserving result accuracy when choosing how to approx- imate focus on the worst case, resulting in overly conser- vative approximation for other cases. Here we discuss the opportunity available in the presence of a technique that dy- namically controls approximation for individual inputs.

2.1 Input Matters for Output Quality

Input is an important part of the accuracy of an approximate computation. To illustrate this, we detail the output quality produced by three different tiling approximations [50] of an image processing application called gamma correction [42]

applied to 800 input images. Tiling is based on the assump- tion that, in many application domains such as image and video processing, elements nearby one another (e.g., pix- els in an image) are likely to have similar values. Instead of computing each element of the output, a tiling approxi-

mation computes a single output element and projects that output onto the surrounding elements to form a tile. Tiling can be tuned to trade off lower accuracy for better perfor- mance by increasing the size of the tile.

Figure 2 presents histograms of the output quality for 800 different images across three tile sizes. For the purposes of illustration, we assume that the target output quality

¹

(TOQ) of the approximation is 90%. As shown in the figure, for all three tile sizes, different inputs can result in very differ- ent output qualities. For example, across these inputs 8 ×8 tiling (Figure 2(b)) results in output qualities ranging from 78%-99% because the assumption made by the approxima- tion technique (that nearby pixels are similar to one another) holds true to a different extent depending on the compo- sition of the input. Furthermore, we have observed that a wide range in output quality across inputs is not unique to tiling approximation and gamma correction, occurring across many applications and approximation techniques.

2.2 Limitations of Conventional Approaches

A common approach used to choose how to approximate is to select a single approximation option for some program and apply that approximation to multiple inputs. This ap- proach to approximation suffers from a form of the prob- lem of aggregation, in which aggregate behavior (average or

1

Target output quality (TOQ) is the minimum acceptable result accu- racy [49], supplied by the user of the application.

163 4x2 tiling, 5.9x speedup 16x16 tiling, 83x speedup

Output Quality

100% 95% 90% 85% 80% 75% 70%

0%

5%

10%

15%

20%

25%

30%

TOQ Violation

Output Quality

100% 95% 90% 85% 80% 75% 70%

0%

2%

4%

6%

8%

10%

12%

14%

TOQ Violation

Output Quality

100% 95% 90% 85% 80% 75% 70%

0%

2%

4%

6%

8%

10%

12%

14%

TOQ Violation

100% 95% 90% 85% 80% 75% 70%

0%

10%

20%

30%

40%

TOQ Violation

0%

2%

4%

6%

8%

10%

0× 100× 200× 300× 400×

0%

5%

10%

15%

2. The Case for Input Driven Dynamism

163

(19)

Why Input Responsiveness?

• Example: gamma correction, 2 tiling approximations, 800 inputs

• What if we take full advantage of the differences among inputs?

Output Quality

100% 95% 90% 85% 80% 75% 70%

0%

5%

10%

15%

20%

25%

30%

TOQ Violation

Output Quality

100% 95% 90% 85% 80% 75% 70%

0%

2%

4%

6%

8%

10%

12%

14%

TOQ Violation

Output Quality

100% 95% 90% 85% 80% 75% 70%

0%

2%

4%

6%

8%

10%

12%

14%

TOQ Violation

100% 95% 90% 85% 80% 75% 70%

0%

10%

20%

30%

40%

TOQ Violation

0%

2%

4%

6%

8%

10%

0× 100× 200× 300× 400×

0%

5%

10%

15%

2. The Case for Input Driven Dynamism

163

Output Quality

100% 95% 90% 85% 80% 75% 70%

0%

5%

10%

15%

20%

25%

30%

TOQ Violation

(a) 4×2 tiling approximation (5.9× speedup)

Output Quality

100% 95% 90% 85% 80% 75% 70%

0%

2%

4%

6%

8%

10%

12%

14%

TOQ Violation

(b) 8×8 tiling (22× speedup)

Output Quality

100% 95% 90% 85% 80% 75% 70%

0%

2%

4%

6%

8%

10%

12%

14%

TOQ Violation

(c) 16×16 tiling (83× speedup)

Figure 2. Histograms of the accuracy of three tiling approximations applied to the same 800 images; some mix of missed opportunities and unacceptably low accuracy are present in each approximation

100% 95% 90% 85% 80% 75% 70%

0%

10%

20%

30%

40%

TOQ Violation

(a) Output quality achieved

Output Quality

0%

2%

4%

6%

8%

10%

0× 100× 200× 300× 400×

(b) Speedup achieved (average is 61×)

Speedup

0%

5%

10%

15%

(c) Tiling method chosen

Figure 3. A dynamic oracle approximation system using the most effective tiling approximation method (fastest without violating TOQ) achieves an average speedup of 61 × and uses 42 different approximation options

results in an average speedup of 10.2 × across our 13 test applications with small accuracy losses in program outputs.

IRA outperforms oracle versions of two classes of prior work while delivering the same level of accuracy, achieving more than 4 × the speedup of oracle calibration techniques and 2 × the speedup of oracle static profiling.

2. The Case for Input Driven Dynamism

The ability of approximate computation to produce high- quality results is one of the keys to making approximation broadly deployable in real systems. Many techniques for preserving result accuracy when choosing how to approx- imate focus on the worst case, resulting in overly conser- vative approximation for other cases. Here we discuss the opportunity available in the presence of a technique that dy- namically controls approximation for individual inputs.

2.1 Input Matters for Output Quality

Input is an important part of the accuracy of an approximate computation. To illustrate this, we detail the output quality produced by three different tiling approximations [50] of an image processing application called gamma correction [42]

applied to 800 input images. Tiling is based on the assump- tion that, in many application domains such as image and video processing, elements nearby one another (e.g., pix- els in an image) are likely to have similar values. Instead of computing each element of the output, a tiling approxi-

mation computes a single output element and projects that output onto the surrounding elements to form a tile. Tiling can be tuned to trade off lower accuracy for better perfor- mance by increasing the size of the tile.

Figure 2 presents histograms of the output quality for 800 different images across three tile sizes. For the purposes of illustration, we assume that the target output quality

¹

(TOQ) of the approximation is 90%. As shown in the figure, for all three tile sizes, different inputs can result in very differ- ent output qualities. For example, across these inputs 8 ×8 tiling (Figure 2(b)) results in output qualities ranging from 78%-99% because the assumption made by the approxima- tion technique (that nearby pixels are similar to one another) holds true to a different extent depending on the compo- sition of the input. Furthermore, we have observed that a wide range in output quality across inputs is not unique to tiling approximation and gamma correction, occurring across many applications and approximation techniques.

2.2 Limitations of Conventional Approaches

A common approach used to choose how to approximate is to select a single approximation option for some program and apply that approximation to multiple inputs. This ap- proach to approximation suffers from a form of the prob- lem of aggregation, in which aggregate behavior (average or

1

Target output quality (TOQ) is the minimum acceptable result accu- racy [49], supplied by the user of the application.

163 4x2 tiling, 5.9x speedup 16x16 tiling, 83x speedup

Output Quality

100% 95% 90% 85% 80% 75% 70%

0%

5%

10%

15%

20%

25%

30%

TOQ Violation

Output Quality

100% 95% 90% 85% 80% 75% 70%

0%

2%

4%

6%

8%

10%

12%

14%

TOQ Violation

Output Quality

100% 95% 90% 85% 80% 75% 70%

0%

2%

4%

6%

8%

10%

12%

14%

TOQ Violation

100% 95% 90% 85% 80% 75% 70%

0%

10%

20%

30%

40%

TOQ Violation

0%

2%

4%

6%

8%

10%

0× 100× 200× 300× 400×

0%

5%

10%

15%

2. The Case for Input Driven Dynamism

163

(20)

Why Input Responsiveness?

Output Quality

100% 95% 90% 85% 80% 75% 70%

0%

5%

10%

15%

20%

25%

30%

TOQ Violation

Output Quality

100% 95% 90% 85% 80% 75% 70%

0%

2%

4%

6%

8%

10%

12%

14%

TOQ Violation

Output Quality

100% 95% 90% 85% 80% 75% 70%

0%

2%

4%

6%

8%

10%

12%

14%

TOQ Violation

100% 95% 90% 85% 80% 75% 70%

0%

10%

20%

30%

40%

TOQ Violation

0%

2%

4%

6%

8%

10%

0× 100× 200× 300× 400×

0%

5%

10%

15%

2. The Case for Input Driven Dynamism

163 High output quality 61x speedup

• Example: gamma correction, 2 tiling approximations, 800 inputs

• What if we take full advantage of the differences among inputs?

Output Quality

100% 95% 90% 85% 80% 75% 70%

0%

5%

10%

15%

20%

25%

30%

TOQ Violation

Output Quality

100% 95% 90% 85% 80% 75% 70%

0%

2%

4%

6%

8%

10%

12%

14%

TOQ Violation

Output Quality

100% 95% 90% 85% 80% 75% 70%

0%

2%

4%

6%

8%

10%

12%

14%

TOQ Violation

100% 95% 90% 85% 80% 75% 70%

0%

10%

20%

30%

40%

TOQ Violation

0%

2%

4%

6%

8%

10%

0× 100× 200× 300× 400×

0%

5%

10%

15%

2. The Case for Input Driven Dynamism

163

Output Quality

100% 95% 90% 85% 80% 75% 70%

0%

5%

10%

15%

20%

25%

30%

TOQ Violation

(a) 4×2 tiling approximation (5.9× speedup)

Output Quality

100% 95% 90% 85% 80% 75% 70%

0%

2%

4%

6%

8%

10%

12%

14%

TOQ Violation

(b) 8×8 tiling (22× speedup)

Output Quality

100% 95% 90% 85% 80% 75% 70%

0%

2%

4%

6%

8%

10%

12%

14%

TOQ Violation

(c) 16×16 tiling (83× speedup)

Figure 2. Histograms of the accuracy of three tiling approximations applied to the same 800 images; some mix of missed opportunities and unacceptably low accuracy are present in each approximation

100% 95% 90% 85% 80% 75% 70%

0%

10%

20%

30%

40%

TOQ Violation

(a) Output quality achieved

Output Quality

0%

2%

4%

6%

8%

10%

0× 100× 200× 300× 400×

(b) Speedup achieved (average is 61×)

Speedup

0%

5%

10%

15%

(c) Tiling method chosen

Figure 3. A dynamic oracle approximation system using the most effective tiling approximation method (fastest without violating TOQ) achieves an average speedup of 61 × and uses 42 different approximation options

results in an average speedup of 10.2 × across our 13 test applications with small accuracy losses in program outputs.

IRA outperforms oracle versions of two classes of prior work while delivering the same level of accuracy, achieving more than 4 × the speedup of oracle calibration techniques and 2 × the speedup of oracle static profiling.

2. The Case for Input Driven Dynamism

The ability of approximate computation to produce high- quality results is one of the keys to making approximation broadly deployable in real systems. Many techniques for preserving result accuracy when choosing how to approx- imate focus on the worst case, resulting in overly conser- vative approximation for other cases. Here we discuss the opportunity available in the presence of a technique that dy- namically controls approximation for individual inputs.

2.1 Input Matters for Output Quality

Input is an important part of the accuracy of an approximate computation. To illustrate this, we detail the output quality produced by three different tiling approximations [50] of an image processing application called gamma correction [42]

applied to 800 input images. Tiling is based on the assump- tion that, in many application domains such as image and video processing, elements nearby one another (e.g., pix- els in an image) are likely to have similar values. Instead of computing each element of the output, a tiling approxi-

mation computes a single output element and projects that output onto the surrounding elements to form a tile. Tiling can be tuned to trade off lower accuracy for better perfor- mance by increasing the size of the tile.

Figure 2 presents histograms of the output quality for 800 different images across three tile sizes. For the purposes of illustration, we assume that the target output quality

¹

(TOQ) of the approximation is 90%. As shown in the figure, for all three tile sizes, different inputs can result in very differ- ent output qualities. For example, across these inputs 8 ×8 tiling (Figure 2(b)) results in output qualities ranging from 78%-99% because the assumption made by the approxima- tion technique (that nearby pixels are similar to one another) holds true to a different extent depending on the compo- sition of the input. Furthermore, we have observed that a wide range in output quality across inputs is not unique to tiling approximation and gamma correction, occurring across many applications and approximation techniques.

2.2 Limitations of Conventional Approaches

A common approach used to choose how to approximate is to select a single approximation option for some program and apply that approximation to multiple inputs. This ap- proach to approximation suffers from a form of the prob- lem of aggregation, in which aggregate behavior (average or

1

Target output quality (TOQ) is the minimum acceptable result accu- racy [49], supplied by the user of the application.

163 4x2 tiling, 5.9x speedup 16x16 tiling, 83x speedup

Output Quality

100% 95% 90% 85% 80% 75% 70%

0%

5%

10%

15%

20%

25%

30%

TOQ Violation

Output Quality

100% 95% 90% 85% 80% 75% 70%

0%

2%

4%

6%

8%

10%

12%

14%

TOQ Violation

Output Quality

100% 95% 90% 85% 80% 75% 70%

0%

2%

4%

6%

8%

10%

12%

14%

TOQ Violation

100% 95% 90% 85% 80% 75% 70%

0%

10%

20%

30%

40%

TOQ Violation

0%

2%

4%

6%

8%

10%

0× 100× 200× 300× 400×

0%

5%

10%

15%

2. The Case for Input Driven Dynamism

163

(21)

Achieving Input Responsiveness

(22)

Achieving Input Responsiveness

• Goal — end-to-end runtime system to choose how to approximate, customized for each input

• Maximize performance given an accuracy target

(23)