On the Complexity of Hardness Amplification
Chi-Jen Lu, Shi-Chun Tsai, Member, IEEE, and Hsin-Lung Wu
Abstract—For 2 (0; 1) and k; n 2 , we study the task of transforming a hard functionf : f0; 1gn ! f0; 1g, with which any small circuit disagrees on(1 0 )=2 fraction of the input, into a harder function f0, with which any small circuit disagrees on (1 0 k)=2 fraction of the input. First, we show that such hard-ness amplification, when carried out in some black-box way, must require a high complexity. In particular, it cannot be realized by a circuit of depthd and size 2o(k )or by a nondeterministic cir-cuit of sizeo(k= log k) (and arbitrary depth) for any 2 (0; 1). This extends the result of Viola, which only works when(1 0 )=2 is small enough. Furthermore, we show that even without any re-striction on the complexity of the amplification procedure, such a black-box hardness amplification must be inherently nonuniform in the following sense. To guarantee the hardness of the resulting functionf0, even against uniform machines, one has to start with a functionf, which is hard against nonuniform algorithms with (k log(1=)) bits of advice. This extends the result of Trevisan and Vadhan, which only addresses the case with(1 0 )=2 = 20n. Fi-nally, we derive similar lower bounds for any black-box construc-tion of a pseudorandom generator (PRG) from a hard funcconstruc-tion. To prove our results, we link the task of hardness amplifications and PRG constructions, respectively, to some type of error-reduction codes, and then we establish lower bounds for such codes, which we hope could find interest in both coding theory and complexity theory.
Index Terms—Computational complexity, hardness amplifica-tion, list-decodable code, pseudorandom generator.
I. INTRODUCTION
A. Background
U
NDERSTANDING the power of randomness in compu-tation is one of the central topics in theoretical computer science. A major open question is the versus question, asking whether all randomized polynomial-time algorithms can be converted into deterministic polynomial-time ones. A stan-dard approach to derandomizing relies on constructingManuscript received March 7, 2007; revised July 9, 2008. Current ver-sion published September 17, 2008. The work of C.-J. Lu was supported in part by the National Science Council of Taiwan under Contract NSC93-2213-E-001-004. The work of S.-C. Tsai was supported in part by the Na-tional Science Council of Taiwan under Contract NSC-93-2213-E-009-035. The work of H.-L. Wu was supported in part by the National Science Council of Taiwan under Contract NSC-97-2218-E-305-001-MY2. The material in this paper was presented at the 20th Annual Computational Complexity Confer-ence, San Jose, CA, June 2005.
C.-J. Lu is with the Institute of Information Science, Academia Sinica, Taipei 115, Taiwan (e-mail: [email protected]).
S.-C. Tsai is with the Department of Computer Science and Information En-gineering, National Chiao Tung University, Hsinchu 30050, Taiwan (e-mail: [email protected]).
H.-L. Wu is with the Department of Computer Science and Information En-gineering, National Taipei University, Taipei, Taiwan (e-mail: hsinlung@mail. ntpu.edu.tw).
Communicated by A. Canteaut, Associate Editor for Complexity and Cryptography.
Digital Object Identifier 10.1109/TIT.2008.928988
the so-called pseudorandom generators (PRG), which stretch a short random seed into a long pseudorandom string that looks random to circuits of polynomial size. So far, all known con-structions of PRG are based on unproven assumptions of the nature that certain functions are hard to compute. The idea of converting hardness into pseudorandomness first appeared in the work of Blum and Micali [2] and Yao [29], who showed how to obtain a PRG from a one-way function. Then, Nisan and Wigderson [18] showed that a PRG can be constructed from a Boolean function, which is hard in average case, and this initi-ated a series of works. To get a stronger result, one would like to weaken the hardness assumption, and [18], [1], [10] showed that, in fact, one can start from a (slightly) hard Boolean func-tion and transform it into a much harder one, before using it to build a PRG. Finally, Impagliazzo and Wigderson [14] proved that one can transform a function in that is hard in worst case into one that is hard in average case, both against circuits of ex-ponential size. As a result, they obtained under the assumption that some function in cannot be computed by a circuit of subexponential size. Simpler proofs and better trade-offs have been obtained since then [23], [13], [22], [26].
Note that hardness amplification is the major step in deran-domizing in the research discussed above, because the step from an average-case hard function to a PRG is relatively simple and has low complexity. We say that a Boolean function is –hard (or has hardness ) against circuits of size if any such circuit attempting to compute must make errors on at least fraction of the input. The error bound is the main param-eter characterizing the hardness; the size bound also reflects the hardness, but it plays a lesser role in our study. Formally, the task of hardness amplification is to transform a function
that is –hard against circuits of size into a function that is –hard against circuits of size , with and close to (usually slightly smaller than) . Normally, one would like to have as close to as possible, preferably with or even
, so that one could have close to ; other-wise, one would only be able to have the hardness of against much smaller circuits. Furthermore, one would like to stay in the same complexity class of , so that one could establish the relation among hardness assumptions within the same com-plexity class.
Two issues come up from those works on hardness amplifi-cation. The first is on the complexity of the amplification pro-cedure. All previous amplification procedures going from worst case hardness ( ) to average case hardness (
) need exponential time [1], [14], [23] (or slightly better, in linear space [16] or [27]). As a result, such a hardness amplification is only known for functions in high complexity classes. Then, a natural question is as follows: Can it be done for functions in lower complexity classes? For
example, given a function in , which is worst case hard, can we transform it into another function in , which is average case hard? Only for some range of hardness [e.g., starting from mild hardness, with ] is this known to be pos-sible [19], [9].
The second issue is that hardness amplification typically in-volves nonuniformity in the sense that hardness is usually mea-sured against nonuniform circuits. In fact, one usually needs to start from a function that is hard against nonuniform circuits, even if one only wants to produce a function that is hard against uniform Turing machines. This is why most results on hardness amplification are based on nonuniform assumptions.
B. Black-Box Hardness Amplification
In light of the discussion above, one would hope to show that some hardness amplification is indeed impossible. However, it is not clear what this means, especially given the possibility (in which many people believe) that average case hard functions may indeed exist.
One important type of hardness amplification is called
black-box hardness amplification. First, the initial function is only given as a black box to construct the new function . That is, there is an oracle procedure such that , so only uses as an oracle and does not depend on the internal structure of . Second, the hardness of the new function is proved in a black-box way. That is, there is an oracle procedure , such that if some algorithm disagrees with on less than fraction of the input, then using as an oracle disagrees with on less than fraction of the input. Again, only uses as an oracle and does not depend on the internal structure of . We call the encoding function and the decoding function. In fact, almost all previous constructions of hardness amplification are done in such a black-box way, so it is nice to establish impossibility results for this type of approaches.
C. Previous Lower Bound Results
Viola [27] gave the first lower bound on the complexity required for black-box hardness amplification. He showed that to transform a worst case hard function into a mildly hard function , both against circuits of size , the en-coding function cannot be realized in the complexity class . This rules out the possibility of doing such hardness amplification in , which explains why previous procedures all require a high computational com-plexity. He also showed a similar lower bound for black-box construction of PRG from a worst case hard function.
Trevisan and Vadhan [25] observed that a black-box hard-ness amplification from worst case hardhard-ness corresponds to an error-correcting code with some list-decoding property. Then, results from coding theory can be used to show that for such amplification from worst case hardness to hardness , the decoding function must need bits of ad-vice in order to compute . This explains why almost all pre-vious hardness amplification results were done in a nonuniform setting, except [15] and [25], which did not work in a black-box way.
There were also impossibility results on weaker types of hard-ness amplification, from worst case hardhard-ness to average case hardness. Bogdanov and Trevisan [3] considered hardness am-plification for functions in in which the black-box require-ment on the encoding function is dropped. They showed that the decoding function cannot be computed nonadaptively in poly-nomial time unless collapses. Viola, in another recent paper [28], considered hardness amplification in which the black-box requirement on the decoding function is dropped. He showed that if the encoding function can be computed in , then there exists an average case hard function in unconditionally. We will not consider such weaker types of hardness amplification in this paper, and hereafter when we refer to hardness amplifi-cation, we always mean the black-box one.
D. Our Results
Previous lower bound results only address hardness in a spe-cific range. However, whether one can amplify hardness beyond this range is also a natural and interesting question. For example, it is known that a black-box hardness amplification from hard-ness to average case hardness can be realized in poly-nomial time [29], [5], [10], [14]. Can such a hardness amplifi-cation be realized in a lower complexity class, such as ? Can it start from hardness below and still be realized in polynomial time? Can it be done in a uniform way (with a uniform decoding function)? In general, how does the quality of a hardness amplification (the amount of hardness increased) determine its inherent complexity or nonuniformity? All these questions will be addressed in this paper. We generalize pre-vious results [27], [25] and consider hardness amplification in a much broader spectrum: from hardness to hardness , for general and . We should stress that our work is mainly inspired by the seminal works of [27] and [25] and we follow closely their approaches; our main effort is to generalize their ideas and to get a more precise picture.
Following [28], we consider a more restricted model called
parallel black-box hardness amplification, in which oracle
queries by the encoding function are done in a nonadaptive way. More precisely, we say that a circuit class realizes a parallel black-box hardness amplification if its encoding function can be implemented in the following way. Given any input , it first generates a circuit together with query inputs , then queries at those inputs, and finally computes as its output. Note that here and only depend on but not . Although this is a more restricted model, almost all previous constructions of hardness amplification can be done in this way, so it would be nice to know its limitation. Furthermore, through a standard simulation [4], [8], negative results in this model can, in fact, be translated to those in the general black-box model.
Our first result addresses both the complexity issue and the nonuniformity issue in the same framework, showing how com-plexity constraints on the encoding function result in the in-herent nonuniformity of the decoding function. Formally, we prove that if such a parallel black-box hardness amplification, from hardness to hardness , is realized by circuits of depth and size , then the decoding func-tion must need an advice of length . Translating this
to the general model, we obtain the same advice lower bound when such a (general) black-box hardness amplification is real-ized in . This implies that no such hardness amplification is possible if the hardness is measured against cir-cuits of size .
Our lower bound is almost tight because the well-knownXOR
lemma [29], [5] gives a way to realize a parallel black-box hard-ness amplification by circuits of depth and size , with using an advice of length . Note that Viola’s result in [27] is a special case of ours, because he only addressed explicitly the specific case with
and (or equivalently,
and ). Although it seems that his technique can be extended to show lower bounds when is small enough, but beyond that, say with , it fails to give a meaningful bound. We can, in fact, cover this case: our result implies that circuits cannot realize a parallel black-box hardness amplification, say, from hardness to hardness . On the other hand, our result when restricted to worst case to average case hardness amplification is incomparable to those of [3] and [28].1Finally, two interesting facts follow from our result. First, it is impossible to produce in a black-box way a function that is –hard against a uniform low complexity class, say , even if we start from a function that is –hard against a uniform but arbitrarily high complexity class equipped with an advice of length , say . On the other hand, it is easy to show that hard functions against do exist.2 This demonstrates one severe weakness of black-box hardness amplifications. Second, when amplifying hardness from to , the complexity of such amplifi-cation is determined mainly by the parameter ; a larger value of results in a higher complexity requirement, for typical values of . Thus, to determine the complexity needed for a hardness amplification process, one should express the initial and final hardness in the forms of and , respectively.3This point was not clear from previous works.
Note that our first result becomes meaningless for as the circuit size becomes . Our second result takes care of this: we show that if a parallel black-box hardness amplification, from hardness to hardness , is realized by nondeterministic circuits of size , even with arbitrary depth, then the decoding function must need an advice of length . For ex-ample, to amplify hardness from to , our second result implies that it cannot be realized by nondetermin-istic circuits of size in a parallel black-box way.
1In [3], the complexity lower bound is given on the decoding function
in-stead, under the unproven (though widely believed) assumption that does not collapse. In [28], a more general type of hardness amplification than ours is considered, but the possibility of such hardness amplification is not ruled out as we do; instead, it was shown that if the encoding function can be computed in
, a hard function in exists unconditionally.
2For example, the parity function is (1=2 0 2 )–hard against
(O(1)). However, according to our result, its hardness cannot be shown in such a black-box way.
3Note that for a function with hardness(1 0 )=2 against small circuits, the
quantity is the maximum correlation of the function with such circuits. There-fore, our result shows that to reduce such correlation from to , the com-plexity is mainly determined byk.
Our third result shows that even without any complexity con-straint on the encoding or decoding function, amplification be-tween certain range of hardness is still inherently nonuniform. For the special case of amplifying hardness beyond 1/4, the need of nonuniformity can be shown using the Plotkin bound [21] from coding theory. We consider hardness amplification in a general range and obtain a quantitative bound on the amount of nonuniformity. More precisely, we show that to amplify hard-ness from to , the decoding function must need an advice of bits. Thus, when , an advice of length is necessary, and when
for some constant , such hardness amplification must be inherently nonuniform. Our result generalizes that of Trevisan and Vadhan [25].
Finally, we derive similar lower bounds on black-box con-structions of PRG from hard functions.
E. Our Techniques
Our results are obtained via a connection between black-box hardness amplifications and some type of “error-reduction” codes, which generalizes the connection given by Trevisan and Vadhan [25] and Viola [27]. A similar observation was also made by Trevisan [24]. Formally, a black-box amplification from hardness to hardness induces a code with the following list decoding property, which is also known as approximate list decoding [11]. Given a corrupted codeword with a fraction of less than errors, we can always find a small list of candidate messages such that one of them is close to the original message, with their relative Hamming distance less than . Therefore, we can focus our attention on such codes, as results on such codes immediately give results on corresponding hardness amplifications.
Our first two results are based on the following idea. A code with such a list-decoding property can only have a small number of codewords close to any codeword, so a random perturbation on an input message is unlikely to result in a close codeword. On the other hand, if such a code is computed by an algorithm that is insensitive to noise on the input, then a random pertur-bation on an input message is likely to result in a close code-word, and we reach a contradiction. Circuits of small size, or circuits of small depth and moderate size can be shown to be insensitive to noise on their input. Thus, they cannot be used to compute such a code and the corresponding hardness amplifica-tion. This basically follows Viola’s idea in [27], but because we consider hardness amplification in a much broader spectrum, a more involved analysis is required. For example, because Viola only considered the case with a small hardness, he only had to deal with noise of a small rate. With such a small noise rate, the output value will only be affected with a small probability, and small loss in his analysis does not matter too much. How-ever, if a large hardness is considered, a high noise rate is needed and then the loss in his analysis will become intolerable, and his bound will become meaningless (see Remark 4 in Section II-D for details). To overcome this problem, we drive another upper bound on noise sensitivity, which works for any noise rate and thus can be used for hardness in a general range.
For the nonuniformity of hardness amplification, we show that given a corrupted codeword with a high fraction
(for a small ) of errors, one may need a long list of candidate messages in order to have one of them within a small relative distance (for a large ) to the original message. To show this, we would like to find a set of messages such that some ball of relative radius in the codeword space contains many of their corresponding codewords, but any ball of relative radius in the message space contains only a small number of messages from that set. We choose these messages randomly and show that they have some chance of satisfying the condition above when is larger than to some extent.
Finally, to prove lower bounds for black-box constructions of PRG from hard functions, we discover that there is also a connection between the error-reduction codes we just consid-ered and such PRG constructions. Then, the results we obtain for such codes immediately yield results for such PRG con-structions. Note that in [27], Viola used a connection between black-box PRG constructions and randomness extractors, and then he proved a separate lower bound for extractors, in addi-tion to that for codes. Our connecaddi-tion, in fact, can be seen as a connection between extractors and codes, and with this connec-tion, we no longer need a separate proof for PRG constructions.
F. Organization of This Paper
First, some preliminaries are given in Section II. Then, in Sections III and IV, we prove the impossibility results of hard-ness amplification by constant-depth circuits and nondetermin-istic circuits, respectively. In Section V, we show that hardness amplification, in general, is inherently nonuniform. Finally, we show the impossibility results for black-box PRG constructions from hard functions in Section VI.
II. PRELIMINARIES
For any , let denote the set and let denote the uniform distribution over the set . When we sample from a finite set, the default distribution is the uni-form one. For a string , let denote the th bit of . All the logarithms in this paper will have base two. Define the binary
entropy function .
We need some standard complexity classes. Let
denote the class of functions computed by alternating Turing machines in time with at most alternations, and let
denote . Let denote the polynomial-time
hier-archy, which is . Let denote
the class of functions computed by nondeterministic Turing ma-chines in time . More information about complexity classes can be found in standard textbooks, such as [20]. The circuits we consider here consist ofAND/OR/NOTgates, allowing unbounded fan-in forAND/ORgates. The size of a circuit is the number of noninput gates it has and the depth of circuit is the number of gates on the longest path from an input bit to the output gate. We call such circuits circuits.
Definition 1: Let denote the class of functions com-puted by circuits of depth and size .
Note that the standard complexity class corresponds to our class . We also introduce the nondeter-ministic version of circuits. An circuit has two parts
of inputs: the real input and the witness input . The Boolean function computed by such a circuit is defined as
if and only if there exists a witness such that .
Definition 2: Let be the class of functions computed by circuits of size .
A function with more than one output bits is said to be com-puted by some type of circuits (e.g., or ) if each output bit can be computed by one such circuit.
A. Black-Box Hardness Amplification and Pseudorandom Generators
Informally speaking, a function is hard if any algorithm without enough complexity must make some mistakes. For-mally, we define the hardness of a function as follows.
Definition 3: We say that a function
has hardness against circuits of size if for any circuit of size
Note that we use the error bound to characterize the hard-ness of a function, and we pay less (sometimes no) attention to the size bound . For hardness amplification, we want to trans-form a function with a smaller hardness into a function with a larger hardness . We will focus on a special type of hardness amplification called black-box hardness amplification, defined next, which consists of two oracle procedures and . We allow to be a nonuniform oracle Turing machine, and we write to denote taking an oracle and an advice string .
Definition 4: A black-box hardness
amplifica-tion consists of an oracle procedure
(called encoding function) and a nonuniform oracle Turing
ma-chine (called decoding function)
with the following property. For any , if a
function satisfies
then there exists an advice string such that
For a complexity class , we say that the black-box hardness amplification can be realized in if for any oracle , the proce-dure can be computed in .
Here, the transformation of the initial function into a harder function is done in a black-box way, as the harder function only uses as an oracle. Moreover, the hardness of the new function is also guaranteed in a black-box way. Namely, any algorithm breaking the hardness condition of can be used as an oracle for a machine to break the hardness condition of . Note that neither of the hardness refers to circuit size, and no constraint is placed on the complexity of the procedure . This freedom makes our impossibility
results stronger. The parameter characterizes the amount of nonuniformity associated with this process. When , we say the hardness amplification is nonuniform.
Remark 1: One can also use the notion of
“advan-tage” to characterize the hardness of a Boolean func-tion. We say that any circuit of size has advantage at most for computing if for any such a circuit , . Clearly, the advantage is related to the hardness in the form . We will focus on the task of amplifying hardness from to , or equivalently, reducing the advantage from to . We choose to present our results in terms of hardness instead of advantage for the following two reasons. First, when talking about hardness amplification, it seems more natural and less confusing to use hardness instead of advantage. Second, as we will see, there is some nice connection between hardness amplifications and error-correcting codes, in which hardness of functions corresponds naturally to distance in codes. However, the drawback of using hardness instead of advantage is that our notation sometimes looks more cumbersome.
Similarly, we can define the notion of black-box construction of pseudorandom generators from hard functions.
Definition 5: A black-box PRG construction
con-sists of an oracle procedure (called encoding function) and a nonuniform oracle Turing machine (called decoding function) with the following property. For any , if a function
satisfies
then there exists an advice string such that
For a complexity class , we say that the black-box PRG con-struction can be realized in if for any oracle , the procedure
can be computed in .
Remark 2: When talking about a black-box hardness
ampli-fication or PRG construction, we usually mean a sequence of them, parameterized by the parameter . Other parameters such as are, in fact, allowed to be functions of
.
In this general model of black-box hardness amplification or PRG construction, we do not put any restriction on how the or-acle is queried by the encoding function ( or ). On the other hand, we will also consider the following more restricted model, first introduced in [28], in which the oracle can only be queried in a nonadaptive way. We call such model a parallel black-box hardness amplification or PRG construction. More precisely, we define the following.
Definition 6: Let be a class of circuits, such as or . We say that realizes a parallel
black-box hardness amplification, if we have a black-box hardness amplification in which the encoding function can be implemented in the following way. Given any oracle
and any input , it first generates a circuit together with query inputs , then queries at those inputs, and finally outputs . The case of parallel black-box PRG construction is defined similarly.
Note that and are produced before the oracle is actually queried, so they depend on but not on the oracle . This restriction makes it easier to obtain negative (or lower bound) results in such a parallel model. Nevertheless, the fol-lowing lemma provides a way to translate such results to those in the general black-box model.
Lemma 1: If a black-box hardness amplification
(or PRG construction) can be realized in , then a parallel black-box hardness amplification (or PRG construction) can be realized in .
Proof: Consider any black-box hardness amplification
(the case of PRG construction is similar) with the encoding function such that for any oracle , belongs to . It is known from [4] and [8] that by adding two alternations (an existential one for guessing the oracle answers along a computational branch and a universal one for verifying the guessed answers), one can transform into another procedure that only queries once in each branch of its computation. Then, by a standard simulation of alternating Turing machines by circuits [4], [8], we know that for any input , the value of can be computed by a circuit in with the answers to the corresponding oracle queries given as part of the input. Note that the circuit and the oracle queries depend only on the input but not the oracle . Thus, we have a parallel black-box hardness amplification
realized in .
B. Codes and Correspondence to Hardness Amplification
We measure the distance between two strings by their relative Hamming distance.
Definition 7: For , define their
dis-tance as their relative Hamming distance, namely, .
According to this distance, we define open balls of radius in the space .
Definition 8: For any , , and ,
let , which
is the open ball in of radius centered at . Let denote the set consisting of all such balls. The following simple fact gives an upper bound on the size of such a Hamming ball.
Fact 1: The size of any ball in is at most .
We borrow the notion of list-decodable codes, but we ex-tend it in a way that leads to some natural correspondence with black-box hardness amplifications.
Definition 9: We call a -list code if for any , there are balls from such that if a codeword is con-tained in , then is contained in one of those balls.
A -list code is related to a standard list-decodable code in the way that each ball in contains at most codewords. Next, we show how such a code arises naturally from a black-box hardness amplification. Let
and . Given any oracle algorithm , let us define the corresponding code
as . That is, seeing any function as a vector in , produces as output the function , which is seen as a vector in . The following is a simple generalization of an observation by Viola [27].
Lemma 2: Let be the encoding
function of a black-box hardness amplification.
Then, , defined as , is a
-list code.
Proof: Let be the encoding function of a black-box hardness amplification, and let be the corresponding decoding function that is an oracle Turing machine with an -bit advice. Consider any ,
seen as . For any codeword
with ,
by Definition 4, there exists an such that . That is, if is in , then is contained in one of the balls of radius centered at for . Therefore, is a -list code.
Remark 3: Note that if a circuit class can realize a par-allel hardness amplification, then every output bit of the corre-sponding code can be computed by a circuit in . This is because for any input , the th output bit of
equals , which is
computed by some circuit on some bits of . In Section VI, we will show that there also exists a natural correspondence between black-box PRG constructions and such list-decodable codes.
C. Noise Sensitivity
Following [19] and [27], we will apply Fourier analysis on Boolean functions. For any and for any
, let Here is a
well-known fact.
Fact 2: For any , .
It is known that for circuits of small depths, the main contribution to the above sum comes from the low-order terms.
Lemma 3 [17]: For any
and for any , .
This can be used to show that circuits of small depth are insensitive to noise on their input. We will need the following more precise relation between the noise sensitivity of a Boolean function and its Fourier coefficients.
Lemma 4: Suppose is sampled from the uniform distribu-tion over and is obtained by flipping each bit of in-dependently with probability . Then, for any
and for any , .
Proof: We know from [19, Prop. 9] that
. Note that
Then, the lemma follows from Fact 2.
Combining Lemmas 3 and 4, we immediately have the following.
Corollary 1: Suppose and are sampled as in Lemma 4.
Then, for any and for any
, .
Remark 4: In [27], Viola derived a weaker bound
, with ,
which becomes vacuous when is not small enough. This prevents him from having a meaningful bound when the hardness is not small enough. The main loss in his derivation comes from his use of the inequality . Our Lemma 4 uses a different inequality to avoid this problem.
III. IMPOSSIBILITY OF AMPLIFICATION BY
SMALL-DEPTHCIRCUITS
In this section, we will show that any parallel black-box hardness amplification realized in
with small and must be highly nonuniform. More precisely, we will prove the following.
Theorem 1: There exist constants such that
for any and any with
and , any parallel
black-box hardness amplification realized in
must have .
Before giving the proof, let us take a closer look at the the-orem itself and discuss some of its consequences. First, note that the conditions on the ranges of and are natural in the fol-lowing sense. When , the initial function is already hard enough, so hardness amplification is usually not needed. When , the resulting function only has a very small hardness, which is rarely what hardness amplifica-tion is used to achieve. Also, as discussed in the Introducamplifica-tion, hardness amplifications normally have close to (preferably with ), therefore , which is at least , would be much larger than .
Although Theorem 1 is on the more restricted parallel model, it in fact implies the following result on the general model of hardness amplification, according to Lemma 1.
Corollary 2: Under the same condition as in Theorem 1, no
black-box hardness amplification can be
Note that Viola’s result [27] is a special case of ours, with initial hardness (amplifying from worst case hard-ness). A closer look at his technique shows that it, in fact, can be extended to cases with small initial hardness. For example, with , his technique can be modified to show the impossibility in to amplify the hardness to with , which also follows from our corollary above. How-ever, as discussed in Remark 4, when the initial hardness grows beyond a certain point, say to , his technique fails to give a meaningful bound. Moreover, our lower bound al-most matches the upper bound given by the well-known XOR
lemma [29], [5], while the technique in [27] does not yield such a bound.
Theorem 2: For any and any , a parallel
black-box hardness amplification can be
real-ized in for .
Proof: The encoding function is
, with , defined as
It is known that the parity of bits can be computed by an circuit (cf., [8]), and note that this circuit and those query inputs do not depend on the oracle . Furthermore, using Levin’s proof for theXORlemma given in [5], one can construct a decoding function that uses an advice of length . Thus, we have the theorem.
Now we proceed to prove Theorem 1.
Proof (of Theorem 1): Consider any parallel black-box
hardness amplification realized in , with for a small enough positive constant . Let and . Recall from Lemma 2 that such a hardness amplification induces a -list code . Then, from Remark 3, it suffices to show that any such code computed by an circuit
must have .
The basic idea behind the proof is the following. Suppose has only a small number of codewords close to any codeword. Then, a random perturbation on an input message is unlikely to result in a close codeword. On the other hand, if is computed by an circuit with small and , which is insensitive to noise on the input, then a random perturbation on an input message is likely to result in a close codeword, and we reach a contradiction.
Now we give the details. Let be sampled from the uniform distribution over and let be the random variable ob-tained by flipping each bit of independently with some prob-ability . We set so that is only slightly larger than .4We call any two codewords close if their (relative) distance is less than . The next lemma gives a lower bound on the probability that is close to , which relies on the fact that such an circuit is insensitive to noise on the input.
4We do not attempt to optimize parameters here, and in fact, it suffices to set
= (1 0 o(1)).
Lemma 5: There exist constants such that for any
and any with , if
, then is close to .
Proof: Suppose for a small enough
constant . Suppose for some constant such that . Then, using Corollary 1 with
, we have that for each
Therefore, , which implies
that is not close to by Markov in-equality. Thus
is close to
for some constant .
Next, we give an upper bound on the probability that is close to , which relies on the fact that each codeword is only close to a small number of other codewords. This requires a more careful analysis than that in [27], in order to get the tighter bound we need.
Lemma 6: For any -list code ,
is close to .
Proof: Consider any fixed . Because is a
-list code, there are at most dif-ferent ’s such that is close to . The lemma would follow easily if each such had a very small probability to occur. However, this may not be the case in general. We will show that although some ’s may occur with higher probability, there are not too many of them, so their overall contribution is still toler-able.
For any ,
, which decreases
as increases. Let .5 Call
good for if and call bad for
otherwise. Note that for any that is good for
5Again, we make no attempt on optimizing the parameter here. In fact, it
On the other hand, is only bad for with a small probability. This is because is obtained by flipping each bit of indepen-dently with probability , so , and by Chernoff bound,
is bad for
Thus, is at most
is close to is good for is bad for
Because this holds for every , the lemma follows.
Suppose and
for suitable constants . Then, from Lemmas 5 and 6, we get
is close to which implies that
Thus, we have the following.
Lemma 7: There exist constants such that for
any and any with and
, if
is a -list code computable by an circuit, then .
Combining this lemma with Lemma 2, we obtain Theorem 1.
IV. IMPOSSIBILITY OF AMPLIFICATION BY
NONDETERMINISTICCIRCUITS
Note that the result in the previous section becomes meaning-less for , as it only rules out circuits in
with . In this section, we show that even without any restriction on the circuit depth, a meaningful lower bound on the circuit size can still be derived. Formally, we have the following theorem.
Theorem 3: There exist constants such
that for any and any with
and , any parallel
black-box hardness amplification realized in
must have .
To the best of our knowledge, no such result has been shown for circuits. From Lemma 1, this implies the following im-possibility result on general black-box hardness amplification.
Corollary 3: Under the same condition as in Theorem 3, no
black-box hardness amplification can be realized in , for some constant .
Now we prove the theorem.
Proof (of Theorem 3): The basic proof idea is similar to
that for Theorem 1. The only difference is to replace Lemma 5 by an analogous one for circuits. Here we use the method of random restriction. A restriction on a set of variables
is a mapping , which ei-ther fixes the value of a variable with or leaves free with . For , let denote the distribution on such restrictions such that each variable
is mapped independently with and
. For a Boolean function and a restriction , let denote the func-tion obtained from by applying the restriction to its
vari-ables. That is, with if
and otherwise. Define the degree of a function as
. It is not hard to verify that a constant function has degree and a function depending on only input bits has de-gree at most . We need the following lemma that bounds the contribution of higher order Fourier coefficients.
Lemma 8 [17]: Let and with
. Then, for any Boolean function , .
The following is the key lemma in this section, which gives a concrete bound on the sum above for circuits.
Lemma 9: For any ,
, when .
Proof: Suppose is computed by an circuit of size , which divides its input into the real input part and the witness part. Let be the set of gates that receive some real input vari-ables directly. Consider applying a random restriction
on the real input variables. We say a gate in is killed if it is anANDgate and receives a real input variable, which is fixed to by , or if it is anORgate and receives a real input variable, which is fixed to by . For a gate , let denote the number of real input variables it receives. For a restriction , let denote the number of remaining real input variables it receives if is not killed by , and let otherwise. Set to be any constant in so that . Then
where the first inequality holds because if no gate exists, then must depend on fewer than variables, and therefore, must have degree less than .
Any with clearly has . On the other hand, any with is likely to be killed, so that
is not killed by
From Lemma 8, we have .
Then, analogously to Lemma 5 (in the previous section), we have the following.
Lemma 10: There exist constants such that for any
and any with , if
, then is close to .
Proof: Suppose ,
for some large enough constant . Using Lemmas 4 and 9 with , we have that for each
when for some suitable constant . Then, the rest is the same as that for Lemma 5, and we can have
is close to for some constant .
Suppose and ,
for suitable constants . By combining Lemma 10 with Lemma 6, we get , which gives the following.
Lemma 11: There exist constant such that for
any and any with and
, if is
a -list code computable by , then .
Combining this with Lemma 2, we obtain Theorem 3.
V. INHERENTNONUNIFORMITY OFHARDNESSAMPLIFICATION
In the previous two sections, we have proven that any black-box hardness amplification must be very nonuniform when the computational complexity of the amplification proce-dure is bounded in certain ways. In this section, we prove that even without any such complexity bound, there still exists some inherent nonuniformity.
First, we state the following simple result that seems to be a folklore. For completeness, we include the proof in the Appendix.
Theorem 4: For some constant and for any ,
no oracle algorithm can realize a black-box hardness amplification with
.
As discussed in the Introduction, hardness amplifications nor-mally have . Thus, the theorem basically says that amplifying hardness beyond must introduce nonuniformity in general. However, the theorem does not provide a quantita-tive bound on the nonuniformity. This is addressed by our next theorem.
Theorem 5: Suppose for some suitable
con-stant , and suppose . Then, any
black-box hardness amplification must have .
Thus, any such hardness amplification, even without any complexity constraint, must be inherently nonuniform, with when for some constant , or with when . Note that our lower bound gen-eralizes that of Trevisan and Vadhan [25]: they only considered
the case with (or equivalently )
and obtained the lower bound , while we consider general and obtain the lower bound .
Now we proceed to the proof of Theorem 5.
Proof (of Theorem 5): Consider an arbitrary code . We would like to show that for some constant to be determined later, one can find a string and a set such that the following two conditions hold.
• For every , is contained in the ball .
• needs balls in to cover with. For this, we first choose uniformly and indepen-dently from to form the set , for some . Call the set -good if (i.e., for any ) and any ball in contains elements of . Later, we will derive the set from a -good .
Lemma 12: When , is -good with
prob-ability .
Proof: First, the probability that for some is at most . Next, the probability that some ball in contains elements of is at
most For
some , both probabilities above are when . This proves the lemma.
We want to choose a string such that the ball contains a lot of codewords coming from a -good . We will fix some of ’s bits first.
Definition 10: For each , let be the bit such that . Call -good for if is
Lemma 13: Suppose . Then, for any , is -good for with probability .
Proof: From Lemma 12, is not -good with probability . Now fix any . Let , for , be the in-dicator random variable such that if and
otherwise. Then
Note that form a sequence of independent identically distributed (i.i.d.) random variables, with for each . Let be the sequence of i.i.d. binary random variables with for each . Then
Therefore, we have
as . Then, is -good for with probability at
least .
An averaging argument immediately gives the following.
Corollary 4: Suppose . Then, there exist a
set with and a set with
such that for every , is -good for . Let us fix the sets and guaranteed by the corollary above. Next, we want to show that many ’s from satisfy the property that the codeword has enough agreement with the vector (with each bit defined in Definition 10) on those dimensions in .
Lemma 14: There exists with such that
for any , .
Proof: For any , is -good for , so
By Markov’s inequality
Thus, there exists of size such that for
any , .
We let the vector inherit from the vector those bits indexed by , and it remains to set the values for the remaining bits. It is easy to show that there exist (in fact, can be
chosen from ) and with such
that for any , , so we just define as if and otherwise. Then, for any
for any large enough constant .
Furthermore, as and is -good, any ball in contains elements of , and hence, must need such balls to cover with. This shows that any -list code must have . Replacing the parameter by , we have the following.
Lemma 15: Suppose for some suitable constant , and
suppose . Then, any -list code
must have .
This, combined with Lemma 2, proves the theorem.
Remark 5: Recently (after the conference version of our
paper), Guruswami and Vadhan [7] used a more involved argument to proved that any -list code must have , which is tight as a matching upper bound (within a constant factor) is known to exist [6], [12]. The proof in [7], in fact, can be extended to show that any -list code must have . Therefore, any such black-box hardness amplification with , for some constant , must be inherently nonuniform.
VI. IMPOSSIBILITYRESULTS ONPRG CONSTRUCTIONS
In this section, we prove lower bound (impossibility) results for black-box PRG constructions from hard functions. For this,
we establish a connection between black-box PRG construc-tions and codes. Then, using those lower bound results for codes in previous sections, we obtain lower bound results for black-box PRG constructions.
Consider any black-box PRG construction with an
en-coding function . We call the
ratio as the stretch factor of the PRG construction. Let and , and define the corresponding code as . That is, seeing any
function as a vector in ,
produces as output the function , which is seen as a vector
in (the concatenation of ’s over
). Analogously to Lemma 2, we have the following connection between PRG constructions and codes.
Lemma 16: Suppose is
the encoding function of a black-box PRG construction with a stretch factor . Then, , defined as , is a -list code.
Proof: Suppose is the encoding function of a black-box PRG construction, and is the decoding func-tion, which is an oracle Turing machine with an -bit advice. Consider any string , which can be seen as a func-tion . We want to show that not many codewords are close to . For this, we show that there exists a distinguisher such that if any is close to , then can distinguish from random.
Define the distinguisher as
if and only if .
Sup-pose , and assume without loss of generality that .6Then
Consider any codeword with .
Now as , by Markov
inequality, we have . Thus
6For a PRGG : f0; 1g ! f0; 1g , one can only expect " 2 0 2 ,
because this can be achieved by a simple distinguisherT defined as T (z) = 1 if and only ifz = G(0 ). Because G is a PRG, r m + 1, " 2 0 2 = 2 , and we have2 .
Therefore, we have
From Definition 5, this implies that there exists an
such that .
We have shown that if is in , then is contained in one of the balls of radius centered at
for . This implies that is a -list code. With the help of this lemma, lower bound results on codes in previous sections now immediately yield results on black-box constructions of PRG.
First, observe that if the PRG construction has a parallel realization in a circuit class, then every output bit of can be computed by a circuit in the class. Then, by combining Lemma 16 with Lemma 7, we have the following theorem on parallel black-box PRG constructions realized by small-depth
circuits.
Theorem 6: There exist constants such that for
any and any with
and , any parallel black-box
realized in with a stretch factor
must have .
Next, by combining Lemma 16 with Lemma 11, we imme-diately have the following theorem on parallel black-box PRG constructions realized by circuits.
Theorem 7: There exist constants such that for
any and any with and
, any parallel black-box
PRG construction realized in with a stretch factor
must have .
Similar to those in Sections III and IV, the two theorems above on the parallel model immediately imply impossibility results on general black-box PRG constructions, via Lemma 1. Finally, by combining Lemma 16 with Lemma 15, we have the following theorem on the inherent nonuniformity of black-box PRG constructions.
Theorem 8: Suppose for some suitable constant , and suppose . Then, any black-box
PRG construction with a stretch factor must have .
APPENDIX
PROOF OFTHEOREM4
From Lemma 2, this reduces to the following coding-theo-retical question: for which values of and do we have an
-list code?
We call an code if the
need the following good code, which can be constructed using, say, the concatenation of Reed–Solomon code with Hadamard code.
Fact 3: codes exist for any .
This says that unique decoding is possible if the fraction of error is slightly smaller than . On the other hand, according to the following Plotkin bound, unique decoding is basically impossible if the fraction of error grows beyond .
Fact 4 (Plotkin Bound [21]): An code with
must have .
Combining these two facts, we have the following.
Lemma 17: For some constant and for any , any
-list code with
must have .
Proof: From Fact 3, there exists a code
with for some constant . Suppose that is a -list code with . If , then
is a code with , which is impossible according to Fact 4. Then, from Lemma 2, we obtain Theorem 4.
ACKNOWLEDGMENT
The authors would like to thank E. Viola for many helpful discussions and anonymous referees for their useful comments.
REFERENCES
[1] L. Babai, L. Fortnow, N. Nisan, and A. Wigderson, “BPP has subex-ponential time simulations unless exptime has publishable proofs,”
Comput. Complex., vol. 3, no. 4, pp. 307–318, 1993.
[2] M. Blum and S. Micali, “How to generate cryptographically strong sequences of pseudo random bits,” in Proc. 23rd Annu. IEEE Symp.
Found. Comput. Sci., 1982, pp. 112–117.
[3] A. Bogdanov and L. Trevisan, “On worst-case to average-case reduc-tions for NP problems,” in Proc. 44th Annu. Symp. Found. Comput.
Sci., Cambridge, MA, 2003, pp. 11–14.
[4] M. L. Furst, J. B. Saxe, and M. Sipser, “Parity, circuits, and the poly-nomial-time hierarchy,” Math. Syst. Theory, vol. 17, no. 1, pp. 13–27, 1984.
[5] O. Goldreich, N. Nisan, and A. Wigderson, “On Yao’s XOR lemma,” Electronic Colloquium on Computational Complexity, Tech. Rep. TR95–050, 1995.
[6] V. Guruswami, J. Håstad, M. Sudan, and D. Zuckerman, “Combinato-rial bounds for list decoding,” IEEE Trans. Inf. Theory, vol. 48, no. 5, pp. 1021–1034, May 2002.
[7] V. Guruswami and S. Vadhan, “A lower bound on list size for list decoding,” in Proc. 8th Int. Workshop Random. Comput., 2005, pp. 318–329.
[8] J. Håstad, “Computational limitations for small depth circuits,” Ph.D. dissertation, Dept. Math., Massachusetts Inst. Technol., Cambridge, MA, 1986.
[9] A. Healy, S. P. Vadhan, and E. Viola, “Using nondeterminism to am-plify hardness,” in Proc. 36th ACM Symp. Theory Comput., 2004, pp. 192–201.
[10] R. Impagliazzo, “Hard-core distributions for somewhat hard prob-lems,” in Proc. 36th Annu. IEEE Symp. Found. Comput. Sci., 1995, pp. 538–545.
[11] R. Impagliazzo, R. Jaiswal, and V. Kabanets, “Approximately list-de-coding direct product codes and uniform hardness amplification,” in
Proc. 47th Annu. Symp. Found. Comput. Sci., 2006, pp. 187–196.
[12] R. Impagliazzo, R. Jaiswal, V. Kabanets, and A. Wigderson, “Uniform direct product theorems: Simplified, optimized, and derandomized,” in
Proc. 40th ACM Symp. Theory Comput., 2008, pp. 579–588.
[13] R. Impagliazzo, R. Shaltiel, and A. Wigderson, “Extractors and pseudo-random generators with optimal seed length,” in Proc. 32nd
ACM Symp. Theory Comput., 2000, pp. 1–10.
[14] R. Impagliazzo and A. Wigderson, “P = Bpp if E requires expo-nential circuits: Derandomizing the XOR lemma,” in Proc. 29th ACM
Symp. Theory Comput., 1997, pp. 220–229.
[15] R. Impagliazzo and A. Wigderson, “Randomness vs. time: De-random-ization under a uniform assumption,” in Proc. 39th Annu. IEEE Symp.
Found. Comput. Sci., 1998, pp. 734–743.
[16] A. Klivans and D. van Melkebeek, “Graph nonisomorphism has subex-ponential size proofs unless the polynomial-time hierarchy collapses,”
SIAM J. Comput., vol. 31, no. 5, pp. 1501–1526, 2002.
[17] N. Linial, Y. Mansour, and N. Nisan, “Constant depth circuits, Fourier transform, and learnability,” J. ACM, vol. 40, no. 3, pp. 607–620, 1993. [18] N. Nisan and A. Wigderson, “Hardness vs randomness,” J. Comput.
Syst. Sci., vol. 49, no. 2, pp. 149–167, 1994.
[19] R. O’Donnell, “Hardness amplification within NP,” J. Comput. Syst.
Sci., vol. 69, no. 1, pp. 68–94, 2004.
[20] C. Papadimitriou, Computational Complexity. Reading, MA: Ad-dison-Wesley, 1994.
[21] M. Plotkin, “Binary codes with specified minimum distance,” IEEE
Trans. Inf. Theory, vol. 6, no. 4, pp. 445–450, Sep. 1960.
[22] R. Shaltiel and C. Umans, “Simple extractors for all min-entropies and a new pseudo-random generator,” in Proc. 42nd Annu. IEEE Symp.
Found. Comput. Sci., 2001, pp. 648–657.
[23] M. Sudan, L. Trevisan, and S. Vadhan, “Pseudorandom generators without the XOR lemma,” J. Comput. Syst. Sci., vol. 62, no. 2, pp. 236–266, 2001.
[24] L. Trevisan, “List decoding using the XOR lemma,” in Proc. 23rd
Annu. IEEE Symp. Found. Comput. Sci., 2003, pp. 126–135.
[25] L. Trevisan and S. P. Vadhan, “Pseudorandomness and average-case complexity via uniform reductions,” Comput. Complex., vol. 16, no. 4, pp. 331–364, 2007.
[26] C. Umans, “Pseudo-random generators for all hardnesses,” J. Comput.
Syst. Sci., vol. 67, no. 2, pp. 419–440, 2003.
[27] E. Viola, “The complexity of constructing pseudorandom generators from hard functions,” Comput. Complex., vol. 13, no. 3-4, pp. 147–188, 2004.
[28] E. Viola, “On constructing parallel pseudorandom generators from one-way functions,” in Proc. 20th Comput. Complex. Conf., 2005, pp. 183–197.
[29] A. C.-C. Yao, “Theory and applications of trapdoor functions,” in Proc.
