National University of Kaohsiung Repository System:Item 310360000Q/10510

全文

(1)國立高雄大學統計學研究所碩士論文. 超更新過程之研究. 研究生：賴佳鈴撰指導教授：黃文璋. 中華民國九十五年六月.

(2) A Study of the Suprenewal Process. by Chia-Ling Lai Advisor Wen-Jang Huang. Institute of Statistics National University of Kaohsiung Kaohsiung, Taiwan 811, R.O.C. June 2006.

(3) Contents. Z`.........................................................................................................ii zZ`........................................................................................................iii 1. Introduction...............................................................................................1 2. Comparisons of {Qt , t ≥ 0} and {Nt , t ≥ 0}...................................................2 3. Some fundamental properties of {Qt , t ≥ 0}...............................................4 4. Limiting and some other related results...................................................15 References.....................................................................................................17. i.

(4) ø?±Ä@~ ¼0>0?Z*}ÿ »ñ{..Tàó. .ßä·Ù »ñ{..Ù.@~X ` 'ðïNgð/ ^2ÿÕùÇÒÎÝÇÒ[/®ÞXnTÝÎ [/XbÇÒmðgóE9óÝË@µðïÿNgðKª ?ÿÕùÇÒãÎÝÇÒ[/®ÞÿÕ·ÍZ&ÆÞ@~×ÍXÛÝø ?±Ä{X i ≥ 1} ×à}ñvb8!5µ ^ó¬S = P X n ≥ 1S = 0EN×Ít ≥ 0LQ = inf{n | n ≥ 0S ≥ t}EÎÝÇÒ[/® ÞQ ðïÕ` t cÝÀÇÒóyTytÝtKðgót ≥ 0&Æ´ f´Ä{Q t ≥ 0}õã!×à{X i ≥ 1}XßW?±Ä{N t ≥ 0}#½Ä {Q t ≥ 0}Ý×°ÃÍP²t¡ôÄ{Q t ≥ 0}Ý×°Á§C8nÝ i. n. 0. t. n i=1. i. n. t. t. i. t. t. t. n " Þ ÇÒ[/®Þ¿¢5µÞ45µ?±ÄøÍ5ø?±Ä . ii.

(5) A Study of the Suprenewal Process Advisor: Dr. Wen-Jang Huang Department of Applied Mathematics National University of Kaohsiung. Student: Chia-Ling Lai Institute of Statistics National University of Kaohsiung. ABSTRACT The classical coupon collector’s problem is concerned with the number of purchases in order to have a complete collection, assuming that on each purchase a consumer can obtain a randomly chosen coupon. For most real situations, a consumer may not just get exactly one coupon on each purchase. Motivated by the classical coupon collector’s problem, in this work, we will study the so-called suprenewal process. Let {Xi , i ≥ 1} be a sequence of independent P and identically distributed random variables, Sn = ni=1 Xi , n ≥ 1, S0 = 0. For every t ≥ 0, define Qt = inf{n | n ≥ 0, Sn ≥ t}. For the classical coupon collector’s problem, Qt denotes the minimal number of purchases, such that the total number of coupons that the consumer has owned until time t is greater than or equal to t, t ≥ 0. First the process {Qt , t ≥ 0} and the renewal process {Nt , t ≥ 0} generated by the same sequence {Xi , i ≥ 1} are compared. Next some fundamental properties of {Qt , t ≥ 0} are provided. Finally limiting and some other related results are obtained for the process {Qt , t ≥ 0}. Keywords: Coupon collector’s problem, geometric distribution, negative binomial distribution, renewal process, sample path, suprenewal process.. iii.

(6) 1. Introduction Starting from the end of April 2005, collecting Hello Kitty magnets became an immensely popular hobby in Taiwan. President Chain Stores Corp., which runs Taiwanese largest convenience store chain, 7-Eleven, was giving away one of a series of commemorative Hello Kitty magnets for each NTD77 a consumer spends at 7-Eleven store. There are 41 different patterns of Hello Kitty magnets in total. Because the cover of each package of magnet is the same, it is reasonable to assume that the magnets are given randomly. We now review the classical coupon collector’s problem. Assume there are N distinct coupons in a collection, and a series of random draws is made with replacement from these. Let T denote the number of draws necessary for all N coupons to have been drawn at least once. Properties of T had been studied by many authors, see e.g. Goodwin (1949) and Feller (1950). Among others, expectation and variance of T can be obtained as follows. For i ≥ 1, let Ci ∈ {1, 2, · · · , N } be the coupon obtained at the i-th draw. The i-th draw is called a success, if Ci has not been obtained before the i-th draw. For 1 ≤ i ≤ N , let Ti denote the number P of draws after the (i − 1)-th success, till the i-th success. Then T = N i=1 Ti . Obviously, T1 , T2 , · · · , TN are independent, and Ti has a geometric distribution with parameter pi = (N −i+1)/N , then E(Ti ) = N/(N −i+1), and Var(Ti ) = (1−(N/(N −i+1))/(N/(N −i+1))2 , 1 ≤ i ≤ N . Thus E(T ) =. N X. E(Ti ) = N HN ,. (1). i=1. where for N ≥ 1, HN =. PN. i=1 1/i. Var(T ) =. is the N -th Harmonic number, and. N X. Var(Ti ) = N 2. i=1. N X 1 − N HN . i2. (2). i=1. The above coupon collector’s problem can be generalized. Assume the i-th coupon has P probability pi of being drawn, where 0 < pi < 1, i = 1, 2, · · · , N , such that N i=1 pi = 1, the pi ’s are allowed to be unequal. This was studied by von Schelling (1954). Some limiting results were derived by Baum and Billingsley (1965) and Hoslt (1971), and others. Related problems had also been discussed, such as the collector’s brotherhood problem. As an example, Foata et al.(2001) and Foata and Zeilberger (2003) considered the situation that the collector shares his harvest with his brothers. They answered the question that when the collection of the collector is completed, the number of coupons each brother still lacks. For our present problem, the expected number of magnets needed for collecting a complete P . set of 41 magnets is 41 41 i=1 1/i = 176.42. For a particular consumer, if his spending is less than NTD77, then he gets 0 magnet, if his spending is at least NTD77 and less than NTD154, than he gets 1 magnet, if his spending is at least NTD154 and less than NTD231, then he gets 2 magnets on that purchase, and so on. Now what is the number of purchases needed in order to get magnets greater than or equal to 176.42? To solve this problem, first we introduce a new process and study some of its properties. 1.

(7) Let {Xi , i ≥ 1} be a sequence of independent and identically distributed (i.i.d.) random P variables, Sn = ni=1 Xi , n ≥ 1, S0 = 0. For every t ≥ 0, define Qt = inf{n | n ≥ 0, Sn ≥ t}. For the magnets problem, Xi can be viewed as the number of magnets received on the i-th purchase, i ≥ 1, and Qt can be viewed as the minimal number of purchases, such that the total number of magnets is greater than or equal to t, t ≥ 0. Recall that the renewal process {Nt , t ≥ 0} generated by the same sequence {Xi , i ≥ 1}, where for t ≥ 0, Nt = sup{n | n ≥ 0, Sn ≤ t}, Nt denotes the number of renewals in [0, t]. Qt can be referred to as the minimal number of renewals in [t, ∞), and we call {Qt , t ≥ 0} the suprenewal process. In Section 2, we compare the suprenewal process {Qt , t ≥ 0} with the renewal process. In Section 3, some fundamental properties of {Qt , t ≥ 0} are studied. Finally, limiting and some other related results are presented in Section 4. 2. Comparisons of {Qt , t ≥ 0} and {Nt , t ≥ 0} Let X1 , X2 , · · · be i.i.d. random variables with the same distribution as X, where X, a nonnegative random variable, has the distribution function F with F (0−) = 0 and F (0) < 1. P Let Sn = ni=1 Xi , n ≥ 1, S0 = 0. Let {Qt , t ≥ 0} and {Nt , t ≥ 0} be the suprenewal process and renewal process generated by {Xi , i ≥ 1} respectively. Obviously, Q0 = 0 and Qt ≥ 1, if t > 0. Also Qt ≤ n if and only if Sn ≥ t. Hence for every t > 0 and integer n ≥ 1, P (Qt = n) = P (Qt ≤ n) − P (Qt ≤ n − 1) = P (Sn ≥ t) − P (Sn−1 ≥ t) = P (Sn−1 < t) − P (Sn < t).. (3). If F is continuous, then Sn is a continuous random variable for every integer n ≥ 0. Consequently, P (Qt = n) = Fn−1 (t) − Fn (t), t > 0, n ≥ 1,. (4). where Fn is the n-fold convolution of F with itself, n ≥ 1, and F0 (t) = 1, t ≥ 0. We now compare the two processes {Qt , t ≥ 0} and {Nt , t ≥ 0}. First instead of having right continuous sample paths for {Nt , t ≥ 0}, {Qt , t ≥ 0} has left continuous sample paths. Next instead of (3) and (4), whether F is continuous or not, P (Nt = n) = P (Sn ≤ t) − P (Sn+1 ≤ t) = Fn (t) − Fn+1 (t), t ≥ 0, n ≥ 0.. (5). On the other hand, {Qt , t ≥ 0} and {Nt , t ≥ 0} have the same jump times. Denote the sequence of jump times by 0 = τ0 < τ1 < τ2 < · · ·. Then Nt = Qt − 1, if t 6∈ {τ0 , τ1 , τ2 , · · ·},. (6). Nτi = Qτi + Yτi − 1, i ≥ 0,. (7). and. 2.

(8) where Yτ0 = 1, Yτi = Qτi + − Qτi = Nτi − Nτi − , i ≥ 1,. (8). denotes the common jump size at τi of the processes {Qt , t ≥ 0} and {Nt , t ≥ 0}. It can be seen that for i ≥ 1, Yτi has a Ge(λ) distribution, where λ = 1 − F (0), if F (0) > 0; and Yτi ≡ 1, hence Qτi = Nτi , i ≥ 1, if F (0) = 0. Yτi and Qτi are independent, and Yτi and Nτi −1 are also independent. If F is continuous, then F (0) = 0, and Nt = Qt − 1 almost everywhere on [0, ∞).. (9). As an example, let F (x) = 1 − e−λx , λ > 0, x > 0, then it is well known that Nt has a P(λt) distribution, t > 0. By (9), Qt − 1 is also P(λt) distributed for almost all t on [0, ∞). Note that except (6) and (7), we also have the following relationship Qτi ≤ Nτi ≤ Qτi+1 , i ≥ 0.. (10). Although Qt may be less than Nt , from the definitions of Qt and Nt , we have SNt ≤ t ≤ SQt , t ≥ 0.. (11). SNτi = SQτi = τi , i ≥ 0.. (12). In particular. Recall that SNt +1 − t, t − SNt , and XNt +1 = SNt +1 − SNt are called residual life at time t, current life at time t, and total life at time t, respectively, for the renewal process {Nt , t ≥ 0}. It is known that P (XNt +1 > x) ≥ P (X1 > x), x ≥ 0, and E(XNt +1 ) ≥ E(X1 ), t ≥ 0. This is the so-called inspection paradox. Similarly, it can be shown P (XQt > x) ≥ P (X1 > x), x ≥ 0.. (13). That is XQt is stochastically larger than X1 . Consequently, E(XQt ) ≥ E(X1 ), t ≥ 0.. (14). Furthermore, using the fact that a renewal process probabilistically starts over when a renewal occurs, for every increasing function g, the following inequality is immediate: E(g(Nt+s − Nt )) ≤ E(g(Ns + 1)), t, s ≥ 0.. (15). E(g(Qt+s − Qt )) ≤ E(g(Qs )), t, s ≥ 0.. (16). Similarly, we have. 3.

(9) In particular E(Qt+s − Qt ) ≤ E(Qs ), t, s ≥ 0.. (17). We give a typical sample paths of {Qt , t ≥ 0} and {Nt , t ≥ 0}, respectively, to illustrate the relationships (6), (7) and (10). Assume X1 = 2, X2 = 0, X3 = 1, X4 = 4, X5 = 0, X6 = 0, X7 = 3, · · ·, then S1 = 2, S2 = 2, S3 = 3, S4 = 7, S5 = 7, S6 = 7, S7 = 10, · · ·. Figure 1 gives the sample paths of {Qt , t ≥ 0} and {Nt , t ≥ 0}. Qt 7 6 6 5 4 3 2 1 b. b b b r. r. r -. r. 0. Nt 7 6 6 5 4 3 2 1. r. r. 0. 1 2 3 4 5 6 7 8 9 10 t. r r r b. r b. b. b. -. 1 2 3 4 5 6 7 8 9 10 t. Figure 1. Sample paths of {Qt , t ≥ 0} and {Nt , t ≥ 0} 3. Some fundamental properties of {Qt , t ≥ 0} There are many investigations for properties of renewal process in the literatures. In this section, we explore some basic properties of the process {Qt , t ≥ 0}, especially for the case that X takes on nonnegative integer values. Throughout this section, let P (X < ∞) = 1, P and P (X = k) = pk , where pk ≥ 0, k = 0, 1, 2, · · ·, p0 < 1, and ∞ k=0 pk = 1. Also let N = sup{i | i ≥ 0, pi > 0}. First we introduce some notation which will be used often in this work. Let dte and btc denote the ceiling function and the floor function, respectively, namely dte = the least integer greater than or equal to t, and btc = the greatest integer less than or equal to t. For example, d3.7e = 4, b3.7c = 3, and d6e = b6c = 6. For integers a, b, c, with a ≤ b ≤ c, and nonnegative integers x0 , x1 , · · · , xN , if N < ∞, let Aa,b = {(x0 , x1 , · · · , xN )|. N X. xi = a, and. i=0. Ba,b,c = {(x1 , x2 , · · · , xN )|. N X. N X. ixi = b},. i=0. xi = a, and. N X. i=1. ixi = b, b + 1, · · · , c},. i=1. and Ca = {(x1 , x2 , · · · , xN )|. N X i=1. 4. xi = a};.

(10) if N = ∞, let A1a,b = {(x0 , x1 , · · ·)|. X. xi = a, and. i≥0. X. ixi = b},. i≥0. and 1 Ba,b,c = {(x1 , x2 , · · ·)|. X. xi = a, and. i≥1. X. ixi = b, b + 1, · · · , c}.. i≥1. Note that if (x0 , x1 , · · · , xN ) ∈ Aa+x0 ,b , then (x1 , x2 , · · · , xN ) ∈ Ba,b,b ; if (x0 , x1 , · · ·) ∈ A1a+x0 ,b , 1 then (x1 , x2 , · · ·) ∈ Ba,b,b , and Ba,a,N a = Ca . We give three simple examples in the following. Example 1. Let p1 = p2 = p3 = 1/3. Then the support of Q3.5 is {2, 3, 4}, and P (Q3.5 = 2) = 2/3, P (Q3.5 = 3) = 8/27, P (Q3.5 = 4) = 1/27. Example 2. Let p0 , p1 , p2 > 0, and p0 + p1 + p2 = 1. Then P (Q2 = i) = (i − 1)pi−2 0 (1 − i−1 p0 )p1 + p0 p2 , i ≥ 1. In particular, if p0 = 0.2, p1 = 0.3, p2 = 0.5, then P (Q2 = 1) = 0.5, P (Q2 = 2) = 0.34, P (Q2 = 3) = 0.116, P (Q2 = 4) = 0.0328, · · · . The next example shows that the family of the distribution Qt contains geometric and negative binomial, the two common statistical distributions. Example 3. Assume 0 < p0 < 1 and p1 = 1 − p0 . In this case τi = i, i ≥ 1. That is X1 , X2 , · · · are i.i.d. Ber(p1 ) random variables, and Sn is B(n, p1 ) distributed, n ≥ 1. Then obviously for every t > 0, SQt = dte, Qt ∼ N B(dte, p1 ), and for every integer k ≥ 2, Q1 , Q2 − Q1 , · · · , Qk − Qk−1 , are i.i.d. random variables with the common Ge(p1 ) distribution. Consequently, for every t > 0, E(Qt ) = dte/p1 , E(SQt − t) = dte − t, and E(t − SQt −1 ) = t − dte + 1. Hence E(XQt ) = E(SQt − SQt −1 ) = 1 > p1 = E(X1 ). Moreover, it can be seen easily, for the above {Xi , i ≥ 1} , for any 0 < p0 < 1, there is an infinite number of positive t’s, such that E(SQt − t) > E(X1 ). Remark 1. As a comparison, for the {Xi , i ≥ 1} defined in Example 3, we have SNt = btc, µ ¶ n btc+1 n−btc P (Nt = n) = p p0 , n ≥ btc, btc 1 and P (Nt = n) = 0, for n < btc. That is Nt + 1 ∼ N B(btc + 1, p1 ), t > 0. This also can be seen by (6), (7) and Example 3. Now E(Nt ) = (btc + p0 )/p1 , E(t − SNt ) = t − btc, E(SNt +1 − t) = btc + 1 − t, and E(XNt +1 ) = 1 > E(X1 ), t > 0. Next we find the distribution of Qt , t > 0.. 5.

(11) Theorem 1. For every integer n ≥ 1 and t > 0,  , p0 = 0,   gn−1,t − gn,t µ ¶ µ ¶ dte−1 X P (Qt = n) = n−1 n  gm,t pn−m−1 ( − p0 ) , 0 < p0 < 1,  0 m m. (18). m=0. where if N < ∞,. gm,t.   (1 − p0 )m   X =   . , 0 ≤ m ≤ b dte−1 N c,. (x1 ,x2 ,···,xN )∈Bm,m,dte−1. N Y pxi i m!( ) , m ≥ b dte−1 N c + 1, xi !. (19). i=1. if N = ∞,     1 gm,t =.   . X. 1 (x1 ,x2 ,···)∈Bm,m,dte−1. , m = 0, Y pxi i m!( ) , m ≥ 1. xi !. (20). i≥1. Proof. Obviously we only need to prove (18) holds for positive integer t. We prove this by induction. (i) First we prove the case p0 = 0 and N < ∞. That (18) holds for t = 1 can be seen as following. From assumptions, we have P (Sn < 1) = 0, n ≥ 1, P (S0 < 1) = 1. Thus P (Q1 = n) = P (Sn−1 < 1) − P (Sn < 1) ( 1 , n = 1, = 0 , n ≥ 2, = gn−1,1 − gn,1 , n ≥ 1, where the last equality holds is because for every n ≥ 1, Bn,n,0 is a null set, hence gn,1 = 0. This together with g0,1 = 1 implies g0,1 − g1,1 = 1, and gn−1,1 − gn,1 = 0, n ≥ 2. Now suppose (18) is true for t = r ≥ 1, i.e. we have P (Qr = n) = gn−1,r − gn,r . Then P (Qr+1 = n) = P (Sn−1 < r + 1) − P (Sn < r + 1) = [P (Sn−1 < r) − P (Sn < r)] + [P (Sn−1 = r) − P (Sn = r)] = [gn−1,r − gn,r ] + [P (Sn−1 = r) − P (Sn = r)] = {. X. (x1 ,x2 ,···,xN )∈Bn−1,n−1,r−1. N Y pxi i (n − 1)!( )− xi ! i=1. 6. X (x1 ,x2 ,···,xN )∈Bn,n,r−1. N Y pxi i n!( )} xi ! i=1.

(12) X. +{. (x1 ,x2 ,···,xN )∈Bn−1,r,r. i=1. N Y. X. =. N Y pxi i (n − 1)!( )− xi !. (n − 1)!(. i=1. (x1 ,x2 ,···,xN )∈Bn−1,n−1,r. pxi i )− xi !. X (x1 ,x2 ,···,xN )∈Bn,r,r. X (x1 ,x2 ,···,xN )∈Bn,n,r. N Y pxi i n!( )} xi ! i=1. N Y pxi i n!( ) xi ! i=1. = gn−1,r+1 − gn,r+1 . This proves (18) holds for t = r + 1. By the induction argument this completes the proof for the case p0 = 0 and N < ∞. The proof of (18) for the case p0 = 0 and N = ∞ is similar to the proof for the case p0 = 0 and N < ∞, hence is omitted. (ii) Next we prove the case 0 < p0 < 1 and N < ∞. The proof of (18) for t = 1 is as following. P (Q1 = n) = P (Sn−1 < 1) − P (Sn < 1) = P (Sn−1 = 0) − P (Sn = 0) µ ¶ µ ¶ n−1 n n−1 n−1 n = p0 − p0 = g0,1 p0 ( − p0 ). 0 0 Now suppose (18) is true for t = r ≥ 1, i.e. we have r−1 X. P (Qr = n) =. gm,r pn−m−1 ( 0. m=0. µ ¶ µ ¶ n−1 n − p0 ). m m. (21). Then P (Qr+1 = n) = P (Sn−1 < r + 1) − P (Sn < r + 1) = [P (Sn−1 < r) − P (Sn < r)] + [P (Sn−1 = r) − P (Sn = r)] = P (Qr = n) + [P (Sn−1 = r) − P (Sn = r)],. (22). and X. P (Sn−1 = r) =. (x0 ,x1 ,···,xN )∈An−1,r. N Y pxi i (n − 1)!( ) xi ! i=0. X. = (. (x1 ,x2 ,···,xN )∈Bb r−1 c+1,r,r. N xi (n − 1)! c−2 Y pi n−b r−1 N p0 ( )) + · · · x! (n − b r−1 N c − 2)! i=1 i. N. +(. X. (x1 ,x2 ,···,xN )∈Br,r,r. ¶ n−1 { b r−1 N c+1. N (n − 1)! n−r−1 Y pxi i p0 ( )) (n − r − 1)! xi !. µ =. µ ¶ n−1 + { r. i=1. N. X. (b. (x1 ,x2 ,···,xN )∈Bb r−1 c+1,r,r. Y pxi r−1 n−b r−1 c−2 i + ··· c + 1)!( )}p0 N N xi ! i=1. N. X (x1 ,x2 ,···,xN )∈Br,r,r. N Y pxi i r!( )}p0n−r−1 . xi !. 7. i=1. (23).

(13) P That the second equality of (23) holds is because if (x0 , x1 , · · · , xN ) ∈ An−1,r , i.e. N i=1 xi = PN PN n − 1 − x0 and i=1 ixi = r, then i=1 xi ≤ r, hence x0 ≥ n − r − 1. On the other hand, if P P x0 ≥ n − b(r − 1)/N c − 1, then N c, and N i=1 xi ≤ b(r − 1)/N i=1 ixi ≤ r − 1 follows. This PN proves n − r − 1 ≤ x0 ≤ n − b(r − 1)/N c − 2. Hence i=1 xi = b(r − 1)/N c + 1, · · · , r. This P together with N i=1 ixi = r implies (x1 , x2 , · · · , xN ) ∈ Bb(r−1)/N c+1,r,r , · · · , Br,r,r . Similarly, µ. ¶ n { b r−1 N c+1. P (Sn = r) =. µ ¶ n + { r. X. (b. (x1 ,x2 ,···,xN )∈Bb r−1 c+1,r,r. N Y pxi i r−1 n−b r−1 c−1 c + 1)!( )}p0 N + ··· N xi ! i=1. N. X (x1 ,x2 ,···,xN )∈Br,r,r. N Y pxi i r!( )}p0n−r . xi !. (24). i=1. Substituting (21), (23) and (24) into (22), it yields µ ¶ µ ¶ µ ¶ r−1 n − 1 n−1 n−1 n−1 P (Qr+1 = n) = { p0 + (1 − p0 )p0n−2 + · · · + (1 − p0 )b N c r−1 0 1 b N c ¶ µ X r−1 r−1 n−1 ( (b c + 1)! ·p0 n−1−b N c + r−1 N b N c+1 (x1 ,x2 ,···,xN )∈Bb r−1 c+1,b r−1 c+1,r. µ ¶ pxi i n−b r−1 n−1 c−2 N ))p + ··· + ( xi ! 0 r−1. N Y. ·(. i=1. N. N. X. (x1 ,x2 ,···,xN )∈Br−1,r−1,r. N Y pxi i )) (r − 1)!( xi ! i=1. µ ¶ µ ¶ N X Y pxi i n−r−1 n−1 n n n−r ·p0 + ( r!( ))p0 }−{ p r x! 0 0 i=1 i (x1 ,x2 ,···,xN )∈Br,r,r µ ¶ µ ¶ r−1 r−1 n n n−1 + (1 − p0 )p0 + · · · + (1 − p0 )b N c p0 n−b N c r−1 1 b N c µ ¶ N X Y pxi i n−b r−1 n r−1 c−1 + r−1 ( (b c + 1)!( ))p0 N b N c+1 N xi ! i=1. (x1 ,x2 ,···,xN )∈Bb r−1 c+1,b r−1 c+1,r N. ¶ n ( +··· + r−1 µ. N. X. (x1 ,x2 ,···,xN )∈Br−1,r−1,r. µ ¶ n + ( r. N Y. X. r!(. (x1 ,x2 ,···,xN )∈Br,r,r. =. (1 −. +. pxi i n−r ))p } xi ! 0. p0 )m pn−m−1 ( 0. m=0 r X. i=1. µ ¶ µ ¶ n−1 n − p0 ) m m. b r−1 c N. X. i=1. N Y pxi i n−r+1 (r − 1)!( ))p xi ! 0. (. X. m=b r−1 c+1 (x1 ,x2 ,···,xN )∈Am,m,r N. 8. µ ¶ µ ¶ N Y pxi i n−m−1 n − 1 n ))p0 ( − p0 ) r!( xi ! m m i=1.

(14) µ ¶ µ ¶ n−1 n − p0 ) m m. r bN c. =. X. (1 −. p0 )m p0n−m−1 (. m=0 r X. +. X. (. r m=b N c+1 (x1 ,x2 ,···,xN )∈Bm,m,r. =. r X m=0. µ ¶ µ ¶ N Y pxi i n−m−1 n − 1 n m!( ))p ( − p0 ) xi ! 0 m m i=1. µ ¶ µ ¶ n−1 n n−m−1 gm,r+1 p0 ( − p0 ), m m. (25). P PN where the third equality holds is because if N i=1 Xi = m, then m ≤ i=1 iXi ≤ N m, hence if r ≥ N m, i.e. m ≤ br/N c, then Bm,m,r = Bm,m,N m and X (x1 ,x2 ,···,xN )∈Bm,m,r. N Y pxi i m!( ) = xi ! i=1. X (x1 ,x2 ,···,xN )∈Bm,m,N m. X. =. (x1 ,x2 ,···,xN )∈Cm. N Y pxi i m!( ) xi ! i=1. N Y pxi i m!( ) xi !. = (p1 + p2 + · · · +. i=1 pN )m. = (1 − p0 )m . This proves (18) holds for t = r + 1, and the proof for the case 0 < p0 < 1 and N < ∞ is completed. (iii) Finally we consider the case 0 < p0 < 1 and N = ∞. The proof of (18) for t = 1 is the same as in (ii). Now suppose the induction statement is true for t = r ≥ 1, i.e. we have r−1 X. P (Qr = n) =. gm,r pn−m−1 ( 0. m=0. µ ¶ µ ¶ n−1 n − p0 ). m m. (26). Then P (Qr+1 = n) = P (Sn−1 < r + 1) − P (Sn < r + 1) = P (Qr = n) + [P (Sn−1 = r) − P (Sn = r)], and X. P (Sn−1 = r) =. (x0 ,x1 ,···)∈A1n−1,r. X. = (. 1 (x1 ,x2 ,···)∈B1,r,r. +(. Y pxi i (n − 1)!( ) xi ! i≥0. (n − 1)! n−2 Y pxi i p ( )) + · · · (n − 2)! 0 xi !. X. 1 (x1 ,x2 ,···)∈Br,r,r. 9. i≥1. (n − 1)! n−r−1 Y pxi i p ( )) (n − r − 1)! 0 xi ! i≥1. (27).

(15) µ ¶ X Y pxi n−1 i = { ( )}p0n−2 + · · · 1 x ! i 1 i≥1 (x1 ,x2 ,···)∈B1,r,r µ ¶ X Y pxi n−1 i + { r!( )}p0n−r−1 , r x ! i 1. (28). i≥1. (x1 ,x2 ,···)∈Br,r,r. where the second equality holds is by using the same argument as in the discussion of the paragraph after (23). Similarly, µ ¶ X Y pxi n i P (Sn = r) = { ( )}pn−1 + ··· 0 x ! 1 i 1 (x1 ,x2 ,···)∈B1,r,r i≥1 µ ¶ Y px i X n i r!( + { )}pn−r . (29) 0 r x ! i 1 (x1 ,x2 ,···)∈Br,r,r. i≥1. Substituting (26), (28) and (29) into (27), it yields µ ¶ µ ¶ X Y pxi n − 1 n−1 n−1 i P (Qr+1 = n) = { p0 + ( ( ))pn−2 + ··· 0 0 1 x ! i 1 i≥1 (x1 ,x2 ,···)∈B1,1,r µ ¶ X Y pxi n−1 i ))p0n−r−1 } + ( r!( xi ! r 1 i≥1 (x1 ,x2 ,···)∈Br,r,r µ ¶ µ ¶ Y pxi X n n n i ( −{ p0 + ( ))pn−1 + ··· 0 0 1 x ! i 1 i≥1 (x1 ,x2 ,···)∈B1,1,r µ ¶ X Y pxi n i ))p0n−r } + ( r!( x ! r i 1 i≥1 (x1 ,x2 ,···)∈Br,r,r µ ¶ µ ¶ n−1 n = pn−1 ( − p0 ) 0 0 0 r X X Y pxi µn − 1¶ µ n ¶ n−m−1 i + p0 ( m!( ))( − p0 ) x m m i! 1 m=1. =. r X m=0. (x1 ,x2 ,···)∈Bm,m,r. i≥1. µ ¶ µ ¶ n−1 n n−m−1 gm,r+1 p0 ( − p0 ). m m. (30). This proves (18) holds for t = r + 1. The proof is completed. Let φt (s) = E(e. −sQt. )=. ∞ X. P (Qt = n)e−sn , s ≥ 0,. n=1. be the Laplace transform of Qt , t ≥ 0. φt (s) and the moments of Qt can be obtained immediately by using Theorem 1. We give the results in the following corollary. 10.

(16) Corollary 1. Let integer n ≥ 1, p0 ≥ 0, and t > 0. (i). dte−1 1 − e−s X e−s φt (s) = 1 − g ( )m , s ≥ 0. m,t 1 − p0 e−s 1 − p0 e−s. (31). m=0. (ii). dte−1. E(Qt ) =. X. m=0. gm,t . (1 − p0 )m+1. (32). (iii). dte−1. Var(Qt ) =. X. m=0. dte−1. X gm,t gm,t (1 + 2m + p0 ) − { }2 , (1 − p0 )m+2 (1 − p0 )m+1. (33). m=0. where gm,t , m ≥ 0, are as defined in (19) and (20). Proof. (i). Due to the expression (18), the proof of (31) is divided into two parts: 0 < p0 < 1, and p0 = 0. First we prove the case 0 < p0 < 1. µ ¶ µ ¶ ∞ dte−1 X X n−1 n n−m−1 φt (s) = { gm,t p0 ( − p0 )}e−sn m m n=1 m=0. ∞ dte−1 ∞ dte−1 X X µn − 1¶ X X µn¶ n−m−1 −sn = gm,t p0 e − gm,t pn−m e−sn 0 m m n=1 m=0. n=1 m=0. ¶ dte−1 dte−1 ∞ µ ¶ ∞ µ X gm,t X X gm,t X n n−1 −s n { (p0 e−s )n } (p0 e ) } − = m m+1 { p0 m m p0 m=0. n=1. m=0. n=1. = A − B.. (34). Now A =. ¶ ¶ dte−1 ∞ µ ∞ µ X gm,t X g0,t X n − 1 n−1 −s n { (p0 e ) } + { (p0 e−s )n } p0 0 m pm+1 0 n=1. m=1. =. =. =. n=1. ¶ X gm,t g0,t e−s n−m n−1 + (p0 e−s )n } m+1 { 1 − p0 e−s m n − m p m=1 0 n=1 µ ¶ dte−1 ∞ X gm,t X g0,t e−s n n−1 p0 e−s m −s m −s n−m + { ( − 1) (1 − p e ) (p e ) }( ) 0 0 1 − p0 e−s n−m 1 − p0 e−s pm+1 n=1 m m=1 0 dte−1. ∞ X. µ. dte−1 X gm,t g0,t e−s 1 p0 e−s m + ( − 1)( ) 1 − p0 e−s 1 − p0 e−s pm+1 1 − p0 e−s m=1 0. 11.

(17) =. dte−1 X g0,t e−s e−s + g ( )m+1 m,t 1 − p0 e−s 1 − p0 e−s m=1. dte−1. =. X. gm,t (. m=0. e−s )m+1 , 1 − p0 e−s. (35). and B = g0,t {. dte−1 ∞ µ ¶ ∞ µ ¶ X X gm,t X n n (p0 e−s )n } + { (p0 e−s )n } 0 pm m 0. n=1. = g0,t. m=1. n=1. µ ¶ n−1 n ( ) (p0 e−s )n } m m−1. dte−1. ∞ X. m=1. n=1. X gm,t p0 e−s + { −s 1 − p0 e pm 0. µ ¶ ∞ X gm,t X p0 n n−1 p0 e−s m −s m −s n−m = g0,t + { ) (1 − p e ) (p e ) }( ) ( 0 0 1 − p0 e−s pm m m−1 1 − p0 e−s 0 dte−1. e−s. m=1. n=1. dte−1. = (−g0,t +. X gm,t g0,t 1 p0 e−s m ( ) + )( ) 1 − p0 e−s pm 1 − p0 e−s 1 − p0 e−s 0 m=1. dte−1. = −g0,t +. X. m=0. gm,t e−s ( )m . 1 − p0 e−s 1 − p0 e−s. (36). The assertion follows by substituting A and B in (35) and (36) into (34), and noting that g0,t = 1. The proof for the case p0 = 0 is given below. φt (s) = =. ∞ X. (gn−1,t − gn,t )e−sn. n=1 ∞ X. gm,t e−s(m+1) − (. m=0. ∞ X. gm,t e−sm − g0,t ). m=0 dte−1. = 1 − (1 − e−s ). X. gm,t e−sm , s ≥ 0.. (37). m=0. (ii). Taking the derivative of φt with respect to s, we obtain for s > 0, φ0t (s) = −. dte−1 e−s e−s (1 − p0 e−s ) − (1 − e−s )p0 e−s X g ( )m m,t (1 − p0 e−s )2 1 − p0 e−s m=0. −. dte−1. X 1− −e−s (1 − p0 e−s ) − e−s (p0 e−s ) m−1 m( ) g { } m,t 1 − p0 e−s 1 − p0 e−s (1 − p0 e−s )2 e−s. e−s. m=0. = −. dte−1 dte−1 e−s e−s (1 − p0 )e−s X e−s (1 − e−s ) X m g ( m( ) + )m−1 gm,t . m,t (1 − p0 e−s )2 1 − p0 e−s (1 − p0 e−s )3 1 − p0 e−s m=0. m=0. 12.

(18) Hence dte−1. E(Qt ) =. − lim φ0t (s) s↓0. X. =. m=0. gm,t . (1 − p0 )m+1. (iii). As in (ii), dte−1. E(Q2t ). =. lim φ00t (s) s↓0. =. X. (1 + 2m + p0 ). m=0. gm,t , (1 − p0 )m+2. hence Var(Qt ) = E(Q2t ) − (E(Qt ))2 dte−1. =. X. dte−1. (1 + 2m + p0 ). m=0. X gm,t gm,t −{ }2 m+2 (1 − p0 ) (1 − p0 )m+1 m=0. as required. Example 3.(Continued) We use Theorem 1 and (i) of Corollary 1, respectively, to demonstrate Qt ∼ N B(dte, p1 ), t > 0. By letting N = 1 in Theorem 1, it yields ( pm , 0 ≤ m ≤ dte − 1, 1 gm,t = (38) 0 , m ≥ dte. Hence µ ¶ µ ¶ n−1 n − p0 ) m m. dte−1. P (Qt = n) =. X. gm,t pn−m−1 ( 0. m=0 dte−1. =. X. µ ¶ µ ¶ µ ¶ n−1 n−1 n−1 −( + )p0 } m m m−1. n−m−1 pm { 1 p0. m=0 dte−1 µ. ¶ dte−2 µ X n − 1¶ n − 1 m+1 n−m−1 = p1 p0 − pm+1 p0n−m−1 1 m m m=0 m=0 µ ¶ n−1 dte n−dte = p p , n ≥ dte, dte − 1 1 0 X. ¡ ¢ where n−1 −1 is defined to be 0. This shows Qt ∼ N B(dte, p1 ), t > 0. Next from (i) of Corollary 1 and (38), we have φt (s) = 1 −. dte−1 dte−1 e−s p1 e−s m 1 − e−s X 1 − e−s X m g ( ( ) = 1 − ) m,t 1 − p0 e−s 1 − p0 e−s 1 − p0 e−s 1 − p0 e−s m=0. = 1−. 1 − e−s 1 − { 1 − p0 e−s. m=0. p1 e−s dte ( 1−p −s ) 0e } −s 1−e −s 1−p0 e. 13. = (. p1 e−s dte ) , s ≥ 0, 1 − p0 e−s.

(19) which is exactly the Laplace transform of a N B(dte, p1 ) distributed random variable. S 1 1 1 1 Note that for integers a, b, with a ≤ b, Ba,a,b = b−a i=0 Ba,a+i,a+i , where Ba,a,a , Ba,a+1,a+1 , ¡ ¢ n−1+k 1 · · · , Ba,b,b , are disjoint. Also let Hkn = , n ≥ 1, k ≥ 0, denote the number of kk combinations with repetition of n distinct things. Before giving Example 4, we need the following lemma. Lemma 1. For integers n ≥ 1, and k ≥ 0, we have X. µ ¶ n−1+k Q = . k i≥1 xi ! n!. 1 (x1 ,x2 ,···)∈Bn,n+k,n+k. (39). Proof. For n ≥ 1, let y1 , y2 , · · · , yn be any n positive integers. For every i ≥ 1, let zi = Pn j=1 I{yj =i} , where ( I{yj =i} =. 1 , yj = i, 0 , yj 6= i,. Pn P P izi = zi = n, and is the indicator function. Then j=1 yj . Conversely, for i≥1 i≥1 P P 1 n, and i≥1 izi = n + k, k ≥ 0, there exists every (z1 , z2 , · · ·) ∈ Bn,n+k,n+k , i.e. i≥1 zi =P exactly one multiset {y1 , y2 , · · · , yn } satisfying nj=1 yj = n+k. Hence for every (z1 , z2 , · · ·) ∈ Q 1 Bn,n+k,n+k , (n!/ i≥1 zi !) is the number of distinct permutations of the corresponding multiset Q P n!/( i≥1 zi !) is the total number of {y1 , y2 , · · · , yn }. For n ≥ 1, k ≥ 0, (z1 ,z2 ,···)∈B1 P n,n+k,n+k combinations of (y1 , y2 , · · · , yn ), such that nj=1 yj = n + k, i.e. X. Q. 1 (z1 ,z2 ,···)∈Bn,n+k,n+k. n!. i≥1 zi !. =. n H(n+k)−n. =. Hkn. µ ¶ n−1+k = , k. as desired. Example 4. Assume p0 = 0 and pk = p(1 − p)k−1 , k ≥ 1, where 0 < p < 1. Then for dte ≥ n, dte−n µ. P (Qt = n) =. X k=0. ¶ ¶ dte−n−1 µ X n − 2 + k n−1 n−1+k n k p (1 − p) − p (1 − p)k , k k k=0. and P (Qt = n) = 0, for dte < n. Proof. For dte < n, the result is obvious. We now prove the case for dte ≥ n. By letting p0 = 0 and pk = p(1 − p)k−1 , k ≥ 1, in Theorem 1, and from Lemma 1, we obtain P (Qt = n) =. gn−1,t − gn,t 14.

(20) X. =. 1 (x1 ,x2 ,···)∈Bn−1,n−1,dte−1. X. −. 1 (x1 ,x2 ,···)∈Bn,n,dte−1. dte−n. =. X k=0. i≥1. Y (p(1 − p)i−1 )xi n!( ) xi ! i≥1. X. {. 1 (x1 ,x2 ,···)∈Bn−1,n−1+k,n−1+k. dte−n−1. −. Y (p(1 − p)i−1 )xi (n − 1)!( ) xi !. X. {. k=0. X. P (n − 1)! Pi≥1 xi Q p (1 − p) i≥2 (i−1)xi } i≥1 xi !. Q. 1 (x1 ,x2 ,···)∈Bn,n+k,n+k. n!. i≥1 xi !. p. P i≥1. xi. P. (1 − p). i≥2 (i−1)xi. }. dte−n µ. =. X k=0. ¶ ¶ dte−n−1 µ X n − 2 + k n−1 n−1+k n k p (1 − p) − p (1 − p)k . (40) k k k=0. 1 That the last equality of (40) holds is because if (x1 , x2 , · · ·) ∈ Bn−1,n−1+k,n−1+k , 0≤k ≤ P P P dte − n, i.e. i≥1 xi = n − 1 and i≥1 ixi = n − 1 + k, then i≥2 (i − 1)xi = k. Similarly, if P P 1 xi = n and i≥1 ixi = n + k, then (x1 , x2 , · · ·) ∈ Bn,n+k,n+k , 0 ≤ k ≤ dte − n − 1, i.e. i≥1 P i≥2 (i − 1)xi = k.. Remark 2. As a comparison, for the pk , k ≥ 0, defined in Example 4, we have btc−n µ. P (Nt = n) =. X k=0. ¶ ¶ btc−n−1 µ X n−1+k n n + k n+1 k p (1 − p) − p (1 − p)k , btc ≥ n, k k k=0. and P (Nt = n) = 0, for btc < n. 4. Limiting and some other related results For the renewal process {Nt , t ≥ 0}, it is known that for every t > 0 and r > 0, a.s. a.s. E(Ntr ) < ∞, and as t → ∞, Nt −→ ∞, Nt /t −→ 1/µ, and E(Nt )/t → 1/µ, where µ = E(X1 ) < ∞. By using (6), (7) and (10), the following consequence is immediate. Theorem 2. Let µ = E(X1 ). (i). For t > 0 and r > 0, E(Qrt ) < ∞. a.s. (ii). If t → ∞, then Qt −→ ∞. a.s. (iii). Let µ < ∞. If t → ∞, then Qt /t −→ 1/µ. (iv). Let µ < ∞. If t → ∞, then E(Qt )/t → 1/µ. Along the lines of the proof for the renewal process {Nt , t ≥ 0}, the central limit theorem also holds for {Qt , t ≥ 0}. 15.

(21) Theorem 3. Let µ = E(X1 ) < ∞, and σ 2 = Var(X1 ) < ∞, then Qt − t/µ p −t−→ −d−∞ → N (0, 1). 3 σ t/µ. (41). For the process {Nt , t ≥ 0}, it is well known E(SNt +1 ) = E(X1 + · · · + XNt +1 ) = E(X1 )E(Nt + 1), t ≥ 0.. (42). Although Nt is not a stopping time, Qt nevertheless is a stopping time. Hence by the Wald equality and (i) of Theorem 2, we have E(SQt ) = E(X1 )E(Qt ), t ≥ 0.. (43). We use Example 3 to illustrate (43). Example 3.(Continued) For t = 0, Q0 = 0, and SQ0 = 0, hence (43) holds. For t > 0, as E(SQt ) = dte, E(Qt ) = dte/p1 , and E(X1 ) = p1 , (43) holds again. We now give an example to present a partial answer of the Hello Kitty magnets example mentioned in Introduction. Note that if one magnet is given at a time, the expected number of purchases to collect a complete set of 41 magnets is t = 176.42. Example 5. Let Y1 , Y2 , · · · be i.i.d. random variables with the same distribution as Y , where for every i ≥ 1, Yi denotes the amount that a consumer spends at the i-th purchase at 7-Eleven. Assume that Y ∼ Uniform{1, 2, · · · , 250}. Then 76 , 250 77 , p1 = P (X1 = 1) = P (77 ≤ Y ≤ 153) = 250 77 p2 = P (X1 = 2) = P (154 ≤ Y ≤ 230) = , 250 20 p3 = P (X1 = 3) = P (231 ≤ Y ≤ 250) = , 250 E(X1 ) = 1.164, Var(X1 ) = 0.905104, and N = sup{i | i ≥ 0, pi > 0} = 3. Thus by routine computations, we obtain p0 = P (X1 = 0) = P (1 ≤ Y ≤ 76) =. b. P (Qt = n) =. dte−1 c N. µ ¶ µ ¶ 76 m 76 n−m−1 n − 1 n 76 (1 − ) ( ) ( − ) 250 250 m m 250. X. m=0. dte−1. X. + m=b. ·(. dte−1 c+1 N. k dte−1−m m−d 2 e. X. X. k=0. v=m−k. m! 77 2m−v−k ( ) v!(2m − 2v − k)!(k − m + v)! 250. µ ¶ µ ¶ 20 k−m+v 76 n−m−1 n − 1 n 76 ) ( ) ( − ), n ≥ 59, 250 250 m m 250 16.

(22) E(Qt ) =. b dte−1 N c+1 + 1 − P0 ·. k dte−1−m m−d 2 e. dte−1. X. m=b. dte−1 c+1 N. X. X. k=0. v=m−k. m! v!(2m − 2v − k)!(k − m + v)!. 250 × 772m−v−k × 20k−m+v , 174m+1. and b. Var(Qt ) =. dte−1 c N. X 81500 + 125000m + 1742. m=0. dte−1. X. m=b. dte−1 c+1 N. k dte−1−m m−d 2 e. X. X. k=0. v=m−k. m! v!(2m − 2v − k)!(k − m + v)!. 2502 × 772m−v−k × 20k−m+1 − (E(Qt ))2 . 174m+2 . . Now for t = 176.42, E(Q176.42 ) = 152.466, and Var(Q176.42 ) = 101.888. Hence the expected number of purchases to get magnets greater than or equal to 176.42 is about 152.466. Furthermore from (43), we have ·. . . E(SQ176.42 ) = E(X1 )E(Q176.42 ) = 1.164 × 152.466 = 177.470, which is slightly greater than 176.42. Recall that, by the definition of Qt , SQt ≥ t, t ≥ 0. Finally, we give the curve of the probability density function of Q176.42 in Figure 2, and plot the probability density functions of Z176.42 and N (0, 1) in Figure 3, where Q176.42 − 176.42/1.164 p Z176.42 = √ 0.905104 176.42/1.1643 is the normalized Q176.42 . As expected, due to Theorem 3, the normal approximation to the probability density function of Z176.42 is very accurate. 0.04 0.035. P(Q176.42=n). 0.03 0.025 0.02 0.015 0.01 0.005 0 60. 80. 100. 120. 140. 160. 180. 200. n. Figure 2. The probability density function of Q176.42. 17.

(23) 0.4 0.35 0.3 0.25 0.2 0.15 0.1 0.05 0 −4. −3. −2. −1. 0. 1. 2. 3. 4. Figure 3. The probability density functions of Z176.42 (dotted line) and N (0, 1)(solid line) References 1. Baum, L.E., Billingsley, P. (1965). Asymptotic distributions for the coupon collector’s problem. Ann. Math. Statist. 36, 1835-1839. 2. Feller, W. (1950). Probability Theory and Its Applications. Vol. 1. John Wiley & Sons, New York. 3. Foata, D., Han GN., Lass, B. (2001). Les nombres hyperharmoniques et la fratrie du collectionneur de vignettes. S´ eminaire Lotharingien de Combinatoire 47, B47a, 20 pp. 4. Foata, D., Zeilberger, D. (2003). The collector’s brotherhood problem using the NewmanShepp symbolic method. Algebra Universalis 49, 387-359. 5. Goodwin, H.J. (1949) On cartophily and motor cars. Math. Gazette. 33, 169-171. 6. Hoslt, L. (1971). Limit theorems for some occupancy and sequential occupancy problems. Ann. Math. Statist. 42, 1671-1680. 7. Von Schelling, H. (1954). Coupon collecting for unequal probabilities. Amer. Math. Monthly 61, 306-311.. 18.

(24)