Bivariate regression splines

(1)

, " , I ¢ I

I

ELSEVIER Computational Statistics & Data Analysis 21 (1996) 399-418

COMPUTATIONAL

STATISTICS

& DATA ANALYSIS

Bivariate regression splines

Lin-An Chen

Institute of Statistics, National Chiao Tung University, Hsinchn 30050, Taiwan

Received May 1993; revised May 1995

Abstract

Towards the construction of multivariate spline functions, we introduce a way to set linear restrictions in the generation of bivariate regression splines. The hyperplanes in R 2 are used in the role of "knot" to slice the domain of explanatory variables; hence, we have the flexibility in domain partition which includes rectangle, parallelogram, trapezoid and trapezium.

Keywords: Bivariate regression spline; Hyperplane; Linear restriction; Piecewise polynomial

1. Introduction

A standard way to approximate the cause-and-effect relationship is a single model over the entire range of explanatory variables, for example, models for linear or polynomial regression. In practice, it might be more realistic to partition the range of explanatory variables as disjoint regimes and to approximate the relationship by a sequence of submodels which is smoothly connected, in some sense, at the boundaries of neighboring regimes. A useful technique for this purpose is the spline function.

Among many approaches to define spline functions, three are widely used. The first is the interpolating spline function, a piecewise polynomial generated com- pletely by interpolating the data points and satisfying conditions of continuous derivatives up to a required order. This method is useful only for fitting nonnoisy data and is then unsuitable for statistical data analysis. The second is the smoothing spline, a solution to an optimization problem of minimizing a sum of a least- squares-like term and a term penalizing roughness. The other is the regression spline which is a piecewise polynomial calculating its parameters by least-squares technique with imposed conditions of continuous derivatives up to a required order. For general accounts of splines, the paper by Wegman and Wright (1983)

(2)

400 Lin-An Chen / Computational Statistics & Data Analysis 21 (1996) 399-418

provides a clear review of splines of these and other kinds; the book by Eubank (1988) provides a good introduction of the theory of smoothing splines.

Being a smooth piecewise polynomial, the regression spline has received much attention from statisticians. Several approaches have been studied to generate regression splines. Poirier (1973) introduced a cubic regression spline with an excellent discussion of basic theory. Buse and Lim (1977) developed linear restrictions on parameter space such that the regression cubic spline is obtained by the restricted least-squares technique. Smith (1979) showed that the cubic regression spline can be obtained by using the " + " function.

About extensions of smoothing splines to the multivariate case, Meingnet (1979), Dyn and Wahba (1982), Cox (1984) and Barry (1986) investigated the multivariate smoothing splines by the method of penalized least squares. Poirier (1975) considered in two articles bilinear splines by the method o f " + " function; one article has a good application of a bilinear spline to formulate the Cobb-Douglas produc- tion function.

Our objective is to propose a class of bivariate regression splines by the technique of restricted least squares. This topic is motivated by several reasons. Buse and Lim (1977) pointed out that formulating a regression cubic spline by the restricted least-squares technique is more general than the approach by Poirier (1973), in that the number of restrictions can be varied and the validity of the restrictions can be tested. However, the regression spline by the technique of restricted least squares has been done for only the case of a cubic single-variable spline. An extension of regression spline to the multivariate case is only the bilinear regression spline of Poirier (1975). The regression regime considered in spline is mostly the rectangular type, but Hamermesh (1970) and Otto et al. (1966) pointed out that many economic structural changes happen only on the axis of a single variable. In Hamermesh's paper he considered estimation of a wage equation for which the consumer price index is the factor affecting the structure. Otto et al. attempted to explain the budgetary process of US government agencies where time is considered as the index of structural changes.

Based on the development of restriction matrices that impose restrictions on the space of regression parameters, we can design regression splines of many types and can extend the polynomial order to arbitrary "k". Of course, the bivariate regression spline that has changed on the axis of a single variable is considered.

The bivariate regression splines to be defined are piecewise bivariate polynomials defined on domain of connected regime sets with continuity condition of partial derivatives on neighborhoods of regime sets. The regime set is a partition of ~2 by the slicing tool of hyperplanes that generate regression splines of many types, including rectangle, trapezoid, trapezium and parallelogram.

In general, a hyperplane in ~2 can be formulated as

~c((~1, 62) = {X = (XI, X2)': 61X 1 "-[- 62X 2 = C} (1.1) with c = 0 or 1. For specification, pairs (FI (61, 0), Fo(61, 0)) and (F1 (0, 62), Fo(0,

62))

with 61 ~: 0 and 62 4:0 include vertical and horizontal hyperplanes, and the pair

(3)

Lin-An Chen / Computational Statistics & Data Analysis 21 (1996) 399-418 401

hyperplanes, we can choose to construct regression splines in m a n y ways. Besides our proposal of regression splines of some types by the restricted least-squares technique, an important part of this work is to find linear restrictions that fulfil the continuity conditions. In contrast to a knot in the role of a change point in a single-variable spline function, we call the hyperplane in R E the knot space also. In Section 2, we introduce m o n o t o n e and bivariate quadrilateral regression splines for which the knot space/'1 (31, 32) with 3a, 32 ~: 0 is used as a slicing tool. In Section 3, rectangle-type bivariate regression spline with knot spaces/'1 (bl, 0) with 61 # 0 and F1 (0, 32) with 32 :# 0 is introduced. The linear restrictions that fulfil the required continuity conditions are derived for all cases. A Bayesian technique for estimating the hyperplanes is introduced in Section 4.

2. Bivariate regression spline with slant knot space

/"1(31, 32)

Let k be a positive integer. The class of degree k bivariate polynomials is formulated as

P

~-"

{ P : P ( x , f ) = ~

~

IJjtJ2 X1 X 2 '

t~

J, J~

~J, J2

's

are real ,

}

(2.1)

l=O jl + j2=l

where x = (Xl, x2)' is a vector of explanatory variables and the vector fl contains coefficients fir, J2 of the bivariate polynomial.

We introduce m o n o t o n e and bivariate quadrilateral regression splines based on slant knot space. Before this, we discuss a spline model lor bivafiate two-phase regression.

Definition 2.1. Let 6 = (31, 32) t with 61, 32 :~ 0. The bivariate two-phase regression model

y = P(x, flo)I(x'3 <_

1) +

P(x, flb))I(x'3

> 1) + e (2.2)

is a bivariate two-phase slant regression spline model if it satisfies the following continuity conditions:

Pj,j2(x,f a)

= P j , j 2 ( x , f b) for XeFl(6) and 0 < j l + j 2 ~ k - 1, (2.3)

where

~J, + J2

Pj, j2(x, f ) - J,xl

P(x, f).

The condition P~k

_j(X, fa) = Pjk-j(X, fb)

is not considered because it would result in the fact that f l j k -

= fljk-j,

b _{j = 0, ..., k, where}

fljk-j

a _andf l j b _ j _{are coefficient para-} meters of P ( . , f~) and P ( . ,

fib)

corresponding to the term x~ x k-j.

To fulfil the continuity conditions for this regression spline, we derive a suffÉcient condition represented by some linear restrictions on the parameter space. The

(4)

402 Lin-An Chen/ Computational Statistics & Data Analysis 21 (1996) 399-418

representation of a differentiated bivariate p o l y n o m i a l on k n o t space FI(6~, 3 2 )

leads us to find those linear restrictions.

L e m m a 2.2. Let P e P and 0 <-jl -F j2 <-- k - 1; the ( j a , j 2 ) t h partial derivative of bivariate polynomial P on hyperplane F1 (61, 32) with 32 4:0 is formulated:

I

k - ( J t +J2)

njlj2 ( X I ' ~ ) = E E E O<_c<_k--(j~+j2 ) l=c O<_d<_c

6~ -a d! ( l - d ) ! d ~J'+eJ~+lr-d)6~-d x~I" (2.4)

C o n s i d e r the vector fl in a fixed p e r m u t a t i o n of parameters flj,j. F o r this

p e r m u t a t i o n there are vectors Lj,j2(c), c = O, 1, ... , k - ( j ~ + j 2 ) such that k-(J,+J2) ( - - 1 ) c - d ( j l + d ) ' ( j 2 + l - d ) ' ( l - d )

L j , j , ( c ) . f = _{l = c}2 _{0 < d < c}Z 6~2 -a d! ( l - d ) ! _{\ c} _{d ]}

x flj, + dj~ + II- d)6~ -a" (2.5)

T h e p e r m u t a t i o n is specified in a convenient way according to its c o r r e s p o n d i n g k n o t space. With the above vector representation.

Pj,j~(xl, fl) = ~ LH2(c)'flx~. (2.6)

O<_c<k-(jl +J2)

Let the p a r a m e t e r vectors /~a and /~b be arranged associated with the same

p e r m u t a t i o n of indices. T h e n the condition P:,j2(x,,6")=Pj,j2(x,~ b) for

0 ~ j l + j 2 ~ k - 1 o n / " 1 ( 6 1 , 6 2 ) is

Lj,j~(c).(fl ~ - IJb)xf = 0 0 _~ C _~ k -- (Jt +J2)

for xl ~ U~ and all 0 --<jl -F j2 --< k - 1, (2.7) which is equivalent to

Lj~j2(c).(~, _ ~b) = O, 0 < c < k - (jl + j 2 ) and 0 ~ j x + J 2 -< k - 1. (2.8) T h e continuity condition (2.3) can be replaced by linear restrictions in (2.8). T h e bivariate two-phase regression spline is then the restricted least-squares estimator

of which the restriction matrix is the vertical joining of all vectors Lj,j2(c),

0 < c < k - ( j l -b j2) a n d 0 < J l + j 2 < k - 1. However, unlike the single-variable

case (see Buse a n d Lim, 1977), the class of Lj,j2(c) in (2.8) is a linearly d e p e n d e n t set

having n u m e r o u s numbers. We seek a m a x i m u m set of linear i n d e p e n d e n t vectors that greatly simplifies the task of finding regression splines.

Definition 2.3. Any m a x i m u m set of linearly i n d e p e n d e n t vectors in set {Lj,j2(c): 0 < e < k - (ja +J2) a n d 0 <J1 + j 2 < k - 1} is called a F1(61, 62)-based restriction basis.

(5)

F o r this k n o t s p a c e / ' 1 (61, 62), the following t h e o r e m exactly explains the de- p e n d e n c e a n d gives a restriction basis.

T h e o r e m 2.4.

Suppose 61 ~ O.

(a)

Fix

(r,j), 1 _< r < k - 1

and 1 <_ j <_ r. For b,

O < _ b < _ k - r ,

L ~ - j j ( b ) = ( - ( b + l ) x L , - j j - x ( b + l ) + L r - j + 1 j - 1 ( b ) ) x ( 6 1 / 6 2 ) .

(b)

The set {Lio(C): 0 < c < k - i , i--O, 1, ... ,k -

1}

of number

(k~2) _ 1

vectors

forms a Fx

(61,

62)-based restriction basis.

Assume that we have n observations

(yi,x~)

with

x~6

< 1 for i _< nl and

x'6 > 1

for i > n a. Let

y = . ,

x'.,i]' x b = l

i }' 13=

, X =

x b .

(2.9,

Moreover, let R be the vertical joinings of vectors of/'1(61, 62)-based restriction basis. T h e n the restricted least-squares estimator of the bivariate two-phase regression spline is

= ~,s - < x ' x ) - 1 R ' ( R ( X ' X ) - 1R')- 1R~,s,

(2.10)

where

~ls = (X'X)- 1X'y,

the ordinary least-squares estimator of/3.

T h e estimated bivariate-two phase regression spline is

y = P(x, fl")I(x'6 <

1) +

P(X,~b)I(x'6

> 1), (2.11)

where/~" and/~" satisfy/~ = (/~,//~b).

T h e n u m b e r of parameters of a bivariate p o l y n o m i a l of order k is (k~2). With a continuity c o n d i t i o n i m p o s i n g (k~ 2) _ 1 linear restrictions, the degrees of freedom of the p a r a m e t e r space of the bivariate two-phase regression spline is then (k S 2) + 1. We extend this idea to a m u l t i p h a s e case.

Definition 2.5. (a) If a set of slant k n o t spaces {Fl(6r), 6~, 6~ ~ 0, r = 0, 1, ... ,a} such that the class of sets

{ x : x ' 6 ~ > l , O < j < r - l a n d x ' 6 r < l } ,

r = l .... ,a (2.12) forms a partition of the d o m a i n of explanatory variables Xl a n d x2, that is, they are m u t u a l l y exclusive, then we call t h e m slant m o n o t o n e regime sets.

(b) A bivariate slant regression spline with regime sets (2.12) is defined as

f ( x ) = ~ P(x,#r)l(x:x'6 j

> 1, 0 < j < r - 1 and x'6' < 1), (2.13) r = l

with continuity c o n d i t i o n s

P/,~2(x,/F -1) =

Pj,j2(x,~ ~)

on

F1(6 "-1)

(2.14)

(6)

404 Lin-An Chen/ Computational Statistics & Data Analysis 21 (1996) 399-418

We add k n o t spaces F~ (8 °) a n d F1 (6") only for convenience. In fact, we assume

that there is no observation falling u n d e r the h y p e r p l a n e

6°'x

= 1 a n d falling above

the h y p e r p l a n e

6"'x

= 1. T h e (k $2) _ 1 linear restrictions are i m p o s e d associated with each neighboring k n o t space and the continuity requirement does not apply on the b o u n d a r y k n o t spaces Fx (6') for r = 0, a. The k n o t spaces and the n u m b e r "a" (or "b" that is used later) are a s s u m e d to be known. Explicit f o r m u l a t i o n of the bivariate slant regression spline is an a n a l o g o u s extending of (2.10) a n d (2.11) to the case of "a" p o l y n o m i a l s that we neglect.

As the parameters for the bivariate slant regression spline n u m b e r s

a(k+2 2)

in

total, so (b) of T h e o r e m 2.4 implies that this spline has degrees of freedom (k~ 2) + (a - 1). A refinement of slant m o n o t o n e regime sets is the quadrilateral set. Definition 2.6. (a) Let {6~} a n d {6 h} represent n o n z e r o k n o t vectors. If the set of slant k n o t spaces {F~ (6~), F1 (6h}; r = 0, 1, ..., a, h = 0, 1 .... , b} such that the class of sets

x 6 v ~ l a n d x ' 6 h < l } , r = l .... , a a n d h = l . . . b

{x:x'6~, -a ~ 1,x'6~ <

1, ' h-1

(2.15) forms a partition of the d o m a i n of explanatory variables x~ and x2, then we call t h e m a quadrilateral regime set.

(b) A bivariate quadrilateral regression spline with regime sets (2.15) is defined as

y = ~

P(x,

flrh)l(X'6r- 1 ~

1, X'6 r (

1,

X'6~-

1 ) . 1 and x'6~ < 1) + E,

h = l r = l

(2.16) with the following continuity conditions.

Fixed h:

p~,j~(x, fl,-1 h) = p~,h(x,p,h)

on F1(6~-1), r = 2 . . . a (2.17)

Fixed r:

Pj,j2(X, fl r h - 1 ) =

Pj,j2(X, fl rh)

on F l ( 6 h - ' ) , h = 2 . . . b (2.18) f o r 0 _< j l -+-J2 ~ k -- 1.

This quadrilateral regression spline has the p r o p e r t y that each piece of polynomial includes all regressor terms x~' x~ 2 for which j l + j2 -< k is satisfied. However, the rectangle regression spline, to be i n t r o d u c e d in the next section, has to sacrifice some regressor terms to fulfil continuity conditions in (2.17) and (2.18). With T h e o r e m 2.4, we have an explicit form of the bivariate regression spline.

Let R~,, R h be the vertical joinings' of F1 (6 r) and

Fl(6h)-based

restriction basis',

r -- 1 .... , a - 1 a n d h = 1, ..., b - 1, respectively. F o r these

ab

polynomials, to fulfil

(7)

includes the s u b m a t r i x that has n o n z e r o elements o n the following matrix, of w h i c h its c o r r e s p o n d i n g p a r a m e t e r v e c t o r s are also listed:

- lj ij j ... IJ ° - ' j I w

-

R ~ - 1 - - R ~ - 1

j = 1 , . . . , b . (2.19)

T h e restriction m a t r i x includes a s u b m a t r i x that has a n o n z e r o subset for c o n t i n u - ities of p o l y n o m i a l s o n the kth c o l u m n regimes as

--ilk1 ilk2 ilk3 ... flkb-1 flkb --

m

k = 1 , . . . , a . (2.20)

T h e restriction m a t r i x is R with R' = ( R u , R u , . . . . ~1, ~2, , R u , R v , ~b, "1, , Rv ). T h e m a t r i c e s in - a ,

(2.19) a n d (2.20) c o n t a i n zeros for which the c o r r e s p o n d i n g p a r a m e t e r vectors are

n o t listed. C o n s i d e r an a r r a n g e m e n t , the set j o i n i n g v e c t o r /~ with

~j' = (ill j , , . . . , flaj,),j = 1 , . . . , b a n d further j o i n i n g fl with/~' = (/~1,, ..., fib,). Let the c o r r e s p o n d i n g o b s e r v a t i o n v e c t o r s a n d m a t r i c e s be ( f h , x r h ) , r = 1 . . . . , a a n d h = 1 , . . . , b . Define the e x t e n d i n g o b s e r v a t i o n v e c t o r s a n d matrices, y with y, = ( y l , , . . . ,yb,) w h e r e yJ satisfies f i ' = (ylj,, . . . . yaj,) a n d X = d i a g ( X 1, ... , X b) w h e r e X ~ = d i a g ( X ~j .... , X~J). T h e restricted l e a s t - s q u a r e s e s t i m a t o r o f the bivariate q u a d r i l a t e r a l regression spline has the f o r m o f (2.10) w h e r e m a t r i c e s y, X, a n d R are the versions listed above.

T h e r e m a i n i n g i m p o r t a n t task is to list explicitly the m a t r i x f o r m of the F1 ( 6 1 , 6 2 ) - b a s e d restriction basis. Before this, we c o n s i d e r a simple p e r m u t a t i o n of the index set {(Jl,

j2):

0 < j l

+ J 2 --<

k}. Define a s u b o r d e r set

S(m): (k - m, O) (k - m - 1, 1) (k - m - 2, 2),... ,(k - m - i, i), ... ,(0, k - m),

(2.20) w h e r e 0 < i < k - m. S ( m ) is an o r d e r e d set of { ( j l , j 2 ) : j l + j = = k - m}. W e define the p e r m u t a t i o n of the index set as the h o r i z o n t a l j o i n i n g of s u b o r d e r set S ( m ) with s e q u e n c e m = k, k - 1 , . . . , O; that is,

(8)

With this index permutation, we set fl as a vector of parameters flJ~J2 permuted according to the order (2.20), (2.21). As an example, we list the matrix form of F1 (61, 62)-based restriction basis for k = 2, 3 and 4 in the following where permutation of vectors is set by way of vertical joining of subvectors:

Lio(1)

. i = O , 1 . . . . , k - 1 .

L i o ( k - i)

The empty cells in the following three matrices represent zero elements and their corresponding parameters according to a linear restriction are also listed.

C a s e 1: k = 2

/ho /~,1 /~o2 /ho /~o,

6, 6 2 1 62 622 1

26,

61

6-~ 622 1 62 1 1 622 62 6, 2 62 1 1 62 C a s e 2: k = 3 f130 fl21 ill2 f103 3 - al ~2 - 6 , 1

62 ,s~

a~

1 - 26, 36e

62 a~

623

1 - -

36,

6~

63

1

/~oo

f120 fill f102 fllO flO! flO0

-- 61 6 2 62 a~ 1 - -

26,

62 622 1

622

-- 61 62 1 62

(9)

Lin-An Chen / Computational Statistics & Data Analysis 21 (1996) 399-418 407 - 2 6 , 6 ~

62 6 2

2 -

26~

62 62 2 1

a2

- 261 a2 2 6-7 2 - 6 1 62 1 62 Case 3: Part 1: [14o k -- 4 ( m a t r i x is h o r i z o n t a l l y s e p a r a t e d into two p a r t s )

fl31

fl22

fl13

f104

f130 f121

fl12

~o3 12 -,51

~2

1 62 - 36~

(52

3 62 - - 6 ( 5 1

(52

6 c~2

ae

- a~

a~

- 2 6 ~ 36e - 4 6 ~ - 6 1 1

a~

1 -- 36, 66~ 1

a~

6~

a~

1 -- 461

a~

6~

1

a~

26~ - 6~

a~

- 46~ 36~

a~

2 -

361 a~

a~

1

61

262

622

-- 461

a~

2

a~

a2

a~

- 261

a~

i

a~

a3

3612

- 361

a2

~

1

a~

- - 2 6 1 6 ~ 2 -- 261 a2 6~ 1 - - 2 6 1

~2

2 ~2

(10)

408 Lin-An Chen / Computational Statistics & Data Analysis 21 (1996) 399 418 24

Part

2:

/h0

- - 661 62 6 62 - 6 1 62 1 62

- 6 1

62

1

62

flo2 fllo flOl floo

zero subvector zero subvector

a~

a22

- 2 6 1 - 61 1

622 a2

1 1 1

a22

62

zero subvector zero subvector zero subvector zero subvector 2 zero subvector zero subvector

where "zero subvector" is a vector of zeros.

We give the restriction basis of order k in a matrix form. The following matrices list only the nonzero parts of which the corresponding parameters are also listed. Let

L(1)

R = . , (2.22)

(11)

Lin-An Chen/ Computational Statistics & Data Analysis 21 (1996) 399-418 4 0 9 w h e r e

El

Lk-

lo(1)

L(O) =

Lk-2O(2)

Loo(k)

F Lk-sO(1)

L (s) = I Lk_s-10

(2) /

~. Loo(k - s +

1) , s = 1 , . . . , k . S u b m a t r i x L(O) is

flkO f l k - l l ilk-22 "'" f l k - i - l i - 1 f l k - i i "'" f l l k - 1 flOk

k! - (k - 1)! 8_£ 82 k! - ( k - 1)!82( k 2 ) ! ~ 2! 81 v2 - - 2 ) . 8 1 ( ( k - i + 1)! k! (k 1)!81 (k - v 2 __ 1 ) i - 1 8 i - 1 i! ( i - 1)!82 ( i - - 2 ) ! 8 2 1! 8 i-1

a~

( -- 1)'(k - i)! 8-T22 (2.23) -- 61 62 ( - - 1 ) k - 1 6 ] - 1 ( -- 1 ) k s ]

82

822 a~

a~

w h e r e i n d e x i = 1 .... , k. = AAr(m) is

W e set

L(s)

= rA'f{°) A•l) _{L ~} _{( s ) , ~v~ I s ) ~ . . .} _{~ ~ v J ( s l J}Aft(s)] w h e r e , for m 0,1, _{. . .} ,s,~,,ts)

1 ~ k - s - l s - m 8s 2 - m ( k -- S)! i l k - s - i s - m + 1 "'' f l k - s - i s + i - m "'" flOk-m

,k

,,,(s m,

1

,,i k

,,,(s+,

(2.24)

1 -1 (s-m-1)g,

_{( - E ' ? -}

a~-o \k_:)a~ -"

(12)

3. Patial bivariate regression spline constructed by knot spaces 111 (61, 0) and

111 (0, 62)

The vertical and horizontal knot spaces include/1(61, 0), i[1(0 , 62) , /"0(61, 0) and 1[o(0, 62). Without cancellation of some polynomial terms the continuity requirement on knot spaces 1[o(61, 0) and 1[o(0, 62) produces unwanted regression splines. To see this, we consider only the knot space Fo(0, 62). The (jl, j2)th derivative of the bivariate polynomial on Fo(0, 62) is

k - ( h + . J 2 )

phj,(xl ' fl) = _~=o~ (jl + c)! j2! _c! _{fir, +,',r~x] •} _(3.1) For fixed (jl, j2), the continuity conditions

Pkj2(x, fl v - l ) = Pj,j2(x, fl r) on 11o(0,62) (3.2) imply that

r r - I

fli, +,J2 = fir, + < f o r 0 ~ C _< k -- ( j l + j 2 ) . (3.3)

Hence, the continuity condition (3.2) of partial derivatives induces an unpleasant result of equating neighboring parameters. By verifying implication of (3.3) for all

0 ~jm +j2--< k - 1, all parameters of neighboring polynomials P(.,fl~) and

p ( . , f l r - m ) except fl~k and fl~)~-i (the term flog corresponding to P(.,fl~) and

P(., fl'-1)) are equal. Because of this unpleasant property, we consider only those regression splines with knot spaces F1 (61, 0) and F1 (0, 62) as the slicing tools. There remain unpleasant implications of the use of these two knot spaces. The unpleasant one is avoided by slight relaxation in deleting some polynomial terms. We will study for only the knot space F1 (0, 62), the case of knot space F~ (61,0) is similar and is skipped.

Lemma 3.1. The (jl, j2)th partial derivative of a bivariate polynomial P in P on knot space Fa (O, 62) with 62 ~ 0 has the form

k - ( J l + J 2 ) Phj2(Xl, fl) = ~_~ ( L ~ o z ( c ) . f l ) x ~ , (3.4) c = O where k IV, +J2t(j 1 + C)!(I -- c + J 2 ) ' Lj, r2(c). = Y ( i - - c)! +'ja + , - c and l = c 0 _< J1 + J2 --< k - 1.

Definition

3.2. Generated by knot spaces FI(0, 62), the maximum set of linearly independent vectors in set {L~o2(c): 0 _< c < k - ( j l +J2), 0 ~ j l + J 2 -< k - l} is called a/'1 (0, 62)-based restriction basis.

(13)

T h e following t h e o r e m states a relation of linear d e p e n d e n c e of vector LJ,j~(c) and

gives a basis set.

T h e o r e m 3.3. Suppose 3 1 = 0 . (a) F i x ( j , b ) where 1 < j < k - b - 1 and

0 < b < k - 2. I f 0 < s < k - ( j + b), then vector Ljb(k - (b + j ) - s) is propor- tional to vector Lob((k - b) - s). (b) T h e set {Lob(C): 0 < c < k - b, 0 _< b _< k - 1}

is the El(O, 32)-based restriction basis.

T h e vector Lok-c(c) has only one n o n z e r o element of the form

/Lk-c (3.5)

(k - c)!

T o set restriction Log-c(c). (fir _ f i r - l ) = 0 implies that tickr -c = tickr--c.1 T h e simple

way to avoid this unpleasant p r o p e r t y is to delete all vectors Lo,-c(c) from the

restriction basis.

Definition 3.4. G e n e r a t e d by k n o t s p a c e / ' 1 (0, 62), the set {Lob(C): 0 <_ c <_ k - 1, 0 < b _< k - c - 1 } is called a F1 (0, 62)-based partial restriction basis and the set which c o r r e s p o n d s to k n o t space Fa(61,0) is called a Fx(61, 0)-based restriction basis.

This partial restriction basis n u m b e r s (k-~ 2) elements.

First we consider a m o n o t o n e regression spline when only the d o m a i n of a single variable is partitioned. W i t h o u t loss of generality, we consider that single variable to be x2.

Definition 3.5. Let R h be the vertical joinings of vectors in/"1(0, 62)-based partial

restriction basis. A x2-segmented bivariate regression spline is defined as b

y = ~ P(x, flh)I(x: 1 < X26~- 1, X26~ < 1) + e (3.6) h = l

with continuity c o n d i t i o n R~- 1 (fib __ flh- ~ ) = 0, h = 2, ..., a.

Joining k n o t spaces F1(61, 0) and /'1(0, 62) produces rectangular regimes. We

define rectangular regression splines as follows.

Definition 3.6. Let g~, R~ be vertical joinings' of vectors in/'1(61, 0) and/'1(0, 62)- based partial restriction basis'. A bivariate rectangular regression spline is defined a s

y = ~ P(x, ~ h ) I ( x : X16]- 1 > 1, X~6] < 1, X26~- 1 > 1 and x26~ < 1} + e

h = l r = l

(3.7) with: (a) F o r h, R ~ - t I J "h = R ~ - l f l r-in, r = 2 , . . . , a a n d

(14)

412 Lin-An Chen / Computational Statistics & D a t a Analysis 21 (1996) 3 9 9 - 4 1 8

The bivariate partial regression splines on regimes slicing on only one variable and the rectangular regimes can be analogously obtained in the form of (2.10). Poirier (1975) considered bilinear splines other t h a n this restricted least-squares m e t h o d . In one article he applied it to the C o b b - D o u g l a s p r o d u c t i o n function.

F r o m e x a m i n a t i o n of the linear restrictions of (a) a n d (b) above, the degree of

freedom of the bivariate rectangle regession spline is "ab(k + 1)". We list the partial

restriction basis in a matrix form as

[ 1 [ ]

Ro Loo(S) R* ₌ RI _. with R~ = Lox(S) _, _{s = 0 , 1 , . . . , k} _- _{1 ,} _{( 3 . 8 )}

Rk-

1

Log

- s - 1 (S) a n d is j~s k - s fls k - s - 1 fls k - s - 2 "'" flsh + l ~ s h 1 1 1 6~-s 6~-~- , 6~-~- ~ k - s k - s - 1 k - s - 2 6 k2 - s - 1 6 k2 - s - 2 (~ k2 - s - 3 ( k - s)! (k - s - 1)[ (k - s - 2)[ 6 ~ - ~ - 2 ( k - s - 2 ) ! 6 ~ - ~ - 3 ( k - s - 3 ) !

6~-s-4(k

- - S

--4)[

/3~,/~, /3so

1 1 1

622 62

2 - - 1 ! 62 2! ( k - s)! ( k - s - 1)! ( k - s - 2)! 6 ~ - ~ - h ( k - s _ h ) ! b t ' 2 - ~ - h - ~ ( k _ s - h _ l)[ 6 k 2 - s - h - 2 ( k _ s _ h _ 2 ) [ (k - s)! (k - s - 1)! (k - s - 2)! 62 62 2!

(k - s)!

62 (h + 1)[ h! 62 (k - s - - 1)! (3.9)

To obtain the restriction basis for a bivariate p o l y n o m i a l on k n o t space Fo(61, 62), we give a representation of the bivariate polynomial.

L e m m a 3.5. Let 0 <-ji <- k - 1, i = 1, 2 and 0 <-jl + j 2 -< k - 1; the ( j b j z ) t h partial derivative of bivariate polynomial P on hyperplane Fo(6) where vector

6' = (61, 62) with 62 ~ 0 is formulated as

5,j,(x) =

_•

Y~

U Y',

_L ( - 1 ) ~ - a ( j l + d ) ! ( j 2 + c - d ) ! _{6c2 - e} _d! _{(c - d)!}

O < _ c < _ k - - ( j l +J2) O<_d<_c

.~-67 ~

(3.10)

(15)

Lin-An Chen / Computational Statistics & Data Analysis 21 (1996) 399-418 4 1 3

Let Li,h (c) be the v e c t o r such that

Lhj2(c). fl

= ,~, ( - - 1)c-a(jx + d ) ! ( j 2 + c - - d)!

6~z -d

d)

(c -- d)!

O _ < d < _ c T h e n

(3.11)

Pi, h(x) =

~

Lj,h(c).flx].

(3.12) O<_c<_k--(jl + J 2 )

T h e relations b e t w e e n the restriction vectors, a n d the basis class are exactly the s a m e as t h a t of T h e o r e m 2.4 with h y p e r p l a n e F1 (61, 62), w h e r e the restriction basis is {Lio(C): 0 < c < k - i, i = 0, 1, ... ,k - 1}. H o w e v e r , unlike

F1(61,

b 2 ) , the h y p e r - plane

Fo(6)

p r o d u c e s Lk-sO(O)'(fl r - - f i r - l ) = O, a n d hence fl/,-so = fl[--lo. W e de-

lete v e c t o r s

Lk-so(O)

for s = 0 .... ,k a n d set a partial restriction basis as the

following matrix: R * * = L ( 1 ) ,

L(k)._J

L(s) =

w h e r e - L k - s - 1 o ( 1 ) - L k - 5 - 2 o (2)

Lk-5-,, o(m)

_

Lo o(k

- s ) _ , s = 0 , 1 . . . . , k - 1 ( 3 . 1 3 ) as follows: ~ k - s - m m "'" ~ l k - s - 1 ~ O k - s

ilk-sO

i l k - s - 1 1 "'"

61

(k - s)! - (k - s - 1)!62 ( k - s ) ! ( k - s - l ) ! 61 - - . , . m! ( m - 1)! 62 ( - l)m(k - s - m)! 6--~ - 6 1 ~2 ( _ l ) k - ~ - ~ 6 ~ - ~ - I (__ l)k-~6~-~ f o r s = O , 1 , . . . , k - 1 a n d w h e r e m = l .... , k - s . (3.14)

(16)

4. A Bayesian approach for estimating bivariate knot

In this section, we address the knot estimation based on a Bayesian method that extends a Bayesian technique by Chin Choy and Broemeling (1980) for switching linear regression model to the restricted linear regression model. For simplicity, we consider only the 4-piece rectangle regression spline with a vertical hyperplane and

X I i X l n X l

a horizontal hyperplane. Let (x21),-.. ,(x2,,) be the observed values of x = (x). For each (u, v), 1 _< u, v _< n, the vector 6'--(61, 6 2 ) = (xl,, x2~) determines a knot vector that slice the space ~z into 4 rectangle pieces. The maximum number of pairwise distinct pairs (xlu, x2~) is n 2. For setting (Xlu, x2~) as a knot, we require that the number of observations xi in each rectangle is large enough so that unique restricted least-squares estimate can be obtained. Denote by J the index set of (u, v) that makes the restricted least-squares estimate uniquely determined. For (u, v) in J, let (y, X) be set under the line in Section 2 with knot spaces F1 ((~")) and F2 (G2,)). To o

Xlu

determine the knot point 6 = (x~,) is now equivalent to determine the index (~) in J. x~°

We further denote by R(~) the restriction matrix with knot vector G~,). Now, we set an assumption set.

Assumption

A. (al) The knot index is uniformly distributed over the set J. (a2) The spline parameter, fl is assigned the improper prior

rc(fl) oz constant.

(a3) The error variable e has normal distribution with mean zero and variance o -2,

w h e n 0 -2 has the well-known noninformative prior distribution

7G(O "2) = 1/O -2 for 0 < 0-2 < ~ .

The following theorem provides a posterior joint probability density function of the knot index and the regression parameters.

Theorem

4.1. Under Assumption A, the posterior probability density function of fl and knot index (~,) is

rc((~), fl lY) ~ [(Y - Xfl)'(y - Xfl)] -(n+ 2)/2 subject to R(~,)fl = O.

To obtain the estimate of the knot index, one way with this posterior density is by solving

arg inb(y - X f i r l s ) ' ( y - - Xfirls),

(17)

Lin-An Chert/Computational Statistics & Data Analysis 21 (1996) 399-418

Aeknowledgements

The author is grateful to the referees for their helpful comments.

415

Appendix

Proof of

L e m m a 2.2. The ( j l , j2)th partial derivative of bivariate polynomial P in

k-lh+J_,l

(jl + b l ) l ( j 2 +b2)[

b, h~

(A.1)

PiaJ2(X) = 2 2 flil +hi.Jz+b2 XI X2"

• / = 0 h ~ + h 2 = / b l ! b 2 ! P is

As we assume that 62 4: 0, the hyperplane/'1 (6) is then

F 1 ( 6 ) = X l , 6 2 / . x I e ~ , (A.2)

where we replace Xz by ((1 -

61xl)/62).

Denote the subpolynomial with degree l of Phh on Fa(6) by

Pl.

Then

1 ( j l + b , ) ' ( J 2 + b 2 ) l ( b n )

p,(xl)

= Z Z ( - - 1)"6b---S ba[ " 6'I

hi +b2=lONn<_ b 2

b2!

X ~j, + hi j2 + b2 Xbl I + ". ( A . 3 )

We claim that

Pt(Xl)

is a one-variable polynomial of degree l. By rearrangement of Pt, we have

p , ( x l ) = ~

~

( - 1 ) "

( j l + c - n ) , ( j 2 + ( l - c + n ) ) , ( l - c + n )

(c

h3i

n

x flJl + , - , I ,.+,,3~x~. (A.4)

For convenience, let d = c - n; we further have

Pt(Xl) = ~

~

( -1)c-a(Jl + d)' (j2 + l - d)' (l - d4")

6t2 -a

dl

(1 - d)!

O<_c<_lO<_d<c

\c

u /

x~-d c (A.5)

X fljl + d l - d U 1 X1.

Then (A.1) and (A.5) further imply L e m m a 2.2. []

(18)

P r o o f of Theorem 2.4. Eq. (2.5) gives

and k h r

L,_jj(b).fl= E

Z

l=bO<dl<_b

x fl~_j+a,j+(;_d,),

(A.6) k-r _ 1)h+ l - d , (r - - j + d l ) ! ( j + 1 - dl)!

L , _ j j _ , ( b + l ) . f l = Z

~"

( (~12-+-l-d'

d ~ ' ( l + l - d , ) !

l=b O<_d~ <_b+ 1 X ( ~ b l + l - d l ~ r _ j + d , j + l _ d l , + l - d 1 k-r b+l ( __ 1)h-d, + I (r - - j + da)!(j + l - dx)!

L~-j+lj_l(b).fl= Z

~ 6/2 -d ' + l (d, 11!(/ d - l + l=b d1 =1 - - - - X (~hl-d' + l f l r _ . j + d , j + l _ d , .

x

dl +

Then + 1 l - - d ' ~j ( b + l - d , ) - 11) (i

and with simplification we have

- (b + 1 ) x L r _ j j _ ~ ( b

+

1).fl + L , - ~ + l j - i ( b ) . f l =

~2L~-jj(b).fl.

k - J l £ ' O(X1) = E ( L h o ( C ) ' f l ) x ~ , c=O where Z J l o ( C ) ' f l = k - j , ( - - l ) c - d ' ( j l d l ' l=c da=O (A.7)

(A.8)

Let L (°), L (1) . . . . , L (k) be defined as in (2.22). It is easy to see that

L (°)

is the matrix stated in (2.23) and obviously vectors of L (°) are linearly independent.

We derive the general form of matrix L (s). The ith row of L (s) is

Lk-~-i(i),

i = O, 1,...,k

-

s. Lk_s_i(i). fl is s+i

~' h~+i_m.fl m,

(A.9)

l=i

where, w.l.o.g, we let

(19)

Lin-An Chen / Computational Statistics & Data Analysis 21 (1996) 399-418 417 m = O, 1, ... ,s, a n d tim' = ( f l k - s s - , , f l k - s - l s - , , , + , , ...,flk-s-,+~ ilk-s-is+,-,,). Lk-s-i(i) is the h o r i z o n t a l j o i n i n g o f {hs+i-m: m = O, 1, ... ,s}. By setting

MI,~) = h~+.l-m ,

h~+i-m__l

we h a v e L ~') _{= t~,,~),~,-~) . . . ~,,~)j. F r o m (A.10), for m = 0,1,}r~xto) A / ( 1 ) /I/f(s)-I . . _.,s,~,,(~)~ ' ) is the m a t r i x s t a t e d in (2.24). The last n o n z e r o elements of L ~) c o r r e s p o n d i n g to the last n o n z e r o elements o f s u b m a t r i x MI]I are {fiR-sO, f l k - ~ - ~ , ...,flOk-s}, respectively. Hence, r o w v e c t o r s of L ~s) are linearly i n d e p e n d e n t . T h e n vectors of m a t r i x R are linearly i n d e p e n d e n t a n d v e c t o r s of R are a Fa (6~, 62)-based restriction basis. [ ]

P r o o f o f T h e o r e m 3.3. P a r t (a) follows easily from (3.4).

F i x ( j , j ' ) , 0 < j , j ' < k - 1 w h e r e j , j' can be equal. W e have, f r o m (3.4),

foj(X, ) = 612- "'(I -- c1)! a n d k-~' -"' (j' + 1 -- Ca)! f ° J ' ( X l ) = ~ (i~=12(~12-c2(1--C2)] flc2j'+l-':'x'i2" C o n s i d e r (Cl, c2) w h e r e 0 < cl < k - j , 0 < c2 < k - j ' a n d c1 # c2. T h e n Loj(Cl).fl = x (1 - - e l ) ! r c ' J + l - c ' I = c tJ2 a n d k - j ' ( j , -4- l -- C2)! Loj'(c2)" fl = Z (~12----~; -(i---~2), fl~J' + ,-c~. l = c 2

Loj(cl).Loj,(c2) = 0 if cl 4: c2, which holds for a r b i t r a r y (j,j'). If cl 4= c2, the sets

{Loj(cl): j = 0,1 .... , k - cl} a n d {Loj,(c2): j = 0 , 1 , . . . , k - c 2 } are then linearly

i n d e p e n d e n t . W e will need to s h o w that, for each c, 0 < c < k, the set {Loj(C): 0 < j < k - c} is a set o f linearly i n d e p e n d e n t vectors. Let c, 0 < c < k; then there are o n l y j for w h i c h j < k - c c o r r e s p o n d to restriction v e c t o r Loj(C). T h e m a t r i x o f linear restriction v e c t o r s is f o r m e d as a vertical j o i n i n g of set {Loj(C): 0 < j < k - c, 0 < c < k} as of (3.8) a n d (3.9).

The fact of linear i n d e p e n d e n c e follows f r o m the fact that Rc is a d i a g o n a l matrix; hence v e c t o r s of R* of (3.8) are a / ' 1 ( 0 , 6z)-based restriction basis.

P r o o f o f T h e o r e m 4.1. This is d o n e b y integrating the j o i n t p r o b a b i l i t y density f u n c t i o n o f (~),//, y, a n d 0 .2 with the t r a n s f o r m a t i o n variable (y - XIJ)'(y - Xll)/tr a.

(20)

418 Lin-An Chen / Computational Statistics & Data Analysis 21 (1996) 399 418 References

Barry, D., Nonparametric Bayesian regression, Ann. Statist. 14 (1986) 934-953.

Buse, A. and L. Lim, Cubic splines as a special case of restricted least squares, J. Amer. Statist. Assoc. 72 (1977) 64-68.

Chin Choy, J.H. and L.D. Broemeling, Some Bayesian inferences for a changing linear model, Technometrics 22 (1980) 71 78.

Cox, D.D., Multivariate smoothing spline functions, S I A M J. Numer. Anal. 21 (1984) 789-813. Dyn, N. and G. Wahba, On the estimation of functions of several variables from aggregated data,

S I A M J. Math. Anal. 13 (1982) 134-152.

Eubank, R.L., Spline smoothing and nonparametric regression (Marcel Dekker, New York and Basel, 1988).

Hamermesh, D.S., Wage bargains, threshold effects and the phillips curve, Q. J. Econom. L X X X X I V (1970) 501-517.

Meinguet, J., Multivariate interpolation at arbitrary points made simple, J. Appl. Math. Phys. 30 (1979) 292-304.

Otto, A. et al., A theory of the budgetary process, Amer. Political Sci. Rev. LX (1966) 529-547. Poirier, D.J., Piecewise regression using cubic splines, J. Amer. Statist. Assoc. 68 (1973) 515-524. Poirier, D.J., On the use of bilinear splines in economics, J. Econometrics 3 (1975a) 23-34. Poirier, D.J., On the use of Cobb-Douglas splines, Int. Econom. Rev. 16 (1975b) 733-744. Smith, P., Splines as a useful and convenient statistical tool, Amer. Statist. 33 (1979) 57-62. Wegman, E.J. and I.W. Wright, Splines in statistics, J. Amer. Statist. Assoc. 78 (1983) 351-365.