An Introduction to Model Building
1.1 An Introduction to Modeling
Operations research (often referred to as management science) is simply a scientific approach to decision making that seeks to best design and operate a system, usually un- der conditions requiring the allocation of scarce resources.
By a system, we mean an organization of interdependent components that work together to accomplish the goal of the system. For example, Ford Motor Company is a system whose goal consists of maximizing the profit that can be earned by producing quality vehicles.
The term operations research was coined during World War II when British military leaders asked scientists and engineers to analyze several military problems such as the de- ployment of radar and the management of convoy, bombing, antisubmarine, and mining operations.
The scientific approach to decision making usually involves the use of one or more mathematical models. A mathematical model is a mathematical representation of an ac- tual situation that may be used to make better decisions or simply to understand the ac- tual situation better. The following example should clarify many of the key terms used to describe mathematical models.
Eli Daisy produces Wozac in huge batches by heating a chemical mixture in a pressur- ized container. Each time a batch is processed, a different amount of Wozac is produced.
The amount produced is the process yield (measured in pounds). Daisy is interested in understanding the factors that influence the yield of the Wozac production process. De- scribe a model-building process for this situation.
Solution Daisy is first interested in determining the factors that influence the yield of the process.
This would be referred to as a descriptive model, because it describes the behavior of the actual yield as a function of various factors. Daisy might determine (using regression methods discussed in Chapter 24) that the following factors influence yield:
■ container volume in liters (V)
■ container pressure in milliliters (P)
■ container temperature in degrees Celsius (T)
■ chemical composition of the processed mixture
If we let A, B, and C be percentage of mixture made up of chemicals A, B, and C, then Daisy might find, for example, that
(1) yield 300 .8V .01P .06T .001T*P .01T2 .001P2
11.7A 9.4B 16.4C 19A*B 11.4A*C 9.6B*C Maximizing Wozac Yield
E X A M P L E 1
To determine this relationship, the yield of the process would have to be measured for many different combinations of the previously listed factors. Knowledge of this equation would enable Daisy to describe the yield of the production process once volume, pres- sure, temperature, and chemical composition were known.
Prescriptive or Optimization Models
Most of the models discussed in this book will be prescriptive or optimization models.
A prescriptive model “prescribes” behavior for an organization that will enable it to best meet its goal(s). The components of a prescriptive model include
■ objective function(s)
■ decision variables
■ constraints
In short, an optimization model seeks to find values of the decision variables that opti- mize (maximize or minimize) an objective function among the set of all values for the decision variables that satisfy the given constraints.
The Objective Function
Naturally, Daisy would like to maximize the yield of the process. In most models, there will be a function we wish to maximize or minimize. This function is called the model’s objective function. Of course, to maximize the process yield we need to find the values of V, P, T, A, B, and C that make (1) as large as possible.
In many situations, an organization may have more than one objective. For example, in assigning students to the two high schools in Bloomington, Indiana, the Monroe County School Board stated that the assignment of students involved the following objectives:
■ Equalize the number of students at the two high schools.
■ Minimize the average distance students travel to school.
■ Have a diverse student body at both high schools.
Multiple objective decision-making problems are discussed in Sections 4.14 and 11.13.
The Decision Variables
The variables whose values are under our control and influence the performance of the system are called decision variables. In our example, V, P, T, A, B, and C are decision variables. Most of this book will be devoted to a discussion of how to determine the value of decision variables that maximize (sometimes minimize) an objective function.
Constraints
In most situations, only certain values of decision variables are possible. For example, cer- tain volume, pressure, and temperature combinations might be unsafe. Also, A B, and C must be nonnegative numbers that add to 1. Restrictions on the values of decision vari- ables are called constraints. Suppose the following:
■ Volume must be between 1 and 5 liters.
■ Pressure must be between 200 and 400 milliliters.
■ Temperature must be between 100 and 200 degrees Celsius.
■ Mixture must be made up entirely of A, B, and C.
■ For the drug to properly perform, only half the mixture at most can be product A.
These constraints can be expressed mathematically by the following constraints:
V 5 V 1
P 400 P 200 T 200 T 100 A 0 B 0 A B C 1 A 5
The Complete Optimization Model
After letting z represent the value of the objective function, our entire optimization model may be written as follows:
Maximize z 300 .8V .01P .06T .001T*P .01T2 .001P2
11.7A 9.4B 16.4C 19A*B 11.4A*C 9.6B*C Subject to (s.t.)
V 5 V 1
P 400 P 200 T 200 T 100 A 0 B 0 C 0 A B C 1 A 5
Any specification of the decision variables that satisfies all of the model’s constraints is said to be in the feasible region. For example, V 2, P 300, T 150, A .4, B .3, and C .1 is in the feasible region. An optimal solution to an optimization model is any point in the feasible region that optimizes (in this case, maximizes) the objective func- tion. Using the LINGO package that comes with this book, it can be determined that the optimal solution to this model is V 5, P 200, T 100, A .294, B 0, C .706, and z 183.38. Thus, a maximum yield of 183.38 pounds can be obtained with a 5-liter
container, pressure of 200 milliliters, temperature of 100 degrees Celsius, and 29% A and 71% C. This means no other feasible combination of decision variables can obtain a yield exceeding 183.38 pounds.
Static and Dynamic Models
A static model is one in which the decision variables do not involve sequences of deci- sions over multiple periods. A dynamic model is a model in which the decision variables do involve sequences of decisions over multiple periods. Basically, in a static model we solve a “one-shot” problem whose solutions prescribe optimal values of decision variables at all points in time. Example 1 is an example of a static model; the optimal solution will tell Daisy how to maximize yield at all points in time.
For an example of a dynamic model, consider a company (call it Sailco) that must de- termine how to minimize the cost of meeting (on time) the demand for sailboats during the next year. Clearly Sailco’s must determine how many sailboats it will produce during each of the next four quarters. Sailco’s decisions involve decisions made over multiple pe- riods, hence a model of Sailco’s problem (see Section 3.10) would be a dynamic model.
Linear and Nonlinear Models
Suppose that whenever decision variables appear in the objective function and in the con- straints of an optimization model, the decision variables are always multiplied by constants and added together. Such a model is a linear model. If an optimization model is not lin- ear, then it is a nonlinear model. In the constraints of Example 1, the decision variables are always multiplied by constants and added together. Thus, Example 1’s constraints pass the test for a linear model. However, in the objective function for Example 1, the terms .001T*P, .01T2, 19A*B, 11.4A*C, and 9.6B*C make the model nonlinear. In general, nonlinear models are much harder to solve than linear models. We will discuss linear models in Chapters 2 through 10. Nonlinear models will be discussed in Chapter 11.
Integer and Noninteger Models
If one or more decision variables must be integer, then we say that an optimization model is an integer model. If all the decision variables are free to assume fractional values, then the optimization model is a noninteger model. Clearly, volume, temperature, pressure, and percentage composition of our inputs may all assume fractional values. Thus, Exam- ple 1 is a noninteger model. If the decision variables in a model represent the number of workers starting work during each shift at a fast-food restaurant, then clearly we have an integer model. Integer models are much harder to solve than nonlinear models. They will be discussed in detail in Chapter 9.
Deterministic and Stochastic Models
Suppose that for any value of the decision variables, the value of the objective function and whether or not the constraints are satisfied is known with certainty. We then have a deterministic model. If this is not the case, then we have a stochastic model. All mod- els in the first 12 chapters will be deterministic models. Stochastic models are covered in Chapters 13, 16, 17, and 19–24.
If we view Example 1 as a deterministic model, then we are making the (unrealistic) assumption that for given values of V, P, T, A, B, and C, the process yield will always be the same. This is highly unlikely. We can view (1) as a representation of the average yield of the process for given values of the decision variables. Then our objective is to find val- ues of the decision variables that maximize the average yield of the process.
We can often gain useful insights into optimal decisions by using a deterministic model in a situation where a stochastic model is more appropriate. Consider Sailco’s problem of minimizing the cost of meeting the demand (on time) for sailboats. The uncertainty about future demand for sailboats implies that for a given production schedule, we do not know whether demand is met on time. This leads us to believe that a stochastic model is needed to model Sailco’s situation. We will see in Section 3.10, however, that we can develop a deterministic model for this situation that yields good decisions for Sailco.
1.2 The Seven-Step Model-Building Process
When operations research is used to solve an organization’s problem, the following seven- step model-building procedure should be followed:
Step 1: Formulate the Problem The operations researcher first defines the organization’s problem. Defining the problem includes specifying the organization’s objectives and the parts of the organization that must be studied before the problem can be solved. In Ex- ample 1, the problem was to determine how to maximize the yield from a batch of Wozac.
Step 2: Observe the System Next, the operations researcher collects data to estimate the value of parameters that affect the organization’s problem. These estimates are used to de- velop (in step 3) and evaluate (in step 4) a mathematical model of the organization’s prob- lem. For example, in Example 1, data would be collected in an attempt to determine how the values of T, P, V, A, B, and C influence process yield.
Step 3: Formulate a Mathematical Model of the Problem In this step, the operations re- searcher develops a mathematical model of the problem. In this book, we will describe many mathematical techniques that can be used to model systems. For Example 1, our optimization model would be the result of step 3.
Step 4: Verify the Model and Use the Model for Prediction The operations researcher now tries to determine if the mathematical model developed in step 3 is an accurate represen- tation of reality. For example, to validate our model, we might check and see if (1) accu- rately represents yield for values of the decision variables that were not used to estimate (1). Even if a model is valid for the current situation, we must be aware of blindly ap- plying it. For example, if the government placed new restrictions on Wozac, then we might have to add new constraints to our model, and the yield of the process [and Equation (1)]
might change.
Step 5: Select a Suitable Alternative Given a model and a set of alternatives, the operations researcher now chooses the alternative that best meets the organization’s objectives.
(There may be more than one!) For instance, our model enabled us to determine that yield was maximized with V 5, P 200, T 100, A .294, B 0, C .706, and z 183.38.
Step 6: Present the Results and Conclusion of the Study to the Organization In this step, the operations researcher presents the model and recommendation from step 5 to the decision- making individual or group. In some situations, one might present several alternatives and let the organization choose the one that best meets its needs. After presenting the results
of the operations research study, the analyst may find that the organization does not ap- prove of the recommendation. This may result from incorrect definition of the organiza- tion’s problems or from failure to involve the decision maker from the start of the project.
In this case, the operations researcher should return to step 1, 2, or 3.
Step 7: Implement and Evaluate Recommendations If the organization has accepted the study, then the analyst aids in implementing the recommendations. The system must be constantly monitored (and updated dynamically as the environment changes) to ensure that the recommendations enable the organization to meet its objectives.
In what follows, we discuss three successful management science applications. We will give a detailed (but nonquantitative) description of each application. We will tie our discus- sion of each application to the seven-step model-building process described in Section 1.2.
1.3 CITGO Petroleum
Klingman et al. (1987) applied a variety of management-science techniques to CITGO Pe- troleum. Their work saved the company an estimated $70 million per year. CITGO is an oil-refining and -marketing company that was purchased by Southland Corporation (the owners of the 7-Eleven stores). We will focus on two aspects of the CITGO team’s work:
1 a mathematical model to optimize operation of CITGO’s refineries, and
2 a mathematical model—supply distribution marketing (SDM) system—that was used to develop an 11-week supply, distribution, and marketing plan for the entire business.
Optimizing Refinery Operations
Step 1 Klingman et al. wanted to minimize the cost of operating CITGO’s refineries.
Step 2 The Lake Charles, Louisiana, refinery was closely observed in an attempt to es- timate key relationships such as:
1 How the cost of producing each of CITGO’s products (motor fuel, no. 2 fuel oil, tur- bine fuel, naptha, and several blended motor fuels) depends on the inputs used to produce each product.
2 The amount of energy needed to produce each product. This required the installation of a new metering system.
3 The yield associated with each input–output combination. For example, if 1 gallon of crude oil would yield .52 gallons of motor fuel, then the yield would equal 52%.
4 To reduce maintenance costs, data were collected on parts inventories and equipment breakdowns. Obtaining accurate data required the installation of a new database-management system and integrated maintenance-information system. A process control system was also installed to accurately monitor the inputs and resources used to manufacture each product.
Step 3 Using linear programming (LP), a model was developed to optimize refinery op- erations. The model determines the cost-minimizing method for mixing or blending to- gether inputs to produce desired outputs. The model contains constraints that ensure that inputs are blended so that each output is of the desired quality. Blending constraints are discussed in Section 3.8. The model ensures that plant capacities are not exceeded and al-
lows for the fact that each refinery may carry an inventory of each end product. Sections 3.10 and 4.12 discuss inventory constraints.
Step 4 To validate the model, inputs and outputs from the Lake Charles refinery were collected for one month. Given the actual inputs used at the refinery during that month, the actual outputs were compared to those predicted by the model. After extensive changes, the model’s predicted outputs were close to the actual outputs.
Step 5 Running the LP yielded a daily strategy for running the refinery. For instance, the model might, say, produce 400,000 gallons of turbine fuel using 300,000 gallons of crude 1 and 200,000 gallons of crude 2.
Steps 6 and 7 Once the database and process control were in place, the model was used to guide day-to-day refinery operations. CITGO estimated that the overall benefits of the refinery system exceeded $50 million annually.
The Supply Distribution Marketing (SDM) System
Step 1 CITGO wanted a mathematical model that could be used to make supply, distri- bution, and marketing decisions such as:
1 Where should crude oil be purchased?
2 Where should products be sold?
3 What price should be charged for products?
4 How much of each product should be held in inventory?
The goal, of course, was to maximize the profitability associated with these decisions.
Step 2 A database that kept track of sales, inventory, trades, and exchanges of all refined products was installed. Also, regression analysis (see Chapter 24) was used to develop forecasts for wholesale prices and wholesale demand for each CITGO product.
Steps 3 and 5 A minimum-cost network flow model (MCNFM) (see Section 7.4) is used to determine an 11-week supply, marketing, and distribution strategy. The model makes all decisions mentioned in step 1. A typical model run that involved 3,000 equations and 15,000 decision variables required only 30 seconds on an IBM 4381.
Step 4 The forecasting modules are continuously evaluated to ensure that they continue to give accurate forecasts.
Steps 6 and 7 Implementing the SDM required several organizational changes. A new vice-president was appointed to coordinate the operation of the SDM and LP refinery model. The product supply and product scheduling departments were combined to im- prove communication and information flow.
1.4 San Francisco Police Department Scheduling
Taylor and Huxley (1989) developed a police patrol scheduling system (PPSS). All San Francisco (SF) police precincts use PPSS to schedule their officers. It is estimated that PPSS saves the SF police more than $5 million annually. Other cities such as Virginia
Beach, Virginia, and Richmond, California, have also adopted PPSS. Following our seven- step model-building procedure, here is a description of PPSS.
Step 1 The SFPD wanted a method to schedule patrol officers in each precinct that would quickly produce (in less than one hour) a schedule and graphically display it. The program should first determine the personnel requirements for each hour of the week. For example, 38 officers might be needed between 1 A.M. and 2 A.M. Sunday but only 14 of- ficers might be needed from 4 A.M. to 5 A.M. Sunday. Officers should then be scheduled to minimize the sum over each hour of the week of the shortages and surpluses relative to the needed number of officers. For example, if 20 officers were assigned to the mid- night to 8 A.M. Sunday shift, we would have a shortage of 38 20 18 officers from 1 to 2 A.M. and a surplus of 20 14 6 officers from 4 to 5 A.M. A secondary criterion was to minimize the maximum shortage because a shortage of 10 officers during a sin- gle hour is far more serious than a shortage of one officer during 10 different hours. The SFPD also wanted a scheduling system that precinct captains could easily fine-tune to produce the optimal schedule.
Step 2 The SFPD had a sophisticated computer-aided dispatch (CAD) system to keep track of all calls for police help, police travel time, police response time, and so on. SFPD had a standard percentage of time that administrators felt each officer should be busy. Us- ing CAD, it is easy to determine the number of workers needed each hour. Suppose, for example, an officer should be busy 80% of the time and CAD indicates that 30.4 hours of work come in from 4 to 5 A.M. Sunday. Then we need 38 officers from 4 to 5 A.M. on Sunday [.8*(38) 30.4 hours].
Step 3 An LP model was formulated (see Section 3.5 for a discussion of scheduling models). As discussed in step 1, the primary objective was to minimize the sum of hourly shortages and surpluses. At first, schedulers assumed that officers worked five consecu- tive days for eight hours a day (this was the policy prior to PPSS) and that there were three shift starting times (say, 6 A.M., 2 P.M., and 10 A.M.). The constraints in the PPSS model reflected the limited number of officers available and the relationship of the num- ber of officers working each hour to the shortages and surpluses for that hour. Then PPSS would produce a schedule that would tell the precinct captain how many officers should start work at each possible shift time. For example, PPSS might say that 20 officers should start work at 6 A.M. Monday (working 6 A.M.–2 P.M. Monday–Friday) and 30 officers should start work at 2 P.M. Saturday (working 2 P.M.–10 P.M. Saturday–Wednesday). The fact that the number of officers assigned to a start time must be an integer made it far more difficult to find an optimal schedule. (Problems in which decision variables must be integers are discussed in Chapter 9.)
Step 4 Before implementing PPSS, the SFPD tested the PPSS schedules against manu- ally created schedules. PPSS produced an approximately 50% reduction in both surpluses and shortages. This convinced the department to implement PPSS.
Step 5 Given the starting times for shifts and the type of work schedule [four consecu- tive days for 10 hours per day (the 4/10 schedule) or five consecutive days for eight hours per day (the 5/8 schedule)], PPSS can produce a schedule that minimizes the sum of short- ages and surpluses. More important, PPSS can be used to experiment with shift times and work rules. Using PPSS, it was found that if only three shift times are allowed, then a 5/8 schedule was superior to a 4/10 schedule. If, however, five shift times were allowed, then a 4/10 schedule was found to be superior. This finding was of critical importance because police officers had wanted to switch to a 4/10 schedule for years. The city had resisted 4/10 schedules because they appeared to reduce productivity. PPSS showed that 4/10 schedules need not reduce productivity. After the introduction of PPSS, the SFPD went
to 4/10 schedules and improved productivity! PPSS also enables the department to exper- iment with a mix of one-officer and two-officer patrol cars.
Steps 6 and 7 It is estimated that PPSS created an extra 170,000 productive hours per year, thereby saving the city of San Francisco $5.2 million per year. Ninety-six percent of all workers preferred PPSS generated schedules to manually generated schedules. PPSS enabled SFPD to make strategic changes (such as adopting the 4/10 schedule), which made officers happier and increased productivity. Response times to calls improved by 20% after PPSS was adopted.
A major reason for the success of PPSS was that the system allowed precinct captains to fine-tune the computer-generated schedule and obtain a new schedule in less than one minute. For example, precinct captains could easily add or delete officers and add or delete shifts and quickly see how these changes modified the master schedule.
1.5 GE Capital
GE Capital provides credit card service to 50 million accounts. The average total out- standing balance exceeds $12 billion. GE Capital, led by Makuch et al. (1989), developed the PAYMENT system to reduce delinquent accounts and the cost of collecting from delinquent accounts.
Step 1 At any one time, GE Capital has more than $1 billion in delinquent accounts.
The company spends $100 million per year processing these accounts. Each day, workers contact more than 200,000 delinquent credit card holders with letters, messages, or live calls. The company’s goal was to reduce delinquent accounts and the cost of processing them. To do this, GE Capital needed to come up with a method of assigning scarce labor resources to delinquent accounts. For example, PAYMENT determines which delinquent accounts receive live phone calls and which delinquent accounts receive no contact.
Step 2 The key to modeling delinquent accounts is the concept of a delinquency move- ment matrix (DMM). The DMM determines how the probability of the payment on a delinquent account during the current month depends on the following factors: size of un- paid balance (either $300 or $300), action taken (no action, live phone call, taped message, letters), and a performance score (high, medium, or low). The higher the per- formance score associated with a delinquent account, the more likely the account is to be collected. Table 1 lists the probabilities for a $250 account that is two months delinquent, has a high performance score, and is contacted with a phone message.
T A B L E 1 Sample Entries in DMM
Event Probability
Account completely paid .30
One month is paid .40
Nothing is paid .30
Because GE Capital has millions of delinquent accounts, there is ample data to accu- rately estimate the DMM. For example, suppose there were 10,000 two-month delinquent accounts with balances under $300 that have a high performance score and are contacted with phone messages. If 3,000 of those accounts were completely paid off during the cur- rent month, then we would estimate the probability of an account being completely paid off during the current month as 3,000/10,000 .30.
Step 3 GE Capital developed a linear optimization model. The objective function for the PAYMENT model was to maximize the expected delinquent accounts collected during the next six months. The decision variables represented the fraction of each type of delinquent account (accounts are classified by payment balance, performance score, and months delinquent) that experienced each type of contact (no action, live phone call, taped mes- sage, or letter). The constraints in the PAYMENT model ensure that available resources are not overused. Constraints also relate the number of each type of delinquent account present in, say, January to the number of delinquent accounts of each type present during the next month (February). This dynamic aspect of the PAYMENT model is crucial to its success. Without this aspect, the model would simply “skim” the accounts that are easi- est to collect each month. This would result in few collections during later months.
Step 4 PAYMENT was piloted on a $62 million portfolio for a single department store.
GE Capital managers came up with their own strategies for allocating resources (collec- tively called CHAMPION). The store’s delinquent accounts were randomly assigned to the CHAMPION and PAYMENT strategies. PAYMENT used more live phone calls and more “no action” than the CHAMPION strategies. PAYMENT also collected $180,000 per month more than any of the CHAMPION strategies, a 5% to 7% improvement. Note that using more of the no-action strategy certainly leads to a long-run increase in cus- tomer goodwill!
Step 5 As described in step 3, for each type of account, PAYMENT tells the credit man- agers the fraction that should receive each type of contact. For example, for three-month delinquent accounts with a small ($300) unpaid balance and high performance score, PAYMENT might prescribe 30% no action, 20% letters, 30% phone messages, and 20%
live phone calls.
Steps 6 and 7 PAYMENT was next applied to the 18 million accounts of the $4.6 billion Montgomery-Ward department store portfolio. Comparing the collection results to the same time period a year earlier, it was found that PAYMENT increased collections by $1.6 million per month (more than $19 million per year). This is actually a conservative esti- mate of the benefit obtained from PAYMENT, because PAYMENT was first applied to the Montgomery-Ward portfolio during the depths of a recession—and a recession makes it much more difficult to collect delinquent accounts.
Overall, GE Capital estimates that PAYMENT increased collections by $37 million per year and used fewer resources than previous strategies.
R E F E R E N C E S
Klingman, D., N. Phillips, D. Steiger, and W. Young, “The Successful Deployment of Management Science Throughout Citgo Corporation,” Interfaces 17 (1987, no. 1):4–25.
Makuch, W., J. Dodge, J. Ecker, D. Granfors, and G. Hahn,
“Managing Consumer Credit Delinquency in the US
Economy: A Multi-Billion Dollar Management Science Application,” Interfaces 22 (1992, no. 1):90–109.
Taylor, P., and S. Huxley, “A Break from Tradition for the San Francisco Police: Patrol Officer Scheduling Using an Optimization-Based Decision Support Tool,” Inter- faces 19 (1989, no. 1):4–24.
Basic Linear Algebra
In this chapter, we study the topics in linear algebra that will be needed in the rest of the book.
We begin by discussing the building blocks of linear algebra: matrices and vectors. Then we use our knowledge of matrices and vectors to develop a systematic procedure (the Gauss–
Jordan method) for solving linear equations, which we then use to invert matrices. We close the chapter with an introduction to determinants.
The material covered in this chapter will be used in our study of linear and nonlinear programming.
2.1 Matrices and Vectors
Matrices
D E F I N I T I O N ■ A matrix is any rectangular array of numbers. ■
For example,
,
,
, [2 1]
are all matrices.
If a matrix A has m rows and n columns, we call A an m n matrix. We refer to m n as the order of the matrix. A typical m n matrix A may be written as
A
D E F I N I T I O N ■ The number in the ith row and jth column of A is called the ijth element of A and is written aij. ■
For example, if
A
then a11 1, a23 6, and a31 7.
3 6 9 2 5 8 1 4 7
a1n
a2n
amn
a12
a22
am2
a11
a21
am1
1
2 3
6 2 5 1 4 2
4 1 3
Sometimes we will use the notation A [aij] to indicate that A is the matrix whose ijth element is aij.
D E F I N I T I O N ■ Two matrices A [aij] and B [bij] are equal if and only if A and B are of the same order and for all i and j, aij bij. ■
For example, if
A
and B
then A B if and only if x 1, y 2, w 3, and z 4.
Vectors
Any matrix with only one column (that is, any m 1 matrix) may be thought of as a column vector. The number of rows in a column vector is the dimension of the column vector. Thus,
may be thought of as a 2 1 matrix or a two-dimensional column vector. Rmwill denote the set of all m-dimensional column vectors.
In analogous fashion, we can think of any vector with only one row (a 1 n matrix as a row vector. The dimension of a row vector is the number of columns in the vector. Thus, [9 2 3] may be viewed as a 1 3 matrix or a three-dimensional row vector. In this book, vectors appear in boldface type: for instance, vector v. An m-dimensional vector (either row or column) in which all elements equal zero is called a zero vector (written 0). Thus,
[0 0] and
are two-dimensional zero vectors.
Any m-dimensional vector corresponds to a directed line segment in the m-dimensional plane. For example, in the two-dimensional plane, the vector
u
corresponds to the line segment joining the point
to the point
The directed line segments corresponding to
u
, v
, w
are drawn in Figure 1.
1
2 1
3 1
2
1 2 0 0
1 2
0 0 1
2
y z x w 2
4 1 3
The Scalar Product of Two Vectors
An important result of multiplying two vectors is the scalar product. To define the scalar prod- uct of two vectors, suppose we have a row vector u = [u1 u2 un] and a column vector
v
of the same dimension. The scalar product of u and v (written u v) is the number u1v1 u2v2 unvn.
For the scalar product of two vectors to be defined, the first vector must be a row vec- tor and the second vector must be a column vector. For example, if
u [1 2 3] and v
then u v 1(2) 2(1) 3(2) 10. By these rules for computing a scalar product, if u
and v [2 3]
then u v is not defined. Also, if
u [1 2 3] and v
then u v is not defined because the vectors are of two different dimensions.
Note that two vectors are perpendicular if and only if their scalar product equals 0.
Thus, the vectors [1 1] and [1 1] are perpendicular.
We note that u v u v cos u, where u is the length of the vector u and u is the angle between the vectors u and v.
3 4 1
2
2 1 2 v1
v2
vn 3
2 x2
x1 1
– 1
– 2 – 2
w
u
v – 1
(–1, –2)
(1, 2) u = 1
2 v =
w =
(1, –3)
1 2
– 3 1
–3 –2 –1
F I G U R E 1 Vectors Are Directed Line Segments
Matrix Operations
We now describe the arithmetic operations on matrices that are used later in this book.
The Scalar Multiple of a Matrix
Given any matrix A and any number c (a number is sometimes referred to as a scalar), the matrix cA is obtained from the matrix A by multiplying each element of A by c. For example,
if A
, then 3A
For c 1, scalar multiplication of the matrix A is sometimes written as A.
Addition of Two Matrices
Let A [aij] and B [bij] be two matrices with the same order (say, m n). Then the matrix C A B is defined to be the m n matrix whose ijth element is aij bij. Thus, to obtain the sum of two matrices A and B, we add the corresponding elements of A and B. For example, if
A
and B
then
A B
.
This rule for matrix addition may be used to add vectors of the same dimension. For ex- ample, if u [1 2] and v [2 1], then u v [1 2 2 1] [3 3]. Vectors may be added geometrically by the parallelogram law (see Figure 2).
We can use scalar multiplication and the addition of matrices to define the concept of a line segment. A glance at Figure 1 should convince you that any point u in the m-dimensional plane corresponds to the m-dimensional vector u formed by joining the origin to the point u. For any two points u and v in the m-dimensional plane, the line segment joining u and v (called the line segment uv) is the set of all points in the m-dimensional plane that correspond to the vectors cu (1 c)v, where 0 c 1 (Figure 3). For example, if u (1, 2) and v (2, 1), then the line segment uv consists
0 0 0 0 0 2 3 3 1 1 2 2
1 1 1 1
0 2
3
1
2 1
1 2 3
1 2
1 1 0
6 0 3
3 2
0 1
1
3
2 x2
x1 1
u u + v
v (1, 2)
(2, 1) (3, 3)
(0, 0) v = 2 1
1 2 3
u = 1 2 u + v = 3 3
F I G U R E 2 Addition of Vectors
of the points corresponding to the vectors c[1 2] (1 c)[2 1] [2 c 1 c], where 0 c 1. For c 0 and c 1, we obtain the endpoints of the line segment uv;
for c12, we obtain the midpoint (0.5u 0.5v) of the line segment uv.
Using the parallelogram law, the line segment uv may also be viewed as the points cor- responding to the vectors u c(v u), where 0 c 1 (Figure 4). Observe that for c 0, we obtain the vector u (corresponding to point u), and for c 1, we obtain the vector v (corresponding to point v).
The Transpose of a Matrix Given any m n matrix
A
the transpose of A (written AT) is the n m matrix
AT
aaa11121n aaa21222n aaam1m2mna1n
a2n
amn
a12
a22
am2
a11
a21
am1 2
x2
u c = 1
c = 0 c =
v
x1 1
1 2
1 2
F I G U R E 3 Line Segment Joining
u (1, 2) and v (2, 1)
x2
u
u
– u v – u v
c = 0
c = 1 c =
v
x1 1
2
F I G U R E 4 Representation of Line Segment uv
Thus, ATis obtained from A by letting row 1 of A be column 1 of AT, letting row 2 of A be column 2 of AT, and so on. For example,
if A
, then AT
Observe that (AT)T A. Let B [1 2]; then
BT
and (BT)T [1 2] B As indicated by these two examples, for any matrix A, (AT)T A.
Matrix Multiplication
Given two matrices A and B, the matrix product of A and B (written AB) is defined if and only if
Number of columns in A number of rows in B (1) For the moment, assume that for some positive integer r, A has r columns and B has r rows. Then for some m and n, A is an m r matrix and B is an r n matrix.
D E F I N I T I O N ■ The matrix product C AB of A and B is the m n matrix C whose ijth element is determined as follows:
ijth element of C scalar product of row i of A column j of B ■ (2) If Equation (1) is satisfied, then each row of A and each column of B will have the same number of elements. Also, if (1) is satisfied, then the scalar product in Equation (2) will be defined. The product matrix C AB will have the same number of rows as A and the same number of columns as B.
Compute C AB for
A
and B
Solution Because A is a 2 3 matrix and B is a 3 2 matrix, AB is defined, and C will be a 2 2 matrix. From Equation (2),
c11 [1 1 2]
1(1) 1(2) 2(1) 5 c12 [1 1 2]
1(1) 1(3) 2(2) 8 c21 [2 1 3]
2(1) 1(2) 3(1) 7
1 2 1 1 3 2 1 2 1
1 3 2 1 2 1 2
3 1 1 1 2 1 2
4 5 6 1 2 3 3
6 2 5 1 4
Matrix Multiplication
E X A M P L E 1
c22 [2 1 3]
2(1) 1(3) 3(2) 11
C AB
Find AB for
A
and B [1 2]
Solution Because A has one column and B has one row, C AB will exist. From Equation (2), we know that C is a 2 2 matrix with
c11 3(1) 3 c21 4(1) 4 c12 3(2) 6 c22 4(2) 8 Thus,
C
Compute D BA for the A and B of Example 2.
Solution In this case, D will be a 1 1 matrix (or a scalar). From Equation (2), d11 [1 2]
1(3) 2(4) 11
Thus, D [11]. In this example, matrix multiplication is equivalent to scalar multiplica- tion of a row and column vector.
Recall that if you multiply two real numbers a and b, then ab ba. This is called the commutative property of multiplication. Examples 2 and 3 show that for matrix multipli- cation, it may be that AB BA. Matrix multiplication is not necessarily commutative. (In some cases, however, AB BA will hold.)
Show that AB is undefined if
A
and B
Solution This follows because A has two columns and B has three rows. Thus, Equation (1) is not satisfied.
1 1 2 1 0 1 2
4 1 3
3 4
6 8 3 4 3
4 8 11 5 7
1 3 2
Row Vector Times Column Vector
E X A M P L E 3
Undefined Matrix Product
E X A M P L E 4
Column Vector Times Row Vector
E X A M P L E 2
Many computations that commonly occur in operations research (and other branches of mathematics) can be concisely expressed by using matrix multiplication.To illustrate this, suppose an oil company manufactures three types of gasoline: premium unleaded, regular unleaded, and regular leaded. These gasolines are produced by mixing two types of crude oil: crude oil 1 and crude oil 2. The number of gallons of crude oil required to manufacture 1 gallon of gasoline is given in Table 1.
From this information, we can find the amount of each type of crude oil needed to manufacture a given amount of gasoline. For example, if the company wants to produce 10 gallons of premium unleaded, 6 gallons of regular unleaded, and 5 gallons of regular leaded, then the company’s crude oil requirements would be
Crude 1 required (34) (10) (23) (6) (14) 5 12.75 gallons Crude 2 required (14) (10) (13) (6) (34) 5 8.25 gallons More generally, we define
pU gallons of premium unleaded produced rU gallons of regular unleaded produced
rL gallons of regular leaded produced c1 gallons of crude 1 required c2 gallons of crude 2 required
Then the relationship between these variables may be expressed by c1 (34) pU (23) rU (14) rL
c2 (14) pU (13) rU (34) rL
Using matrix multiplication, these relationships may be expressed by
Properties of Matrix Multiplication
To close this section, we discuss some important properties of matrix multiplication. In what follows, we assume that all matrix products are defined.
1 Row i of AB (row i of A)B. To illustrate this property, let
A
and B
Then row 2 of the 2 2 matrix AB is equal to
1 3 2 1 2 1 2
3 1 1 1 2
pU
rU
rL
1 4
3 4
2 3
1 3
3 4
1 4
c1
c2
T A B L E 1
Gallons of Crude Oil Required to Produce 1 Gallon of Gasoline
Crude Premium Regular Regular
Oil Unleaded Unleaded Leaded
1 34 23 14
2 14 13 34
[2 1 3]
[7 11]
This answer agrees with Example 1.
2 Column j of AB A(column j of B). Thus, for A and B as given, the first column of AB is
Properties 1 and 2 are helpful when you need to compute only part of the matrix AB.
3 Matrix multiplication is associative. That is, A(BC) (AB)C. To illustrate, let A [1 2], B
, C
Then AB [10 13] and (AB)C 10(2) 13(1) [33].
On the other hand,
BC
so A(BC) 1(7) 2(13) [33]. In this case, A(BC) (AB)C does hold.
4 Matrix multiplication is distributive. That is, A(B C) AB AC and (B C)D BD CD.
Matrix Multiplication with Excel
Using the Excel MMULT function, it is easy to multiply matrices. To illustrate, let’s use Excel to find the matrix product AB that we found in Example 1 (see Figure 5 and file Mmult.xls). We proceed as follows:
Step 1 Enter A and B in D2:F3 and D5:E7, respectively.
Step 2 Select the range (D9:E10) in which the product AB will be computed.
Step 3 In the upper left-hand corner (D9) of the selected range, type the formula
MMULT(D2:F3,D5:E7)
Then hit Control Shift Enter (not just Enter), and the desired matrix product will be computed. Note that MMULT is an array function and not an ordinary spreadsheet func- tion. This explains why we must preselect the range for AB and use Control Shift Enter.
7 13
2 1 3
5 2 4
5 7 1 2 1 2 3 1 1 1 2
1 3 2 1 2 1
1 2 3 4 5 6 7 8 9 10 11
A B C D E F
MatrixMultiplication
1 1 2
A 2 1 3
B 1 1
2 3
1 2
5 8
C 7 11
F I G U R E 5 Mmult.xls
2.2 Matrices and Systems of Linear Equations
Consider a system of linear equations given by
a11x1 a12x2 a1nxn b1
a21x1 a22x2 a2nxn b2
(3)
am1x1 am2x2 amnxn bm
In Equation (3), x1, x2, . . . , xnare referred to as variables, or unknowns, and the aij’s and bi’s are constants. A set of equations such as (3) is called a linear system of m equa- tions in n variables.
D E F I N I T I O N ■ A solution to a linear system of m equations in n unknowns is a set of values for the unknowns that satisfies each of the system’s m equations. ■
To understand linear programming, we need to know a great deal about the properties of solutions to linear equation systems. With this in mind, we will devote much effort to studying such systems.
We denote a possible solution to Equation (3) by an n-dimensional column vector x, in which the ith element of x is the value of xi. The following example illustrates the con- cept of a solution to a linear system.
1 For A
and B
, find:
a A b 3A c A 2B
d AT e BT f AB
g BA
2 Only three brands of beer (beer 1, beer 2, and beer 3) are available for sale in Metropolis. From time to time, people try one or another of these brands. Suppose that at the beginning of each month, people change the beer they are drinking according to the following rules:
30% of the people who prefer beer 1 switch to beer 2.
20% of the people who prefer beer 1 switch to beer 3.
30% of the people who prefer beer 2 switch to beer 3.
30% of the people who prefer beer 3 switch to beer 2.
10% of the people who prefer beer 3 switch to beer 1.
For i 1, 2, 3, let xibe the number who prefer beer i at the beginning of this month and yibe the number who pre- fer beer i at the beginning of next month. Use matrix mul- tiplication to relate the following:
xxx123
y1
y2
y3
2
1 2 1 0 1 3
6 9 2 5 8 1 4 7
P R O B L E M S
Group A Group B
3 Prove that matrix multiplication is associative.
4 Show that for any two matrices A and B, (AB)T BTAT. 5 An n n matrix A is symmetric if A AT.
a Show that for any n n matrix, AATis a symmet- ric matrix.
b Show that for any n n matrix A, (A AT) is a symmetric matrix.
6 Suppose that A and B are both n n matrices. Show that computing the matrix product AB requires n3 multiplications and n3 n2additions.
7 The trace of a matrix is the sum of its diagonal elements.
a For any two matrices A and B, show that trace (A B) trace A trace B.
b For any two matrices A and B for which the products AB and BA are defined, show that trace AB trace BA.
Show that
x
is a solution to the linear system
x1 2x2 5 2x1 x2 0 (4) and that
x
is not a solution to linear system (4).
Solution To show that
x
is a solution to Equation (4), we substitute x1 1 and x2 2 in both equations and check that they are satisfied: 1 2(2) 5 and 2(1) 2 0.
The vector
x
is not a solution to (4), because x1 3 and x2 1 fail to satisfy 2x1 x2 0.
Using matrices can greatly simplify the statement and solution of a system of linear equations. To show how matrices can be used to compactly represent Equation (3), let
A
, x
, b
Then (3) may be written as
Ax b (5)
Observe that both sides of Equation (5) will be m 1 matrices (or m 1 column vec- tors). For the matrix Ax to equal the matrix b (or for the vector Ax to equal the vector b), their corresponding elements must be equal. The first element of Ax is the scalar product of row 1 of A with x. This may be written as
[a11 a12 a1n]
a11x1 a12x2 a1nxn
This must equal the first element of b (which is b1). Thus, (5) implies that a11x1 a12x2 a1nxn b1. This is the first equation of (3). Similarly, (5) implies that the scalar
x1
x2
xn
b1
b2
bm
x1
x2
xn
a1n
a2n
amn
a12
a22
am2
a11
a21
am1
3 1 1 2 3 1 1 2 Solution to Linear System
E X A M P L E 5
product of row i of A with x must equal bi, and this is just the ith equation of (3). Our dis- cussion shows that (3) and (5) are two different ways of writing the same linear system. We call (5) the matrix representation of (3). For example, the matrix representation of (4) is
Sometimes we abbreviate (5) by writing
Ab (6)
If A is an m n matrix, it is assumed that the variables in (6) are x1, x2, . . . , xn. Then (6) is still another representation of (3). For instance, the matrix
represents the system of equations
x1 2x2 3x3 2 x2 2x3 3 x1 x2 x3 1
P R O B L E M
Group A
2 3 1 3 2 1 2 1 1 1 0 1
5 0 x1
x2
2
1 1 2
2.3 The Gauss–Jordan Method for Solving Systems of Linear Equations
We develop in this section an efficient method (the Gauss–Jordan method) for solving a system of linear equations. Using the Gauss–Jordan method, we show that any system of linear equations must satisfy one of the following three cases:
Case 1 The system has no solution.
Case 2 The system has a unique solution.
Case 3 The system has an infinite number of solutions.
The Gauss–Jordan method is also important because many of the manipulations used in this method are used when solving linear programming problems by the simplex algo- rithm (see Chapter 4).
Elementary Row Operations
Before studying the Gauss–Jordan method, we need to define the concept of an elemen- tary row operation (ERO). An ERO transforms a given matrix A into a new matrix A via one of the following operations.
1 Use matrices to represent the following system of equations in two different ways:
x1 x2 4 2x1 x2 6 x1 3x2 8
Type 1 ERO
A is obtained by multiplying any row of A by a nonzero scalar. For example, if
A
then a Type 1 ERO that multiplies row 2 of A by 3 would yield
A
Type 2 ERO
Begin by multiplying any row of A (say, row i) by a nonzero scalar c. For some j i, let row j of A
rows of A.
For example, we might multiply row 2 of A by 4 and replace row 3 of A by 4(row 2 of A) row 3 of A. Then row 3 of A becomes
4 [1 3 5 6] [0 1 2 3] [4 13 22 27]
and
A
Type 3 ERO
Interchange any two rows of A. For instance, if we interchange rows 1 and 3 of A, we obtain
A
Type 1 and Type 2 EROs formalize the operations used to solve a linear equation sys- tem. To solve the system of equations
x1 x2 2 2x1 4x2 7 (7)
we might proceed as follows. First replace the second equation in (7) by 2(first equa- tion in (7)) second equation in (7). This yields the following linear system:
x1 x2 2
(7.1) 2x2 3
Then multiply the second equation in (7.1) by 1
2, yielding the system x1 x2 2
(7.2) x232
Finally, replace the first equation in (7.2) by 1[second equation in (7.2)] first equa- tion in (7.2). This yields the system
3 6 4 2 5 3 1 3 2 0 1 1
4 6 27 3 5 22 2 3 13 1 1 4
4 18 3 3 15 2 2 9 1 1 3 0
4 6 3 3 5 2 2 3 1 1 1 0
