Source: www.usnews.com
U.S. News ranks business colleges in United States in 2004 and listed 82 of them.
They have used three major sections with total of eight criteria for the entire ranking process. These criteria are listed below with their weightings and descriptions.
(1) Quality Assessment (total 40%):
I. Peer Assessment (25%) – Deans and directors from business schools of accredited programs were asked to rate programs from marginal (1) to outstanding (5). Notice that 56% of them have returned the survey.
II. Recruiter Assessment (15%) – Corporate recruiters were also asked to rank the programs which they have hired employee from in the previous year. However, only 32% of them replied the survey.
(2) Placement Success (total 35%):
II. Percentage of Graduates Employed at Graduation (7%) – The percentage of emplacement rate is measure before the students actually graduate from full-time MBA program.
III. Percentage of Graduates Employed 3 Months after Grad (14%) – The percentage of employed graduates three months after
completing the full-time MBA program.
(3) Student Selectivity (total 25%):
I. Average Undergrad GPA (7.5%) – The average GPA of new students.
II. Average GMAT (16.25%) – Average GMAT score of new students who are accepted to the full-time MBA program.
III. Acceptance Rate (1.25%) – Percentage of accepted applications.
From their hard data, we have tried to duplicate their ranking formula and have found a very similar ranking result with identical overall scores. The formula
should be very close to
∑
and minimum values of kth criteria.Financial Times
Source: www.ft.com
Unlike U.S. News & World Report, Financial Times (FT) has ranked business schools from all over the world and has listed 100 of them. FT has also selected twenty criteria for the ranking process. The following are those criteria and their weightings.
(1) Weighted Salary (20%) – This is the average salary today with adjustment for different industries. Also, this figure is the average salary three years after graduation. (in US dollars)
(2) Salary Percentage Increase (20%) – The percentage increase in salary from beginning of MBA program to three years after graduation.
(3) Value for Money (3%) – This is calculated by the salary earned by MBA graduates three years after graduation with the course costs and the opportunity cost, while still in school and not employed.
(4) Career Progress (3%) – The degree to which alumni have moved up the career ladder three years after graduating. Progression is
measured through changes in level of seniority and the size of company in which they are employed.
(5) Aims Achieved (3%) – The extent ot which alumni fulfilled their goals or reasons for doing an MBA. This is measured as a percentage of total returns for a school and presented as a rank.
(6) Placement Success (2%) – The percentage of 2000 alumni that gained employment with the help of career advice. The data is presented as rank.
(7) Alumni Recommendation (2%) – Alumni of 2000 were asked to name three business schools from which they would recruit MBA
graduates. The figure represents the number of votes received by each school. The data is presented as a rank.
(8) International Mobility (6%) – A rating system that measures the degree of international mobility based on the employment movements of alumni between graduation and today.
(9) Employed at Three Months (2%) – the percentage of the most recent graduating class that had gained employment within three months.
(11) Women Students (2%) – Percentage of female students.
(12) Women Board (1%) – Percentage of female members in the advisory board.
(13) International faculty (4%) – The percentage of international students.
(14) International Students (4%) – Percentage of the board whose nationality differs from their country of employment.
(15) International board (2%) – Percentage of the board whose nationality differs from their country of employment.
(16) International Experience (2%) – Weighted average of three criteria that measure international exposure during the course.
(17) Languages (2%) – Number of additional languages required on completion of the MBA. Where a proportion of students required another language due to an additional diploma or degree chosen that figure is included in the calculations but not presented in the final table.
(18) Faculty with Doctorates (5%) – Percentage of faculty with a doctoral degree.
(19) FT Doctoral Rating (5%) – Number of doctoral graduates from the last three academic years with additional weighting for those graduates taking up a faculty position at one of the top 50 school in this year’s ranking.
(20) FT Research Rating (10%) – a rating of faculty publications in 40 international academic and practitioner journals. Points are accrued by the business school at which the author is presently employed.
Adjustment is made for faculty size.
The results and hard data of both U.S. News and World Report and Financial
Times are attached in the Appendix section. Both publishers have worked with other companies for data collection. However, they did not explain how the weightings for the criteria were decided. Moreover, perhaps because U.S. News and World Report is the most recognized publisher in university ranking, it receives many criticisms on both the changes on weightings from year to year and the correctness of hard data. On the contrary, Financial Times has fixed their weightings. However the way hard data is presented has been modified from year to year. For example, the criterion “value for money” was a score ranging from 1 to 5 in year 2002 and 2003 ranking. In 2004, this criterion has been changed into “value for money rank”. When it was a score from 1 to 5, there can be only 50 different scores and is unlikely that all the variation of the score will be assigned. Hence there are many schools with the same scores. When it changed to rank, only few schools are being ranked as the same, so the variation is larger. This problem arises on more than one criterion in Financial Times’ ranking.
2.2 Data Envelopment Analysis
Data Envelopment Analysis (DEA) is a method for evaluating the activity performance, especially for organizations such as business firms, government
agencies, hospitals, educational institutions, and etc (Cooper etc. 1999). A commonly used measure for efficiency is the output-input ratio. Number of items sold in a store will be an example of the output; number of sales clerk in the store will be the input.
Hence, the efficiency of this store, basing on only these two criteria, will simply be NumberOfGoodsSold / NumberOfClerk. These comparable entities are often called
The purpose of DEA is to empirically estimate the efficient frontier based on the set of available DMUs and assumes that each performance measure can be
categorized as either an input or an output (Schrage, 1997). It provides the user information about both efficient and inefficient units along with the efficiency scores and reference sets for inefficient units (Halme etc, 1999). An Efficient Frontier is a line that has at least one DMU point touching it. The DMUs, who touch the EF line, are the most efficient DMUs. The idea of Production Frontier is first discussed by Farrell in 1975 which has three assumptions. The attractive feature of DEA is that it produces efficiency score between 0 and 1.
In 1978, Charnes, Cooper, and Rhodes proposed a DEA model called the CCR model basing on Farrell’s single input-output model in 1975. CCR model is designed to measure the cases of multi input and multi output. The following is the
pseudo-code for the CCR model. Ur represents the weighting for rth output criterion and Vi represents the weighting for ith input criterion. They are automatically generated when the score of kth DMU is maximized. Yr and Xi are the output and
Where
Yr is the rth output of DMU Xi is the ith input of DMU Ur is the weighting for rth output Vi is the weighting for ith input
In this CCR model, it will calculate the score of each DMU based on the weightings that can maximize the score of current DMU, which means that the nth DMU can obtain the best score with nth set of weightings. Hence, if there are n numbers of DMUs, then there will have n set of weightings. kth set of weighting is determined under the condition that they can maximize the Scorek. All the scores have to be between 0 and 1. Once score of each DMU is determined, it then compares all of them again with their score. The DMU with highest score is the most efficient one.
2.3 Analytic Hierarchy Process
The Analytical Hierarchy Process (AHP) was proposed by Saaty in 1980 and his collaborators as a method for establishing priorities in multi-criteria decision making contexts based on variables that do not have exact numerical consequences (Genest, 1996). It also helps people set priorities and make the best decision when both
qualitative and quantitative aspects of a decision need to be considered. AHP not only helps decision makers arrive at the best decision, but also provides a clear rationale that it is the best.
AHP can be conducted in three steps:
Setp 1: Perform pairwise comparisons between each DMU on every criterion
In this step, the goal is to obtain the priorities between DMUs for each criterion. To do so, a pairwise comparison has to take place between each DMU with respect to each criterion. For each criterion, a m by m matrix, where m is the number of DMUs, will be generated and the priority vector will be calculated from this matrix. Priority vector displays the preference orders for each DMU with respect to criteria. Since there are n numbers of criteria, n number of priority vector will be generated at the end.
Step 2: Perform pairwise comparison between each criterion
In the decision making process, not every criterion is quantitatively measurable, so a pairwise comparison between each criterion has to take place in order to specify the importance between each criterion. From the comparison, a set of weightings can be found for score calculation at the last step.
Step 3: Compute final scores for DMUs
With the priority vectors and the weightings for criteria, DM can now calculate the score for each DMU. DMU with the higher score should be the better alternative for the Decision Maker.
Following is an illustration of an example of a student, John, wanting to purchase a car. Due to his financial limitation, John can only buy a second hand car, and only
has few things that he really car. He wants to buy a car that is cheap, nice out look, and comfortable. However, among the three cars he has in mind, none of them has best score on each of these criteria. He has decided to use AHP to help him select a car from these three. Table 2.1 lists all the data he gathered about these three cars.
Table 2.1 Hard data provided by John on cars.
Price Look Comfort
Car 1 13100 Good Very good
Car 2 12000 Fair Good
Car 3 9800 Good Fair
To perform pairwise comparison between each car with respect to each criterion, a priority score has to be assigned to each comparison. The scores can range from 1 to 9, where 9 is the most satisfactory score. Notice that if a DM compare A1 to A2 and assigns a score of 4, then the score between comparison of A2 and A1 will be the inverse of A1 and A2’s, which will be 1/4. This property can ensure the logical consistency for each comparison.
1 Choice i and j are equally important 3 Choice i is weakly more important than j 5 Choice i is strongly more important than j 7 Choice i is very strongly more important than j 9 Choice i is absolutely more important than j 2, 4, 6, 8 are intermediate values
After finishing pairwise comparisons, matrixes with these priority scores will be generated (Table 2.2).
Table 2.2 Comparison score for each car with respect to each criterion
Criteria Price Look Comfort
Car1 Car2 Car3 Car1 Car2 Car3 Car1 Car2 Car3
Car1 1 1/3 1/8 1 3 1 1 3 6
Car2 3 1 1/6 1/3 1 1/4 1/3 1 4
Car3 8 6 1 1 4 1 1/6 1/4 1
From these matrixes, normalization has to be done before the priority vectors can be calculated (Table 2.3). Normalization is simply divides each value by the sum of corresponding column. For example, the normalized value between car2 and car3 with respect to price is calculated by
(1/6) / (1/8 + 1/6+ 1) = 0.1290.
Table 2.3 Normalized comparison table
Criteria Price Look Comfort
Car1 Car2 Car3 Car1 Car2 Car3 Car1 Car2 Car3 Car1 0.0833 0.0454 0.0967 0.4286 0.375 0.4444 0.6666 0.7059 0.5454 Car2 0.250 0.1363 0.1290 0.1428 0.125 0.1111 0.2222 0.2352 0.3636 Car3 0.6666 0.8182 0.7742 0.4286 0.5 0.4444 0.1111 0.0588 0.0909
Each criterion has its own priority vector and the values in the vector can be seen as the score of each DMU on corresponding criterion. The values in the priority vectors are the sum of rows from the normalized pairwise comparison matrix and divided by the number of DMUs, as in Table 2.4. The values in priority vector for price is calculated as follow:
(0.0833 + 0.0454 + 0.0976) / 3 = 0.2254 (0.2500 + 0.1363 + 0.1290) / 3 = 0.5153 (0.6666 + 0.8182 + 0.7742) / 3 = 2.2590
Table 2.4 Priority vectors with respect to each criterion
Priority Vector for Price Priority Vector for Look Priority Vector for Comfort
Car1 0.0751 0.4160 0.6393
Car2 0.1717 0.1263 0.2736
Car3 0.7530 0.4576 0.0869
After the values of priority vector is calculated, pairwise comparison has to perform on criteria to obtain the weightings for each criterion. Similar to previous steps, a 3 by 3 matrix, with criteria on both row and column, will be created. Using the same calculation method for priority vector, the weighting for each criterion can also be found (Table 2.5).
Table 2.5 Comparison tables and weightings for criteria
Comparison Matrix Normalized Comparison Matrix
Price Look Comfort Price Look Comfort Weighting
Price 1 1/5 3 0.1579 0.1489 0.2727 0.1931
Look 5 1 7 0.7894 0.7447 0.6363 0.7234
Comfort 1/3 1/7 1 0.0526 0.1064 0.0909 0.0833
The weightings on Table 2.5 suggest that Look is the most important criterion for John. Price is the next concern and comfort is the last. With the weightings on the criteria and the priority vectors on each criterion, the score for each car can now be calculated as follow:
Car 1: (0.0751 * 0.1931)+(0.4160 * 0.7234)+(0.6393 * 0.0833) = 0.3687 Car 2: (0.1717 * 0.1931)+(0.1263 * 0.7234)+(0.2736 * 0.0833) = 0.1473 Car 3: (0.7530 * 0.1931)+(0.4576 * 0.7234)+(0.0896 * 0.0833) = 0.4839
From the calculation, Car 3 has the highest score and should be the best choice for John to consider.
2.4 Intransitivity
When Decision Makers are making decisions, some do a pairwise comparison with AHP before they make the actual decision. However, AHP does not have a means for detecting an intransitivity situation. An intransitivity is when A > B, B > C, but C
> A. This situation is also called logically inconsistent. When there is a cycle exists in the decision process and is not very logical. Hence, the intransitivity detection is a very important process before the any decision is made.
In Gass’ study (1998), he presented a way to detect the intransitivity with simple matrix operation.
Theorem:
Let P be the preference matrix of a preference diagram D. Then in Pk, the (i,j) entry, denoted by Pi,j(k) , is the number of sequences in D of length k from node vi to node vj. (Pk is the kth power of P)
The theorem states that Pi,jk
denotes the number of cycles, with different sequence. Take a preference graph shown in Figure 2.1 as an example. We can generate a tournament matrix from this preference graph. The preference matrix P, Table 2.11, has values of 0 or 1. Pi,j is set to 1 if i is smaller than j.
1
2
3
4 5
6
Figure 2.1 Preference Graph of six nodes
Table 2.6 Preference matrix on six nodes P1 P2 P3 P4 P5 P6
P1 0 0 0 1 0 1
P2 1 0 0 0 0 0
P3 1 1 0 0 1 1
P4 0 1 1 0 0 1
P5 1 1 0 1 0 0
P6 0 1 0 0 1 0
From this preference matrix, we can apply the theorem to this matrix and look for the cycles. Since the theorem said that the value of Pijk
means there are the same numbers of combinations of sequences in the preference graph of length k from node i to node j. Similarly, if we look at Piik
, then this will mean the sequence start at node i and come back to node i with the length of k. Hence, we can simply check the diagonal of each Pk for k = 3 up to k = n, where n is the number of nodes.
Table 2.7a to Table 2.7d are the power of preference matrix from P3 to P6. In Table 2.7a, we can see that the diagonal has nonzero values. P113
is 4, so there are four cycles with the length of 3 and the starting and ending node is P1. The cycles are (P1, P2, P4, P1), (P1, P3, P4, P1), (P1, P2, P6, P1), and (P1, P5, P6, P1). With the same
technique, it is very easy to find the existence of cycles for any given preference graph. From Table 2.7b to Table 2.7d, it is clear that there are cycles with the length of 4, 5, and 6.
Table 2.7 Preference Matrixes
Clustering involves dividing a set of data points into non-overlapping into groups, where points in each group are more similar to each other than to points in other groups (Faber, 1994). When a set of data is clustered, every point is assigned to a group and every group can be characterized by a single reference point, normally the average of points in the same group.
There are several techniques in the field of clustering. General clustering
techniques are Hierarchical clustering, K-Mean clustering, Incremental clustering, and Probability-based clustering. K-mean clustering is also called Iterative Distance-based clustering. The character “k” in the name of K-mean is the number of groups, or clusters, DM wants to make. The basic idea for K-mean is randomly start with k number of points and assign each data point to one of the reference point in k by calculating the minimal total distance. Once the groups are determined, it then tries to adjust the position of the reference points so that it will locate in the center of
corresponding group. The algorithm for the k-mean clustering is shown below.
Algorithm for K-mean Clustering:
(1) Choose k centroid points.
(2) Calculate the distance of each point to all centroids.
(3) Get the minimum distance. This data is said belong to the cluster that has minimum distance from this data
(4) Adjust the centroid location based on the current data updated data.
(5) Assign all the data to this new centroid.
(6) Repeat until no data is moving to another cluster anymore.
In this study, the proposed model will be able to generate a set of weightings for criteria based on the preferences given by the decision makers. The model has applied similar idea from Data Envelopment Analysis. In DEA, it is trying to measure the efficiency based on maximizing the score of DMU. However, in the proposed model, it will try to maximize the rank for each DMU instead of score. The concept from Analytic Hierarchy Process is also used to create tournament matrix for ranking by doing pairwise comparison. Gass’ technique is also used to ensure the non-existence of intransitivity. Last but not least, the concept from K-mean clustering will be modified to help this ranking method to present the data points on a 3D ball to help DM make decisions.
3. Ranking and Grouping Models
In this chapter, the ranking and grouping process can be break down into two major parts. First part will deal with the actual ranking and score calculation. The second part is mapping each school onto a 3D ball and clustering these data points.
Figure 3.1 shows the entire process of proposed ranking and grouping model.
Figure 3.1 Flowchart
3.1 Common Weight Model
As discussed in chapter 2, DEA is mainly used for efficiency measurement. The concept of DEA is to calculate the ratio between inputs and outputs, and rank each DMU (Data Making Unit) by their maximized scores. In this ranking objective, however, DEA is not the perfect tool for the ranking process because the most efficient DMU might not be the best choice for DM (Decision Maker). Moreover, , sometimes criteria are hard to distinguish from input or output, the proposed method has modified the traditional DEA method to meet the DMs’ requirement without the need to identify inputs and outputs for criteria. This model will automatically ranks and groups the DMUs based on the absolute dominance relationships found in the hard data, so the DMs do not need to worry about assigning weightings for each criterion. This is a big improvement from the traditional ranking systems, which often have controversy on weighting settings.
In the experiments, Lingo8.0 is used as the optimization tool. Given the correct model and inputs, the system will calculate the ideal weights for each criterion, which will allow us to rank the DMUs and map each DMU to a coordinate on 3D ball to help DM visualize the relationships between DMUs, as well as the correlation between DMUs. In this section, the mathematical model and the concept behind it
In the experiments, Lingo8.0 is used as the optimization tool. Given the correct model and inputs, the system will calculate the ideal weights for each criterion, which will allow us to rank the DMUs and map each DMU to a coordinate on 3D ball to help DM visualize the relationships between DMUs, as well as the correlation between DMUs. In this section, the mathematical model and the concept behind it