The BSC index is obtained through the following steps:
Step 1. Planning
The goal of the e-learning system to the investigated organization is to enhance the teaching effectiveness and increase the general reputations to the society. The four perspectives are Reaction, Learning, Behavior and Value. The perspective weights are individually determined by the decision maker in order to represent their importance, for example, PWC=0.25. In order to achieve the e-learning project’s goal, the objectives of each perspective are determined, as for example, C1: “Enjoyment of current e-learning environment”. The questionnaire evaluated this objective as “Do you feel more satisfied about our e-learning environment compared to last month? [□Strongly Disagree □Disagree □Neutral □Agree □Strongly Agree]”. According to the previous investigations, it is difficult to successively obtain “satisfied” (questionnaire score larger than 3.0) every month. This study objectively setup the success point of C1 is “at least 3 cumulated satisfied score must be achieved”.
After determining the objectives, this study objectively assigned objective weights to each objective for illustration, for example, WC1=0.09. Note that the objective weights should be determined according to the e-learning project’s mission and goals.
In order to keep a neutral perspective, the anticipated growth rate rΓ are set to zero. The initial value and K value of each objective are set to 10.
Step 2. Obtain objectives scores
With the questionnaire collected at 3/4/2006, this study set Strongly Disagree = 0, Disagree = 1, Neutral = 2, Agree = 3 and Strongly Agree = 4 to score each question.
The questionnaires were then issued to randomly selected students and staff members.
All questionnaires were summarized with Microsoft Excel to determine the mean value of each objective. For example, the objective score of C1 is the mean value of question C1 collected from the questionnaires. As a result, the questionnaire score of C1 equals to 2.26 in the investigation performed at 3/4/2006. Based on the success point “at least 3 cumulated satisfied score must be achieved”, the objective score of C1 can be thus calculated as illustrated in Example 3. The objective score of C1 is 1.94.
Step 3. Calculate current measures
The current measures were calculated in this step. For example, the previous measure of C1 collected at 1/2/2006 and 3/3/2006 is 10.08 and 10.12, the current measure of C1 is SC1= 10.12(0.95+1.94*0.025) = 10.11. The volatility of C1 can thus be calculated as σC1’=Std(10.08, 10.12, 10.11)= 0.0208. Because we collect the
questionnaires every 30 days (t = 30), thus, 0.0727
Step 4. Apply B & S formula
Because the decision maker want to explore if the performance can exceed the organization’s goals at 1/1/2007, the time to maturity can be calculated as T = (1/1/2007 – 3/4/2006)/365 = 0.75 years. The objective index of C1 can be calculated by the B & S formula with parameters (S, K, σ, T, r) = (SC1, KC1, σC1, T, rC1)= (10.11, 10.0, 0.0727, 0.75, 0). Applying the B & S formula, (4.5) d1 = 0.1977 and (4.6) d2 = 0.1347, thus GC1= 0.3086 *0.09= 0.0278 by (4.4).
Step 5. Calculate the BSC index value
The BSC sheet can be completed by applying (4.11) with objective measures and its objective weight as shown in Table 5. For example, the perspective index of Reaction is (GC1 + GC2 + … + GC9) = (0.0278 + 0.0838 + … + 0.0467) = 0.5511. The perspective fail index of Reaction is (ZC1 + ZC2 + … + ZC9) = (0.0183 + 0.0822 + .. + 0.0487)= 0.5711 and the perspective success value equals ln(0.5511/0.5711) = -0.0356.
Table 8 The completed BSC analyzing sheet.
Objective Objective Expected Volatility Previous Current Target Objective Objective Fail Success Score Growth Rate Measure Measure Value Weight Index Index Value
(X) (rΓ) (σΓ) (PΓ) (SΓ) (KΓ) (WΓ) (GΓ) (ZΓ) (VΓ)
Reaction (Customer, ex. students, trainee) Perspective weight: 0.25
Perspective Index: 0.5511, Perspective Fail Index: 0.5711, Perspective Success Value:-0.0356
C1:Increase the enjoyment-of-use of e-learning environment 1.95 0 0.0727 10.12 10.11 10.00 0.09 0.0278 0.0183 0.4169 C2:Provide friendly user interface, style and functionality 1.61 0 0.1716 10.11 10.01 10.00 0.14 0.0838 0.0822 0.0187
C3:Enhance organizational collaboration capability 1.78 0 0.1270 10.15 10.10 10.00 0.08 0.0394 0.0314 0.2259
C4:Improve the communication between trainees and trainers 1.75 0 0.1239 10.11 10.05 10.00 0.13 0.0587 0.0529 0.1050
C5:Increase user satisfaction 1.61 0 0.2229 10.01 9.91 10.00 0.12 0.0866 0.0973 -0.1169
C6:Increase the flexibility of learning time arrangement 1.58 0 0.1857 10.04 9.94 10.00 0.12 0.0730 0.0804 -0.0961 C7:Provide abundant linkage to related learning materials 1.61 0 0.2002 9.99 9.89 10.00 0.09 0.0571 0.0668 -0.1566
C8:Provide sufficient example and case studies 1.58 0 0.2075 9.98 9.87 10.00 0.12 0.0781 0.0931 -0.1758
C9:Provide vivid illustrations of rich multimedia materials 1.71 0 0.1256 10.05 9.98 10.00 0.11 0.0467 0.0487 -0.0426 Learning and Growth (Learning and Growth)
Perspective weight: 0.27
Perspective Index: 0.5226, Perspective Fail Index: 0.4962, Perspective Success Value:0.0519
L1:Continuously improve knowledge by e-learning systems 1.92 0 0.1042 10.08 10.06 10.00 0.025 0.0098 0.0083 0.1658
L2:Ensure consistent support by organization for e-learning project 2.15 0 0.2944 10.13 10.17 10.00 0.1 0.1107 0.0942 0.1615 L3:Enhance the innovation and seniority capability of all organization members 1.92 0 0.0556 10.03 10.01 10.00 0.1 0.0199 0.0185 0.0721
L4:Improve IT capability of all members 2.25 0 0.2979 10.15 10.21 10.00 0.05 0.0573 0.0468 0.2021
L5:Improve professional skills of all members 1.88 0 0.1363 10.11 10.08 10.00 0.025 0.0128 0.0108 0.1672
L6:Enhance personalization learning capabilities 1.82 0 0.0973 10.09 10.04 10.00 0.025 0.0090 0.0079 0.1296
L7:Ensure logical sequence among learning materials 1.68 0 0.1412 10.06 9.98 10.00 0.15 0.0718 0.0744 -0.0364
L8:Ensure learners realize the learning subjects 1.95 0 0.0724 10.08 10.07 10.00 0.15 0.0430 0.0328 0.2722
L9:Increase knowledge comprehension 1.95 0 0.0388 10.06 10.05 10.00 0.005 0.0008 0.0006 0.3719
L10:Increase e-learning system usage frequency 1.75 0 0.1118 10.02 9.95 10.00 0.1 0.0362 0.0409 -0.1210
L11:Increase e-learning system operational familiarities 1.75 0 0.1121 10.04 9.98 10.00 0.27 0.1018 0.1070 -0.0495
L12:Create satisfied learning results 1.68 0 0.1500 10.03 9.95 10.00 0.1 0.0495 0.0540 -0.0874
Behaviour (Internal Business Processes) Perspective weight: 0.24
Perspective Index: 0.9568, Perspective Fail Index: 0.7951, Perspective Success Value:0.1851
I1:Provide differentiated e-learning environment 1.97 0 0.3437 10.29 10.28 10.00 0.075 0.1010 0.0797 0.2366
I2:Provide incentive systems for users 1.76 0 0.2733 10.22 10.16 10.00 0.2 0.2061 0.1745 0.1662
I3:Generate healthy organization cultures 2.24 0 0.3886 10.24 10.30 10.00 0.15 0.2264 0.1819 0.2189
I4:Motivate effective learning activities 1.43 0 0.2708 10.22 10.07 10.00 0.05 0.0487 0.0451 0.0770
I5:Improve the role playing capabilities during learning activities 1.76 0 0.1232 10.11 10.05 10.00 0.2 0.0901 0.0806 0.1112
I6:Encourage knowledge sharing between members 2.24 0 0.1527 10.11 10.17 10.00 0.1 0.0619 0.0453 0.3129
I7:Generate better learning behavior change 1.70 0 0.1361 10.14 10.07 10.00 0.1 0.0506 0.0439 0.1421 I8:Provide efficient and effective learning environment 2.17 0 0.3626 10.18 10.22 10.00 0.125 0.1720 0.1441 0.1771 Value (Finance, Results)
Perspective weight: 0.24
Perspective Index: 0.9308, Perspective Fail Index: 0.8208, Perspective Success Value:0.1259
F1:Increase organization value 2.71 0 0.6595 10.39 10.57 10.00 0.05 0.1304 0.1019 0.2463
F2:Share knowledge to other organizations 2.10 0 0.3082 10.31 10.34 10.00 0.125 0.1571 0.1149 0.3125
F3:Provide high quality knowledge and information platform 2.37 0 0.2767 10.20 10.29 10.00 0.025 0.0280 0.0207 0.3036
F4:Raise organization reputations 2.17 0 0.1860 10.18 10.22 10.00 0.1 0.0767 0.0543 0.3447
F5:Improve organization's competitiveness 1.70 0 0.1828 10.18 10.10 10.00 0.2 0.1375 0.1167 0.1640
F6:Increase knowledge absorption 1.43 0 0.2846 10.14 10.00 10.00 0.1 0.0980 0.0981 -0.0015
F7:Increase the number of research achievements 1.70 0 0.3139 9.98 9.90 10.00 0.1 0.1027 0.1126 -0.0914
F8:Increase the number of online users 1.29 0 0.3156 10.11 9.93 10.00 0.1 0.1048 0.1119 -0.0660
F9:Reduce learning costs 1.70 0 0.1339 10.11 10.03 10.00 0.2 0.0957 0.0896 0.0658
Applying (4.8), the BSC index value is 0.5511 * 0.25 + 0.5226 * 0.27 + 0.9568 * 0.24 + 0.9308 * 0.24 = 0.7319. Similarly, the BSC fail index is 0.5711 * 0.25 + 0.4962
* 0.27 + 0.7951 * 0.24 + 0.8208 * 0.24 = 0.6645 and the BSC success index is ln(0.7319/0.6645) = 0.0966. These values indicate that with an initial value of 10, the expected value that the overall e-learning performance exceeds its target value at 1/1/2007 is 0.7319, the expected fail value is 0.6645 and the entire project is nearly neutral to success (slightly success but possibly fail), if everything remains unchanged.
According to Table 5, the weakest perspective is Learning & Growth, which obtained the lowest perspective index value (0.5226) while the most dangerous perspective is Reaction because its success value is negative. If the manager wants to keep the e-learning performance balanced, he must inject more efforts to increase the customer (students and trainees) satisfaction.
The E-learning Project Performance Summary Report
Global View
Current Status: 0.0966.
Neutral, but possible to fail
Perspective View Reaction: -0.0356
Current status: Slightly under performance, possible to fail.
Alerting objectives: C5, C7, C8 Notice objectives: C6, C9 Learning & Growth: 0.0519
Current status: Neutral, but possible to fail Alerting objectives: L10
Notice objectives: L7, L11, L12 Behavior: 0.1851
Current status: Optimistically above neutral.
Alerting objectives: None Notice objectives: None Value: 0.1259
Current status: Above neutral Alerting objectives: None Notice objectives: F6, F7, F8
Overall performance contributions:
Reaction
Top 10 performers:
Objective Objective Fail Success
Index Index Value
C1:Increase the enjoyment-of-use of e-learning environment 0.0278 0.0183 0.4169 C3:Enhance organizational collaboration capability 0.0394 0.0314 0.2259 L8:Ensure learners realize the learning subjects 0.0430 0.0328 0.2722
L9:Increase knowledge comprehension 0.0008 0.0006 0.3719
I1:Provide differentiated e-learning environment 0.1010 0.0797 0.2366 I6:Encourage knowledge sharing between members 0.0619 0.0453 0.3129
F1:Increase organization value 0.1304 0.1019 0.2463
F2:Share knowledge to other organizations 0.1571 0.1149 0.3125
F3:Provide high quality knowledge and information platform 0.0280 0.0207 0.3036
F4:Raise organization reputations 0.0767 0.0543 0.3447
Last 10 performers:
Objective Objective Fail Success
Index Index Value
C8:Provide sufficient example and case studies 0.0781 0.0931 -0.1758 C7:Provide abundant linkage to related learning materials 0.0571 0.0668 -0.1566 L10:Increase e-learning system usage frequency 0.0362 0.0409 -0.1210
C5:Increase user satisfaction 0.0866 0.0973 -0.1169
C6:Increase the flexibility of learning time arrangement 0.0730 0.0804 -0.0961 F7:Increase the number of research achievements 0.1027 0.1126 -0.0914
L12:Create satisfied learning results 0.0495 0.0540 -0.0874
F8:Increase the number of online users 0.1048 0.1119 -0.0660
L11:Increase e-learning system operational familiarities 0.1018 0.1070 -0.0495 C9:Provide vivid illustrations of rich multimedia materials 0.0467 0.0487 -0.0426