In this chapter, the research framework, research design, measurement of the study, and research sample are described. The measurement includes, task technology fit of e-learning, learning outcomes, training transfer of S Company, and control variable.
Research Framework
According to the literature review, this study suggests that there are positive relationship between employees’ perception of task technology fit of the e-learning system and their learning effectiveness including learning outcomes and training transfer. The research framework is shown in Figure 3.1.
Figure 3.1. Research framework Task technology fit of
e-learning system
Learning outcome
Training transfer Learning
style
H1 H2
H3 H4
44
Research Hypotheses
As discussed in chapter two, the research hypotheses tested in this study are stated as following:
H1: Perception of task-technology fit of an e-learning course has a positive influence on employees’ learning outcome.
H2: Perception of task-technology fit of an e-learning course has a positive influence on employees’ training transfer.
H3: Employees’ learning outcome mediates the relationship between task-technology fit of an e-learning course and training transfer.
H4: Employees’ learning style will moderate the relationship between task-technology fit of an e-learning course and learning outcome.
Research Design
Sampling and Data Collection
Quantitative method is implemented in this study by collecting data from 151 new real estate agents of S Company. The data of task technology fit were collected from questionnaires of employees’ self-report. The learning outcomes and training transfer scores were provided by S Company. Before formally administering the questionnaires a pre-test and exploratory factor analysis (EFA) were conducted to ensure validity of the measures.
Sample Description
In order to obtain multiple sources of information to minimize the effect of common method variance (CMV), the researchers worked with one of the largest real estate agencies in Taiwan, which was nicknamed the S Company for anonymity, to collect data needed. The rationales for choosing this company as research target are: first, the target is a multinational
company in real estate industry, where competition is crucial. Second, the target company ranks top one in the real estate industry in Taiwan and is known for its well-organized and systematic training; therefore, the result can be used as benchmark for other realty companies.
Third, the company is very advanced in developing their official website and internet applications for customers to access transaction information; thus, it has a higher level of acceptance and experience in the use of e-learning for real estate agents.
As for the uniqueness of their tasks on job, real estate agents are the best choice of being the research sample. It is because they are not employees that do their job at office desks;
instead, they need to go out and spend most of their working hours with customers. As a result, real estate agents are suitable sample applying task technology fit model for this research.
The research sample is the new-hired real estate agents in S Company, who were recruited from June to August, 2011. Out of 151 questionnaires distributed, 151 responded, which is a rate of 100%. The sample description is shown in Table 3.1 The majority of the new agents has a bachelor degree (81%), majored in business, management, literature, or social science (53%), and came to S Company with no previous sales experience (67%).
Table 3.1. Sample Description Sample Description
Sample
Characteristics Frequency Percentage
Gender Female 49 32.7
Male 102 67.3
Age
21-25 84 55.6
26-30 53 35.1
31-35 13 8.6
Above 35 1 0.7
(table continues)
46
Table 3.1. (continued) Sample
Characteristics Frequency Percentage
Education Background
Senior /Vocational high school 1 0.7
Bachelor 140 92.7
Master 10 6.6
Doctor 0 0
Subject Majored
Literature/Social Science 42 27.8
Science 33 21.9
Measurement
Task Technology Fit of E-learning System
Goodhue’s (1998) development of Task Technology Fit measurement was adopted in this research. The scale has 29 items in 13 dimensions. After an expert review and pilot test with 25 new-hired employees in S Company, the questionnaire items were modified. Twelve of the 13 dimensions of TTF measurement were adopted for this study with word changes to reflect the e-learning context, including “the right data”, “level of detail”, “accuracy”,
“compatibility”, “locatability”, “accessibility”, “meaning”, “assistance”, “ease of use”,
“systems reliability”, “currency”, and “presentation”. Table 3.2 lists a sample item for each of the dimensions.
Table 3.2. Sample Item for Each Dimension of Task Technology Fit Sample Item for Each Dimension of Task Technology Fit
Dimension Item
The Right Data The information maintained by the e-learning courses is exactly what I need to carry out my tasks.
Level of Detail The e-learning courses maintain information at an appropriate level of detail for my purposes.
Accuracy The information that I use or would like to use is accurate enough for my purposes.
Compatibility The information is with consistency when it is cited from two different sources.
Locatability It is easy to locate the e-learning courses on corporate portal.
Accessibility I can get in the e-learning courses quickly and easily when I need it.
Flexibility
It is easy to change the selection and format of information made available by our e-learning systems. (personal settings: font styles or the amount of charts or text)
(table continues)
48
Table 3.2. (continued)
Dimensions Item
Meaning The instructions of all operational functions and commands on the e-learning system are clear.
Assistance It is easy to get assistance when I am having trouble finding or using information.
Ease of Use It is easy to learn how to use the e-learning system that gives me access to information.
Systems Reliability I can count on the system to be “up” and available when I need it.
Currency The information is up-to-date enough that reflects the current market.
Presentation The information is presented in a readable and useful format.
Items of each dimension are in the form of statements and responses are designed to be on a 5-Point Likert-type scale, ranging from 1 = strongly disagree to 5 = strongly agree.
Therefore, the higher the scores the employee rated, the higher the degree of TTF they perceive.
The task technology fit questionnaire was developed by Goodhue (1998) with 15 dimensions and total 44 specific questions. The researcher will modify some of the items and each question will take the form of a declarative statement suggesting when needs were being met or not being met, with respondents agreeing or disagreeing on a 5-point scale. Each dimension had a cronbach’s alpha of more than .70 in the Goodhue (1998) study as shown in Table 3.3.
Table 3.3. Reliabilities of Dimensions of Task Technology Fit Model Reliabilities of Dimensions of Task Technology Fit Model
Dimensions of Task Technology Fit Reliabilities
Number of Questions Cronbach’s Alpha
Lack of Confusion 2 .73
Level of Detail 3 .85
Locatability 3 .77
Meaning 3 .78
Right Data 4 .83
Accessibility 3 .84
Assistance 3 .87
Ease of Use 3 .77
Flexibility 4 .70
System Reliability 3 .77
Accuracy 3 .83
Compatibility 4 .82
Currency 3 .73
Presentation 2 .86
Source: Adopted from “Development and measurement validity of a task-technology fit instrument for user evaluations of information systems,” by Goodhue, D. L., 1998, Decision Sciences, 29(1), p.116.
The researchers translated the questionnaire items into Chinese and conducted a pilot test with 22 employees of S Company. The pilot test showed that reverse-coded items were not responded well by pilot participants. These items were rephrased in the opposite direction to get rid of negation which seemed to cause misinterpretation of the reverse-coded questions (Swain, Weathers, & Niedrich, 2008). Before distributing the questionnaires to the study sample, an expert review was performed again to ensure clarity and face validity of all questions.
After data were collected, the researchers ran an exploratory factor analysis (EFA) on the TTF items to explore the factor structure of the modified scale. Three factors emerged which represented the “content”, “technology”, and “presentation” aspects of an e-learning
50
system respectively. Items of the original TTF dimensions: “level of detail”, “accuracy”, and
“compatibility” are now loaded onto a higher level “content” construct. Items of dimensions:
“locatability”, “accessibility”, “meaning”, “assistance”, “ease of use”, “systems reliability”, and “currency” are now loaded onto a higher level “technology” construct. Items of the dimension “presentation” remain as a separate construct. Two items were deleted because of cross loading on two factors. Hence, the final TTF scale of this research comprises 8 questionnaire items in the “content” factor with cronbach’s alph at .930, 12 items in the
“technology” factor with α= .951, and 3 items in the “presentation” factor with α= .890. Table 3.4 shows the results of exploratory factor analysis, while Table 3.3 summarizes the reliability of the TTF measures.
Table 3.4. Summary of Exploratory Factor Analysis Results for Task Technology Fit Summary of Exploratory Factor Analysis Results for Task Technology Fit
Factor Loadings
Item Content Technology Presentation
1 The e-learning courses provide critical information that would be very useful to me in my job.
.699 .150 .457
2 The information maintained by the e-learning courses is exactly what I need to carry out my tasks.
.739 .154 .439
3 The e-learning courses provide useful information and increase my job efficiency.
.752 .096 .423
4 The e-learning courses maintain information at an appropriate level of detail for my purposes.
.667 .261 .249
5 The information that I use or would like to use is accurate enough for my purposes.
.716 .367 .240
6 There is no accuracy problem in the information I use or need.
.707 .410 .186
7 The e-learning courses maintain consistent information throughout.
.748 .432 .089
8 The e-learning courses maintain information that is consistent with other sources of information.
.655 .540 .039
(table continues)
Table 3.4. (continued)
Factor Loadings
Item Content Technology Presentation
9 It is easy to locate the e-learning courses on corporate portal.
.169 .589 .265
10 I can find specific topics in the e-learning courses quickly and easily when I need it.
.391 .534 .486
11 It is easy to change the selection and format of
information made available by our e-learning systems.
(personal settings: font styles or the amount of charts or text)
.410 .656 .156
12 The instructions of all operational functions and commands on the e-learning system are clear.
.295 .734 .040
13 The instructions on how to operate the e-learning system is easy to find.
.302 .680 .314
14 I can get the help I need in accessing and understanding the information.
.203 .773 .271
15 It is easy to get assistance when I am having trouble finding or using information.
.229 .845 .154
16 It is easy to learn how to use the e-learning system. .171 .783 .299 17 The e-learning system is convenient and easy to use. .176 .777 .345
18 The e-learning system including software and hardware works stably.
.369 .750 .232
19 I can count on the system to be “up” and available when I need it.
.316 .663 .401
20 I can get information that is current enough to meet my needs.
.212 .696 .280
21 The information is up-to-date enough that reflects the current market.
.380 .304 .700
22 The information in the e-learning courses is organized in a way that is easy to understand.
.318 .388 .720
23 The information is presented in a readable and useful format.
.287 .376 .718
52
Table 3.5 shows the reliability of the three dimensions and total task technology fit score.
The cronbach’s alpha values are higher than .890, which shows a high reliability in this measurement.
Table 3.5. Reliability Reliability
Cronbach’s Alpha M SD
Content Factor .930 4.125 0.591
Technology Factor .951 3.952 0.667
Presentation Factor .890 3.916 0.741
Total TTF Score .965 3.997 0.602
Note. N=151
The researchers calculated a simple aggregation of the three factors to come up with a total TTF score which is defined as a total TTF value. The TTF value has a cronbach’s alpha of .965 which indicates a very high internal consistency among items.
Learning Outcome
The learning outcomes are in the form of scores generated from participants’ online tests right after finishing the e-learning courses. Contents of the S Company’s e-learning courses encompass three different aspects, which are knowledge of agency, basic skills, and attitude (KSA). For example, “agency process”, “real estate contract introduction”, “house condition inspection”, “guarantee of contract” and “deed introduction” are courses that relate to agency knowledge. “Paperwork introduction”, “blueprint and floor plan mapping” and “standard operating procedure” relate to agency skills. “Etiquette and customer service” and “time and project management” help agents to construct positive image of S Company. The test is composed of multiple choices questions for learners. The objective is simply to test agents’
recall of knowledge acquired from the e-learning courses. The score ranges from 0 to 100.
Training Transfer
The training transfer scores are provided by learners’ supervisors and assigned senior agents after observing the learner’s learning application on job. Before the 6-month trial period ends, new agents are rated by managers and mentors in their assigned agency store every month. The training transfer scores derived from supervisory reports, agents’ case reports and some training tests, each is in certain proportion. The rating ranges from 0 to 100 by evaluating new agent’s overall performance during client service processes. That is, whether the new agent can demonstrate good attitude toward customers, sufficient agency knowledge such as house condition introduction, and smooth agency process will be rated by sales managers. To be noted, there are still smaller proportions of the scores that take agents’
training performances into accounts.
Learning Style
The index of learning styles (ILS) is self-report dichotomous response option survey instrument with 44 items (11 items for each of 4 dimensions) that classifies students along four dimensions: sensing-intuitive, visual-verbal, active-reflective, and sequential-global (Solomon & Felder, 2002). A response to an item is scored for one of the two poles of the relevant bipolar scale according to a set pattern (i.e., response “a” is scored for Polar A, and response “b” is scored for Polar B). Scores on dimension can range from 0 to 11. Scores on a scale of 10 and 11 reflect a “strong preference” for the relevant pole, scores of 8 and 9 indicate a “moderate preference,” and cores of 6 and 7 reflect a “mild” or “balanced preference” for both poles of dimension.
As this study aims at the presentation characteristic of an e-learning course, the scale of visual-verbal preference is adopted. Table 3.6 lists sample items of visual-verbal preference of learning style scale.
54
Table 3.6. Sample Items for Learning Style: Visual-verbal Learners Sample Items for Learning Style: Visual-verbal Learners
1 When I think about what I did yesterday, I am most likely to get (a) a picture.
(b) words.
2 I prefer to get new information in
(a) pictures, diagrams, graphs, or maps.
(b) written directions or verbal information.
3 In a book with lots of pictures and charts, I am likely to
7 When I see a diagram or sketch in class, I am most likely to remember (a) the picture.
(b) what the instructor said about it.
8 When someone is showing me data, I prefer (a) charts or graphs.
(b) text summarizing the results.
9 When I meet people at a party, I am more likely to remember (a) what they looked like.
(b) what theysaid about themselves.
10 For entertainment, I would rather (a) watch television.
(b) read a book.
11 I tend to picture places I have been (a) easily and fairly accurately.
(b) with difficulty and without much detail.
In addition, according to the result of the reliability and validity test of ISL from previous literature, active-reflective scale has α values ranged from .60 to .76, visual-verbal from .64 to .71, sensing-intuitive from .63 to .71, and sequential-global from .47 to .62.
(Bacon, 2004; Genovese, 2004; Hosford & Sider, 2010). Except the sequential-global scale which has unacceptable α value, other scales have stable internal reliability.
Since this study focuses on the alignment between characteristics of an e-learning course and the tendency of sensory channel through which learners receive information, the researcher chose the visual-verbal scale considering the fitness between characteristics of ILS and TTF scales.
Control Variable
After running the correlation of data as well as achieving the purpose of separating individual differences from the main effect, demographic factors such as age, gender, job experience, and e-learning experience were considered as control variables in this study.
56