In this chapter, the methods of the research are reviewed as research framework together with hypotheses, procedure, measurement, data analysis as well as demographic variables are presented. At last, reliability and validity are tested in the end of this chapter.
Research Framework
Research framework of this study is illustrated in Figure 3.1 below as it demonstrates the relation between variables in the study. The main purpose of the research is to investigate the differences between RJPs and TJPs in terms of their effects on individuals’ unmet expectation in pre-hired phase of recruitment; in addition, the current research also aims to understand the effect of job preview videos on individuals’ perception of role clarity, which may also indirectly and negatively influence their unmet expectation. As a result, unmet expectation is designed as a dependent variable of job preview. Furthermore, role clarity is assumed to serve as a mediator between the independent variable, job preview, and the dependent variable, unmet expectation.
Figure 3.1. Research framework H1b (RJP > TJP)
H3
H2
H1a (RJP > TJP) Job Preview
Realistic Job Preview Traditional Job Preview
Role Clarity
Unmet Expectation H5
H4
20
Research Hypotheses
Based on literature review, the hypotheses of the research are proposed as following:
Hypothesis 1a: RJPs have a stronger effect on increasing individuals’ perception of unmet expectation than TJPs do during pre-hire phase.
Hypothesis 1b: RJPs have a stronger effect on increasing individuals’ perception of role clarity than TJPs do during pre-hire phase.
Hypothesis 2: Job preview has a positive effect on unmet expectations during pre-hire phase.
Hypothesis 3: Job preview has a positive effect on role clarity during pre-hire phase.
Hypothesis 4: Role clarity has a negative effect on unmet expectation during pre-hire phase.
Hypothesis 5: Role clarity mediates the relationship between job preview and unmet expectation.
Research Procedure
Research procedure illustrates the steps taken from the beginning of the research to the conclusion of the research. The first step taken by the researcher was reviewing literature in order to frame up ideas about the overall study. After the researcher identified the research topic, the research purposes and questions were proposed. By framing up the research topic as well as questions, the researcher then was able to develop the research framework and hypotheses based on the literature. Afterward, research instruments were designed for data collection. To ensure the reliability and validity, instruments were examined through conducting expert review and pilot test. After collecting data, the research had to conduct a series of analyses to answer prosed research questions. In the end, the research findings were reported based on the results of analyses and the literature, and the research could make a
21
conclusion of the study. The research procedure is shown in Figure 3.2.
Figure 3.2. Research procedure
Research Design
The research was conducted through quasi-experimental method using scenarios in the survey questionnaire to collect data for statistical analysis, the scenarios simulated the process of recruiting during the pre-hire phase, and the participants received either realistic job preview video or traditional job preview video. At first, participants filled out pre-test of survey questionnaire, and they subsequently reviewed either RJP video or TJP video. Afterwards, they are again required to fill out the post-test survey questionnaire. Through comparing the differences between pre-test and post-test, researcher will be able to evaluate the change of participants’ attitude caused by the job preview video.
22
Sampling and Data Collection
The sample population of the present study was individuals who were either about to enter job market or currently employed. As the purpose of the study aimed to understand job market in Taiwan, the population of the research was all Taiwanese. The researcher chose a quantitative method through distributing online survey questionnaires to targeted participants and examined the different effects of RJPs and TJPs on their perceptions of role clarity as well as unmet expectations. In addition, to avoid bias among targeted sample, both individuals with and without fulltime job experiences were invited to answer the survey questionnaire.
As the quasi-experimental approach was adopted, the research used snowball sampling and convenient sampling while distributing online questionnaire, and collected data by contacting friends to share the link of online questionnaire with qualified samples. Through social media platform such as Facebook and Line, the total sample collected were 302, and 298 of the total samples were qualified.
During the data collection phase, each participant firstly answered the pre-test of self-report role clarity; later on, they were given a scenario described as following: the participant was looking for a job related to marketing, and he or she saw the vacancy of marketing specialist, which fit his or her major, posted on website by a virtual company, the industry of which also fit his or her interest. In addition, it was a growth company providing employees competitive wage rate and benefit. By reviewing the given scenario, participants filled out the pre-test perception of job attraction to the virtual company. Later on, participants were asked to watch either an RJP video or a TJP video of the marketing specialist in the company, which illustrated the job content and responsibilities in the position. After reviewing the job preview video, participants again filled out the post-test of role clarity and job attraction. Through the comparison between pre- and post-test, the researcher were able to find out the effect of job preview videos; also, the comparison between RJP and TJP was also discussed. The collected
23
descriptive statistics of the sample is shown in Table 3.1.
Table 3.1.
Descriptive Statistics of the Sample
Variable Item Frequency Percentage (%)
Job Preview
24
The instruments of the study consisted of job preview videos, measures of role clarity and unmet expectation. As the purpose of the study is to investigate phenomenon in Taiwan, all instruments are translated in Mandarin Chinese. Each participant reviewed either a realistic job preview (RJP) video or a traditional job preview (TJP) video of a marketing specialist. As for the other two variables, the research adopted the scale developed by Highhouse, Lievens, and Sinar in 2003 for the measure of unmet expectation and the scale developed by Saks and Cronshaw in 1990 for the measure of role clarity.
Job Previews (Realistic Job Preview Video & Traditional Job Preview Video)
RJP and TJP were designed according to the characteristics of job previews, and the researcher made two job preview videos (https://youtu.be/gdTyRBL3LT8;
https://youtu.be/-25
9NoOpQF_Xo) of marketing specialist based on the information provided by job banks as well as interviews with marketing majors. The way an RJP video was different from TJP video was because it presented both positive and negative aspects of the job and it included information job applicants were unlikely to know or were likely to have unrealistic expectations about, and also explained what was done on the job and why it should have been done; in addition, an RJP started job description with positive and exciting aspects of the job but did not hide negative ones. On the contrary, a TJP video of the marketing specialist in the study only presented positive aspects of the position instead of providing a realistic view of the position. For example, the major differences the RJP video had from TJP video were as following: (1) encountering complaints from customers (2) experiencing busy working conditions (3) having a meeting that ran overtime until late evening (4) sacrificing time with family members during business travels.
Except for the four main characteristics above, the two job preview videos both presented positive working environment with friendly colleagues, and both RJP and TJP videos illustrated how employee are going to sharpen their skills, grow and are also provided with good compensation and benefits from the company. The comparison between two job preview videos is shown in Table 3.2.
Table 3.2.
Comparison between RJP Video and TJP Video
26
RJP Video TJP Video
Encountering complaints from customers. Not shown in the video.
Experiencing busy working conditions. Not shown in the video.
Having a meeting that ran overtime until late evening.
Having meeting which brought creativity and teamwork.
Sacrificing time with family members during business travels.
Anticipating business travels.
Measurement
Unmet Expectation
Unmet expectation in this study was measured through comparing the differences between pre-test and post-test of participants’ perception of organizational attractiveness. Because organizational attractiveness is a crucial factor affecting job applicants’ decision on job choice, the researcher aimed to examine the differences between pre-test and post-test of individuals’
perception to it. By comparing different results of organizational attractiveness, one was measured before reviewing job preview video and the other was measured after reviewing job preview video, the discrepancies between two responses illustrated the outcome of unmet expectations.
The measurement for organizational attractiveness was provided to respondents twice, once after reviewing the introduction to the organization in the given scenario and before the job preview video (pre-test), another after viewing the job preview video (pos-test). The measurement of organizational attractiveness was derived from the scale developed by Highhouse et al. (2003). The original measurement used by Highhouse et al. (2003) included
27
three dimensions, including general attractiveness, prestige, and intention to pursue; there were five items in each dimension. Since this research used a simulated company instead of a real one, the dimension of prestige is not applicable, therefore the researcher adopted only two of three dimensions, general attractiveness and intention to pursue, which included ten questions, for the measure of the organizational attractiveness. Questions measuring general attractiveness includes five questions such as “For me, this company would be a good place to work;” while questions measuring individuals’ intention to pursue includes five questions such as “I would accept a job offer from this company.” All items used a 7-point Likert scale from 1 (strongly disagree) to 7 (strongly agree). Cronbach's alpha for this scale in previous study (Fang, 2016) was 0.88 for general attractiveness and 0.82 for intention to pursue.
Role Clarity
Role clarity in the current study was measured through the three-item scale developed by Saks and Cronshaw (1990). Originally, the three questions were designed to measure subjects who worked as hotel desk clerks; as a result, the example of the sample question was like, “I am well aware of the duties required of a hotel desk clerk.” For the present study, the researcher gave participants scenario of seeking for a marketing-related position, so the job in each item was transferred to marketing specialist. For example, the sample item was as following: “I am well aware of the duties required of a marketing specialist.” Cronbach’s alpha for the scale in previous study was 0.89. This study also measured role clarity twice, before (pre-test), and after job preview video (post-test).
Control Variable
Career decision-making self-efficacy.
28
According to previous research, it was believed individuals’ career decision-making self-efficacy (CDMSE) was a significant predictor of individuals’ future planning and clarity (Michael, Cinamon, & Most, 2015). Moreover, it was found that CDMSE affected a person’s vocational identity as well as the degree of engagement in career-related activities (Gushue, Scanlan, Pantzer, & Clarke, 2006). As a result, the current study also conducted pre- and post-test of participants’ perception of CDMSE as a control variable.
The measurement of CDMSE was firstly developed by Taylor and Betz in 1983 and it involved 50 tasks or behaviors required questions. Later on, there was the short form of career decision-making self-efficacy scale (CDMSE-SF) introduced by Betz, Klein, and Taylor in 1996 with 25 items evaluating individuals’ s confidence in making career decisions.
The major purpose of the CDMSE was to control the effect of students’ perception of career-related self-efficacy on the dependent variable. Therefore, the present study adopted the scale of brief decision-making self-efficacy developed by Lent et al. (2016). The brief measure of CDMSE included 8 items enquiring respondents’ confidence in their ability to make career-related behaviors or decisions, and each item was measured using a 7-point Liker scale from 1 (no confidence) to 7 (very confident). Cronbach's alpha for this scale in previous study (Lent et al., 2016) was 0.96.
Reliability and Validity
Exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) were both used to test the validity of the measurement. EFA was used to ensure the uni-dimensionality and construct validity of the measurements. As for CFA, it was performed in AMOS in order to ensure that data collected fit the theoretical measurement model of the study. As question items for variables were self-reported, a Harman’s one-factor test using EFA technique was also
29
conducted to observe common method variance (CMV). In addition, Cronbach Alpha reliability test was performed in order to ensure the internal consistency.
Exploratory Factor Analysis
By conducting SPSS, the exploratory factor analysis was performed to check construct validity of measurement. Five main variables were analyzed, including pre- and post-test of job attraction, pre- and post-test of role clarity, and control variable, career decision-making self-efficacy. The Kaiser-Meyer-Olkin Measure of Sampling Adequacy for EFA of job attraction (pre-test) and role clarity (pre-test) were .935 and .638. The Bartlett’s test of sphericity were both significant. On the other hand, the Kaiser-Meyer-Olkin Measure of Sampling Adequacy for EFA of job attraction (post-test), role clarity (post-test), and career decision-making self-efficacy respectively were .950, .543, and .931. The Bartlett’s test of sphericity for the three variables were all significant. As a result, these outcomes indicated that the data was suitable for the EFA analysis. Factors were extracted by using the criterion of eigenvalue larger than 1 in Principal Component Analysis with Varimax rotation. Table 3.3 and 3.4 show the EFA results of each item by performing SPSS.
Table 3.3.
EFA Result: Rotated Component Matrix for Job Attraction (pre-test) and Role Clarity (pre-test)
30
(n=298)
Component
1 2
JA5_ A job at this company is very appealing to me. .892 JA9_I would exert a great deal of effort to work for this company. .877 JA3_ This company is attractive to me as a place for employment. .874 JA6_ I would accept a job offer from this company. .841 JA4_ I am interested in learning more about this company. .836 JA7_ I would make this company one of my first choice as an employer. .816 JA8_ If this company invited me for a job interview, I would go. .792 JA1_ For me, this company would be a good place to work. .749 JA10_ I would recommend this company to a friend looking for a job. .630 JA2_ I would not be interested in this company except as a last resort. .423
RC2_I am well aware of the duties required of a marketing specialist. .927 RC3_I have a very good idea of what the job of marketing specialist
entails.
.914
RC1_I really do not have a clear idea of what a marketing specialist does.
.749
Note. Extraction Method: Principal Component Analysis. Rotated Method: Varimax.
Table 3.4.
EFA Result: Rotated Component Matrix for Job Attraction (post-test), Career Decision-Making Self-Efficacy, and Role Clarity(post-test) (n=298)
31
Component
1 2 3
JA6_ I would accept a job offer from this company. .920 JA3_ This company is attractive to me as a place for employment. .903 JA5_ A job at this company is very appealing to me. .900 JA7_ I would make this company one of my first choice as an
employer.
.882
JA9_I would exert a great deal of effort to work for this company. .868 JA4_ I am interested in learning more about this company. .852 JA8_ If this company invited me for a job interview, I would go. .777 JA10_ I would recommend this company to a friend looking for a job. .754 JA1_ For me, this company would be a good place to work. .753
JA2_ I would not be interested in this company except as a last resort. .514 CDMSE3_ Pick the best-fitting career option for you from a list of
your ideal careers.
.884
CDMSE5_ Match your skills, values, and interests to relevant occupations.
.881
CDMSE2_ Identify careers that best use your skills. .852 CDMSE4_ Learn more about careers you might enjoy. .834 CDMSE8_ Identify careers that best match your interests. .831 CDMSE6_ Make a well-informed choice about which career path to
pursue.
.829
(continued) Table 3.4. (continued)
CDMSE1_ Figure out which career options could provide a good fit for your personality.
.811
32
CDMSE7_ Learn more about jobs that could offer things that are important to you.
.761
RC2_I am well aware of the duties required of a marketing specialist. .873 RC3_I have a very good idea of what the job of marketing specialist
entails.
.869
RC1_I really do not have a clear idea of what a marketing specialist does.
.523
Note. Extraction Method: Principal Component Analysis. Rotated Method: Varimax.
Confirmatory Factor Analysis
Using AMOS, Confirmatory Factor Analysis (CFA) was performed in order to ensure the construct validity of the measurement. To determine the goodness of model fit, it was needed to check the result of CFA and ensure it meet the criteria for the goodness of fit, such as Chi-Squared test, GFI, AGFI, RMESEA, and SRMR. The criteria for model fit indices are shown in Table 3.5.
Table 3.5.
Summary of Goodness of Fit Criteria
33
>.05 but <.08 Acceptable fit
Diamantopoulos & Siguaw, 2000; Hu & Bentler, 1999
RMSEA
<.05 Close fir
>.05 but <.08 Reasonable fit
>.10 Poor fit
MacCallum, Brown &
Sugawara, 1996
Note. Adapted from “Structural Equation Modelling: Guidelines for Determining Model Fit,” by D. Hooper, J. Coughlan, and M. Mullen, 2008, Electronic Journal of Business Research Method, 6(1), 53-60. Copyright 2008 by The Academic Conference Ltd.
CFA for Research Model
The measurement model of current study was tested through conducting CFA in AMOS in order to ensure the construct validity, and the results of model fit in pre-test were as following:
(1) 2/df was 3.216, which fits the criterion (< 5.0), (2) GFI was .893, which was acceptable (>.80), (3) AGFI was .847, which was acceptable (>.80), (4) RMSEA was .086, which was a reasonable fit, and (5) SRMR was .0432, which was a well fit (<.05). On the other hand, the results of model fit in post-test were as following: (1) 2/df was 2.802, which fits the criterion (< 5.0), (2) GFI was .856, which was acceptable (>.80), (3) AGFI was .821, which was acceptable (>.80), (4) RMSEA was .078, which was a reasonable fit, and (5) SRMR was .0561,
34
which was also an acceptable fit (>.05 but <.08). The summary of model fit in pre-test and post-test was shown in Table 3.6, 3.7., 3.8., and 3.9.
Table 3.6.
Model Fit Summary of Pre-Test
2 df P 2/df GFI AGFI RMSEA SRMR
205.823 64 .000 3.216 .893 .847 .086 .0432
Table 3.7.
Model Fit Summary of Post-Test
2 df P 2/df GFI AGFI RMSEA SRMR
521.156 186 .000 2.802 .856 .821 .078 .0561
Table 3.8.
Summary of Average Variance Extracted (AVE) and Composite Reliability (CR) in Pre-Test
AVE CR
Job Attraction .61 .94
Role Clarity .66 .84
Table 3.9.
35
Summary of Average Variance Extracted (AVE) and Composite Reliability (CR) in Post-Test
AVE CR
Job Attraction .71 .96
Role Clarity .72 .95
CDMSE .67 .84
Note. CDMSE= Career Decision-Making Self-Efficacy.
Figure 3.3. Measurement model of job attraction (pre-test), and role clarity (pre-test)
36
Figure 3.4. Measurement model of job attraction (post-test), role clarity (post-test) and career decision-making self-efficacy
Alpha Coefficient Test
In order to ensure the reliability of each variable, alpha coefficient test was conducted.
Except role clarity (post-test) which has a Chronbach’s alpha of .69 which is very close to the threshold of .7, the results of the analysis show all Cronbach’s alpha’s were larger than .7
37
(Nunnally, 1978), which means that all variables in the present study possessed internal consistency reliability. The results are presented in Table 3.10.
Table 3.10.
Cronbach’s Alpha of Research Variables (n=298)
Variable Cronbach’s alpha
Job Attraction (pre-test) .92
Job Attraction (post-test) .95
Role Clarity (pre-test) .83
Role Clarity (post-test) .69
Career Decision-Making Self-Efficacy .94
Harmon’s Single Factor Test
Since the items for each variable were self-reported in the current study, a Harmon’s single factor test was conducted to examine the common method variance (CMV) problem. The result showed that the largest un-rotated factor explained 42.95% of the variances, which was lower than the suggested ceiling of 50% (Podsakoff, MacKenzie, Lee, & Podsakoff, 2003). Therefore, CMV in the current study may not be a serious concern.
38