3.4 Definitions and Measures of Variables
3.4.6 Workplace Innovation Opportunity (WIO) Measurement
Five survey items that measure workplace innovation opportunity are originally from items identified by Hinton (Hinton, 1968, 1970) and then the construct validity was tested by DiLiello and Houghton (2008). In these items the employee‘s perception of the perceived opportunities to use their creative potential is measured.
The five survey items were developed that describe a variety of opportunities in the workplace to use one‘s expertise, creativity skills and abilities. All items were measured using a five-point Likert (1932) scale ranging from Strongly Agree to Strongly Disagree.
1. I have opportunities to use my design thinking skills and abilities at work 2. I am invited to submit ideas for improvements to the use of design
thinking in the workplace
51
3. I have the opportunity to participate in team work involving design thinking
4. I have the freedom to decide how to integrate design thinking into my work
5. My design thinking abilities are used to my full potential at work 3.4.7 Qualifying Questions.
To obtain ethics review committee approval the survey had to have the first question in the survey to gain the respondents consent to take part in the study and have the results shared. The first question also further ensures the respondent that their answers will stay anonymous. The second qualifier was that the respondent had to have undergone training in design thinking at Singapore Polytechnic. Over 900 staff had been trained at the time of this survey.
The two qualifying questions are as follows:
1. I consent for the data that I ANONYMOUSLY contributes to the results of this survey to be shared with management and for the purposes of publication.
2. I have participated in a Design Thinking or Business Design course and/or workshop at Singapore Polytechnic.
3.5 Control Variables.
Consistent with previous research (George & Zhou, 2007) the variables gender, approximate age, job tenure, education level and field education were controlled. The first control variables to discuss are gender and age. It has been shown in previous research that the role of gender and age has often been overlooked in the investigation of organizational creativity (Binnewies et al., 2008). When age and gender have been examined in previous research it has shown that these factors may account for
52
differences in practiced creativity to potential creativity (Amabile et al., 2005).
Furthermore is has been shown that age can make it more difficult for people to adapt and learn new skills. This can hinder the creative process.
Job tenure was measured in approximate number of years at the organization.
This item was controlled because in previous studies creative potential is describe to consist of domain specific knowledge. This knowledge takes time to build and may come with a longer time in an organization (Oldham & Cummings, 1996). However, the longer an employee has been doing the same job, there is an assumption that the more resistant to change they may become. Therefore, the longer tenure may reduce practiced design thinking. However, Kunze, Boehm, and Bruch (2010) found that sometimes it is younger employees who are more resistant to change. Either way, using age as a control variable will give us more details on the subject.
The educational level was controlled because this may influence a person‘s practiced creativity. Employees with higher education levels are often the ones in positions to innovate and develop strategies. This is not commonly a position that a junior staff would be given to do the high importance and risk to the organization.
The educational field was measured to understand the spread of respondents throughout the organization. It was also measured because the type of expertise or field a person is employed in may affect their ability or desire to be creative. For example, there are ubiquitous thoughts that people in finance would probably be less creative than someone in the arts, or design field. Roger Martin describes traditional business executives to be very different from designers and often less creative (2007a). This is a common discussion on ‗Suits‘ versus ‗Creatives‘. For this study the position or department at the organization was not asked to keep the respondents privacy and ensure no repercussions for honest answers.
53
3.6 Questionnaire Design and Testing.
3.6.1 Questionnaire.
A survey questionnaire includes 51 main questions, 2 qualifying questions and 5 control variables used for this study. The questionnaire has four main parts. The first part is testing the individuals design thinking potential. Second, is to test the moderating effects of workplace context. The third measurement is of the dependent variable, practiced design thinking. The last part includes basic descriptive statistics in order to directly compare differences in individuals, including their age and tenure.
The questions for the survey use self-perception to measure the items. Each respondent will use a 5-point Likert scale to adequately answer each item. The measure will be self-administered over the internet using Survey Monkey.
3.6.2 Data Collection.
A preliminary version of the questionnaire was designed by the author and tested with a small group, mostly in management, at the sample organization. Questionnaire items were revised based upon the feedback of the test sample of employees. The survey was assessed and approved by the internal Ethics Review Committee.
To administer the survey the introduction was carefully constructed to ensure the respondents felt safe being honest in their replies. The introduction is also used to help motivate the staff to take the time to respond to the survey. The person is thanked for taking part in the organizations design thinking journey and prompted to respond to the survey to give their feedback. The hope is that employee will view this survey as a safe way to voice their opinions about their work and the use of design thinking in their organization. The letter is displayed in Appendix 1.
The first notice was sent out to over 400 staff by the Director of the Educational Department at the organization. This department has a goal to implement and support
54
design thinking at SP. Colleagues in management positions at the organization later helped to send the survey out to their networks. The author also sent out two more reminders and request for the staff to complete the survey. The response rate was quite slow; it is predicted this is because of the very busy schedules of staff. It may also be because staffs do not feel comfortable sharing their feedback.
3.7 Methods of Analysis.
After collecting the data several methods of analysis will be employed to test the research framework and hypotheses. This study will use comprehensive statistical analyses including factor analysis, multiple regression analysis, and SEM. These tests will help to explore the relationships between the constructs and to evaluate the overall model. Chapter four will give more details on the methods of statistical analysis.
3.7.1 Descriptive Analysis.
To understand a summary of the sample and data descriptive analysis will be conducted. Descriptive statistical analysis used in this study includes mean, median, standard deviation, range and variance. The results from this data will help to understand the control variables used in the study; age, tenure, sex, education level and education field.
3.7.2 Correlation Coefficient Analysis.
The Pearson correlation coefficient analysis is a preliminary analysis used to explore the relationship between variables on a one-by-one basis. It will be used to obtain a general measure of the strength of linear dependence between two separate variables (Hair, Black, Babin, & Anderson, 2006). Pearson correlation coefficient analysis, or Pearson‘s product-moment correlation, assumes that two variables are measured at least on interval scales. The coefficients measure how variables are
55
proportionally correlated to each other and whether proportional means there is linear relationship. The test will show if there is a high correlation, if it can be estimated by a regression line (straight line). The correlation coefficient also measures the fit of predicted value with actual data.
3.7.3 Reliability Test.
Reliability test is the extent to which a variable or set of variables is consistent in the intended measures. Reliability is an assessment of the degree of consistency between multiple measurements of a variable. If several measurements are taken, the reliable measures will all be consistent in their values. Internal consistency is one of the commonly used measures of reliability for each of the research factors. The individual items of the summated scale should all be measuring the same construct and therefore be inter-correlated. In the research of Hair et al. (2006) they set the average figures for coefficient correlation in term of coefficient alpha or Cronbach‘s alpha value. If the alpha value is higher than 0.7 means this means there is high reliability and lower than 0.3 means that there is low reliability.
3.7.4 Confirmatory Factor Analysis (CFA).
The confirmatory factor analysis test, or CFA, explains how well the theoretical specification of the factors matches reality (the actual data) and provides a confirmatory test of measurement theory. For this study CFA is used to confirm the construct validity of the model and to confirm the factor loadings (correlations between factor and variables). It will also reflect group effects on factors.
Construct validity is measured by factor loading estimates, average variance extracted (AVE – a summary measure of convergence among a set of items representing a latent construct), construct reliability (CR – measure of reliability and internal consistency to the measured variables representing a latent construct). The
56
average figures for the above criteria are standardized loading estimates that should be higher than 0.5, AVE should be 0.5 or higher, and construct reliability should be 0.7 or higher.
3.7.5 Hierarchical Multiple Regression Analysis.
For testing the hypotheses, this research will employ a moderated hierarchical regression analysis. In this study there are several variables that can interact, therefore several regression diagnostics will be used to assess if the modeling assumptions were satisfied. Hierarchical multiple regression analysis will be applied to examine hypothesis derived on the design thinking potential (IEC, DTWS) effect on practiced design thinking (PDT) with the moderating effect of the workplace context (WA, WIO, WIA). This analysis is used to find the interaction effects of the moderator variables or multi-collinearity setback that prevent independent variables to separate its effects in multiple regression analysis.
57
CHAPTER FOUR
RESEARCH RESULTS
4.1 Descriptive Statistic Analysis.
Characteristics of respondents are recorded in descriptive analysis. The results show that about 42 percent of respondents are female. Ages range from 20 – 69 years old and majority of respondents‘ ages is between 30 – 59 years old. Participants having master degree and majoring in engineering dominate the sample, which account for about 64 percent and 42 percent, respectively. Generally, about one-fourth of the sample respondents have been working in the industry for more than 20 years.
Table 4-1
Characteristics of Research Respondent (N=160)
Question Categories Frequency Percentage
Gender Blank 4
Male 91 58.3
Female 65 41.7
Age Blank 4
20-29 years old 3 1.9
30-39 years old 54 34.6
40-49 years old 49 31.4
50-59 years old 47 30.1
60-69 years old 3 1.9
Education Blank 4
Diploma 0 0
Bachelor Degree 38 24.4
Master Degree 100 64.1
Doctorate 16 10.3
Other 2 1.3
58
Question Categories Frequency Percentage
Major Blank 4
Engineering 65 41.7
Maths and Science 20 12.8
Design 10 6.4
Business 21 13.5
Information Technology 40 25.6
Other 0 0 consistency among multiple measurements of a variable. Coefficient alpha (Cronbach‘s α) is used as a valuable measurement of reliability since the individual scale items should be all measured in the same construct to achieve highly inter-correlated results. The threshold of coefficient alpha is generally set at .70, if it is higher this suggests good reliability for confirmatory analysis (Hair et al., 2006).
Internal consistency exists as there is high construct reliability, meaning that the measures all consistently represent the same latent construct. In contrast, low level of the Cronbach‘s Alpha (lower than 0.3) implies that there is low reliability. In addition, the second measurement scale is Kaiser-Meyer-Olkin‘s Measure of Sampling adequacy (KMO). The threshold of KMO to guarantee the high reliability level is 0.5.
Table 4-3 demonstrates the reliability analysis of each construct.
59
As shown in the Table 4-2, all the KMO and coefficient alpha surpass the threshold, showing high construct reliability and inter-correlated items to construct.
Table 4-2 Reliability Test
Construct KMO Cronbach's Alpha
Individual Employee Creativity 0.894 0.872
Design Thinking Working Style 0.893 0.881
Practiced Design Thinking 0.638 0.772
Workplace Atmosphere 0.743 0.738
Workplace Innovative Activity 0.811 0.884
Workplace Innovative Opportunity 0.831 0.818
For this study AMOS 18.0 was used to examine the viability of the research model. Confirmatory Factor Analysis (CFA) was conducted to analyze the validities of each individual construct and the full measurement model. Generally, items above the criterion (0.4) were employed in CFA. In order to achieve the best model fit, some items might need to be deleted. Moreover, modification indices were used to adjust the goodness of fit.
Seven indicators are used to determine the goodness of fit for the overall model and individual constructs. The first criterion is the Chi-Square value. It should be notes that a low value of Chi-Square is considered as goodness of fit to the data. If the Chi-Square value is close to zero this means that there is little difference between expected and observed covariance matrices. Furthermore, the value of Chi-Square fit index divided by degree of freedom should be less than 3. The resulting value of this demonstrates the dependence of the model on the sample size. The lower this value is the better the model is. The Comparative Fit Index (CFI) should be equal or greater 0.9 to achieve acceptable model fit. The sixth index is the Root Mean Square Error of
60
Approximation (RMSEA), which has the range from 0 to 1. To accept the model fit RMSEA should be lower than 0.8. The last criterion, Root Mean Square Residuals (RMR) value should lower than 0.05. The lower RMSEA and RMR have the implication of a better model fit. In short, the rules of thumb for the good model fit are χ2/D.F < 3, CFI > 0.9, RMSEA < 0.08, and RMR < 0.05.
4.2.1 Individual Employee Creativity.
As shown in Table 4-3, seven out of ten items in Individual Employee Creativity exceed the threshold of 0.4. Although items IEC1, IEC2, IEC4 are lower than 0.4, they are still higher than 0.3, which fall in the acceptable (Dillon & Goldstein, 1984;
Molz, 1988a). Thus, those items are kept to analyze the overall goodness of fit of the model. The rules of thumb for the good model fit are χ2/D.F < 3, CFI > 0.9, RMSEA
< 0.08, and RMR < 0.05. The result confirms that this construct has a good model fit since all the criteria are satisfied.
Table 4-3
Confirmatory Factor Analysis Result for Individual Employee Creativity
Constructs Items Factor
I am on the lookout for new ideas from all the people with whom I interact as part of
my job
0.398*** 5.009
IEC3 I believe that I am currently very creative in my
work 0.592*** 7.958
IEC4 My work is so personally rewarding for me that… 0.317*** 3.925 IEC5 I feel that I am good at generating novel ideas. 0.772*** 11.401 IEC6 I have confidence in my ability to solve… 0.825*** 12.606 IEC7 I have a knack for further developing ideas of… 0.645*** 8.881 IEC8 I am good at finding creative ways to solve… 0.859*** A IEC9 I have the talent and skills to do well in my work 0.699*** 9.889 IEC10 I feel comfortable trying new ideas 0.621*** 8.454
61
Table 4-4 showed that DTWS 2, DTWS 7, and DTWS 18 were deleted because of the low factor loadings, which leads to low model fit. In this construct, DTW 5 and DTWS 16 are kept because their factor loadings (DTWS 5: β=0.368, p<0.001;
DTWS 16: β=0.393, p<0.001) exceed 0.3 (Dillon & Goldstein, 1984) and their existence does not affect the goodness of fit indices. The goodness of fit of this construct is satisfied (χ2/D.F = 0.062, RMR = 0.053, CFI =0.958, RMSEA = 0.047).
Table 4-4
Confirmatory Factor Analysis Result for Design Thinking Working Style
Constructs Items Factor DTWS3 I am comfortable with open-ended and
exploratory research 0. 601*** 6.614
DTWS4 I often think of solutions holistically, as
systems 0.620*** 6.852
DTWS5 I often understand how others are feeling and
want to help them when possible 0.368*** 4.247 DTWS6 I analyze problems in depth 0.605*** 6.686 DTWS7 I learn and understand best through
experience Deleted -
DTWS8 I use all my senses and intuition when trying
to understand or learn 0.535*** 5.974 DTWS9 I like to be challenged in my work 0.741*** 9.366 DTWS10 When problem solving I try to put myself in 0.670*** 7.309
62 DTWS13 I don‘t often work in a linear fashion 0.643*** 7.029 DTWS14 I am comfortable visualizing my ideas 0.640*** 7.034 DTWS15 I have a predisposition toward
multi-functionality 0.626*** 6.906
DTWS16 I often work in a playful way 0.393*** 4.516 DTWS17 I am comfortable inferring the cause or
reason for something 0.462*** 5.276
DTWS18 I am practical in my work Deleted - DTWS19 I use sketching to convey my ideas 0.467*** 5.304
Fit Index
Table 4-5 shows that all factor loadings of 4 items in Practiced Design Thinking surpass the threshold of 0.4 (PDT1: β=0.821, p<0.001; PDT2: β=0.805, p<0.001;
PDT3: β=0.587, p<0.001; PDT4: β=0.647, p<0.001). Fit indices (χ2/D.F = 0.276, RMR = 0.006, CFI =1.000, RMSEA = 0.090) show moderate goodness of model fit.
Table 4-5
Confirmatory Factor Analysis Result for Practiced Design Thinking
Constructs Items Factor
PDT1 I have successfully integrated design
thinking… 0.821*** A
PDT2 I feel that design thinking has improved my… 0.805*** 8.084 PDT3 I have used design thinking in my project… 0.587*** 6.616 PDT4 ® I did not find design thinking to be useful… 0.647*** 6.103
Fit Index
63 surpass the 0.4 threshold. WA1 and WA2 were deleted due to low factor loadings to guarantee the goodness of fit in an acceptable range. Fit indices (χ2/D.F = 0.8438, RMR = 0.011, CFI = 1.000, RMSEA = 0.000) prove the good model fit of this construct.
Table 4-6
Confirmatory Factor Analysis Result for Workplace Atmosphere
Constructs Relations Factor WA4 There is free and open communication… 0.768*** A WA5 People are quite concerned about negative… 0.447*** 4.729 WA6 In my organization, there is an atmosphere of… 0.534*** 5.494 WA7 The members of my workgroup feel a strong… 0.439*** 4.5888
Fit Index
64
4.2.5 Workplace Innovative Activity.
Table 4-7 indicates that all items have factor loadings exceeding the threshold of 0.4. Fit indices (χ2/D.F = 1.739, RMR = 0.044, CFI =0.992, RMSEA = 0.068) depicts a moderate goodness of model fit.
Table 4-7
Confirmatory Factor Analysis Result for Workplace Innovative Activity
Constructs Items Factor
WIA1 New ideas are always being tried out in my
organization 0.696*** 10.105
WIA2 In my organization, a lot of ideas are generated 0.697*** 10.127 WIA3 New workplace processes are often
implemented in my organization 0.575*** 7.778 WIA4 Compared to other organizations in Singapore,
my organization is one of the most innovative 0.817*** 12.982 WIA5 My organization can respond quickly to changes
in the external environment 0.905*** A WIA6 My organization regularly introduces new
products/services into the marketplace 0.794*** 12.408 Fit Index
65
Table 4-8
Confirmatory Factor Analysis Result for Workplace Innovative Opportunity
Constructs Items Factor
WIO1 I have opportunities to use my design thinking
skills and abilities at work 0.699*** 7.780 WIO2 I am invited to submit ideas for improvements to
the use of design thinking in the workplace 0.642*** 7.216 WIO3 I have the opportunity to participate in team
work involving design thinking 0.668*** 7.477 WIO4 I have the freedom to decide how to integrate
design thinking into my work 0.687*** 7.669 WIO5 My design thinking abilities are used to my full
potential at work 0.743*** A we ran the CFA for the whole model. Table 4-9 discloses the results of the CFA. In this step, DTWS5 was deleted since its factor loading is lower than 0.4 and its existence affected the overall goodness of fit indices (Dillon & Goldstein, 1984; Molz, 1988b). Among those remaining items, the factor loadings of individual employee creativity IEC8, workplace innovative activity WIA, and practiced design thinking PDT2 are highest: 0.884, 0.862, and 0.836, respectively at p<0.001. Design thinking working style DTWS16 has the lowest factor loading (β = 0.4; p<0.001).
66
As for the goodness of fit, the overall result is moderately acceptable (χ2/D.F=
1.747632; IFI = 0.910; TLI = 0.899; CFI = 0.927, RMR = 0.042, RMSEA = 0.0047) In short, the model is moderately accepted for the fitness to explain the relationship of observed constructs.
Table 4-9
Confirmatory Factor Analysis Result for Full Measurement Model
Confirmatory Factor Analysis Result for Full Measurement Model