Identifying and assessing critical risk factors for BIM projects: Empirical study

(1)

Identifying and assessing critical risk factors for BIM projects:

Empirical study

Kuo-Feng Chien

a

, Zong-Han Wu

b

, Shyh-Chang Huang

a,

⁎

a

Department of Civil Engineering in National Chiao Tung University, Hsinchu, Taiwan b_{Department of Civil Engineering in National Central University, Taoyuan, Taiwan}

a b s t r a c t

a r t i c l e i n f o

Article history:

Received 24 September 2013 Received in revised form 13 April 2014 Accepted 19 April 2014

Available online 18 May 2014 Keywords:

Building information modelling (BIM) Decision-making trial and evaluation laboratory (DEMATEL)

Risk assessment Critical risk factors (CRFs)

Building information modelling (BIM) technology exhibits strong potential to become the core technology used in the construction industry. However, the process of implementing new technology involves numerous challenges, and the performance of new technology can be impaired when unidentified risk factors are pres-ent during implempres-entation. A complete understanding of the risk factors can enable BIM users to execute early responses to the potential risks, thus increasing the possibility that BIM is implemented successfully. In this study, 13 risk factors related to the technical, management, personnel,financial, and legal aspects of BIM adoption were identified. Based on the results of a questionnaire survey distributed to architects, engineering consultants, academics, and construction companies in the architecture, engineering, and construction industry in Taiwan, relationships between risk factors were identified using the decision-making trial and evaluation laboratory method. This study identified the critical risk factors of BIM projects at various levels and proposes relative risk-response strategies for a case study project.

1. Introduction

1.1. Background of building information modelling implementation

Building information modelling (BIM) is an emerging technology in which digital information models are employed in a virtual space to achieve high-quality and efﬁcient construction and management throughout the life cycle of a facility. Several general construction projects worldwide, such as the EMP Museum at Seattle Center, Walt Disney Concert Hall, Shanghai World Expo Cultural Center, Shanghai World Expo China Pavilion, Washington National Park, the Bird's Nest and Water Cube constructed for the Beijing Olympics, and Shanghai Tower, have been successfully completed by implementing BIM tech-nology. The digitized and parameterized characteristics of BIM enable project designers to fully analyze the inﬂuences of the environment and energy, and the parametric design facilitates the production of highly accurate results and instant feedback to changing variables when construction personnel encounter complex geometric designs. By implementing a three-dimensional (3-D) visual BIM model platform, project construction teams can assess clash detection in advance and adopt four-dimensional (4-D) information to facilitate construction management when facing tight deadlines.

Using BIM tremendously beneﬁts construction projects in various aspects. By analyzing 32 major projects, the Stanford University

Center for Integrated Facility Engineering revealed that using BIM yields numerous bene_{ﬁts, including an up to 40% elimination of} un-budgeted change, cost-estimation accuracy within 3%, an up to 80% reduction in cost estimate generation time, savings of up to 10% of the contract value through clash detection, and an up to 7% reduction in project time[13]. Suermann surveyed 105 people in the Facility Information Council National BIM Standards Committee at the Na-tional Institute of Building Sciences to understand the effects of BIM on construction and obtained the following responses: 76% of the respondents indicated that BIM can facilitate reducing unit la-bour hours, 70% stated that BIM can facilitate reducing unit costs, 84% reported that BIM facilitates reducing project costs, 90% stated that BIM facilitates the timely completion of projects, and 94% reported that BIM facilitates ensuring that the quality of project designs is high[42]. In four detailed case studies, Kaner et al. ob-served apparent improvements in engineering design quality, in-cluding error-free drawings and steadily increasing improvement in labour productivity, when BIM was applied[28]. Barlish and Sulli-van also indicated that there is a high potential for BIM beneﬁts to be realized in RFIs (requests for information), change orders, schedule and cost savings[6]. Giel and Issa researched case studies that indi-cated that the return of investment of BIM varied greatly from 16% to 1654%. In addition, the total number of requests for information in a small tilt-wall project, a three-story assisted living facility pro-ject, and a midrise commercial condominium project decreased by 34%, 68%, and 43%, and the number of change orders decreased by 40%, 48%, and 37%, respectively[22].

⁎ Corresponding author. Tel.: +886 3 5731717. E-mail address:[email protected](S.-C. Huang).

Contents lists available atScienceDirect

Automation in Construction

(2)

1.2. Challenges of building information modelling risk assessment Zavadskas et al. (2010) mentioned that the size and complexity of construction projects are increasing, thus increasing risks[50]. BIM pro-jects are often the largest investments or the most prominent propro-jects undertaken in the construction industry. However, although most con-struction organisations are highly experienced in managing traditional construction projects, BIM projects may involve new challenges and risk factors that must be managed differently from those of traditional construction projects. Thus, risk management is more complex and cru-cial in BIM construction projects than in conventional construction projects.

Insufﬁcient risk management knowledge and techniques are the pri-mary barriers to risk management[43]. Several public BIM publications have emphasized the potential risks of BIM implementation. For exam-ple, the BIM Handbook (2011) described the barriers associated with work process changes and technological risks in implementing BIM

[16]; the architecture, engineering, and construction (AEC) (UK) BIM Protocol (2012) provided guidance for BIM interoperability problems

[1]; the BIM Planning Guide for Facility Owners presented a structured approach for effectively planning the integration of BIM into an organisation's ownership of model, data reuse, and data security[7]; and the Singapore BIM Guide v. 2.0 provided a reference for risk alloca-tion and intellectual property rights[41].

Projects are commonly inﬂuenced by multiple risk factors. To sys-tematically manage large and complex BIM projects, the potential risk factors must be identiﬁed during the risk-management process. Because of the limits on project resources and awareness of suggested BIM risks, new questions have arisen among adherent BIM practitioners:“What are the major risk factors associated with BIM projects?”, “Is every level of the AEC industry facing a similar risk situation?”, and “What are the risk-response strategies used for addressing BIM risks associated with BIM projects?”

1.3. Research objectives

To provide a broader context in which to investigate these questions, this paper introduces a comprehensive risk-factor identiﬁcation and assessment method for BIM construction projects. First, we reviewed the literature to identify the risks affecting general construction projects, information technology (IT) and software projects, and BIM projects as well as the risk factors associated with BIM projects. Second, the decision-making trial and evaluation laboratory (DEMATEL), an effective risk-factor assessment method, was applied to determine the critical risk factors (CRFs) in BIM projects. Finally, a case study was con-ducted, and risk-response strategies for BIM projects that involve allo-cating risk among project partners were proposed.

2. Literature review

2.1. Risk management

Ghosh suggested that risk is a factor that can jeopardize the success-ful completion of a project by causing cost overruns, time overruns, and underspeciﬁcation[23]. Chapman and Ward noted that project risk

implies the existence of substantial uncertainty regarding the level of achievable project performance [10]. Wang et al. indicated that a systematic approach to risk management in the construction industry consists of three main stages: risk identification, risk analysis and eval-uation, and risk response[45]. The risk-management process begins with identifying the relevant and potential risks associated with the construction project; this stage is crucial because the processes of risk analysis and response management are applied only to the potential risks identified. Risk analysis and evaluation is the intermediate process between risk identification and management in which uncertainty is measured quantitatively and qualitatively to assess the potential impact of the risk. The evaluation should generally focus on risks with high probabilities, highfinancial consequences, or combinations thereof. Once the risks of a project have been identified and analyzed, an appro-priate method for eliminating risks must be adopted.

Al-Bahar and Crandall stated that the risk analysis and evaluation process is the vital link between the systematic identiﬁcation of risks and the rational management of the key risks, and that the process forms the foundation of decision making regarding the various manage-ment strategies that should be used. Risk analysis and evaluation is de-ﬁned as “a process which incorporates uncertainty in a quantitative manner, using probability theory to evaluate the potential impact of risk.”[2].

Risk analysis generally includes qualitative analysis, semiquantita-tive analysis, quantitasemiquantita-tive analysis, common construction engineering risk analysis, and methods of operation (Table 1).

Zhi noted that the level of risk is evaluated according to several criteria, such as the probability of an undesirable occurrence, the degree of severity, and the subsequent impact of an undesirable event[51]. Williams suggested that a risk can be expressed as R = P × I, where R is the degree of risk (between zero and one), P is the probability of the risk occurring (between zero and one), and I is the degree of impact of the risk (between zero and one)[46].

Risk probability can be assessed using two methods: subjective judgment and objective analysis. Subjective judgment involves estimat-ing the probability that a risk factor is present directly; this is simple and practical for construction projects, but requires experience. Subjective judgment can be used to assign probability to certain risks that appear frequently and for which numerous comparable experiences exist. Ob-jective analysis is used widely to estimate the probability of a risk factor. However, to use this approach, historical data are required, occasionally rendering the application of this method impractical in construction practice; this is especially true for BIM projects because BIM projects are often new and unfamiliar, and comparable information required for undertaking such projects is not easily found.

The ranking of risk factors provides a basis for prioritizing responses to various risks in a construction project. Several methods can be used to assess risk impact. In this paper, we suggest using the DEMATEL method.

2.2. Decision-making trial and evaluation laboratory method

The DEMATEL method, which was created by the Geneva Research Center of the Battelle Memorial Institute[18–20], is practical and useful for visualizing the structure of complex causal relationships by using

Table 1

Construction risk analysis methods.

Qualitative analysis Semi-quantitative analysis Quantitative analysis

Description Using the text or the descriptive classiﬁcation level to describe the extent and the impact of risk that may affect the probability of risk occurrence.

Using actual values to determine the level, but each level is not equal to the actual value that directly affects risk impact and probability.

Using actual data (rather than the semiquantitative analysis used for descriptive classiﬁcation) to describe risk impact and probability.

Analysis method Checklists, inﬂuence diagrams, hazard and operability analysis, brainstorming, cause and effect analysis, risk breakdown structure, failure mode and effects analysis, SWOT analysis, and the Delphi method.

The matrix method, fuzzy analysis method, connection method, and statistical inference.

Fault tree analysis, hierarchical or network analysis, event tree analysis, the Monte Carlo simulation, sensitivity analysis, DEMATEL, and the Poda equation.

(3)

matrices or digraphs. The matrices or digraphs portray contextual rela-tionships among the elements of a system and contain numerals that represent the strengths of the in_{ﬂuences. Thus, the DEMATEL method} can convert the relationship between the causes and effects of criteria into an organised structural model of the system. The DEMATEL method has been successfully applied in various_ﬁelds[11,12,26,36]. Samani used a crisp DEMATEL method to assess risk in the Persian Gulf Sea Bridge project and produced valuable results[4]. A procedure used for assessing risk that is based on the approach of Fontela and Gabus

[18–20]is detailed as follows.

Step 1 Determine the criteria that influence the building energy effi-ciency and measure the relationships between the criteria. The comparison scale consists of five levels: 0 (no influence), 1 (low influence), 2 (medium influence), 3 (high influence), and 4 (extremely high influence). Experts generate sets of pairwise comparisons based on the direction of influence of the relation-ship between the criteria.

Step 2 Generate the direct-relation matrix. From the analysis in Step 1, initial data can be obtained to create a direct-relation matrix, an n × n matrix A, in which Aijis the degree to which criterion i

affects criterion j. A¼ 0 a1;2 … a1;n a2;1 0 … a2;n ⋮ an;1 ⋮ an;2 ⋮ … 0⋮ 2 6 6 4 3 7 7 5 ð1Þ

Step 3 Normalize the relation matrix. Based on the direct-relation matrix A, the normalized direct-direct-relation matrix X can be obtained using the following formulae:

X¼ k A ð2Þ k¼ 1 max 1≤ i ≤ n Xn j¼1ai; j; i; j ¼ 1; 2; …; n: ð3Þ

Step 4 Calculate the total-relation matrix. Once the normalized direct-relation matrix X is obtained, the total-direct-relation matrix T can be

acquired by accumulating all direct and indirect inﬂuences, as shown in Formula(4).

T¼ X þ X2

þ ⋯ þ Xn

ð4Þ According to Formula(3), we have

T¼ XI−X

n−1

1−X ð5Þ

where I denotes the identity matrix.

After normalization, for all the elements (xij) of matrix X, 0b xi,j

b 0. Therefore, when n → ∞, Xn− 1_{→ 0. Thus, Formula}₍₅₎_{can be}

converted into Formula(6).

T¼ X I−Xð Þ−1 ð6Þ

Step 5 Produce a causal diagram. The sum of rows and the sum of col-umns are denoted separately as vector Diand vector Rjby

using Formulas(7)–(9). T¼ ti; j h i nxni; j ¼ 1; 2; …n ð7Þ Di¼ Xn j¼1 ti; j; i ¼ 1; 2; …n ð8Þ Rj¼ Xn i¼1 ti; j; j ¼ 1; 2; …; n ð9Þ

where vector Diand vector Rjrespectively denote the sum of the

rows and the sum of the columns in the total-relation matrix T. The vector (Di+ Rj), named“Prominence,” is then generated

by adding Dito R, thus revealing the importance of the criterion

weight. Similarly, the vector (Di− Rj), named“Relation,” is

gen-erated by subtracting Rjfrom Di, thus dividing the criteria into a

cause group and an effect group.

When (Di− Rj) is positive, the criterion generally belongs to the

cause group, and when (Di− Rj) is negative, the criterion

Second quadrant (Driving factor): Low Prominence, High Relation If independent, will affect a small number of other factors

Second place to use management resources

First quadrant (Core factor): High Prominence, High Relation Key factor affecting research, should be classified as priority target

First place to use management resources

Third quadrant (Independent factor)

Low Prominence, Low Relation

Low interaction with other factors, a separate control factor can be suitable for this area Third place to use management resources

Fourth quadrant (By impact factor)

High Prominence, Low Relation

Factors must be managed, but not improved directly

Last place to use management resources

Low influence Factor

High influence Factor

Avg

(D+R)

Low

High

(D-R)

0.00 Result Factor

Cause Factor

(4)

belongs to the effect group. Thus, (Di+ Rj) indicates how

critical factor i is to the entire system. Conversely, the differ-ence (Di− Rj) indicates the net effect that factor i contributes to

the system. Speciﬁcally, when (Di− Rj) is positive, factor i is a

net cause, whereas factor i is a net receiver or result when (Di

− Rj) is negative.

Each quadrant in the causal diagram represents the meaning and characteristics of factors (Fig. 1). Factors can be classiﬁed ac-cording to their location in the diagram, and determining factors should be attributed to the cause of characteristic classes or the result class, and factors affecting and being affected by proper-ties. By acquiring this information, appropriate decisions can be made to solve problems. To optimize the effective use of management resources, the processing sequence should pro-ceed from the ﬁrst quadrant (core factor) to the second

quadrant (driving factor), then to the third quadrant (indepen-dent factor), andﬁnally to the fourth quadrant (impact factor). 3. Building information modelling project risk assessment

3.1. Risk assessment process

The BIM project risk assessment process used in this study is illustrated inFig. 2

3.2. Identifying potential risk factors

According to Lyons and Skitmore, the most commonly used risk as-sessment technique is based on previous surveyﬁndings, intuition, judgments, and experience[31]. Therefore, identifying and assessing

General Project risk factor IS/IT Project risk factor BIM Project risk factor DEMATEL Questionnaires

Risk Identification Risk Assessment Risk Response

Market Level DEMATEL Risk Assessment Core Risk Factors Traditional Risk Assessment Project Level Case Study BIM Implementation literature Industry Level Organization Level Response Risk Strategy

Fig. 2. BIM project risk assessment process in this study.

Table 2

Risk factors for general construction projects.

Risk categories Description Representative reference

Technical risk Design changes; equipment failure; insufficient or incorrect design information; shortage of skills or techniques; improper design; improper quality control; poor definition of scope; and conflicts in documents.

[33,39,40,45,43,52,15]

Management risk Unclear speciﬁcations; labour disputes and strikes; defective work; labour injuries; theft; damages caused by human error; poor project programme accuracy; improper project feasibility and planning; poor relationship with stakeholders; poor labour productivity; safety (accidents); poor coordination; and inadequate insurance.

[33,51,39,40,23,43,52,15]

Financial risk Inflation; funding storage; unbalanced cash flow; labour and material cost increase; and financial failure of subcontractors. [33,51,40,23,43,52,15] Legal risk Breach of contract (claims and disputes) and intellectual property protection. [40,45,23,43,15] Environmental risk Natural force damage; changes in laws and regulations; weather; cultural differences; nonstandard contract form; human

resource shortage; material shortage; and unforeseen site conditions.

[33,51,39,40,45,23,43,52,15]

Political risk War; inconsistency of government policies; and corruption and bribery. [33,51,45,52,15]

Table 3

Risk factors in IT or software projects.

Scholar Risk factors

Schmidt et al. (2001)[38] Top management lacks commitment to the project; failure to gain user commitment; misunderstanding the requirements; inadequate user involvement; project personnel lack the required knowledge and skills; lack of frozen requirements; changing scope or objectives; introduction of new technology; failure to manage end-user expectations; insufficient or inappropriate staffing; and conflicts between user departments.

Fairley et al. (2003)[17] Excessive schedule pressure; changing requirements; lack of technical speciﬁcations; lack of a documented project plan; excessive and secondary innovations; requirements creep up; lack of scientiﬁc methods; ignoring the obvious; and unethical behaviour.

Wallace (2004)[44] Change in organisational management during the project; corporate politics that negatively affect projects; unstable organisational environment; organisation undergoing restructuring during the project; users resistant to change; conflict among users; users with negative attitudes towards the project; users not committed to the project; lack of cooperation from users; continually changing system requirements; system requirements not adequately identified; unclear system requirements; incorrect system requirements; project involves using new technology; high level of technical complexity; immature technology; project involves using technology that has not been used in prior projects; lack of an effective project management methodology; project progress not monitored closely enough; inadequate estimation of required resources; poor project planning; project milestones not clearly defined; inexperienced project manager; ineffective communication; inadequately trained development team members; inexperienced team members; and team members lack specialised skills required by the project.

Kappelman et al. (2006)[27] Lack of top management support; weak project manager; no stakeholder involvement and participation; weak commitment of the project team; team members lack requisite knowledge and skills; subject matter experts are overscheduled; lack of documented requirements and success criteria; no change-control process (change management); ineffective schedule planning and management; communication breakdown among stakeholders; resources assigned to a project with a higher priority than that of the current project; and no business case for the project.

(5)

risk factors effectively can facilitate understanding the types of risks that may exist as well as guiding the management of these risks at the contracting and construction stages.

3.2.1. General construction risk factors

Identifying risks can be challenging for managers, especially because risks must be distinctly described and categorized. Terms such as risk factors, critical success factors, and uncertainty factors are often used to convey the same concept. Therefore, these factors were combined and considered identical factors in this study.Table 2shows the risk fac-tors for general construction projects that several studies have identi-ﬁed and classiidenti-ﬁed.

3.2.2. Information technology project risk factors

Risk factors for software and IT development projects have been identi_{ﬁed and classiﬁed in several studies}[17,27,38,44](Table 3). Stud-ies have suggested that inadequate risk assessment is a major source of problems in IT and software projects. From a management perspective, IT project risks have managerial, strategic, personnel-related, and tech-nical dimensions. Thus, IT risk assessment involves various changes in managerial and working progress.

3.2.3. Building information management project risk factors

Numerous governmental, public, and private construction clients have established formal requirements for BIM use in their projects

[29]. Hartmann et al. indicated that implementing BIM in a construction project requires practitioners to conﬁgure and align the BIM-based tools, work processes, and business models of the companies that col-laborate on these projects[25].

Adopting BIM involves specific challenges concerning teamwork, collaboration, and information sharing. Bryde et al. indicated that the major benefits of BIM use are cost reduction and control, time reduction and control, and communication improvement. Software problems were identified to be the most prevalent negative factor among all of the criteria[9]. Won et al. identified 17 barriers to BIM adoption by reviewing the general barriers, limitations, challenges, and consider-ation factors reported in the literature[47]. In this study, the risks that have been identified in BIM-related studies were first noted and then similar risks were identified. Thus, a total of 13 risks were identified from the reviewed literature (Table 4).

Based on the summarized risk categories listed inTable 2, a typical classi_{fication of BIM risks, including technical, management,} environ-mental,financial, and legal risks, is proposed. The risk dimensions and definitions compiled based on the BIM literature and on interviews with members of consultant companies as well as the public and academic sectors are summarized inTable 5.

Table 4

Risk factor identiﬁcation in BIM projects.

Risk factors A B C D E F G H I

F1. Project experience inadequate • • • • • •

F2. Lack of software compatibility • • • •

F3. Model management difﬁculties • • • •

F4. Inefﬁcient data Interoperability • • •

F5. Management process change difﬁculties • • •

F6. Inadequate top management commitment • •

F7. Workﬂow transition difﬁculties • • •

F8. Lack of available skilled personnel • • •

F9. Increase in short-term workload • • • • •

F10. Rise in short-term costs • • • •

F11. Additional expenditure • • •

F12. Lack of BIM Standards • •

F13. Unclear legal liability • • •

A = Won et al.(2013)[47]; B = Bryde et al.(2013)[9]; C = Migilinskas et al.(2013)[32]; D = Eadie et al.(2013)[14]; E = Porwal and Hewage (2013)[35]; F = Azhar (2011)[3]; G = Gu and London (2010)[24]; H = Luthra (2010)[30]; I = Yan and Damian (2008) [48].

Table 5

Summary of risk factors.

Dimension Factor Description

Technical risk F1 Inadequate project experience •Because of inadequate experience with projects that implement BIM technology, the unclear business value and unknown risk results could lower the willingness to apply BIM.

F2 Lack of software compatibility •Most project participants are accustomed to working with particular tools (software and hardware); consequently, data transfer is often limited because of incompatibility, which affects the transmission of consistent information to other participants. The untransferred data must be recovered and additional efforts must be made to recover it or add the information for other particular tools.

F3 Model management difﬁculties •As the model is updated continually to create new versions, version control problems will likely occur •Accurate data entry strictly required

•Compiling a single ﬁle that shows a virtual 3-D model can easily cause company knowledge to be leaked; hence, information security must be readjusted.

•Software unable to handle large amounts of data

F4 Inefficient data interoperability •During BIM-IFC file exchange or when reading BIM models on distinct software files, data loss will occur after file conversion

Management Risk F5 Management process change difﬁculties

•Managers still follow traditional 2D design-management models to manage 3D design workﬂow, could make it difﬁcult for department or division to clarify responsibilities and thus result in incomplete assignments. •Reluctance to openly share information

•Shift of liability among project participants F6 Inadequate top management

commitment

•Insufﬁcient commitment of top management would lead to poor performance

F7 Workflow transition difficulties •Lack the ability to integrate traditional 2D workflow such as design and review process with new 3D design tools, would lead to ineffective collaboration between people with distinct roles

•Shift of liability among project participants

Environmental Risk F8 Lack of available skilled personnel •BIM knowledge and ability of existing staff not fully established •Lack of technical personnel familiar with BIM

F9 Increase in short-term workload •Compiling a BIM library early on in the process increases the initial workload •A considerable amount of time required to become familiar with software operation •Existing staff require training to learn new techniques

Financial Risk F10 Rise in short-term costs •Initial BIM implementation could increase expenses related to BIM model review, personnel training, hardware and software acquisition, and other processes

F11 Additional expenditures •Additional funds are required for legal disputes, software updates, and other expense Legal Risk F12 Lack of BIM standards •No clear product delivery and acceptance criteria and no clear criteria for model building

F13 Unclear legal liability •Standard contract, insurance policy, intellectual property ownership, dispute-settlement mechanisms, and other laws and lines of responsibility still in the discussion stage

(6)

3.3. Decision-making trial and evaluation laboratory survey 3.3.1. Survey respondents

The survey was conducted between March 2013 and April 2013 among owners (public agencies) in the Taiwanese AEC industry, architects, three major construction engineering consultant companies, construction contractors, academics, and BIM service providers. The survey respondents were people who have shared successful BIM pro-ject operating experiences in BIM propro-ject publications or on domestic BIM forums. Before the questionnaire was administered, an invitation letter containing details on the survey was sent to every potential respondent. After we received confirmation by e-mail or telephone, the official questionnaire was administered to the respondents through e-mail. In addition to increasing the instrument reliability, this careful selection and confirmation of respondents was necessary because BIM adoption in the Taiwanese AEC industry was still at an early adoption stage, and collecting suitable and coherent information from this small and selective sample was necessary to understand the industry.

To distinguish between the risks of distinct roles, three groups were identiﬁed. Type A represented all BIM practitioners in the AEC industry. Type B represented the marketing level and was divided into a design team and a construction team. Type C represented the organisation level and was divided into groups of architects, construction personnel, owners, and scholars.

3.3.2. Survey questionnaire content

The questionnaire contained three main sections. Thefirst section consisted of questions used to profile the respondents and their compa-nies. The second section introduced the definitions of BIM risk criteria and risk factors. In the third section, the respondents rated the CRFs for BIM projects by using a 5-point Likert scale. A risk matrix was pro-vided to help respondents rate the criticality of the risk factors that were identified by conducting a comprehensive literature review.

Because evaluating the criticality of risk is complex and using vague qualitative linguistic terms is unavoidable[45], we adopted a 5-degree rating system (0 = lowest; 1 = low; 2 = moderate; 3 = high; 4 = extreme) to assess risk criticality (Appendix A).

3.3.3. Survey response

We distributed 45 questionnaires and collected 42, yielding a re-sponse rate of 93% (Table 6). Most of the respondents were men and were aged 30 to 39 years old. All of the respondents (excluding the architects) had more than 5 years of experience in BIM projects and all of them played crucial roles in ongoing BIM projects in their companies.

3.3.4. Reliability and validity tests

Questionnaires must be subjected to statistical analysis to conﬁrm their reliability and validity[5,8,23,49]. To validate the CRFs, reliability and validity tests of the instrument were conducted using the approach adopted by Saraph et al.[37].

Reliability test: internal consistency analysis

According to Foster, reliability refers to the consistency of the results among various items in a test[21]. To understand the relationship be-tween various data items, the reliability of the data must be quantified. For reliability analysis, Cronbach'sα was used to verify the internal con-sistency among the items constituting each factor. This coefficient is used to determine whether a questionnaire measures success factors effectively and to determine the extent to which items within a factor (or construct) are related to each other; moreover, this coefficient facil-itates identifying problematic items (elements) that must be excluded from the scale to improve reliability. The items of each critical factor were separately subjected to an internal consistency analysis by using

Table 6

Personal information of the respondents.

Basic information BIM service providers Construction engineering consultants Architects Constructors Owners Scholars

Sex Male 3 6 8 7 8 3 Female 2 1 1 2 1 0 Title Manager 2 2 1 2 0 0 Architect 0 1 7 0 1 0 Engineer 3 4 1 7 8 0 Professor 0 0 0 0 0 3

BIM experience (in years) 1–2 1 2 8 6 8 1

3–4 2 0 1 2 1 0 5~ 2 5 0 1 0 2 Questionnaires Distributed 6 9 9 9 9 3 Questionnaires Collected 5 7 9 9 9 3 Response rate 83% 78% 100% 100% 100% 100% Table 7

BIM risk factor inﬂuence analysis.

Risk factors Inﬂuence rank (D) Being affected rank (R) D + R D− R Causal diagram quadrant

F1. Project experience inadequate 11.71582 11.49729 23.21311 0.219 1

F2. Lack of software compatibility 10.38877 9.311877 19.70065 1.077 2

F3. Model management difﬁculties 10.0304 9.47345 19.50385 0.557 2

F4. Inefﬁcient data interoperability 11.30403 9.517939 20.82197 1.786 2

F5. Management process change difﬁculties 10.72723 11.51414 22.24136 −0.787 4

F6. Inadequate top management commitment 11.2124 12.63928 23.85168 −1.427 4

F7. Workﬂow transition difﬁculties 10.92502 11.47065 22.39567 −0.546 4

F8. Lack of available skilled personnel 11.30673 10.82031 22.12704 0.486 1

F9. Increase in short-term workload 10.87014 11.77913 22.64927 −0.909 4

F10. Rise in short-term costs 10.64427 11.978 22.62227 −1.334 4

F11. Additional expenditure 10.12844 10.71158 20.84002 −0.583 3

F12. Lack of BIM standards 11.095 9.701903 20.7969 1.393 2

F13. Unclear legal liability 9.152091 9.084814 18.23691 0.067 2

(7)

the SPSS reliability analysis procedure. Nunnally recommended using a minimal Cronbach's_{α value of 0.7}[34]. The Cronbach's_{α values for the} questionnaire used in this study ranged from 0.78 to 0.80, indicating that all of the factors exhibited high internal consistency and that the questionnaire was thus reliable.

Validity test: content validity

Content validity refers to the success of researchers in creating mea-surement items that cover the content domain of the variable being measured[5]. The content validity of the questionnaire was based on the literature review and on the opinions of several experts who exam-ined the items. Thus, we concluded that the CRFs had content validity. According to Nunnally, an instrument has content validity when it con-tains a representative collection of items and when appropriate methods were used to construct the test[34]. We concluded that the CRF section of this survey had content validity because it was approved by the pilot respondents.

4. Research results

4.1. Type A (architecture, engineering, and construction industry level)

The inﬂuence and being-affected rank of the BIM risk factors of Type A obtained from the DEMATEL total-relation matrix are shown in

Table 7.

The calculated values of (D + R) and (D− R) were used to generate the causal diagram shown inFig. 3. According to the causal diagram, CRFs of BIM were F1 (inadequate project experience) and F8 (lack of available skilled personnel).

4.2. Type B (market level)

4.2.1. Design team (architects, consultants, owners, and building information modelling service providers)

The received inﬂuence rank and being-affected rank of the BIM risk factors that were identiﬁed by the design team and ob-tained from the DEMATEL total-relation matrix are shown in

Table 8.

The calculated values of (D + R) and (D− R) were used to gener-ate the causal diagram shown inFig. 4. According to the causal diagram, a core risk factor of BIM was F8 (lack of available skilled personnel).

4.2.2. Construction team (construction personnel, owners, building information modelling service providers)

The inﬂuence rank and being-affected rank of the BIM risk factors that were identiﬁed by the construction team and obtained from the DEMATEL total-relation matrix are shown inTable 9.

The calculated values of (D + R) and (D− R) were used to generate the causal diagram shown inFig. 5. According to the causal diagram, core risk factors of BIM were F1 (inadequate project experience) and F8 (lack of available skilled personnel).

Table 8

F1. Project experience inadequate 11.70467 12.25224 23.95691 −0.548 4

Causal diagram First quadrant (Core factor):1; Second quadrant (Driving factor):2; Third quadrant (Independent factor):3; Fourth quadrant (By impact factor):4.

18 20 22 24

D+R

-2 -1 0 1 2

D-R

F1 F8 F12 F2 F3 F4 F5 F6 F7 F9 F10 F11 F13

Industry

Fig. 3. Casual diagram of the industry-level BIM risk factors.

9 10 11 12

D+R

-2 -1 0 1 2

D-R

F8 F1 F2 F3 F4 F5 F6 F7 F8 F9 F10 F11 F12 F13

Design Team

(8)

4.3. Type C (organisation level)

4.3.1. Designers (architects and design consultants)

The inﬂuence rank and being-affected rank of the BIM risk factors that were identiﬁed by designers and were obtained from the DEMATEL total-relation matrix are shown inTable 10.

The calculated values of (D + R) and (D− R) were used to generate the causal diagram shown inFig. 6. According to the causal diagram, a core risk factor of BIM was F8 (lack of BIM standards).

4.3.2. Construction personnel

The inﬂuence rank and being-affected rank of the BIM risk factors that were identiﬁed by the construction personnel and ob-tained from the DEMATEL total-relation matrix are shown in

Table 11.

The calculated values of (D + R) and (D− R) were used to generate the causal diagram shown inFig. 7. According to the causal diagram, core risk factors of BIM were F1 (inadequate project experience), F4 (insuf_{ﬁcient data interoperability), F5 (management process change} difﬁculties), and F8 (lack of available skilled personnel).

4.3.3. Owners

The inﬂuence rank and being-affected rank of the BIM risk factors that were identiﬁed by owners and obtained from the DEMATEL total-relation matrix are shown inTable 12.

The calculated values of (D + R) and (D− R) were used to gener-ate the causal diagram shown inFig. 8. According to the causal diagram, a core risk factor of BIM was F1 (inadequate project experience).

4.3.4. Academics

The inﬂuence rank and being-affected rank of the BIM risk factors that were identiﬁed by academics and obtained from the DEMATEL total-relation matrix are shown inTable 13.

The calculated values of (D + R) and (D− R) were used to generate the causal diagram shown inFig. 9. According to the causal diagram, a core risk factor of BIM was F7 (workﬂow transition difﬁculties). 4.4. Critical risk factor analysis and discussion

According to the DEMATEL causal diagram results, the risk factors lo-cated in theﬁrst quadrant were deﬁned as the core factors affecting all

Table 9

Risk factors Inﬂuence rank (D) Being affected Rank (R) D + R D− R Causal diagram quadrant

F4. Inefﬁcient data Interoperability 8.802813 7.21911 16.02192 1.584 2

F12. Lack of BIM Standards 8.868577 7.63167 16.50025 1.237 2

Causal diagram First quadrant (Core factor):1; Second quadrant (Driving factor):2 Third quadrant (Independent factor):3; Fourth quadrant (By impact factor):4.

7 8 9 10 11

D+R

-0.8 -0.4 0 0.4 0.8

D-R

F1 F8 F12 F2 F3 F4 F5 F6 F7 F9 F10 F11 F13

Construct Team

Fig. 5. Causal diagram of the market-level (construction team) BIM risk factors.

Table 10

F3. Model management difﬁculties 10.00963 10.19299 20.20262 −0.183 3

(9)

of the factors, and must be classiﬁed as priority targets for resource management. In this study, we also identiﬁed these core risk factors as CRFs. After we interviewed the survey respondents and BIM experts, the CRFs of Types A, B, and C were summarized, as shown inTable 14.

4.4.1. Common risk factors: F1 (inadequate project experience) and F8 (lack of available skilled personnel)

1 BIM adoption is still at the early stages in the Taiwanese AEC industry. Without sufﬁcient project benchmarking information, unknown risks might lower the potential willingness of BIM adopters to use BIM. Thus, project experience is still extremely inadequate.

2 Because government policy regarding BIM is also at the early stages, educational institutes, such as universities and professional schools, are not fully prepared to create BIM educational programmes for training the skilled BIM students that the industry requires. More-over, the demand of the AEC industry for skilled BIM personnel is ex-tremely high, thus generating problems in project in which an insufﬁcient number of skilled personnel is employed.

4.4.2. Individual risk factors: F4, F5, F7, and F12

1 Designers: F12 (lack of BIM standards): Because academic research and information companies promote BIM use, owners will gradually become thoroughly familiar with the beneﬁts of using BIM. Public and private owners speciﬁcally required using BIM to undertake sev-eral early projects. However, because the government has not established adequate BIM standards, designing and delivering 3-D results when the industry is still using the two-dimensional (2-D)

examination process causes difficulty for architects. Defining and communicating the work scope and quality to owners and fellow design technicians is also difficult for architects.

2 Construction: F4 (inefficient data interoperability) and F5 (manage-ment process change difficulties) The majority of the construction in Taiwan is still executed using the design-bid-build (DBB) process. Currently, several construction projects require the delivery of as-built BIM models when construction is completed. Without the in-tegrated project delivery (IPD) process or using turnkey project type, construction personnel encounter problems regarding BIM-IFCfile exchange or reading BIM models on different softwarefiles. Partial data loss occurs afterfile conversion, especially between different brands of BIM software. Moreover, most of the contractors have extremely insufficient BIM project experience, and managers are still accustomed to communicating with subcontractors by using tra-ditional 2-D information. Unfamiliarity with BIM technology can causes difficulties for contractors in clarifying the responsibilities of each partner and coordinating the process effectively.

3 Academics: F7 (workflow transition difficulties) Disregarding the ef-fects of external BIM environment development, from a midterm or long-term BIM development perspective, workflow transition is a common practical problem with which BIM academics are con-cerned. Although BIM has been implemented in several real projects, empirical data are still necessary for researchers to determine the most suitable practice for the local industry. Therefore, workflow transition dif_{ficulties are the BIM risk factor with which academics} are the most concerned.

14 16 18 20 22

D+R

-2 -1 0 1 2

D-R

F12

Desinger

F4 F13 F2 F3 F5 F1 F11 F7 F8 F9 F6 F10

Fig. 6. Causal diagram of the organisation-level (designers) BIM risk factors.

Table 11

BIM risk factors inﬂuence analysis.

F5. Management process change difﬁculties 4.309444 4.077317 8.386761 0.232 1

F11. Additional expenditure 3.606926 3.571955 7.178881 0.035 2

F12. Lack of BIM standards 2.957156 3.525028 6.482184 −0.568 3

F13. Unclear legal liability 2.269367 3.490655 5.760022 −1.221 3

5 6 7 8 9

D+R

-2 -1 0 1 2

D-R

Constructor

F1 F5 F3 F2 F11 F12 F13 F9 F8 F4 F6 F7 F10

(10)

5. Empirical case study 5.1. Project case introduction

The largest military technology research institute in Taiwan recently implemented BIM technology in one of its construction projects. The project construction site was located in Southern Taiwan. The DBB pro-curement process was applied in the project, the construction budget was nearly 40 million U.S. dollars (the BIM model budget was approxi-mately 0.4% of the construction fee), and the expected construction schedule was 1,000 days. It was constructed by 50 members of the

contractor's project team and 5 members completed the BIM-related work. BIM was implemented during the construction phase only.

5.2. Traditional risk assessment method (risk impact/probability chart) The project team managerfirst used the traditional risk assessment method (a risk impact and probability chart) and determined that the six major BIM risk factors were F8 (lack of available skilled personnel), F2 (lack of software compatibility), F4 (inefficient data interoperability), F1 (inadequate project experience), F3 (model management dif ficul-ties), and F10 (rise in short-term costs) (Table 15).

5.3. Decision-making trial and evaluation survey results for the project The inﬂuence rank and being-affected rank of the BIM risk factors for the project obtained from the DEMATEL total-relation matrix are shown inTable 16.

The calculated values of (D + R) and (D− R) were used to generate the causal diagram shown inFig. 10. According to the causal diagram, a core risk factor of BIM was F1 (inadequate project experience).

In this project, the research team worked closely with the project team and summarized the DEMATEL analysis results as well as identi-fied the factors as CRFs of BIM projects. The six major CRFs were F1 (inadequate project experience), F2 (lack of software compatibility), F3 (model management difficulties), F4 (inefficient data interoperabili-ty), F8 (lack of available skilled personnel), and F12 (lack of BIM standards).

5.4. Difference analysis

The differences between the DEMATEL analysis results and the infor-mation in the risk impact and probability chart are shown inTable 17.

Table 12

BIM risk factors inﬂuence analysis.

F8. Lack of available skilled personnel 4.301063 4.347384 8.648448 −0.046 3

F11. Additional expenditure 4.360251 4.283127 8.643378 0.077 2

7 8 9 10 11

D+R

-2 -1 0 1 2

D-R

F1

Owner

F2 F3 F4 F5 F6 F7 F8 F9 F10 F11 F13 F12

Fig. 8. Casual diagram of the organisation-level (owners) BIM risk factors.

Table 13

F7. Workﬂow transition difﬁculties 3.550754 3.342008 6.892762 0.209 1

(11)

Common project risk factors

By using DEMATEL analysis and a risk impact and probability chart, theﬁve most common project risk factors were identiﬁed: F1, F2, F3, F4, and F8.

Project risk factor differences Only two risk factors differed.

1. F10. Rise in short-term costs (score in the risk impact and probability chart higher than the DEMATEL score)

Because this is theﬁrst project in which the contractor was required to deliver BIM data (such as clash detection, 4-D), it spent more time and

money preparing a BIM infrastructure (software and hardware) than it spends in a typical construction project. When the risk impact and prob-ability chart was created, every risk item was considered independent. When the project was still at the early stages, a rise in short-term costs re-ceived more scores based on impact; this was directly reflected in the risk assessment results. During the DEMATEL test, the team members needed to compare relative influence. The rise in short-term costs seemed to have a less substantial effect than the other risk items did when we considered that every risk could influence the other risks.

2. F12. Lack of BIM standards (DEMATEL score higher than the score in the risk impact and probability chart)

When the DEMATEL method was conducted, the lack of BIM stan-dards was justified with driving risk factors, which could directly affect a small number of factors. However, the project owner had already con-ducted the feasibility research. Every BIM work item was clearly defined in the contract. From the perspective of the construction personnel, as long as the requirements were fulfilled and they were working with the assistance of a BIM service provider, a lack of BIM standards must produce a low score in the risk impact and probability chart.

5.5. Risk response

Risk response strategies are approaches that can be implemented to manage the identiﬁed risks. The objective of these strategies is to reduce the effects of potential risks and increase the control over risks[2]. Early responses to these risks can increase the possibility of successful BIM implementation. Risks can be managed using several strategies, such as avoidance, reduction, acceptance, and transfer. After the project teams, consultants, and BIM experts were interviewed, the suggested

2 3 4 5

D+R

-2 -1 0 1 2

D-R

Academics

F1 F5 F4 F2 F3 F13 F12 F8 F9 F7 F11 F10 F6

Fig. 9. Causal diagram of the organisation-level (academics) BIM risk factors.

Table 14

Critical risk factors analysis.

Critical risk factors Industry level Market level Organisation level

AEC Design Construction Designer Constructor Owner Academics

Common F1, F8 F8 F1, F8 F8 F1, F8 F1

Difference F12 F4, F5 F7

Table 15

Risk impact and probability chart.

Category Impact analysis Probability

Performance Time Cost

1 No impact or inﬂuence small No impact or inﬂuence small No impact or

inﬂuence small

Extremely unlikely 2 Acceptable, lower part of the margin Achievable due date shall be additional

resources

b5% Unlikely

3 Acceptable, dramatically reducing the

margin

Important milestones in slightly behind 5–7% Possible

4 Acceptable, no margin Important milestone in signiﬁcantly behind 7–10% Very likely

5 Unacceptable Due date cannot be reached, the main project

milestones behind

N10% Almost certainly occurred

Factor Impact analysis Probability Risk index Risk ranks

F1. Project experience inadequate 2 4 8 4

F2. Lack of software compatibility 2 5 10 2

F3. Model management difﬁculties 2 4 8 4

F4. Inefﬁcient data Interoperability 2 5 10 2

F5. Management process change difﬁculties 1 5 5 10

F6. Inadequate top management commitment 1 2 2 13

F7. Workﬂow transition difﬁculties 2 3 6 7

F8. Lack of available skilled personnel 3 4 12 1

F9. Increase in short-term workload 1 4 4 12

F10. Rise in short-term costs 2 4 8 4

F11. Additional expenditure 2 3 6 7

F12. Lack of BIM Standards 1 5 5 10

(12)

relative risk-response strategies used for the case project were identi-ﬁed (Table 18).

6. Limitations of the study

1. Risk is a complex construct and this research may not have addressed every aspect of BIM project risk, despite applying an effective method

to ensure that the speciﬁcations of the construct domain were as complete as possible and that acceptable measurement properties were obtained.

2. Another limitation of the study is that it focused on one country. Therefore, determining the extent to which theﬁndings can be gen-eralized to other countries is difﬁcult. Furthermore, we can rely only on our personal knowledge of the cultures involved to infer the dif-ferences observed among the respondents that were observed in this study.

7. Conclusions

1 Assessing risks and developing strategies to counteract the risks requires an understanding of the characteristics of the risks, which risks deserve attention, and how to respond to the risks. Based on a survey and analysis, we identiﬁed the risks that BIM practitioners in Taiwan consider to be the most critical for execut-ing BIM projects. A total of 13 risk factors were identiﬁed through a literature search.

2 Sufﬁcient historical risk and research data were unavailable; there-fore, the DEMATEL method was introduced and applied to assess the risk of BIM projects. Because factors can interact, DEMATEL is a risk assessment method that is suited to the real-world project envi-ronment. In this study, a DEMATEL questionnaire was designed to gather data from BIM projects, and the data were analyzed to identify the CRFs.

3 Our research results indicated that, in the overall Taiwanese AEC in-dustry, F1 (inadequate project experience) and F8 (lack of available skilled personnel) are CRFs that affect other risk factors; therefore, allocating management resources to addressing these risk factors should be a priority. This result reﬂects the problems prevalent in all countries in which BIM development is at the early stage. More-over, this result is consistent with those of previous studies in

Table 16

F12. Lack of BIM Standards 4.199355 3.263608 7.462963 0.936 2

6 7 8 9 10

D+R

-2 -1 0 1 2

D-R

F1

Project

F2 F3 F4 F5 F6 F7 F8 F9 F10 F11 F13 F12

Fig. 10. Causal diagram of case project.

Table 17

Distinct roles and team strategies for responding to risk.

Selection factors DEMATEL rank Risk impact/probability chart rank Top 5 match

F1. Project experience inadequate 1 4 ✓

F2. Lack of software compatibility 2 2 ✓

F3. Model management difﬁculties 2 4 ✓

F4. Inefﬁcient data interoperability 2 2 ✓

F5. Management process change difﬁculties 9 10

F6. Inadequate top management commitment 9 13

F7. Workﬂow transition difﬁculties 9 7

F8. Lack of available skilled personnel 2 1 ✓

F9. Increase in short-term workload 7 12

F10. Rise in short-term costs 7 4 x

F11. Additional expenditure 7 7

F12. Lack of BIM standards 2 10 x

(13)

which a lack of required knowledge and skills among project person-nel and a commitment of top management to the project were con-sidered among the topﬁve factors in IT and software projects. 4 In a similar external environment, every level shares common CRFs.

However, different roles involve various individual CRFs because working resource capabilities differ. Designers considered F12 (lack of BIM standards) as a CRF because of the legal design responsibility. Contractors identified F4 (inefficient data interoperability) and F5 (management process change difficulties) as CRFs based on opera-tional management concerns.

5 BIM practitioners must fully understand the identified risk profile to adopt appropriate risk-management strategies. Our research in-volved implementing a DEMATEL risk assessment method in a real BIM project and comparing the DEMATEL method with a traditional risk assessment method, the risk impact and probability chart. The analysis results indicated that, in most instances, the same CRFs were identified when using either the DEMATEL method or the tradi-tional method, part for risk factors is different because of the unique project characteristics. In certain instances, the short-term project phenomenon reflected more on the impact when using the tradition-al assessment method than when using the DEMATEL method. Based on the DEMATEL analysis results, we identified the top six risk factors and proposed appropriate response strategies that BIM practitioners and software partners can use to develop an in-depth understanding of the risk environment encountered when implementing BIM projects.

6 BIM technology must be integrated into the project management and working process, and potential legal andﬁnancial risks must be con-sidered. Our research results suggested that F1 (inadequate project experience) and F8 (lack of available skilled personnel) are the most common risk factors at all levels; therefore, the government should establish and sponsor education programmes for industry professionals and prepare a knowledge portal for historical data on performance and lessons learned to determine the most effective methods for implementing BIM.

Appendix A

1. First Section

• Instructions for ﬁlling out the questionnaire • Basic personal data

A. Name:

B. Gender: Male Female C. BIM working experience: D. Service Unit:

E. Service Dept.: F. Job Title:

G. Age: Under 30 years old (including) 30–35 years old (including) 1. 35–40 years old (including) 40–50 years old (including)

2. Over 50 years old 2. Second Section

• Descriptions of criteria

All decision dimensions and criteria are shown inTable 5. • Method for ﬁlling out

A. Filling factors influence level: 0. No influence; 1. Minor influ-ence; 2. Middle influence; 3. High influence; 4. Extreme influ-ence. For example: The influence degree of A to B is extreme influence, then filling 4 under B column.

B. Examples:

(1) The inﬂuence degree of ingredient consistency to process capability is extreme thenﬁling 4 into the cross blank of F1 and F2.

Table 18

Distinct roles and team strategies for responding to project risk.

Item Factors Risk Possible counteractions