Evaluation

During the field test, the three BIM engineers handled all the BIM-related work. The BIM works included the as-built schedule integrated with BIM models creation, modification, and revision. The twelve onsite project engineers and the general contractor handled the entire construction project up to its completion using the as-built schedule.

The ConBIM-SKM prototype was installed on the general contractor’s main server during the test. A user guide was given and two ConBIM-SKM prototype workshops were held to show how the ConBIM-SKM prototype could be integrated with the BIM approach. Furthermore, BIM-related software (Autodesk Revit) was selected to create and revise the as-built models. All BIM files were converted into DWF files for review and markup in the ConBIM-SKM prototype.

In this case study, the BIM engineer handled and assisted the production and revision of BIM models for BIM-assisted as-built scheduling. The onsite engineers developed and updated the BIM-assisted as-built schedule. Finally, the project managers and onsite

58

engineers accessed and tracked the updated as-built schedule with the BIM approach in the ConBIM-SKM prototype.

During the field test, the verification test was carried out through the assessment of whether the ConBIM-SKM prototype performed tasks as specified in its design. During the validation test, selected case participants were asked to use the system; project teams then provided feedback via a questionnaire. The case participants consisted of three BIM engineers of general contractor with three years’ experience, six onsite engineers with five years’ experience, two senior subcontractor engineers with ten years’ experience, and two general contractor project engineers with ten years’ experience. Questionnaires were distributed to evaluate system function and user satisfaction with system capabilities.

System users were asked to grade the system's usage, functionality, and capability separately, based on a comparison with the previous meeting approach on a five-point Likert scale, ranging from 1 for “not useful” to 5 for “very useful.” Questionnaire results indicated that enhancing the management of the visual as-built updated schedule with graph visualization using the BIM approach was significantly improved through the use of the proposed system. Comments regarding possible improvements to the ConBIM-SKM prototype were also obtained from project participants. The ConBIM-ConBIM-SKM prototype was demonstrated to the respondents, who were asked to express their opinions towards the system by completing the questionnaire. Table 5 shows the results of the testing of the system. Based on the questionnaire result, respondents thought BIM should be a helpful tool in construction schedule management. Obviously, most of the respondents agreed the use of BIM to support as-built schedule illustration and improve updating as-built schedule sharing visually.

59

Table 5. System Evaluation Result for schedule management Using BIM System Evaluation Item Mean Score Enhance visual as-built schedule illustration 4.5

Applicability to Construction Industry 4.8

Reduces Mistakes in As-built schedule 4.2

Enhances Virtual As-built Schedule Sharing 4.3 Enhances Tracking As-built Schedule Status Virtually 4.2 Improves Updating As-built Schedule Sharing 4.5 Enhances BIM-assisted Schedule Communication 4.4 Enhances Updated Parts of As-built Schedule 4.1 Enhances Managing As-built Schedule Virtually 4.3 Note: the mean score is calculated from respondents' feedback on a

five -scale questionnaire: 1(Strongly Disagree), 2, 3, 4 and 5 (Strongly Agree)

The principal advantages of the ConBIM-SKM prototype, based on questionnaire results, The principal advantages of the ConBIM-SKM prototype, based on questionnaire results, are as follows: (1) The ConBIM-SKM prototype allowed project engineers and managers to track and manage the visual BIM-assistant as-built schedule information during construction (92% agreed). (2) The ConBIM-SKM prototype allowed onsite engineers to provide as-built schedule feedback with color-coded statuses through the ConBIM-SKM prototype (89% agreed). (3) The ConBIM-SKM prototype enabled the project engineers to track and view the updated contents of the as-built schedule when using it with the BIM approach (88% agreed). (4) The ConBIM-SKM prototype enhanced visual management of the as-built schedule effectively in the visual environment (92%

agreed).

60

The following recommendations are based on user feedback. (1) Successful ConBIM-SKM adoption should be supported by the individuals of highest authority in a firm and on a jobsite. (2) Policy and strategy must be considered to encourage use of the ConBIM-SKM prototype at the jobsite during the construction phase. (3) Further efforts and additional approaches are required to overcome unwillingness to adopt BIM software (such as Autodesk Design Review) at a construction jobsite. (4) Initial case study results should be used to educate users about adoption of BIM software, and additional staff training is needed. (5) Further training and workshops on the ConBIM-SKM prototype and BIM software are necessary for all users.

Regarding to knowledge management section, Questionnaire results from the case study evaluation reveal that the ConBIM-SKM prototype effectively shares knowledge for construction projects. A verification test was performed by checking whether the ConBIM-SKM prototype could perform tasks as specified in the system analysis and design. The validation test was undertaken by requesting selected case project practitioners to use the system, and the provide feedback by answering a questionnaire.

There were 18 respondents; two project managers with ten years of experience; six onsite engineers with ten years of experience; seven onsite engineers with five years of experience; two assistants with three years of experience, and one CKO with ten years of experience. The ConBIM-SKM prototype was demonstrated to the respondents, who were requested to give their opinions on it by completing the questionnaire. Table 6 shows the results of the system evaluation. Based on the questionnaire result, respondents thought BIM should be a helpful tool in construction knowledge management. Obviously, most of the respondents agreed the use of BIM to support knowledge illustration and improve knowledge sharing visually.

61

Table 6. System Evaluation Result for knowledge management Using BIM System Evaluation Item Mean Score

Enhances visual knowledge illustration 4.5

Applicability to construction knowledge management 4.8

Reduces rework percentage 4.2

Reduces percentage of mistakes occurring 4.3

Improves the knowledge collection 4.2

Enhances knowledge communication 4.4

Improves knowledge sharing 4.7

Enhances learning performance 4.3

Note: the mean score is calculated from respondents' feedback on a

five -scale questionnaire: 1(Strongly Disagree), 2, 3, 4 and 5 (Strongly Agree)

Overall, the onsite engineers’ feedback for the case study was positive. Most engineers and project managers agreed that the ConBIM-SKM prototype helps them to view all collected knowledge and experience in the BIM environment. Questionnaire results indicate that the primary advantages of using the ConBIM-SKM prototype are as follows: (1) it provides 3D visual illustration of knowledge regarding to project-based knowledge (86% agreed); (2) it provides BIM-assisted animation easily and effectively (89% agreed); and (3) it clearly identifies available knowledge and different status of different knowledge in the BIM environment (90% agreed).

Questionnaire results indicate that the primary advantages of the application of BIM in BIM-based knowledge models are as follows: (1) it provides clear 3D representations,

62

thus identifying knowledge and experience feedback relevant to an object or activity (92% agreed); (2) one can generate a visual object of BIM model illustrations of knowledge, thus identifying acquired knowledge and experience feedback relevant to tasks and projects (90% agreed); (3) it allows one to view knowledge and information provided by onsite engineers easily and effectively (90% agreed); and (4) it enables engineers to trace and manage any acquired BIM-based knowledge feedback (88%

agreed).