Tao, Y.-H., Yeh, R., S.-I. Sun, Improving training needs assessment processes via the Internet: System design and qualitative study, Internet Research, Vol. 14, No. 4, 2006, 427-449.
Improving Training Needs Assessment Processes via the Internet:
System Design and Qualitative Study
Structured Abstract
Purpose
The goal of this research is to demonstrate how Web-based applications improve the effectiveness of a very important HR function. It proposes a framework of a Web-based training needs assessment system for human resource (HR) professionals to effectively assess employee needs for competency-based training.
Design/methodology/approach
A conceptual framework of a survey-based training needs assessment using the Internet was developed. A use case detailing a walkthrough of a prototype system developed under this proposed framework was created to illustrate how this system works. A two-phased operation test of the prototype system was then conducted to assess how well HR managers would accept and adopt this technology to develop organizational competencies. The first phase involved demonstrating the use case to a group of experienced HR managers and polling their opinion toward the system. The second phase involved one-on-one interviews with three select HR managers who were asked to experience the prototype system hands-on.
Findings
HR managers from both phases of validation demonstrated positive acceptance of both the needs assessment model and the process improvement generated from the Web-based prototype system.
Research limitations/implications
The qualitative validation test was conducted to a small sample ofTaiwan’sHR managers. Caution is advised when generalizing the positive results to other regions or countries with more advanced IT applications in HR practices.
Practical implications
This study contributes to the HR practice in several ways. First, it qualitatively confirms that HR professionals accept a competence-based approach to build company’straining curriculum. Second, it observes a lack of effective tools to help HR professionals in the task of training needs assessment. Third, it shows that HR professionals do recognize the power of Web technology in helping them become more efficient.
Originality/value
Formal competency-based training programs are rarely implemented in the real world because the process required is both tedious and time-consuming. This paper adopts Internet
technologies in a conceptual model for effective competency-based training needs assessment, and presents an efficient Web-based tool to assist HR professionals in the needed analysis.
Keywords Internet, Assessment, Training needs, Human resource management
Introduction
The notion of a resource-based perspective in a knowledge economy reinforces the idea of employees being the most important asset of an organization. Human resource departments in many companies are compelled to look for ways to increase human capital (Kessels, 2001). One viable way to increase a company’s human capital in order to gain competitive advantage is through carefully implemented HR practices such as training and development, performance evaluation and planned manpower changes (Schuler & MacMillan, 1984; Lado
& Wilson, 1994; Hwang, 1999). Training, especially, is an important strategic practice in the development of internal competence.
Training activities precede any knowledge management practices currently in fashion as the primary means to transfer programmed organizational knowledge and know-how to employees. However, unlike knowledge management practices, there seems to be a lack of efficient use or coordination in modern technologies in training related processes. Though we have seen an increase in the utilization of human resource management information systems, from a humble 7% in the seventies, to 68% in the eighties, and to a whooping 88% in the early nineties, the applications are still limited to the more traditional aspects of personnel management, such as compensation and hiring (Kinnie and Arthurs, 1993). Also, although e-learning is receiving much attention from academics and practitioners, the applications deal primarily with the delivery of learning contents. The process of deciding who should get training on what, i.e., the planning process, is still done manually today. Blanchard & Thacker (1999) highlighted the importance of structure and technology as the two internal factors which strongly affect an organization’s competitive position. To improve an organization’s competitive stand, we call attention to the use of the latest information technologies to support this strategic planning process called training needs assessment.
Swanson (1987) proposed a five-step technical training system model which includes analysis, design, development, implementation and evaluation. Needs assessment, which occurs during the initial planning and analysis of a training program, is the most vigorous and important step in the training and development process. The accuracy of the analysis determines the legitimacy and effectiveness of training program design and subsequent deliveries in the latter stages of the training process. There are various ways to determine
employees’developmental needs. The most commonly used method is a survey to the employees and their managers measuring the employees’skill levels. However, as with any survey research, this method in its traditional manner is extremely tedious and time consuming. From a resource-based view, this paper adopts Internet technologies in a conceptual model for effective training needs assessment, and presents an efficient Web-based tool to assist human resource development (HRD) professionals in the needed analysis. The proposed Web-based training needs assessment system was qualitatively tested by a select group of experienced human resource managers. The latter sections of this paper describe the testing process as well as managers’suggestions for improvement.
Theoretical Background
RBV and the Competency-Based Perspective
The resource-based view (RBV), which shifted organizations’strategic perspective from external competitive environment (Porter, 1980) to internal resources (Wernerfelt, 1984; Barney 1991), has become an important foundation for empirical work in strategy management and firm governance. In a review of the RBV literature, Priem & Butler (2001) discovered that “arguments have been advanced that single-business firms can achieve sustainable competitive advantage from such resources as information technology, strategic planning, organizational alignment, human resources management, trust, organizational culture,administrativeskills,top managementskills,and guanxi,among others.”Barney’s
organizing framework--that organizational resources that are valuable, rare, difficult to imitate and non-substitutable can yield sustained competitive advantage (Barney 1991)--has been the most cited RBV literature. The RBV framework implied the need for developing firm-specific capabilities and competencies in order to stay ahead, hence the core competence concept.
Rooted in the resource-based view, identifying and developing organizational core competencies is a viable approach to build competitive resources. Lado & Wilson (1994) defined organizational competencies as “firm-specific resources and capabilities that enable the organization to develop, choose, and implement value-enhancing strategies.”These resources and capabilities “include all firm-specific assets, knowledge, skills, and capabilities embedded in the organization’s structure, technology, processes, and interpersonal relationships.”They are often referred to as “core competencies”because they are basic to firm survival (Baghai et al., 1999), and critical elements for many companies to gain competitive advantage in their business (Kessels, 2001).
Companies can buy core competencies but, to maintain sustainable competitive advantage as the RBV suggested, it is better to develop them internally. Researchers have also attempted to suggest a list of common “core competencies”(Lado & Wilson, 1994; Baghai et al., 1999), but adopting the whole list would contradict the RBV notion of resource uniqueness. Hafeez et al. (2002) suggested a structural method using collectiveness, uniqueness and strategic flexibility as criteria to identify core competencies. Once identified, a systematic way to developing, maintaining or upgrading these core competencies is through competency-based training. (Lado & Wilson, 1994)
Training Needs Assessment
Training resources, like other resources in the organization, are limited. To utilize training resources to the best advantage, companies must put a fair amount of effort in training needs assessment. Gilley & Eggland (1989) defined need as “a gap between a current set of circumstances and some changed or desirable set of circumstances”, and needs assessment as the process of “measuring (as scientifically as possible) or appraising that gap.”
The word “circumstance” in this definition can be substituted with words such as “proficiency”(knowledge, skills, and attitudes), “performance”or “situation”.
Results of a needs assessment can be used in a lot of areas. They have been documented to “help curriculum planning, diagnose individual problems, assess student learning, demonstrate accountability, improve practice and safety, or offer individual feedback and educational intervention”(Grant, 2002). The most widely accepted application of needs
assessment is to ground training and development efforts to specific organizational objectives. Without a clear understanding of needs, organizations’training efforts may completely miss the mark resulting in a total waste of valuable resources. There is also psychological merit to doing needs assessment. Because the process involves employees and management, a well-executed needs assessment also increases credibility and support for the design of the subsequent training program among those who participated. (Gilley & Eggland, 1989)
In 1961, McGehee & Thayer laid the foundation for the core framework of needs assessment as consisting of three levels of analysis: organization analysis, operations analysis, and man analysis. Today, researchers and practitioners still abide by the three-level framework although terms may have changed over time--operations analysis is now task or work analysis and man analysis is individual or person analysis. (O’Connor et al., 1996, Holton III, Bates and Naquin, 2000) The most effective needs assessment should take into consideration all three levels of analysis starting from the organization level.
Methods of Needs Assessment
There are numerous reported “methods”for assessing needs in the literature. Gilley & Eggland (1989) believed that the ways, strategies, and schemes which were normally referred
to collectively as “methods”may aggregate to a total of 20 or 30. Grant (2002) diagramed 46 formal and informal methods of self assessment from The Good CPD Guide. Among those, Grant listed formal needs assessment methods often used to identify group needs, including critical incident techniques, gap analysis, objective knowledge and skills tests, observation, revalidation, self assessment, video assessment, and peer review. Gilley & Eggland (1989) also pointed out the six most useful methods in human resource development including interviews, questionnaires, tests, group problem analyses, records and report studies, and job analysis and performance reviews.
Although the literature generally reports only the more formal methods of needs assessment, HRD practitioners use a wide range of formal or informal ways of identifying training needs as part of their ordinary practice. Depends on the method used, the needs data collected can be categorized as “feltneeds(whatpeoplesay they need),expressed needs (expressed in action), normative needs (defined by experts), and comparative needs (group comparison).”(Grant, 2002) Among those, the “felt-needs”methodology, which usually asks
employees to simply list or rank desired training courses, has often been used to assess needs of large numbers of employees. (Holton III, Bates and Naquin, 2000) However, this approach is also most criticized for not being able to gauge true needs because employees usually don’t know what the organizational objectives are and often report “wants”instead of “needs”.
Gap analysis, though not commonly practiced in assessing training needs, is actually a widely used tool for assessing discrepancies between “what is”in reality and “what was intended to be”in many fields. (Kochhar et al., 1991; Bannister & Jesuthasan, 1997; Khan & Hafiz, 1999) It is a formal method to quantify discrepancies and has the added advantage of prioritizing assessed items to expedite action planning. When used in assessing
competency-based training needs, gap analysis “involvescomparing performancewith stated intended competencies by self assessment, peer assessment, or objective testing”.(Grant, 2002) Data collection for gap analysis is usually administered via a survey-type questionnaire to the employee, the employee’s manager, or other intended personnel.
Automated Needs Assessment Tools
There is a general lack of development and practice in technology-assisted training support. Guo (1989) advocated that though companies may adopt various training models or processes, they all need to establish a training information system as a reference for determining an effective training plan. However, a 1998 survey indicated that among regular human resource information system (HRIS) functions, adoption rate for employee training and development was merely 30%, which was only higher than the 25% adoption rate for employee relations and 16% for diversity programs (Greengard, 1999).
Advances in Web technologies in recent years provide a promising new avenue for the development of training support applications. Attributes such as instant communication and capability to send information back and forth without errors are two important advantages of incorporating Web technologies in training needs assessment. Meade (2000) emphasized that Web-based HRIS software provides self-service convenience to the employees and managers via Internet for mutual communication. Evans and Mathur (2005), in an article that discussed the role of Internet in survey research, presented 16 major strengths of online survey including global reach, flexibility, speed and timeliness, ease of data entry and analysis, and low administration cost. The only hurdle to collecting data via WWW is the design and development of Web pages, which can be very time-consuming. Luckily, this can easily be supported by questionnaire design tools (Tao and Yang, 2000).
A Framework for Competency-based Training Needs Assessment
The Framework
In light of the resource-based view and a competency-based perspective, Yeh (2000) developed a comprehensive needs assessment process for competency-based training. The process was developed from years of field experience and was validated by HR managers and HRD professionals across multiple industries (Yeh, 2000). This training needs assessment process was adopted as the framework of this research and the skeleton of the proposed prototype system.
The framework contains three stages of a competency-based training assessment process with specific output items signifying each stage: 1) competency identification stage, 2) gap identification stage, and 3) training curriculum stage. As seen in Figure 1, the first stage of the process starts with identifying the company’s vision, its mission statements, and business strategies, in order to flush out the company’s core competencies. These steps assure that core competencies will support company strategies in order to fulfill corporate vision and mission statements. Deploying from the identified competencies, in Figure 2, a task analysis is conducted to further identify required knowledge, skills and abilities/attitudes (KSA’s) of a corresponding competency. Then needs assessment is conducted to measure the gaps between “expected performance”and “actual performance”of an employee on each of the required
KSA’s. Finally, as shown in Figure 3, after ruling out non-trainable gaps, the HRD professional plots the training curriculum for each employee by filling each individual gap with a learning activity.
Corporate Vision Corporate Mission #1 Core Competency #2 Core Competency #3 Core Competency Business Strategies
Figure 1 Identify company core competencies
Needs Assessment
#1 Gap #2 Gap #3 Gap Task Analysis #1 Core Competency #2 Core Competency #3 Core Competency
Knowledge Skills Abilities
Figure 2 Perform task analysis and needs assessment
#1 Learning Activity
#1 Gap #2 Gap #3 Gap
#2 Learning Activity #3 Learning Activity Training Issue? Learning Curriculum Yes
Figure 3 Design the learning curriculum
System Description
program. This research converted the above conceptual model into an automated process with necessary activities for the HRD professional to accomplish the task of analyzing training needs. A supporting Web-based environment is then constructed to assist the HRD professional to design the training curriculum more efficiently. The functional structure chart as shown in Figure 4 illustrates the five modules for training needs assessment, which includes 1) employee data management, 2) core competence management, 3) survey questionnaire management, 4) needs assessment management, and 5) information management. Web-based Training Needs Assessment System Employee Data Management Core Competence Management Survey Questionnaire Management Needs Assessment Management Information Management Basic Data Management Training Record Management Competence Data Management Questionnaire Design Online Management Distribution Management Employee Analysis BBS Message Board Group Analysis
Figure 4 Functional Structure Chart
The employee data management module, which includes basic employee data (e.g., name, title, department, e-mail, etc.) and their training records, is regularly updated through routine HR management activities. The system can also retrieve these data from a subset of a larger HRIS. The core competence management module stores the most current competence data identified by the company using a competence identification process similar to what we described in Figure 1. Accompanying each competence is the associated KSA’s resulted from
a task analysis (as shown in Figure 2).
The survey questionnaire management module includes questionnaire design, online management and distribution management sub-modules. This is where the training needs assessment task is initiated and administered. The questionnaire design sub-module assists the HRD professionals generating a questionnaire template with questions matching the KSAs stored in the core competence management module. Once finalized, the questionnaire is posted online with the online questionnaire management sub-module. The HRD professional then selects and sends the appropriate employees and their managers an electronic notice and survey instruction through the distribution management sub-module which accesses employee contact information from the employee data management module.
Once the questionnaires are completed online, data is automatically collected and stored in a database. Using the employee analysis and group analysis sub-modules, the needs assessment management module accesses the survey data, calculates the competence gaps, reports these gaps for an individual employee or a group of employees, and recommends appropriate training. The information management module includes a BBS sub-module where HR professionals can use to disseminate or discuss training related information, and a message board sub-module to post policy announcements.
Potential Benefits of Process Improvements
Selander and Cross (1999) proposed a two-stage process redesign analysis, which includes process mapping and value analysis, to evaluate the improvements and cost-savings of the new process. This research adopted their ideas by first laying out the process of traditional/manual training needs assessment, identifying the tasks and errors that were not value added, and then improving them in the new process. The proposed Web-based prototype thus attempts to increase the effectiveness of the HRD professionals by relieving them from
some of the tedious, time-consuming, and repeated tasks in training needs assessment procedures, which include designing, printing, and mailing the training needs questionnaires; collecting returned questionnaires, coding and entering the data, calculating survey results, and matching employee needs to training classes (See Figure 5.) These tasks have no value-added portions and are converted to an automated process. With the proposed Web-based system the needs assessment tasks are minimized to a few key strokes. Once the needs questionnaire is finalized, all the HRD professionals have to do are posting the questionnaire, setting up the mailing list, scheduling the survey, selecting the mode for calculation, and matching intended training classes to needs questions; the automated system would then do the rest.
Figure 5 compares the procedure using the proposed prototype against that of the more traditional, manual method on the four phases of a needs assessment process (i.e., task analysis, data collection, data analysis, and training recommendation). The Web-based environment is designed to assist the HRD professionals to effectively perform training needs assessment with the aid of Internet technology. The proposed prototype does not replace the role of a training expert. That is, a HRD professional still needs to be skillful with the tasks of a training needs assessment, but the proposed Web-based information system will make the execution of the process faster, easier and with less data handling errors.
Data Collection Task Analysis K n o w le d g e T ra d it io n a l/ M a n u a l P ro ce ss Questionnaire
Data Analysis RecommendationTraining
S k il ls A tt it u d e P ro to ty p e S y st e m P ro ce ss D es ig n Q u es ti o n n a ir e D es ig n P ri n t D is tr ib u te C o m p le te R et u r n D is tr ib u te th r o u g h e -m a il C o m p le te o n -l in e A n a ly ze d a ta O b ta in g a p d a ta R ec o m m e n d tr a in in g R e p o rt R e p o rt R ec o m m e n d tr a in in g M a tc h g a p s w it h tr a in in g O b ta in g a p d a ta C o d e d a ta E n te r d a ta . E x ec u te st a ts p a c k a g e. A n a ly ze d a ta .
Figure 5 A comparison of a manual needs assessment process and the prototype system process
The improvement of the needs assessments process can be clearly seen from the graphical comparison of a manual-based and the Web-based approach in Figure 5. The main aspect of the claimed effectiveness using our Web-based prototype comes from the time saved for performing some of the tedious, time-consuming, and repeated tasks in training needs assessment processes. Taking a company of 120 employees for example, a conservative estimation of the time saved could be 17 man-hours, which could mean several days or even weeks of time span in real-life business operations due to the discontinuous process nature (Tao et al., 2002). The direct impact on the HRD professionals is that they can be more effective by focusing only on their core tasks, which in return to the business organization are a higher satisfaction level and a low retention rate of HRD professionals.
This simple process improvement can also bring out many potential benefits that were not considered before. First of all, training needs assessment is usually stereotyped as a periodic task in training and development, done once a year for example. With a Web-based tool, training needs assessment can become a more common task which is especially helpful
to those business organizations with high employee turnover rates. Secondly, because many business organizations have adopted or plan to adopt e-learning, the complete corporate training cycle can be integrated from the Web-based training needs assessment to e-training course delivery. This creates an opportunity for do-it-yourself training program by any employee at all levels. Thirdly, training needs assessment can actually be used in new employee selection for desirable competence of any specific job title. This not only expands the use of this Web-based needs assessment tool into another HRD function, but also further improves the overall HRD processes from hiring, selection to training by acquiring employee competence data from the very beginning of the hiring process.
In theory, all these benefits can be realized by any business organization, with the emphasis on maximizing the overall organizational competencies using the Internet.
The Web-Based Prototype System
To better illustrate how the proposed framework works, a use case is provided below with screen shots captured during a walkthrough of the prototype system. The prototype system was implemented using Microsoft IIS Web server, ASP Web programming language and Access 2000 database.
A Use Case
A use case is illustrated via a use case diagram complied to the format of Unified Modeling Language (Satzinger et al., 2002), as seen in Figure 6. This use case diagram helps to define the system scope by identifying the actors that interact with the system and the functions provided to those actors.
Employee Data Management
Core Competence Management
Survey Questionnaire Management
Needs Assessment Management
Create new employees
Maintain personal data and training records Training staff
Enter new skills or competencies Maintain skill and
competence data Training staff
Create a new questionnaire with groups of items
Setup and schedule for online Setup and deliver questionnaires to target employees Training staff
Map questionnaire items with corresponding skills Execute the gap analysis at
employee level Training staff
Employees Fill out online questionnaires
Automated system boundar y
Figure 6 A use case diagram for the operational testing
In Figure 6, the automated system boundary represents the interface between the system and the two actors in this case: the training staff and the employees. Within the automated system boundary, the first four management modules of the Web-based training needs assessment system are embedded. Most of these functional modules are used by the training staff in the HR department except for the employee interface to fill out the online questionnaire. The use case described below complements the conceptual walkthrough of the system functionalities in previous paragraphs related to Figure 4.
Walkthrough of the Use Case
competency needs of the information system (IS) staff in a large integrated circuit testing corporation in Southern Taiwan. We adopted a generic core competency model from a study sponsored by the Ministry of Economic Affairs in Taiwan on IS staff in the small and medium enterprises (Yeh, 2003). In this report, three core competencies were identified, including professional knowledge and skills, professional attitudes and traits, and interpersonal skills. Under these three competencies, sixteen skills and eighty-two behavioral indicators were identified from the literature, focus group interviews with subject matter experts, and surveys to academia and industry. Normally, the behavioral indicators are used to develop questionnaire items in a competence-based needs survey. However, to make our illustration concise, we use the sixteen skills instead as the questionnaire items. A series of screen shots of the prototype system are presented with brief explanations below.
The screen layout, as can be seen in Figure 7, shows the main hierarchical function menu on the left side of the screen. When any of the first-level modules is clicked, a tree structure expands to display additional sub-menu selections. In this case, “Basic Data”and “Training Record”functions are shown under “Employee Data Management”module. Meanwhile, in the center of the screen under the system banner, a third-level sub-menu of the second-level function of “Employee Basic Data”shows up. Beneath that is the main user interaction area. All the important database tables in the prototype system have insert, delete, update, and query functions which are shown in Figure 7 as the third-level sub-menu under the banner.
Figure 7 Screen shot of “updating an employee’s basic data”
Figure 7 displays the first task of the training staff creating a new data entry for an employee. The training staff can also maintain existing employee basic data and training records using the update, delete and query functions which work similarly to most Web-based systems.
The core competence management module is where the training staff stores and manages corporate competence data. Figure 8 demonstrates the operation of inserting a new skill. In this case, a user can input a skill name corresponding to one of the pre-specified core competence options from a pull-down menu. If there is no appropriate category of core competence, the user can add a new core competence first using the “Insert new competence”
option on the right hand side of the pull-down menu.
Figure 8 Screen shot of “inserting a new skill”
Via a Web-based questionnaire design aid, the training staff uses the questionnaire design management (QDM) module to design appropriate data collection instruments for the needs assessment. There are three parts in QDM module, including design, online schedule and distribution setup. Figure 9 shows a list of existing questionnaires with an “Update” button on the right-hand-side of each questionnaire, and another option at the bottom of the list to insert a new questionnaire. When the “Update”button is clicked, the complete questionnaire shows up (as seen in Figure 10). Figure 10 is similar to what an employee would see when filling out the questionnaire.
Figure 9 Screen shot of “listing questionnaires for update and insert options”
When adding a new questionnaire, users are prompted to enter questionnaire topic (e.g., Core Competence of Information System Staff), item group name (e.g., Information Professional Knowledge and Skills) and type of question item (e.g., Likert scale single choice), as shown in Figure 11. This system employs a special design to make Likert-type scale development easier. When a question is entered, for example “I have information system customization capability”, the user is prompted to select three parameters—an adverb, a verb and the number of scale points (usually 3, 5, or 7), as seen in Figure 12—to automatically generate a Likert-type question. In the case the user cannot find a suitable adverb or verb in the selection menu, a hyperlink next to the pull-down menu is available to add new ones. When all the parameters are entered as required, the question and answers of the Likert-type question item will appear (see Figure 13) for user confirmation before they are inserted into the database.
Figure 12 Screen shot of “inserting a new question item”
Figure 13 Screen shot of “confirming a questionnaire item”
The function of online schedule management comes into play after the questionnaire development is ready for administration. The online schedule management module allows the user to rearrange groups of questionnaire items, if needed, and to schedule online availability. With appropriate setup, the prototype system can then automatically generate the access URL of the questionnaire, and initiate the questionnaire linkage to administer the survey to target employees. The schedule setup screen, as seen in Figure 14, allows the
user to select the online period, currently available in 7, 14, or 21 days, and the offline option. Current online/offline status and days remaining if currently online are displayed for every questionnaire in this screen. The setup button in the last column allows the user to switch the survey online or offline.
Figure 14 Screen shot of “scheduling online survey”
After the questionnaire is developed and before it goes online, the target employees need to be informed of the questionnaire URL. Figure 15 shows the questionnaire distribution menu which allows the user to specify who will receive notification of the survey. As the prototype system adopts a gap analysis approach to uncover an employee’s training needs, it is designed to collect the assessment of both the current skill level and the expected skill level. Current skill level data is obtained through employee’s self assessment, while expected skill level data can be collected from the employee or the employee’s manager. As shown in Figure 15, the “expectation by”column for each questionnaire has three choices--the employee, the manager, or none. The administration mode specifies whether this survey will be done by selected employees, by department, or
by all employees. The Setup button confirms and stores the parameter settings.
Figure 15 Screen shot of “distributing the questionnaire”
The distribution mode option in the fourth column of Figure 15 will take the training staff to a popup screen to select employees to be surveyed, as can be seen in Figure 16. The system also provides an automatic follow-up mailing function after the questionnaires are sent. Users can specify the number of follow-ups to be sent in the last column in Figure 16.
Figure 16 Screen shot of “email setup function”
The needs assessment management module manages the calculation of gaps between current and expected skill level, and then maps a gap to a prescribed course. The mapping relationship between a questionnaire item and a competence can be easily specified via a pull-down menu of company competencies, which is shown in Figure 17. Figure 18 shows the result of the gap analysis with suggested courses as needed.
Figure 17 Screen shot of “mapping questionnaire items with competencies”
Operation Testing of the Web-Based System
We tested the general acceptance of the proposed framework through a two-phased operation test.
Phase One—Group Demonstration and Evaluation
The purpose of this phase-one evaluation was to test the operations of the competency-based needs assessment model and the prototype system from a practical standpoint. We looked for experienced HR practitioners as our study sample because they were most qualified to judge whether the system was valid to use in the field as a needs assessment tool. We carefully selected a class of experienced managers who were registered in a MIS (Management of Information Systems) course offered through the HRM program in a major university in Taiwan. These managers were studying for their master’s degree in a graduate program designed specifically for experienced human resource managers and professionals. We conducted a Q&A session toward the end of the MIS course to ensure that the study sample had adequate knowledge of the use of IS in human resource management. Thirty-one managers participated in the study, and most of whom had experiences in training related jobs. The complete process of a needs assessment using the prototype system was demonstrated in the MIS class. The demonstration took 40 minutes followed by a 20-minute Q&A. Then the class was asked to fill out a questionnaire. The questionnaire contains seven questions using a Likert-type scale of 1 to 5 (1 means strongly disagree and 5 means strongly agree).
Table 4 presents the results of the descriptive statistics of mean and standard deviation (S.D.). Respondents showed positive reactions toward each of the seven measures. The concept of a competency-based training needs assessment received most favorable ratings, followed by the use of gap analysis to assess these needs. Respondents were also impressed with the ability of the system to shorten operation time required of a competency-based
training needs assessment.
Table 4 Results of Validity Measures
Questionnaire Items Mean* S.D.
1. Accept the concept of a competency-based training needs assessment
4.23 0.88
2. Accept the concept of using gap analysis to assess training needs 3.84 0.90 3. The system effectively assist the planning phase of a
competency-based training
3.32 0.94 4. The system simplifies the process of a competency-based training
needs assessment
3.48 0.93 5. The system shortens operation time required to conduct a
competency-based training needs assessment
3.71 0.97
6. The system effectively expedites the transmission of training needs assessment information
3.65 0.91 7. The system makes analysis results of employees’competence
gaps more accessible to HRD professionals
3.48 1.06
*Average score ranging from 1 strongly disagree to 5 strongly agree
Phase Two—Individual Tryout Sessions
The purpose of this phase-two evaluation is to test the functionality and usability from a more realistic operation of the prototype system. To this end, we conducted one-on-one interviews with three training and development (T&D) managers after they had experienced the operation of the prototype system hands-on.
The three T&D managers were from different industry backgrounds. Manager A was a T&D manager for a major department store and had 8 years of working experience in HR-related functions. Manager B worked as a T&D supervisor for a government agency for more than 10 years. Manager C was in charge of training for an area hospital and had 3 years of HR-related experience. Each manager received a 10-minute introduction of the competency-based training needs assessment model, then worked through each of the five modules of the prototype system to complete a hypothetical needs assessment process like the
use case. The hands-on experience took 15 to 20 minutes, followed by a 30-minute semi-structured interview. Interview questions included the following: (See table 7 for a complete list of interview questions.)
1.
Effectiveness, time-saving and accuracy of the questionnaire management module2.
Effectiveness and accuracy of the needs assessment module to calculate competencegaps
3.
Whether the prototype system was easy to follow, straight-forward and user-friendly4.
Consistency of layout, appropriate size and amount of texts, appropriate use of graphics5.
Content accuracy, flow of information and legibility of the prototype systemIn general, all three managers agreed that the Web-based prototype system provides a good model to assess employees’training needs, reinforces the concept of competency-based training, and simplifies the analysis stage of a planning process for training. While the training managers were positive about the effectiveness and the process improvement prospects of the system, they also stressed the importance of customization to realize these benefits. The extent of customization included company organization and communication channel, a company-wise competency databank, training curriculum, process linkage to training course registration, and administration, etc. Concerns of employees mistaking a training needs analysis to performance evaluation were raised. Though not a system problem, one of the managers-Manager C-suggested adding this message to the survey: “For development purpose only, and will not be used in performance evaluation”to minimize the possibility of employees sending falsified information.
In terms of the functionality of the system, the managers suggested the following improvements:
Provide multiple methods for data analysis or allow exporting of data to other
Send an automatic confirmation when the survey is received by the intended
employee.
Provide better tracking mechanism, such as status on send, reply and follow-up. Provide better navigation functions, such as backtrack, system maps and on-line
instructions.
Provide options to setup the order of how respondents fill out the question items. Remind respondents of missing answers.
Table 7 Interview Findings
Interview Questions Results Improvement Suggestions
1. Do you agree with the concepts imbedded in the prototype system?
Positive
2. Will the system facilitate the use of
competence-based training needs assessment in your company?
Positive - Customization is required prior to implementation.
3. Will the system help save time in planning employee training?
Positive
4. Will the system simplify the assessment process for employee training?
Positive - Provide choices of analytical tools
5. Is the system effective in assisting a
competency-based training needs assessment?
Positive
6. Effectiveness of questionnaire management module: Design? Dissemination? Collection?
Positive - Automatic confirmation when message is received by the receiver. - Provide stats on send, reply and
follow-up status. 7. Effectiveness of needs assessment module in
quantifying employees’competence gaps?
Positive - Stress “the survey is for
developmental purpose only, and will not be used in performance evaluation”
8. Is the prototype system easy to follow and user-friendly?
Positive - Provide system maps and on-line instructions.
9. Aesthetic appeals of the screen display:
Consistency? Appropriate placement and size of text and graphic?
Positive
10. Legibility, accuracy and flow of contents in the system?
Positive - Provide option to setup the order of how respondents fill out the questions.
- Remind respondents of missing answers.
Implications and Conclusion
Conclusion
We have presented an integrated framework of a Web-based training needs assessment system to effectively and efficiently assist organizations in their pursuit of competitive core competencies. Neither the competence-based training approach nor the needs assessment method is new to seasoned HR professionals. The method of using survey questionnaires to uncover skill gaps in the organization has been around for decades. Competence-based theories, or more specifically, building organization core competencies for sustained competitive advantage, have been the topic for discussion since the 80’s. But these concepts are rarely implemented in the real world (Selmer, 2000) because the process required is both tedious, and time-consuming. (Gilley & Eggland, 1989; Goldstein, 1993; Guo, 1987)
This study contributes to the HR practice in several ways. First, it qualitatively confirms that HRD professionals accept a competence-based approach to build company’s training curriculum. Second, it observes a lack of effective tools to help HRD professionals in the task of training needs assessment. Third, it shows that HRD professionals do recognize the power of Web technology in helping them become more efficient. Lastly, A Web-based training
needs assessment system for competence-based training, like the one designed for this study, is well-received by the HRD professionals. These claims are further evidenced by several eager HR managers who came up to the researchers after the system demonstration of first-phase evaluation to express interest in utilizing a complete system, based on knowledge of the prototype. Among them, one is the head of HR department in a large regional hospital. We have donated this prototype system for their internal use with our authorization to change the program codes in order to integrate with their internal systems.
Implications and Future Work
Needs assessment for general purpose. Because of its versatility, the proposed
prototype system can be used with any kind of survey instruments to determine deficiencies of any subject under investigation (e.g., cross-cultural understandings, technical skills, product knowledge, etc). It is designed to efficiently manage organization-wide as well as departmental and group assessments. It also has a high potential of serving as a testing tool for selection. Just enter any test items which can be answered with a Likert-type scale and pre-program the desired answers in the “expected level”. The system will automatically calculate the gap between a job candidate’s test score with the desired answer for each test item. The application is not limited within the HR functions. Using similar approach, the system can also assist in analyzing knowledge deficiencies in an organization for knowledge management purpose.
Additional Usages of the Assessment Results. First of all, the aggregated result also
serves as an indication of the firm’s overall competence level, a clear abstraction of the firm’s strengths and weaknesses. Secondly, quantifiable and concrete competence data from the assessment can help managers establish credibility in the performance evaluation and feedback process. Thirdly, when conducted periodically, the competence assessment allows tracking of individual employees’progress in terms of competence development, so both the
firm and the employees are better informed when making employee career decisions. Finally, the result can be used in the selection process to search for candidates who will complement the existing work force in those weaker areas.
The goal of this research is to demonstrate how Web-based applications improve the effectiveness of a very important HR function. Future work includes the enhancement of the conceptual model and the prototype structure with various activities commonly seen in a practical training process, and an expansion of the prototype to include the upstream of hiring and the downstream of evaluation related to human resource management.
References
Baghai, M.A., Coley, S.C., & White, D. (1999), “Turning capabilities into advantages”, The
McKinsey Quarterly, Vol. 1, pp. 100–109.
Bannister, R.J. & Jesuthasan, R. (1997), “Is your company ready for value-based management?”The Journal of Business Strategy, Vol. 18, No. 2, pp. 12-15.
Barney, J. B. (1991), “Firm resources and sustained competitive advantage”, Journal of
Management, Vol. 17, pp. 99-120.
Blanchard, P. N. & Thacker, J. W. (1999), Effective Training: Systems, Strategies, and
Practices, Prentice Hall.
Evans, J.R. & Mathur, A. (2005), “The Value of Online Surveys”, Internet Research, Vol 15, No. 2, pp 195-219.
Gilley, J. W. & Eggland, S. A. (1989), Principles of Human Resource Development, Addison Wesley.
Goldstein, I. L. (1993), Training in Organizations: Needs Assessment, Development, and
Evaluation. Monterey, CA: Brooks-Cole.
Vol. 324, No. 7330, pp. 156-159.
Greengard, S. (1999), “How to fulfill technology’s promise”, Workforce and Software, February, pp. 10-18.
Guo, F. (1987), How to Conduct a Good Educational Training. Taipei: Management-Expansion Management Consulting and Publishing Company.
Hafeez, K., Zhang, Y. B., & Malak, N. (2002), “Core competence for sustainable competitive advantage: A structured methodology for identifying core competence”, IEEE
Transactions on Engineering Management, Vol. 49, No. 1, pp. 28-35.
Hennman, R.L., (1999), “Design for usability: process, skills and tools”, Information
Knowledge, System Management, Vol. 1, pp.133-144.
Holton III, E., Bates, R. & Naquin, S. (2000), “Large-scale performance-driven training needs assessment: A case study”, Public Personnel Management; Vol. 29, No. 2, pp. 249-267. Hwang, I. (1999), “Effective training approach for corporate human development”, Metal
Industries, Vol. 33, No. 6, pp. 116-123.
Kessels, J. W. M. (2001), “Learning in organizations: a corporate curriculum for the knowledge economy”, Future, Vol. 33, pp. 497-509.
Khan, M.K. & Hafiz, N. (1999), “Development of an expert system for implementation of ISO 9000 information systems”, Total Quality Management, Vol. 10, No. 1, pp.47-59. Kinnie, N. & Arthurs, A. (1993), “Will personnel people ever to love the computer?”
Personnel Management, Vol. 25, No. 6, pp. 46-51.
Kochhar, A.K. Suri, A.I. & Hether, R. (1991), “Design and implementation of a general purpose knowledge gap analysis system with particular reference to the implementation of material requirement system”, in Proceeding of the Mechanical Engineers, Part C, pp.129-133.
advantage: A competency-based perspective”, Academy of Management Review, Vol. 19, No. 4, pp. 699-727.
McGehee, W. & Thayer, P.W. (1961), Training in Business and Industry. New York: Wiley. Meade, J. (2000), “Web-based HRIS meets multiple needs”, HR Magazine, August, pp.
129-133.
O’Connor, B. N., Bronner, M. & Delaney, C. (1996), Training for Organizations, South-Western Educational Publishing, p.7.
Pettit, F. A. (1999), “Explore the use of World Wide Web as a psychology as a data collection tool”, Computers in Human Behaviors, Vol. 15, pp. 67-71.
Porter, M. E. (1980), Competitive strategy. New York: Free Press.
Pressman, R.S. (1992), Software Engineering: A Practitioner’s Approach, McGraw-Hill Inc.
Priem, R.L. & Butler, J.E. (2001), “Is the resource-based ‘view’a useful perspective for strategic management research? ”The Academy of Management Review. Vol, 26, No. 1, pp. 22-40.
Satzinger, J. W., Jackson, R. B. and Burd, S. D. (2002), System Analysis and Design in a Changing World, 2ndEdition, Course Technology.
Schuler, R.S. & MacMillan, I.C. (1984), “Gaining competitive advantage through human resource management practices”, Human Resource Management, Vol. 23, No. 3, pp. 241-255.
Selander, J. P. & Cross, K. F. (1999), “Process redesign: is it worth it?”Management
Accounting, Vol. 80, No, 7, pp. 40-44.
Selmer, J. (2000), “A quantitative needs assessment technique for cross-cultural work adjustment training”, Human Resource Development Quarterly, Vol, 11, No. 3, pp. 269-281.
training problems in industry and business”, Journal of Industrial Teacher Education, Vol. 24, pp. 7-17.
Tao, Y. & Yang, G.. (2000), “A design of a Web-based survey system”, International
Conference of Overlook New Century, Ming-Chuang University, Taoyuan, Taiwan,
March 23.
Tao, Y.-H., Sun, S. & Yeh, R. (2002), “Efficient Web-based planning support for training”, The
Second International Conference on Electronic Business, Taipei, Taiwan, Dec. 10-13.
Wernerfelt, B. (1984). “A resource-based view of the firm”, Strategic Management Journal, Vol. 5, pp. 171-180.
Yeh, C. R. (2000), “Planning in training and development”, Conference Series, Formosa
Society of Training and Development, Kaohsiung, Taiwan.
Yeh, C. R. (2003), The Design and Development of a Competence Model for Information
Department Staff in Small and Medium Enterprises. A project report for the “Office Automation Training and Web-Based Learning for Small and Medium Enterprises Project”, Ministry of Economic Affairs, Taiwan, R.O.C. (In Chinese)
