Information commitments: evaluative standards
and information searching strategies in
web-based learning environments
Ying-Tien Wu*w & Chin-Chung Tsai*
Institute of Education, National Chiao Tung University, Hsinchu, Taiwan wDepartment of Earth Sciences, National Taiwan Normal University, Taipei, Taiwan
Abstract ‘Information commitments’ include both a set of evaluative standards that Web users utilize to assess the accuracy and usefulness of information in Web-based learning environments (implicit component), and the information searching strategies that Web users use on the Internet (explicit component). An ‘Information Commitment Survey’ (ICS), consisting of aforementioned components, was developed. The participants of this study were 610 uni-versity students, coming from four universities in Taiwan, and their information commit-ments were surveyed. Through a series of structural equation modelling (SEM) analyses with Linear Structure RELationships (LISREL), this study confirmed the reliability and validity of ICS. More importantly, the causal relationships between the two components of ICS were also examined. The findings showed that learners’ evaluative standards on Web materials (implicit component) had significant effects on their information searching strategies in Web-based learning environments (explicit component). For example, learners who utilized an advanced evaluative standard, such as using multiple sources to judge the accuracy of Web materials, were significantly more oriented toward using a sophisticated information searching strategy, such as carefully exploring and elaborating information in Web-based learning environments. In other words, learners’ evaluative standards for Web materials should be viewed as important predictors for their searching strategies in Web-based learning environments.
Keywords evaluative standards, information commitments, information searching strategy, LISREL analysis, questionnaire, Web-based learning
Introduction
Nowadays, the Internet is increasingly being used for educational purposes, and it has made a considerable impact on educational practice (Jonassen et al. 1999; Knezek & Christensen 2002). Many researchers have
also proposed that the Web-based instruction, con-forming to the constructivist perspectives on teaching and learning, can not only provide learners with dis-tant, interactive, broad, individualized and inquiry-oriented learning activities but also promote their knowledge construction and meaningful learning (e.g., Relan & Gillani 1997; Tsai 2001a; Chou & Tsai 2002). At all levels of schools, the Internet has been gradually utilized by teachers in their teaching practice to im-prove students’ learning outcomes (e.g., Kadijevich 2002; Lin et al. 2002). Moreover, it has also been
Correspondence: Prof. Chin-Chung Tsai, Institute of Education & Center for Teacher Education, National Chiao Tung University (Taiwan), 1001 Ta Hsueh Rd., Hsinchu 300, Taiwan. E-mail: cctsai@mail.nctu.edu.tw
pointed out that students’ usage of the Web as a main information resource is growing overall (e.g., Kuech-ler 1999; McDowell 2002; Jean-Francois 2003).
When students search for resources and information on the Internet to complete some learning tasks, their usage of the Internet, at least, involves two parts: the first part is the search of Web information, while the second part is the utilization or application of in-formation that they have searched on the Internet. In the process of information searching (the first part), learners may use various searching strategies, guiding their searching behaviours, to obtain information that they need on the Internet. Moreover, before using the information they have searched (the second part), they may evaluate the accuracy and usefulness of the Web-based materials they have searched.
While learners are increasingly taking advantages of the Internet or Web in their learning, some situations are especially worth gaining attention from educators. For example, many learners may lack proper searching strategies to guide their information-searching beha-viours efficiently when seeking information on the Internet. And, they may depend on the Internet to provide accurate information without carefully ensur-ing the accuracy of the information they obtain (Metzger et al. 2003). However, studies about the nature of students’ Web use have not kept pace with their usage of the Internet (Metzger et al. 2003).
As learners’ activities in Web-based learning en-vironments often involve information-searching tasks, learners’ information-searching processes on the In-ternet become relatively a newer and more important research issue for researchers interested in the nature of students’ Web use. In these tasks, they may utilize different searching strategies to guide their searching behaviours on the Internet. Consequently, they may have different searching results, which can be viewed as an important indicator of their performances and learning outcomes derived from Web-based learning environments (Lin & Tsai 2005). In other words, learners’ information searching strategies on the In-ternet should be one of the important factors influen-cing their performances on these tasks. During the last decade, some studies have been conducted to in-vestigate learners’ searching strategies on the Internet (e.g., Hill 1999; Beaufils 2000; Tsai & Tsai 2003). However, still not much is known about the nature of learners’ information-searching strategies on the
In-ternet. Therefore, more studies should also be con-ducted to investigate learners’ information searching strategies in Web-based learning environments.
Moreover, unlike other filtered information that has been assessed by experts (e.g., books and newspapers), most Internet users have to evaluate the Internet in-formation by themselves (Brandt 1996). However, it has been found that while learners appreciate the usage of the Internet to provide them with several information resources, the quality of these resources is not necessarily important to them (Metzger et al. 2003). In other words, learners may rely heavily on the Internet for gathering information despite the evidence that it is potentially inaccurate and biased (Brandt 1996; Flanagin & Metzger 2000). Recently, evaluation of information on the Internet has been recognized as an important issue, and some related studies have been conducted (e.g., Brandt 1996; Flanagin & Metzger 2000). Among these studies, still little research, especially addressing how students evaluate the ac-curacy and usefulness of Web-based learning materi-als, has been conducted. Therefore, more studies should be undertaken to explore students’ views about the accuracy and usefulness of information on the Internet.
In sum, the information-searching strategies in which learners use in Web-based learning environ-ments and how they evaluate Web-based information have become two essential issues for educators. Therefore, the present study was conducted to address these.
Previous research
Recently, learners’ epistemological beliefs have re-ceived more attention from educators, particularly those in science education. The ‘epistemological commitments’ include a set of evaluative standards used to judge the merits of knowledge (such as its generalizability, internal consistency and parsimony), and these commitments will guide the individual learner’s process and outcomes of knowledge con-struction (Hewson 1985; Tsai 2001b, 2002). Similar to the conception of epistemological commitments, Tsai (2004a) has defined the idea of ‘information commit-ments’ for Web learners to further elaborate the aforementioned important research issues. ‘Informa-tion commitments’ include both a set of evaluative
standards in that Web users utilize to assess the ac-curacy and usefulness of information in Web-based learning environments, and the information-searching strategies that Web users use on the Internet. Ac-cording to Tsai’s (2004a) perspectives, different lear-ners may use varying evaluative standards for assessing the information in Web-based learning en-vironments, and these evaluative standards are likely guiding learners’ processes and outcomes of knowl-edge construction in Web-based learning environ-ments. Tsai (2004a) also pointed out that it was plausible to assume that different evaluative standards would lead to different types of learners’ information-searching strategies on the Web. Moreover, learners’ evaluative standards for Web materials are implicit, while their information-searching strategies on the Web are explicit. Therefore, learners’ information commitments include two components: one consists of evaluative standards for Web materials (the implicit component), while the other contains learners’ in-formation searching strategies on the Web (the explicit component).
Tsai (2004a) also proposed a theoretical framework for describing Web users’ information commitments, including the following three aspects (for further de-scription of this theoretical framework, please refer to Tsai 2004a):
(1) Standards for accuracy: The standards in which learners utilize to evaluate the accuracy of Web in-formation. The possible orientations are ‘multiple sources’ versus ‘authority’. Some Web users usually use ‘multiple sources’, such as other websites, prior knowledge, peers or other printed materials, to ex-amine the accuracy of Web information, while others use the ‘authority’ of the website as a major criterion for the accuracy.
(2) Standards for usefulness: The standards in that learners use to assess the usefulness of Web in-formation. The possible orientations are ‘content’ versus ‘technical (functional)’. The ‘content’ indicates that the learners use the relevance of Web content as the major standard for evaluating its usefulness, while the ‘technical (functional)’ indicates that the learners perceive the technical and functional issues of the Web (e.g., the ease of retrieval, the ease of search or the ease of providing rich information) as major standards for evaluating its usefulness.
(3) Searching strategy: The searching strategies that Web users use to seek Web information. The possible orientations are ‘elaboration and exploration’ versus ‘match’. The ‘elaboration and exploration’ indicates that learners may have purposeful thinking when na-vigating in the Web environments, and try to integrate Web information from several websites to find the best fit that fulfils their purpose, while the ‘match’ indicates that learners may be eager to match their searching purposes by finding only a few websites that contain the most fruitful and relevant information.
Among these three aspects, the first two are evaluative standards, categorized as the ‘implicit component’, and the third one is the information-searching strategy utilized by Web users, classified as the ‘explicit component’. Tsai (2004a) also concluded that the three information commitments, labelled as ‘multiple sources’, ‘content’ and ‘elaboration and exploration’, commonly expressed by experts, were advanced in-formation commitments, while the others were per-ceived as less sophisticated.
On the basis of Tsai’s (2004a) theoretical framework for describing Web users’ information commitments, Wu and Tsai (2005) have developed an Information Commitment Survey (ICS) to survey university stu-dents’ information commitments in Web-based learn-ing environments. To state more clearly, as shown in Fig. 1, there are two components involved in the ICS developed by Wu and Tsai (2005): one was the implicit component, and the other was the explicit component. The implicit component assessed the implicit standards that learners utilize to judge the accuracy and useful-ness of Web information (such as multiple sources of the website and the relevance of Web content), while the explicit component explored how Web users per-ceived their explicit action in Web-based learning en-vironments (i.e., their possible orientations for utilizing different searching strategies on the Internet).
However, in their follow-up study, Wu and Tsai (2005) found that university students may likely uti-lize both the information commitments, ‘multiple sources’ and ‘authority’, at the same time when evaluating the accuracy of the materials on the Web, and they also probably utilized both the information commitments, ‘content’ and ‘technical issues’, to evaluate the usefulness of the materials on the Web. Therefore, Wu and Tsai (2005) proposed that many
learners expressed mixed standards of evaluating the accuracy or the usefulness of Web materials. In other words, Tsai’s (2004a) original framework, possibly suggesting that ‘multiple sources’ and ‘authority’ are opposite orientations, and and that ‘content’ ‘technical (functional)’ are reverse orientations, needs some modifications.
Purposes of the study
As already mentioned, the present study was con-ducted to address two important issues in Web-based learning environments (that is, learners’ evaluative standards for Web materials and their information-searching strategies). And, it is clear that an explora-tion of learners’ informaexplora-tion commitments concurs with these issues. In their previous study, Wu and Tsai (2005) had developed an ICS to assess university students’ information commitments in Web-based learning environments. However, they examined the validity and reliability of ICS only by exploratory
factor analyses (EFA). Therefore, one of the major purposes of this study was to further confirm the va-lidity and reliability of ICS.
In addition, the relationships between two compo-nents of students’ information commitments (i.e., the implicit component and the explicit component) are also worth investigating. Some previous studies may have been conducted to explore learners’ information-searching strategies (e.g., Hill 1999; Tsai & Tsai 2003). However, the possible variables influencing learners’ information-searching strategies have not been investigated in these studies. Moreover, Wu and Tsai (2005) have also suggested that further studies be conducted to examine carefully the interplay between learners’ evaluative standards on Web materials and their online searching strategies. Therefore, another major purpose of the current study was to examine the possible interplay between learners’ evaluative stan-dards on Web materials and their searching strategies on the Internet. This study asserted that students’ evaluative standards regarding the accuracy and the
includes sources includes Information Commitments assessed by ICS
ICS by confirmatory factor analysis?
Q1: How are the reliability and validity of
Implicit Component Explicit Component
Q2: What are the
relationships between these two components?
includes scales of Standards for accuracy Standards for usefulness
Elaboration includes scales of
includes scales of
Authority Multiple
Content Technical Match
usefulness of Web materials might influence the in-formation-searching strategies they used in Web-based learning environments. For instance, students who utilize multiple sources and the relevance of Web content to evaluate the accuracy and usefulness of Web-based material may tend to use the ‘exploration and elaboration’ Web-seeking strategy, while those who perceive the authority of the website and the technical issues of Web as essential standards for the accuracy and usefulness of Web-based material may tend to use the ‘Match’ searching strategy.
The present study would address the above issue by using structural equation modelling (SEM) techniques. The use of SEM techniques in the social science is rapidly increasing. One of the important uses of the SEM techniques is that it can provide a flexible and powerful means of simultaneously assessing the quality of measurement and examining predictive re-lationships among constructs (Kelloway 1998).
Moreover, unlike exploratory factor analysis (EFA), which is guided by intuitive and ad hoc rules, SEM casts factor analysis in the tradition of hypothesis testing, with explicit tests of both the overall quality of the factor solution and specific parameters (e.g., factor loadings) composing the model (Kelloway 1998). In other words, confirmatory factor analysis (CFA), an application of SEM, is more rigorous than the tech-niques of exploratory factor analyses. Therefore, CFA was used in this study to confirm the validity and re-liability of ICS.
Research questions
On the basis of the ICS developed by Wu and Tsai (2005), the research questions of this study were as follows:
1. Through CFA, is the ICS developed by Wu and Tsai (2005) sufficiently valid and reliable for as-sessing university students’ information commit-ments in Web-based learning environcommit-ments? 2. By SEM analysis, do university students’
evalua-tive standards utilized to assess the accuracy and the usefulness of Web materials have effects on their searching strategies on the Internet?
These research questions and relevant conceptual framework are illustrated as Fig. 1.
Method Sample
The participants of this study included 610 volunteer university students with Internet experiences (includ-ing 395 college students and 215 graduate students), coming from four well-known national (science- and technology-oriented) universities. Many of them ma-jored in science or engineering, so the sample of this study consisted of relatively more males (including 405 males and 205 females). In general, a sample size of at least 200 observations would be an appropriate minimum for SEM analyses (Kelloway 1998). Therefore, the sample size of 610 was sufficient for the SEM of ICS in this study.
Instrument
The ICS developed by Wu and Tsai (2005) included six scales: ‘multiple sources as accuracy’, ‘authority as accuracy’, ‘content as usefulness’, ‘technical issues as usefulness’, ‘elaboration and exploration as searching strategy’ and ‘match as searching strategy’. By EFA, the ICS is deemed sufficiently reliable for assessing learners’ information commitments in Web-based learning environments (Wu & Tsai 2005).
The ICS, as shown in Table 1, consist of two components and a total of six scales, and there are, respectively, three, four, five, four, five and three items in each scale. The items of ICS in this study were exactly the same as those in Wu and Tsai (2005). The first component of the ICS (Implicit Component) ex-plores students’ evaluative standards utilized to assess the accuracy and usefulness of Web-based materials. This component includes four scales: ‘multiple sources as accuracy’ (Multiple sources), ‘authority as accu-racy’ (Authority), ‘content as usefulness’ (Content) and ‘technical issues as usefulness’ (Technical). The second component (Explicit Component) investigates the information-searching strategies that learners use when seeking information on the Internet. This com-ponent includes two scales: ‘exploration and elabora-tion as searching strategy’ (Exploraelabora-tion) and ‘match as searching strategy’ (Match). The items of ICS were presented with bipolar strongly agree/strongly disagree statements in a six-point Likert scale (i.e., strongly agree, agree, somewhat agree, somewhat disagree, disagree and strongly disagree). The use of such a
six-point Likert scale could not only avoid totally neutral responses but also differentiate students’ variations of agreement in proper details. In addition, the ICS im-plemented in this study was presented in digital for-mat, and the participants were asked to fill out their responses on the questionnaire on the Internet.
Data analysis
To fulfil the main purposes of this study, the SEM techniques were conducted as the major statistical analysis method in this study. Moreover, the data collected in this study were analysed with Linear Structure RELationships (LISREL), which is a widely
available software package for SEM. LISREL is commonly used for the analysis of latent variables, and it can be used to confirm the validity of the scales of an instrument and assess the structural relationships among the scales (Joreskog & Sorbom 1989; Kello-way 1998). Basically, two parts of results can be in-volved in a LISREL analysis: one was a measurement model, which is obtained throughout CFA, specifying the relations of the observed measures (i.e., items) to their posited underlying constructs (i.e., scales), and the other was a structural model specifying the causal relations of the constructs to one another (Anderson & Gerbing 1988). To answer the first research question of this study, a series of CFA were conducted with
Table 1. Components, scales and items of the ICS
Implicit Component: Evaluative standards utilized to assess the accuracy and usefulness of Web materials When I view some information unknown on the Internet
Multiple sources as accuracy (Multiple Sources)
Multiple 1 I will discuss with teachers or peers, and then judge whether the information is correct Multiple 2 I will explore relevant content from books (or print materials), and then evaluate
whether the information is correct
Multiple 3 I will try to find more websites to validate whether the information is correct Authority as accuracy (Authority)
Authority 1 I will believe in its accuracy if the information is posted in well known websites Authority 2 I will believe in its accuracy if the information appears in government websites Authority 3 I will believe in its accuracy if the information is posted in professional (official) websites Authority 4 I will believe in its accuracy if the information appears in some websites recommended by experts
When I view or navigate the information on the Internet Content as usefulness (Content)
Content 1 If its content fits my searching goal, I will consider the information as useful to me Content 2 If it can provide more related links, the information for me is useful
Content 3 If it can help me search relevant information further, I will think the information is useful to me Content 4 If it is closer to my searching purpose, I will more believe in its usefulness
Content 5 If it is highly related to my intended searching content, the information for me is useful Technical issues as usefulness (Technical)
Technical 1 If it is presented by animation, I will think the information is useful to me Technical 2 If it does not take much time to be retrieved, the information is useful to me
Technical 3 If it does not require a password or registration, I will think the information is useful to me Technical 4 If it is shown in more beautiful websites, I will believe in its usefulness
Explicit Component: Information searching strategies used in seeking information on the Internet When I need to search information on the Internet
Elaboration and exploration as searching strategy (Elaboration)
Elaboration 1 I am used to summarizing a variety of information
Elaboration 2 I can use some acquired information for advanced search to find the most-fit information Elaboration 3 I can integrate the information obtained from a variety of websites
Elaboration 4 I can keep reminding myself about the purpose of my search Elaboration 5 I can compare different information from related websites (or pages) Match as searching strategy (Match)
Match 1 I usually only use a search engine to find the most-fit websites (or pages) Match 2 If I find the first relevant website, I will not search others
LISREL to confirm the validity and reliability of ICS. Also, in response to the second research question, a structural model was revealed by a series of structural modelling analyses with LISREL to examine the re-lationships between the scales (components) of ICS. In this study, LISREL version 8.54 was used throughout the conduct of SEM analyses with LISREL.
Moreover, a variety of fit indices are currently available for researchers to assess the fit of their models obtained by SEM analyses. For example, the w2value is one of the most commonly used indices for assessing the fit of a structural model when the study involves a large sample (Kelloway 1998). However, as the w2value is very sensitive to sample size, Joreskog and Sorbom (1989) also suggested that the degree of freedom can be used as an adjusting standard by which to judge whether the w2 value is large or small. As suggested by Kelloway (1998), the w2/DF ratios of less than 5 have been interpreted as indicating a good fit to the data. In addition, the following recommended va-lues of different commonly used indices were also suggested to evaluate a satisfactory fit between the specified model and the sample data in previous stu-dies: root mean square error of approximation (RMSEA) of 0.08 or below, root mean square residual (RMR) of 0.08, goodness of fit index (GFI) of 0.9 or greater, adjusted goodness of fit index (AGFI) of 0.9 or greater normed fit index (NFI) of 0.9 or greater, non-normed fit index (NNFI) of 0.9 or greater, comparative fit index (CFI) of 0.9 or greater and incremental fit index (IFI) of 0.9 or greater (Joreskog & Sorbom 1993; Hoyle & Panter 1995; Selim 2003; Wen et al. 2004). In particular, Hu and Bentler (1999) proposed the use of joint criteria in determining the fit of a model to reduce the threat of erroneously rejecting or retaining the wrong model: CFI X0.96 and standard root mean square residual (SRMR) 40.10. To ex-amine the model that was a better fit to the data in this study, various measures described above were utilized to evaluate the fitness of the structural model of ICS.
Results
Measurement model: instrument validation and its reliability
Throughout a CFA by LISREL, the fitness of items for each scale of ICS was examined, and the results are
presented in Table 2. Convergent validity examines the relationship among items in the same construct (scale). Bagozzi (1980) has recommended that for convergent validity to occur, all factor loadings (i.e., l’ values) must be large and statistically significant. According to Table 2, all the t-values of 24 items on the six scales of ICS showed significance at the 0.5 level, indicating that all of the items within each scale were highly correlated with each other. Therefore, the ICS revealed a high convergent validity.
Moreover, except for the factor loading of the item ‘Technical 4’ ( 5 0.49), those of the other 23 items were higher than 0.5. In general, each item of ICS showed high factor loading in its measuring scale. Also, the reliability of ICS was demonstrated by as-sessing the composite reliability coefficient on each scale. Table 2 shows the composite reliability coeffi-cients of the six scales, which were 0.86, 0.71, 0.74, 0.82, 0.89 and 0.77, respectively, all more than 0.6. In sum, the results in Table 2 revealed high validity and reliability of the ICS instrument.
Structural model: the causal relations of the scales Through the SEM analysis with LISREL, the causal relationships of the scales were explored. The four scales related to students’ evaluative standards of as-sessing the accuracy and the usefulness of Web ma-terials (described in Component I) were used as the predictor variables, while the other two scales re-garding students’ information-searching strategies used in seeking information on the Internet (described in Component II) were used as the outcome variables for the analysis. The structural model of this study is presented in Fig. 2. Figure 2 also reveals the summary of the maximum parameter estimates and the sig-nificance of the t-value as indicated by the asterisks for the model. In Fig. 2, the statistically significant re-lationships are shown in solid lines. Moreover, the fit measures for the ICS model are summarized in Table 3, indicating that the model in Fig. 2 had a highly satisfactory fit.
According to Fig. 2, ‘Multiple Sources’ and ‘Con-tent’ were significant predictors in explaining the variation of ‘Exploration’. ‘Authority’ and ‘Technical’ were significant predictors to ‘Match’. In addition, ‘Content’ was a negatively related predictor to ‘Match’, and ‘Exploration’ was also a significantly
negative predictor to ‘Match’. In other words, ‘Mul-tiple Sources’ and ‘Content’ had a significantly posi-tive effect on ‘Exploration’, while ‘Authority’ and ‘Technical’ imposed a positive effect on ‘Match’. Moreover, both ‘Content’ and ‘Exploration’ had sig-nificantly negative effects on ‘Match’. Tsai (2004a) has proposed that ‘multiple sources’ and ‘content’ evaluative standards and the ‘elaboration & explora-tion’ searching strategy, which were commonly ex-pressed and used by experts, were advanced information commitments, while ‘authority’, ‘techni-cal’ and ‘match’, which were more often expressed and utilized by novices, were less sophisticated. Ac-cording to the results in this study and the perspectives of Tsai (2004a), the advanced evaluative standards (i.e., ‘multiple sources’ and ‘content’) commonly held by experts (learners) did cause their utilization of the advanced information-searching strategy (i.e., ‘ex-ploration and elaboration’) in Web-based learning environments. On the other hand, the ‘authority’ and
‘technical’ evaluative standards commonly held by novice learners did lead to use of the less sophisticated information searching strategy (i.e., ‘match’). In sum, learners’ evaluative standards did have significant ef-fects on the searching strategies that they used in the Web-base learning environment.
Moreover, it was also found that ‘Exploration’ had a significant negative effect on ‘Match’ in Fig. 2. The results indicated that the searching strategy categor-ized as ‘exploration and elaboration’, commonly held by experts, and the searching strategy categorized as ‘Match’, frequently expressed by novices, were op-posite, as suggested by Tsai (2004a). But ‘Multiple sources’ was positively correlated with ‘Authority’ (r 5 0.17, Po0.05), and ‘Content’ was positively correlated with ‘Technical’ (r 5 0.42, Po0.05). In other words, ‘multiple sources’ and ‘authority’ are not opposite orientations, and ‘content’ and ‘technical (functional)’ are not reverse orientations. This finding concurs with what was suggested by Wu and Tsai
Table 2. Instrument validation
Scale Item Mean SD Factor loading t-value Composite reliabilitya
Multiple Sources Multiple 1 4.36 0.91 0.73 17.85 0.74
Multiple 2 4.32 0.94 0.77 18.92 Multiple 3 4.70 0.92 0.58 13.72 Authority Authority 1 4.48 0.95 0.75 20.06 0.82 Authority 2 4.54 1.01 0.80 21.62 Authority 3 4.76 0.88 0.73 19.10 Authority 4 4.39 0.89 0.62 15.50 Content Content 1 5.09 0.74 0.75 21.11 0.89 Content 2 4.94 0.75 0.83 24.35 Content 3 5.03 0.70 0.89 27.13 Content 4 5.14 0.76 0.77 21.84 Content 5 5.02 0.75 0.70 19.23 Technical Technical 1 3.65 1.00 0.50 12.35 0.77 Technical 2 4.48 1.12 0.88 23.84 Technical 3 4.48 1.17 0.79 21.01 Technical 4 3.35 1.08 0.49 12.00 Elaboration Elaboration 1 4.88 0.89 0.79 22.46 0.86 Elaboration 2 4.96 0.81 0.77 21.79 Elaboration 3 4.97 0.78 0.80 22.83 Elaboration 4 4.62 0.91 0.61 15.94 Elaboration 5 4.77 0.87 0.73 20.18 Match Match 1 3.14 1.22 0.56 12.97 0.71 Match 2 2.42 1.04 0.81 18.77 Match 3 2.90 1.19 0.62 14.44 Po0.05. aInstrument reliability: 0.82.
(2005), that learners may have mixed standards when searching Web information.
Discussion and conclusion
This study was conducted to confirm the reliability and validity of ICS and examine the relationships
between two components of ICS (i.e., the implicit component and the explicit component). Through a series of SEM analyses with LISREL, the results showed high validity and reliability of the ICS in-strument for evaluating learners’ information com-mitments in Web-based learning environments. Also, a structural model was developed to represent the causal relationships between the implicit component and the explicit component of ICS. The fit measures for this model also showed that this structural model of ICS had a highly acceptable fit. This model in-dicated that learners’ evaluative standards on Web materials (implicit component) had significant effects on their information-searching strategies in Web-based learning environments (explicit component). Therefore, learners’ evaluative standards on Web materials should be viewed as important predictors for their information-searching strategies in Web-based learning environments.
How different factors influence the selection and use of sources for information seeking and retrieval is al-ways an important issue in information seeking and retrieval research (e.g., Fidel et al. 1999; Bilal 2000, 2001; Pharo & Jarvelin 2004). Relevant studies have identified many factors that influence learners’ searching behaviour. For example, Pharo and Jarvelin
Exploration The Explicit Component The Implicit Component
Match 0.27* 0.04 −0.00 0.12* 0.36* −0.12* −0.00 0.12* −0.51* 0.42* 0.43* 0.31* 0.17* 0.42* 0.11* *Significant t-value, p< 0.05. Multiple Sources Content Authority Technical
Fig. 2 Structural model and Linear Structure RELationships estimates of the Information Commitment Survey.
Table 3. Fit measures for the structural model of ICS
Fit index ICS Recommended
value
Chi-square (w2) 709.79 –
Degree of freedom (DF) 237 –
w2/DF 3.34 45
Root Mean Square Error of Approximation (RMSEA)
0.064 40.08
Root Mean Square Residual (RMR) 0.055 40.10
Standard RMR (SRMR) 0.060 ! 0.05
Goodness-of-Fit Index (GFI) 0.90 X0.90 Adjusted Goodness-of-Fit
Index (AGFI)
0.87 X0.80
Normed Fit Index (NFI) 0.95 X0.90
Non-Normed Fit Index (NNFI) 0.96 X0.90 Comparative Fit Index (CFI) 0.96 X0.90 Incremental Fit Index (IFI) 0.96 X0.90
(2004) have summarized five categories that affect information searching: the work task, the searcher, the social/organizational environment, the search task and the search process. This study believes that learners’ information commitments, which may vary across in-dividuals, will guide their Web-seeking behaviours, and they can be conceptualized as a factor of ‘the searcher’ among the five categories summarized in Pharo and Jarvelin (2004). Moreover, further research is suggested to investigate the interplay between in-formation commitments and the other factors in the five categories summarized by Pharo and Jarvelin (2004).
Moreover, it should be acknowledged that the cur-rent study, clearly, is an initial attempt to explore the possible variables influencing learners’ searching strategies in Web-based learning environments, and this part of the findings was obtained throughout a series of SEM analyses with LISREL. In addition, as the sample in this study came from science- and technology-oriented universities, the degree to which the results can be generalized should be acknowl-edged. Therefore, there is a call for further research on search habits of students in social science and huma-nities, and more follow-up studies are needed to fur-ther examine these perspectives. In addition, the present study only examined whether learners’ eva-luative standards for Web materials influenced their searching strategies. One may be interested in: could their searching strategies also, in turn, influence their evaluative standards for Web materials in Web-based learning environments? Further studies are required to address this issue. In addition, the current study has assessed a group of both undergraduates and graduate students’ information commitments. Further studies may be conducted to investigate how their expertise plays a role in their information commitments.
Moreover, educators have distinguished students’ learning approaches between deep and surface ap-proaches in a conventional educational context (e.g., Marton 1983; Biggs 1987). The results in this study also showed that ‘exploration and elaboration’ and ‘match’ were likely two opposite searching strategies. Accord-ing to the above results, we can, similarly, distAccord-inguish students’ searching strategies in processing Web-learning tasks between a deep searching strategy (i.e., ‘elaboration and exploration’) and a surface searching strategy (i.e., ‘match’). Learners utilizing the deep searching strategy (i.e., ‘elaboration and exploration’)
can be viewed as utilizing the advanced learning strat-egy, and they may use the deep approach to learning in Web-based academic tasks. On the other hand, the surface searching strategy utilized by learners, i.e., ‘match’, can be viewed as a less sophisticated learning strategy in Web-based academic tasks. However, more studies are necessary to examine the relationships be-tween learners’ searching strategies on the Web and their learning in Web-based academic tasks.
The information commitments, as proposed by Tsai (2004b), shape some epistemological standards for Web information and are related to learners’ episte-mological beliefs. Recently, how learners’ epistemo-logical beliefs influence their Web-based learning has received increasing attention among educational re-searchers. For example, Hofer (2004) has argued that student use of the Internet as a medium for learning involves a host of epistemological judgements that deserve more attention. Hofer (2004) also suggested that students’ awareness in their epistemological jud-gements could enhance their ability to think critically about seeking and evaluating information on the In-ternet. Therefore, learners’ epistemological beliefs may influence their ways of seeking and evaluating in-formation on the Internet. Tsai and Chuang (2005) have also found that there were interplays between students’ epistemological beliefs and their preferences towards some metacognitive features of the Internet learning environments. Moreover, some studies have also been conducted to explore the relationships between college students’ epistemological beliefs and their information-seeking behaviours (e.g., Whitmire 2003, 2004). For instance, it has been revealed that college students having more advanced epistemological beliefs would exhibit the ability to handle conflicting informa-tion sources and to evaluate informainforma-tion sources (Whitmire 2004). In sum, learners’ epistemological beliefs may be correlated with their ways of seeking and evaluating information on the Internet. In other words, an interplay between learners’ epistemological beliefs and their information commitments may exist. However, more studies are required to confirm this perspective.
Moreover, as educators have proposed that episte-mological beliefs guide students’ conceptual devel-opment and the manner of knowledge growth (e.g., Tsai 2000; Hofer 2001, 2004), whether the informa-tion commitments held by learners will guide their
processes and outcomes of knowledge construction in Web-based learning environments still needs careful exploration. Therefore, further research is also sug-gested to investigate the relationships between lear-ners’ information commitments, searching behaviours and their learning outcomes in Web-based learning environments.
This study also revealed some educational im-plications for instructors. With the implementation of Web-based learning instruction, the significance of learners’ information commitments on their Web-based learning should be highlighted. For example, educators should consider learners’ different in-formation commitments when designing Web-based courses or learning activities. Moreover, educators should also try to help learners develop better eva-luative standards for Web materials (e.g., multiple sources) by providing more proper opportunities in Web-based learning environments. These well-devel-oped evaluative standards may improve learners’ use of advanced information searching strategies, and then help them attain better learning outcomes in Web-based learning environments.
Acknowledgements
Funding of this research work was supported by Na-tional Science Council, Taiwan, under grant numbers NSC 93-2524-S-009-003 and NSC 94-2524-S-009-003. The authors also express their gratitude to the Editor, and two anonymous reviewers’ helpful com-ments for further development of this article.
References
Anderson J.C. & Gerbing D.W. (1988) Structural equation modeling in practice: a review and recommended two step approach. Psychological Bulletin 103, 411–423. Bagozzi R. (1980) Causal Modeling in Marketing. Wiley &
Sons, New York.
Beaufils A. (2000) Tools and strategies for searching in hypermedia environments. Journal of Computer Assisted Learning 16, 114–124.
Biggs J. (1987) Student Approaches to Learning and Studying. Australian Council for Educational Research, Melbourne.
Bilal D. (2000) Children’s use of the Yahooligans! Web search engine: I. Cognitive, physical and affective
be-haviors on fact-based search tasks. Journal of the American Society for Information Science 51, 646–665. Bilal D. (2001) Children’s use of the Yahooligans! Web
search engine: II. Cognitive and physical behaviors on research tasks. Journal of the American Society for In-formation Science and Technology 52, 118–136. Brandt D.S. (1996) Evaluating information on the Internet.
Computers in Libraries 16, 44–46.
Chou C. & Tsai C.-C. (2002) Developing Web-based cur-ricula: issues and challenges. Journal of Curriculum Studies 34, 623–636.
Fidel R., Davies R.K., Douglass M.H., Holder J.K., Hopkins C.J., Kushner E.J., Miyagishima B.K. & Toney C.D. (1999) A visit to the information mall: web searching behavior of high school students. Journal of the Amer-ican Society for Information Science 50, 24–37. Flanagin A.J. & Metzger M.J. (2000) Perceptions of Internet
information credibility. Journalism and Mass Commu-nication Quarterly 77, 515–540.
Hewson P.W. (1985) Epistemological commitments in the learning of science: examples from dynamics. European Journal of Science Education 7, 163–172.
Hill J.R. (1999) A conceptual framework for understand-ing information seekunderstand-ing in open-ended information sys-tems. Educational Technology Research & Development 47, 5–27.
Hofer B.K. (2001) Personal epistemology research: im-plications for learning and teaching. Educational Psy-chological Review 13, 353–383.
Hofer B.K. (2004) Epistemological understanding as a me-tacognitive process: thinking aloud during online searching. Educational Psychologist 39, 43–55. Hoyle R.H. & Panter A.T. (1995) Writing about structural
equation models. In Structural Equation Modeling: Concepts, Issues and Applications (ed. R.H. Hoyle). Sage, London.
Hu L.T. & Bentler P.M. (1999) Cut-off criteria for indexes in covariance structure analysis: conventional criteria ver-sus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal 6, 1–55.
Jean-Francois R. (2003) What was I looking for? The in-fluence of task specificity and prior knowledge on stu-dents’ search strategies in hypertext. Interacting with Computers 15, 409–428.
Jonassen D.H., Peck K.L. & Wilson B.G. (1999) Learning with Technology: A Constructivist Perspective. Merrill, Upper Saddle River, NJ.
Joreskog K.G. & Sorbom D. (1989) LISREL 8: User’s Re-ference Guide, 2nd ed. Lincolnwood, IL, SSI.
Joreskog K.G. & Sorbom D. (1993) LISREL 8: Structural Equation Modeling with the SIMPLIS Command Lan-guage. Erlbaum, Hillsdale, NJ.
Kadijevich D. (2002) An Internet-based collaborative en-vironment for the learning of mathematics. Journal of Computer Assisted Learning 18, 48–50.
Kelloway E.K. (1998) Using LISREL for Structure Equation Modeling: A Researcher’s Guide. Sage, Newbury Park, CA. Knezek G. & Christensen R. (2002) Impact of new in-formation technologies on teachers and students. Edu-cation and Information Technologies 7, 369–376. Kuechler M. (1999) Using the Web in the classroom. Social
Science Computer Review 17, 144–161.
Lin C.-Y., Cheng Y.-J., Chang Y.-T. & Hu R.-P. (2002) The use of Internet-based learning in biology. Innovations in Education and Teaching International 39, 237–242. Lin C.-C. & Tsai C.-C. (2005) A ‘‘navigation flow map’’
method of representing students’ searching strategies on the Web. Paper presented at 2005 World Conference on Educational Multimedia, Hypermedia & Tele-communications, Montreal, Canada.
Marton F. (1983) Beyond individual differences. Educa-tional Psychology 3, 289–303.
McDowell L. (2002) Electronic information resources in undergraduate education: an exploratory study of op-portunities for student learning and independence. British Journal of Educational Technology 33, 255–266. Metzger M.J., Flanagin A.J. & Zwarun L. (2003) College
student Web use, perceptions of information credibility, and verification behavior. Computers & Education 41, 271–290.
Pharo N. & Jarvelin K. (2004) The SST method: a tool for analyzing Web information search process. Information Processing and Management 40, 633–654.
Relan A. & Gillani B.B. (1997) Web-Based Instruction and traditional classroom: similarities and difference. In Web-Based Instruction (ed. B.H. Khan), pp. 41–46. Educa-tional Technology, Englewood Cliffs, NJ.
Selim H.M. (2003) An empirical investigation of student acceptance of course websites. Computers & Education 40, 343–360.
Tsai C.-C. (2000) The effects of STS-oriented instruction on female tenth graders’ cognitive structure outcomes and the role of student scientific epistemological beliefs. In-ternational Journal of Science Education 22, 1099–1115. Tsai C.-C. (2001a) The interpretation construction design model for teaching science and its applications to
inter-net-based instruction in Taiwan. International Journal of Education Development 21, 401–415.
Tsai C.-C. (2001b) A review and discussion of epistemolo-gical commitments, metacognition, and critical thinking with suggestions on their enhancement in Internet-assisted chemistry classrooms. Journal of Chemical Education 78, 970–974.
Tsai C.-C. (2002) Nested epistemologies: science teachers’ beliefs of teaching, learning and science. International Journal of Science Education 24, 771–783.
Tsai C.-C. (2004a) Information commitments in Web-based learning environments. Innovations in Education and Teaching International 41, 105–112.
Tsai C.-C. (2004b) Beyond cognitive and metacognitive tools: the use of the Internet as an ‘‘epistemological’’ tool for instruction. British Journal of Educational Technol-ogy 35, 525–536.
Tsai C.-C. & Chuang S.-C. (2005) The correlation between epistemological beliefs and preferences toward Internet-based learning environments. British Journal of Educa-tional Technology 36, 97–100.
Tsai M.J. & Tsai C.-C. (2003) Information searching stra-tegies in web-based science learning: the role of Internet self-efficacy. Innovations in Education and Teaching International 40, 43–50.
Wen M.L., Tsai C.-C., Lin H.-M. & Chuang S.-C. (2004) Cognitive-metacognitive and content-technical aspects of constructivist Internet-based learning environments: a LISREL analysis. Computers & Education 43, 237–248.
Whitmire E. (2003) Epistemological beliefs and the in-formation-seeking behavior of undergraduates. Library & Information Science Research 25, 127–142.
Whitmire E. (2004) The relationship between under-graduates’ epistemological beliefs, reflective judgment, and their information-seeking behavior. Information Processing and Management 40, 97–111.
Wu Y.-T. & Tsai C.-C. (2005) Developing the Information Commitment Survey for assessing college and graduate students’ evaluative standards for Web information and their searching strategies in Web-based learning en-vironments. Paper presented at 2005 World Conference on E-Learning in Corporate, Government, Healthcare, & Higher Education, Vancouver, Canada.
