In order to echo the educational reform that is focused on improving students’
abilities of reasoning and problem solving, the assessment tools nowadays should enable teachers to evaluate these abilities of their students (Pellegrino, 2001). At the same time, during the whole process of assessment, teachers not only need to observe and record their students’ behavior when performing specific tasks, but also need to evaluate their students’ abilities on the basis of their performance of these tasks. Of course, the justification for doing the evaluation should be the theory of students’
cognition and learning (Pellegrino, 2001). However, the problem is that even though teachers can evaluate students’ performance on the basis of the learning theory, it is almost impossible for any teacher to record students’ every single action and manage the longitudinal tracking data of students without the help of technology tools (Brown, Hinze, & Pellegrino, 2008).
It is worth noting that the improvements in information and communications technology in education make both measuring and evaluating students’ performance easier and more reliable (Brown et al., 2008). Firstly, technology-based assessments (TBAs) can help teachers to collect students’ data or observable performance more easily by decreasing the time required to collect and input test data (Brown et al., 2008; Kuo & Wu, 2013; Pellegrino, 2001). The observable performance includes students’ interactions with the simulation tasks during their processes of problem solving and the submitted answers as final solutions to the problem (De Klerk, Veldkamp, & Eggen, 2015). Secondly, teachers can diagnose students’ learning problems by retrieving and comparing the process data of the target performance (Brown et al., 2008). All of these potentials of TBAs make the measurement of complex abilities or skills possible and reliable.
6
Furthermore, an increasing number of TBAs have been adopted by large-scale and international evaluation programs in recent years (Quellmalz, Timms, Silberglitt,
& Buckley, 2012), such as the Programme for International Student Assessment. This means that the study of TBAs has become an important issue for both researchers and teachers.
However, the infrastructure needed, the preparation time requirements (Cheon, Lee, Crooks, & Song, 2012; Pellegrino, 2001) and especially teachers’ beliefs about assessment could all have an influence on how TBAs are implemented (Black &
Wiliam, 1998; Cheon et al., 2012).
Generally, previous educational studies have articulated that teachers make decisions on the basis of their beliefs (Hart, 2002; Pajares, 1992) and their practice is guided by their beliefs about teaching and learning (van der Schaaf, Stokking, &
Verloop, 2008). Similarly, other studies have also indicated that teachers’ beliefs impact on their acceptance and implementation of innovations in curricula, especially on their use of new technology tools (De Smet, Bourgonjon, De Wever, Schellens, &
Valcke, 2012; Pynoo et al., 2011; Teo, 2011).
Furthermore, some studies have contributed to portraying the relationships between teachers’ beliefs about TBAs and their assessment practice (Cheon et al., 2012; Graham, 2005; Stiggins, 2004). On the one hand, Lee, Feldman, and Beatty (2012) found that teachers without the necessary resources and time tended not to try to implement TBAs. Teachers with enough resources but who did not believe in the usefulness of TBAs would also not integrate TBAs into their assessment practice.
Only those teachers who could both perceive the usefulness of TBAs and find the way to overcome the problem of resources would actively use TBAs in their classrooms.
On the other hand, Stiggins (2004) asserted that the passive beliefs held by teachers may constrain their involvement in the use of TBAs.
7
It is worth noting that Stiggins’ (2004) assertion was based on his review, and the findings of Graham (2005) and Lee et al. (2012) were based on their coding and interpretation of the qualitative data. Even though these studies revealed that teachers’
beliefs about TBAs might relate to their assessment practice, and articulated some features of teachers’ beliefs about TBAs, they did not articulate the causal relationship between teachers’ beliefs about TBAs and their assessment practice.
In other words, if researchers want to not only portray the features of teachers’
beliefs about TBAs, but also test the relationship between teachers’ beliefs about TBAs and their assessment practices, and identify the mechanism of how they interact with each other, a study combining qualitative data analysis, confirmatory factor analysis (CFA) , regression analysis, and path analysis is needed.
At the same time, the studies nowadays have also
1. Teachers’ beliefs impact on their acceptance and implementation of innovations in curricula, especially on their use of new technology tools (Churchill, 2006; Kim, Kim, Lee, Spector, & DeMeester, 2013; Shin, 2015; Stylianidou, Boohan, &
Ogborn, 2005; Zacharia, 2003).
2. Several theoretical models such as the technology acceptance model (TAM) and the theory of planned behavior (TPB) have been utilized to analyze and estimate the effects of beliefs on how teachers adopt technology for use in instruction (Ajzen, 1991; Davis & Venkatesh, 1996; Liu, 2012; Taylor & Todd, 1995; Teo, 2011).
3. Some studies have indicated that teachers’ technology-infused practice has an effect on their students’ learning performances (Comi, Argentin, Gui, Origo, &
Pagani, 2017; McNeill, Pimentel, & Strauss, 2013).
8
4. However, compared to the substantial amount of research on how teachers use technology in general, there has been relatively little research into how teachers use technology-based assessments in their teaching.
To summarize the above considerations, compared to the substantial amount of research on how teachers use technology in general, there has been relatively little research into teachers’ beliefs about TBAs and their usage about TBAs.
Thus, this series of studies aimed at filling this gap and the purpose of each study was listed as below. Besides, a mixed method design combining qualitative data analysis, a structural equation modeling (SEM) analysis that combines both CFA and path analysis and the hierarchical linear modeling (HLM) was conducted in this series of studies to achieve these aims. More details about the research design will be
introduced in the section of methods.