This study attempted to investigate factors that affect the item difficulty of cube enumeration test and to evaluate the performance of VIB-based cube computer based enumeration test (VIB-based CBT) and the learning system on improving student’s cube enumeration ability. In Chapter 1 of this thesis, the background, motivation and purpose of this research were introduced. Chapter 2 reviewed literatures on classical test theory (CTT), item response theory (IRT), virtual item bank (VIB), automatic item generation (AIG) and modern technologies used to generate items. Chapter 3 described a series of experiments designed to investigate the factors that affect the item difficulty of cube enumeration test, the item exposure rate of VIB-based CBT and the performance of the learning system. Chapter 4 showed the results of experiment.
Chapter 5 attempted to answer and discuss proposed research questions and propose suggestions for further research.
1.1. Background and Motivation
Like Zhouyi is used to predict one’s fate in ancient China, IQ tests are used to evaluate one’s IQ, future direction and potential for development in modern days. The present study builds two cube enumeration tests and a VIB-based cube enumeration test to evaluate an examinee’s spatial ability and develop a learning system to enhance the spatial ability of students. Moreover, as there are different combinations of the eight trigrams in Zhouyi, the present study also combines different basic graphic elements for different tests, hoping to estimate an examinee’s spatial ability accurately with this item generation system.
Spatial ability is not only an essential part of everyday life but also an important skill for mechanists, architects, chemists, surgeons, surveyors, and cartographers.
Nowadays, the mathematical programs in elementary and junior high schools implementing in the country or abroad list spatial ability as one of the geometry course
2
materials. Researchers were interested in how to measure and improve spatial ability.
Currently there are plenty of related tests that can be used to evaluate spatial ability.
However, it is very difficult to develop and only very experienced test constructors can do this job. Moreover, the feasibility can only be evaluated with massive test administration. For the improvement of spatial ability, now there are lots of learning systems and software that we can use, but their effectiveness in improving spatial ability has to be further analyzed and verified.
According to IAEP (1992), the fifth and sixth graders used cube enumeration to learn the concepts of volume because spatial ability was an important concept of geometric. IAEP indicated that spatial ability was positively correlated with the concept of volume. Since cube was the basic unit of volume, cube enumeration tests are often used to assess one's mental ability and operational skills of 3-D geometric objects. Cube enumeration tests are usually kept confidential and exempt from public access. The main reason for this limitation is to keep the items bank private. Another important reason is that it’s highly difficult to develop cube enumeration test. To develop a cube enumeration test, there is a specific process to follow to ensure the consistency of test content and test purpose, and to accurately measure examinees’
ability. In other words, much manpower, time and budget were required to construct enough items to avoid the reduction of test quality due to item exposure.
As information rapidly evolves, how to apply information technology in research becomes a significant issue. This study tried to apply information technology in the evaluation and learning of spatial ability. By combining the VIB, CBT, and AIG, this study intended to establish a whole set of cube enumeration test. With the VIB system, the test would no longer consisted of any substantial forms of the items. Within this system, the item bank only consisted of cubes as the basic component and no longer encountered the risk of the item exposure due to the fact that the items would be dynamically generated by the basic components and the generate methods.
3
1.2. Purposes
Based on the above-mentioned background, this study aimed to investigate related literature and develop a virtual item bank of computerized cube test. By using this item bank the researcher could further generate a system to measure spatial ability.
The research purposes of this study were:
1. To analyze the influence of the allowance of item rotation on item difficulty.
2. To analyze the influence of the number of invisible cubes on item difficulty.
3. To analyze the influence of the integrity of cubes on item difficulty.
4. To develop difficulty formula.
5. To develop a VIB-based cube enumeration test and evaluate its performance.
6. To evaluate the item exposure rate of the VIB.
7. To investigate whether the examinees’ performance of the computerized cube enumeration test is related to their spatial ability.
8. To construct an online cube learning system and perform teaching by using the strategy of rotatable cubes.
9. To evaluate the examinees’ learning performance by using the online learning system.
1.3. Research Questions
According to the above research purposes, the following research questions would be answered in this study:
1. Does the allowance of cube rotation influence item difficulty?
2. Does the integrity of cubes influence item difficulty?
4
3. Does the number of invisible cubes influence item difficulty?
4. What is the difficulty formula derived from the above data?
5. Can we calculate the examinees’ spatial ability by using the VIB-based cube enumeration test constructed and derived from the above data?
6. Is there a difference of the examinees’ spatial ability in gender calculated by the cube enumeration test?
7. Does this virtual item bank constructed by the above formula have a good item exposure rate?
8. Is the learning system constructed by using the rotation strategy effective?
5