In this systematic design of the instruction, pretest and posttest were employed with 16 questions. They were the same questions with different time of implementation. In addition, the concept of entry behavior test and embedded test was derived from pretest and
posttest. The test items of the entry behavior and embedded test were the same as in pretest and posttest. In this study, the tests questions corresponding performance objectives 2.1.1, 2.1.2, 3.1.1, 3.1.2, 4.1.1, and 5.1.1 were entry behavior test, and other questions were items of embedded test which appeared in the units of the instructional material to give students immediate practice of the learned contents. Overall, the results of these four tests would provide practical scores for the purpose of formative evaluation.
Complete test questions of pretest and posttest:
1. Use a line to connect the terms on the left with the correct answers on the right.
(1) modern a. present and recent time.
(2) primitive b. the first and earliest period of the world, before recorded events.
2. Use a line to connect the terms on the left with the correct answers on the right.
(1) modern housing a. dangerous and uncomfortable.
(2) primitive shelter b. safe and comfortable.
3. Use a line to connect the terms on the left with the correct answers on the right.
(1) modern transportation a. slow and not handy.
(2) primitive transportation b. speedy and handy.
4. Write T in the blank line if it is true.
Write F in the blank line if it is false.
____(1) Eating raw meat is dangerous.
____(2) Modern people use ma chines to make clothes.
____(3) Primitive people went from one place to another place by car or by airplane.
Circle the correct answer in the following questions.
5. What period of time do we live in?
a. modern society b. primitive society
6. What is the characteristic of modern food?
a. raw b. cooked and/or prepared 7. What kind of food did primitive people eat?
a. hamburger b. raw meat c. fried chicken 8. What kind of equipment did primitive people use to make food?
a. fire b. bone knife c. stove 9. What kind of tools did primitive people use to make clothes?
a. sewing machines b. animal bones
10. What kind of clothes did primitive people wear?
a. leaves or animal skins b. T-shirts c. sweaters 11. Where did primitive people live?
a. caves or big trees b. houses or apartments 12. What does the word "cave" mean?
a. a mountain b. a hollow in a mountain or big tree c. a big tree 13. Which pictures show modern food? (Write M in the ( ) )
Which pictures show primitive food? (Write P in the ( ) )
( ) ( ) ( ) ( ) 14. Which pictures show modern clothes? (Write M in the ( ) ) Which pictures show primitive clothes? (Write P in the ())
( ) ( ) ( ) ( ) 15. Which pictures show modern houses? (Write M in the ( ) ) Which pictures show primitive shelters? (Write P in the())
( ) ( ) ( ) ( )
16. Which pictures show modern transportation? (Write M) Which pictures show primitive transportation? (Write P)
( ) ( ) ( ) ( )
Results
After students read the designed materials and took the tests, the results of the entry behavior test, pretest, and posttest were analyzed in the following Tables.
As shown on Table 3 for student performance on the entry
behavior test, S3, S4, and S6 missed one question (3.1.2) while others got 100% correct. S3 and S6 were aim-medium level in language arts and S4 was aim-high level. Three of them circled the word
"manually" in their text which indicated they did not understand what it meant. Overall, all of them possessed the entry behavior and had the basic ability to study this lesson by themselves.
Table 3 Student Performance on the Entry Behavior Test
Student S1 S2 S3 S4 S5 S6 Mean
O 2.1.1 + + + + + + 100
b
j 2.1.2 + + + + + + 100
e
c 3.1.1 + + + + + + 100
t
i 3.1.2 + + - - + - 50
v
e 4.1.1 + + + + + + 100
s
5.1.1 + + + + + + 100
Percent 100 100 83 83 100 83 92
of objectives
passed by each learne
"+" indicates objective was achieved;
"-" indicates objective was not achieved.
The average of the pretest was 54% which indicated that children had a little knowledge about this lesson (see Table 4). S1 and S6 got high correct percentage--94% and 83% respectively. Both of them missed the same question: "What kind of tools did primitive people use to make clothes? (a) sewing machine (b) animal bones" (3.2.2).
This indicated that they were not familiar with the tools which rarely related to their experiences. On the other hand, the least correct item (17%) was the objective (2.2.3): "What kind of equipment did
primitive people use to make food? (a) bone knife (b) fire (c) stove."
Similarly, students seldom used equipment and cooked food.
Therefore, their performance was reasonable in reflection of their living experiences.
Table 4 Student Performance on the Pretest
"+" indicates objective was achieved.
"-" indicates objective was not achieved.
* S4 got 6% on the pretest
*
In terms of the embedded test, students made a great progress by the correct answers of 85% (see Table 5). Individually, S1, S2, and S6 got 100% correct while S4 and S5 got 94%. S4, an aim-high class student, only missed one question and made a great improvement from 6% on the pretest to 94% on the embedded test. He really learned the whole concept from the pr int delivery context gradually by reading the lesson, doing the practice, and getting feedback immediately.
However, S3, an aim-medium student, got one more correct answer on the embedded test. This approved what his mother said about his reading problem in learning new materials.
The target subjects got 89% correct on the posttest (see Table 6).
After they did each practice and got the correct answers immediately, they made a little improvement from 85% to 89% on the embedded test and posttest respectively. S1 and S2 both missed one question which they passed on the embedded test. The reason why they made
mistakes was perhaps that they did too fast and/or with careless on the posttest. Both of S4 and S5 made a little progress from 94% to 100%.
Also, S3 made improvement from 22% to 39% which meant he got 3 more correct questions on the posttest.
As shown in Table 7, all of the six tryouts made improvement after they finished this lesson. Further explained by percentage data, each child's performance on the pretest, and posttest was as follows:
S1 , 94% and 94%; S2 , 61% and 94%; S3 , 17% and 39%; S4 , 6%
and100%; S5 , 67% and 100%; S6 , 83% and 100%. Obviously, S3, S4, S5, and S6 made a positive progress. S2 made a progress from the pretest to the posttest with one item incorrect. Apparently, S4 made the most improvement among these tryouts. But, why did S4 earn the lowest score of 6% on the pretest? Maybe S4 did not possess the knowledge of utensils, and tools were not familiar to or rarely related to his life experience. On the other hand, S3, an aim-medium
Chinese boy, only made a little improvement from this lesson. He got 17%, and 39% on the pretest, and posttest respectively. Probably, that is due to his language inefficiency.
Table 5 Student Performance on the Embedded Test
"+" indicates objective was achieved.
"-" indicates objective was not achieved.
* S4 got 94% on the embedded test
*
Table6 Student Performance on the Posttest
"+" indicates objective was achieved.
"-" indicates objective was not achieved.
* S4 got 100% on the embedded test.
Tab l e 7 C o m p a r i s o n o f E a c h St u d e n t P er f o r m a n c e o n th e P r e t e s t an d P o s t t es t
Stu d en t S 1 S 2 S 3 S 4 S 5 S 6 M e a n
P r e t e s t 9 4 6 1 1 7 6 6 7 8 3 5 4
P o s t t e s t 9 4 9 4 3 9 1 0 0 1 0 0 1 0 0 8 9
Unit: Percentage (%)
Appealingly, children really gained help from showing pictures (see Table 8). They got all 67% on the pretest of the objectives 2.1.4
& 2.2.4, 3.1.4 & 3.2.3, 4.1.3 & 4.2.4, and 5.1.3 & 5.2.3 which have attached pictures. In other words, they got more correct answers on these items than others without pictures. Also, they got 100% on three of them and the other one 83% on the posttest. Therefore, iconic pictures with tests really helped children answer the
question/guess the answer and learned the lesson. In addition, it was apparent that children got better scores on the lower subordinate skills than the higher ones. For instance, they got 83% and 83% correct on the objective (2.2.2) while only getting 17% and 67% on the objective (2.2.3). The same situation happened on the objectives (3.2.1) and (3.2.2), and (4.2.1) and (4.2.2). However, the objectives (2.2.1) and (2.2.2) were not the same situation. Children got 33% correct on the (2.2.1) and 83% on the (2.2.2) for the pretest. The objective (2.2.1) was: "What kind of food did primitive people eat? (a) pizza (b) raw meat (c) fried chicken." Most of them marked (a) pizza. The reason why they made mistakes was probably they did not read the question carefully and just marked the first item they were familiar. The objective (2.2.2) was a true or false question: "We eat raw meat because that is safe." Apparently, this question was related to children's daily life, so it was not a difficult question. Finally, they got the same scores (83%) of these two objectives on the posttest.
Table 8 Summary of Pretest and Posttest Scores for Each Objective
Conclusion
Based on the organized text which included Jacky Lu's story, pictures, entry behavior test, pretest, embedded test, answers to embedded test, and posttest, children really learned the life styles of primitive and modern societies through this study. In addition, six tryouts reflected satisfactory opinions after they took the lesson.
Overall, according to the principles of Dick, Carey, and Carey’s approach, this research project applied the essential components of instructional systems design to accomplish a practical teaching contexts for comparing the life styles of modern and primitive
societies. After examining the effect of the systematic approach in this empirical study, the concluding remarks were accordingly denoted as follows:
1. Iconic pictures were useful to promote students’ learning interests.
2. Story-telling technique could serve as a binding thread to sequence the teaching contexts.
3. Students scored 92% correct in the entry behavior test which recognized that the six tryouts owned beginning ability to learn the designed materials.
4. The total mean scores of the pretest were 54% correct and raised to be 85% in the embedded test. This revealed that immediate practice helped students for learning teaching contents.
5. The pretest scores of 54% correct were eventually raised to 89% correct in the posttest. The obvious improvement in correct rate showed that systematic approach was effective in teaching and curriculum development.
As the systems approach grew rapidly in the field of instructional design, Dick, Carey, and Carey (2005) claimed that their model should be effective for any particular learner population in both school and nonschool settings. From the above findings of this study, it was
convincible that the application of their systematic process did work as an effective tool to develop schoolchildren’s teaching materials for satisfactory learning outcomes.
References
Armstrong, A. M. (2004). Instructional design in the real world: A view from the trenches. Hershey, PA: Idea Group.
Chevalier, R. D. (1990). Improving efficiency and effectiveness of training: A six year case study of systematic change. Performance and Instruction, 29 (5), 21-23.
Dick, W., & Carey, L. (1985). The systematic design of instruction (2nd ed.). Glenview, IL: Scott Foresman.
Dick, W., & Carey, L. (1996). The systematic design of instruction (4th ed.). New York: Harper Collins.
Dick, W., Carey, L., & Carey, J. O. (2005). The systematic design of instruction (6th ed.). Boston, MA: Allyn and Bacon.
Dong, L. (2006). Writing Chinese America into words and images:
Storytelling and retelling of the song of Mu Lan. The Lion and the Unicorn, 30(2), 218-233.
Gagne, R. M. (1985). The conditions of learning (4th ed.). New York:
Holt, Rinchart & Winston.
Gay, L. R., & Airasian, P. W. (2003). Educational research:
Competencies for analysis and applications (7th ed.). Upper Saddle River, NJ: Merrill/Prentice Hall.
Hales, C. M., & Jones, M. E. (1983). Managing your reading program-A total system. Washington, DC: Department of Education.
Lu, C. H. (1994). A study of theory and practice on instructional systems design. International Journal of the Humanities, 3, 31-60。
Lu, C. H. (2006). Implementation of multicultural curriculum and instruction at an American elementary school. Paper presented at the 2006 Eastern Taiwan Educational Symposium, Hualien, Taiwan.
Morrison, G. R., Ross, S. M., & Kemp, J. E. (2001). Designing effective instruction (3rd ed.). New York: John Wiley & Sons.
Office of Education (1975). Direction for Alabama reading teachers-A handbook. Washington, DC: Author.
Opie, C. (2004). Research approaches. In C. Opie (Ed.), Doing
educational research: A guide to first time researchers (pp. 73-94).
Thousand Oaks, CA: Sage.
Piaget, J., & Inhelder, B. (1969). The psychology of the child. (H.
Weaver, Trans.). New York: Basic Books.
Robinson, V., & Lai, M. K. (2006). Practitioner research for
educators: A guide to improving classrooms and schools. Thousand Oaks, CA: Sage.
Rothwell, W. J., & Kazanas, H. C. (2004). Mastering the instructional design process: A systematic approach (3rd ed.). San Francisco, CA: Pfeiffer.
Tripp, S. D., & Bichelmeyer, B. (1990). Rapid prototyping: An alternative instructional design strategy. Educational Technology, 38 (1), 31-44.
Watts, J. E. (2006). Benefits of storytelling methodologies in 4th and 5th grade historical instruction. Dissertation Abstract International, 45(1), 0140. (UMI No. 1437706)
Yackley, L. E. (2006). Storytelling: A key to adult learning.
Dissertation Abstract International, 67(12), 4415A. (UMI No.
3247568)