Based on the PISA 2012 digital problem-solving assessment log files, as well as responses to the student, parent and school questionnaires, the present study used the data mining technique of classification and regression tree (CART) to discern which factors classify whether a student is a high- performing problem-solving expert or low-performing novice in the two
groups of students of the top ten high-performing Eastern and Western economies. The results showed that: (1) in the seven Eastern economies, amongst the 91 variables under examination, three factors that are found to have major influence in determining students’ problem-solving proficiency levels are: Discovery of the optimal solution path of the problem task,
Mathematics self-efficacy, and Experience with pure mathematics tasks at school; (2) in the three Western economies, the factors which have influence
in classifying students’ problem-solving proficiency levels are: Mathematicsself-efficacy, Discovery of the optimal solution path of the problem task, Familiarity with mathematical concepts, and Mathematics work ethics. All of
the factors identified are student-level variables.With regard to the factors identified, there are some subtle differences between the high-performing Eastern and Western economies. Firstly,
Discovery of the optimal solution path of the problem task has the strongest
influence in classifying whether a student is a problem-solving expert or a problem-solving novice in the Eastern economies, while Mathematics self-efficacy has a greater influence in the Western economies. In addition, besides
the factors of Discovery of the optimal solution path of the problem task, andMathematics self-efficacy, student problem-solving proficiency is also
affected by Experience with pure mathematics tasks at school in Eastern economies, and Familiarity with mathematical concepts and Mathematicswork ethics in Western economies. The implication is that teachers should
design problem-based learning based on these findings, with due attention paid to helping students organize and acquire essential knowledge, learn the task-specific problem-solving strategies, and develop self-efficacy and work ethics purposefully.Educators have long argued that the traditional mode of teaching with a focus on knowledge acquisition is not beneficial for students to develop integrative competence, such as the digital problem-solving competence in this study (Blank, 1982). One viable approach for contemporary teaching is to emphasize the cultivation of students’ positive learning attitudes and beliefs.
According to the results of this study, teachers should not only focus on disciplinary knowledge and skills, but also attitudes and beliefs. In the Eastern cultural context, enhancing students’ task-specific problem-solving skills is more effective in raising student problem-solving competence, while in the
Western cultural context, helping students to develop self-efficacy is more efficient to improve student problem-solving ability. Increasing student mathematics work ethics also has a moderate positive effect on student problem-solving performance in the high-performing Western economies.
Among the many factors related to student attitudes and beliefs, self- efficacy is confirmed in this study to have the strongest influence on students’
digital problem-solving competence. Therefore, how can teachers develop students’ self-efficacy? According to Bandura’s (1977b) social learning theory, an individual’s self-efficacy may come from four main informational sources:
(1) An individual’s performance accomplishment provides the most influential efficacy because it is based on personal mastery experiences; (2) vicarious experiences of observing others succeed through their efforts; (3) verbal persuasions in order to cope with circumstances successfully; and (4) states of physiological arousal from which people judge their level of anxiety and vulnerability to stress. In school environments, teachers can provide more opportunities for students to master problem-solving tasks that are appropriately set at their ability levels. Collaborative learning is also an effective way to help students gain successful experience from others, and meanwhile develop their own self-efficacy.
References
Aiken, L. R. (1980). Attitude measurement and research. New Directions for Testing and Measurement, 7, 1-24.
Alivernini, F., & Manganelli, S. (2015). Country, school and students factors associated with extreme levels of science literacy across 25 countries.
International Journal of Science Education, 37(12), 1992-2012.
doi:10.1080/09500693.2015.1060648
Allore, H., Tinetti, M. E., Araujo, K. L., Hardy, S., & Peduzzi, P. (2005). A case study found that a regression tree outperformed multiple linear regression in predicting the relationship between impairments and social and productive activities scores.
Journal of Clinical Epidemiology, 58(2), 154-161.
doi:10.1016/j.jclinepi.2004.09.001
Baker, R. S. J. D. (2010). Data mining. In P. Peterson, E. Baker, & B. McGaw (Eds.), International encyclopedia of education (3rd ed., Vol. 7, pp. 112-118). Oxford, England: Academic Press.
Bandura, A. (1977a). Self-efficacy: Toward a unifying theory of behavioral change.
Psychological Review, 84(2), 191-215. doi:10.1037/0033-295X.84.2.191 Bandura, A. (1977b). Social learning theory. New York, NY: General Learning Press.
Biggs, J. (1994). What are effective schools? Lessons from East and West. The Australian Educational Researcher, 21(1), 19-39.
Blank, W. E. (1982). The competency-based approach to education and training. In Deakin University Course Development Centre (Ed.), A collection of readings related to competency-based training (pp. 3-27). Victoria, Australia: Deakin University.
Boaler, J. (2013). Ability and mathematics: The mindset resolution that is reshaping education. FORUM, 55(1), 143-152.
Bousbia, N., & Belamri, I. (2014). Which contribution does EDM provide to computer-based learning environments? In A. Peña-Ayala (Ed.), Educational data mining (pp. 3-28). Cham, Switzerland: Springer.
Brand-Gruwel, S., Wopereis, I., & Vermetten, Y. (2005). Information problem solving by experts and novices: Analysis of a complex cognitive skill. Computers in Human Behavior, 21(3), 487-508. doi:10.1016/j.chb.2004.10.005
Breiman, L., Friedman, J., Olshen, R., & Stone, C. (1998). Classification and regression trees. Boca Raton, FL: CRC Press.
Cai, J. (1995). A cognitive analysis of US and Chinese students’ mathematical performance on tasks involving computation, simple problem solving, and complex problem solving. Journal for Research in Mathematics Education, Monograph, i-151. doi:10.2307/749940
Casner-Lotto, J., & Barrington, L. (2006). Are they really ready to work? Employers’
perspectives on the basic knowledge and applied skills of new entrants to the 21st century workforce. Washington, DC: Conference Board.
Chang, L., Mak, M. C., Li, T., Wu, B. P., Chen, B. B., & Lu, H. J. (2011). Cultural adaptations to environmental variability: An evolutionary account of East-West differences. Educational Psychology Review, 23(1), 99-129.
doi:10.1007/s10648-010-9149-0
Chi, M. T. H., Feltovich, P. J., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5(2), 121-152.
doi:10.1207/s15516709cog0502_2
Chiou, H. J. (2015, December). From small data to big data: The role of psychometrics in data science. Paper presented at the 2015 Global Chinese Conference on Educational Information and Assessment & Chinese Association of Psychological Testing 2015 Annual Conference, Taichung, Taiwan.
Cooper, H., Robinson, J. C., & Patall, E. A. (2006). Does homework improve academic achievement? A synthesis of research, 1987-2003. Review of Educational Research, 76(1), 1-62. doi:10.3102/00346543076001001
De Mauro, A., Greco, M., & Grimaldi, M. (2015). What is big data? A consensual definition and a review of key research topics. Proceedings of the AIP Conference, 1644(1), 97-104. doi:10.1063/1.4907823
Dweck, C. S. (2006). Mindset: The new psychology of success. New York, NY:
Balantine Books.
Elio, R., & Scharf, P. B. (1990). Modeling novice-to-expert shifts in problem-solving strategy and knowledge organization. Cognitive Science, 14(4), 579-639.
doi:10.1207/s15516709cog1404_4
Goos, M., & Galbraith, P. (1996). Do it this way! Metacognitive strategies in collaborative mathematical problem solving. Educational Studies in Mathematics, 30(3), 229-260. doi:10.1007/BF00304567
Greiff, S., Wüstenberg, S., & Avvisati, F. (2015). Computer-generated log-file analyses as a window into students’ minds? A showcase study based on the PISA 2012 assessment of problem solving. Computers & Education, 91, 92-105.
doi:10.1007/BF00304567
Han, J., Kamber, M., & Pei, J. (2011). Data mining: Concepts and techniques.
Waltham, MA: Morgan Kaufmann Publishers.
Hess, R. D., & Azuma, M. (1991). Cultural support for schooling: Contrasts between Japan and the United States. Educational Researcher, 20(9), 2-8.
doi:10.3102/0013189X020009002
Ho, D. Y. F. (1991, June-July). Cognitive socialization in Confucian heritage cultures.
Presented at the Workshop on Continuities and Discontinuities in the Cognitive Socialization of Minority Children, Department of Health and Human Services, Washington, DC.
Kaur, B., & Har, B. Y. (2009). Mathematical problem solving in Singapore schools. In B. Kaur, Y. B. Har, & M. Kapur (Eds.), Mathematical problem solving:
Yearbook 2009, Association of Mathematics Educators (pp. 3-13). Singapore:
World Scientific.
Klassen, R. M. (2004). Optimism and realism: A review of self-efficacy from a cross-cultural perspective. International Journal of Psychology, 39(3), 205-230.
doi:10.1080/00207590344000330
Larkin, J. H., McDermott, J., Simon, D. P., & Simon, H. A. (1980). Models of competence in solving physics problems. Cognitive Science, 4(4), 317-345.
doi:10.1207/s15516709cog0404_1
Lee, Y. J. (2015). Analyzing log files to predict students’ problem solving performance in a computer-based physics tutor. Journal of Educational Technology & Society, 18(2), 225-236.
Lewis, R. J. (2000). An introduction to classification and regression tree (CART) analysis. Paper presented at the 2000 Annual Meeting of the Society for Academic Emergency Medicine, San Francisco, California.
Mandler, G. (1989). Affect and learning: Causes and consequences of emotional interactions. In D. B. McLeod & V. M. Adams (Eds.), Affect and mathematical problem-solving: A new perspective (pp. 3-19). New York, NY: Springer-Verlag.
Mayer, R. E. (1998). Cognitive, metacognitive, and motivational aspects of problem solving. Instructional Science, 26(1-2), 49-63. doi:10.1023/A:1003088013286 Neale, D. C. (1969). The role of attitudes in learning mathematics. The Arithmetic
Teacher, 16(8), 631-640.
Nicolaidou, M., & Philippou, G. (2003). Attitudes towards mathematics, self-efficacy and achievement in problem solving. In M. A. Mariotti (Ed.), European research in mathematics education III. Pisa, Italy: University of Pisa.
Organization for Economic Cooperation and Development (2004). Problem solving for tomorrow’s world: First measures of cross-curricular competencies from PISA 2003. Paris, France: Author.
Organization for Economic Cooperation and Development (2013). PISA 2012 assessment and analytical framework: Mathematics, reading, science, problem solving and financial literacy. Paris, France: Author.
Organization for Economic Cooperation and Development (2014a). PISA 2012 results:
Creative problem solving: Students’ skills in tackling real-life problems (Vol. 5).
Paris, France: Author.
Organization for Economic Cooperation and Development (2014b). PISA 2012 technical report. Paris, France: Author.
Organization for Economic Cooperation and Development (2016). Global competence for an inclusive world. Paris, France: Author.
Peña-Ayala, A. (2014). Educational data mining: A survey and a data mining-based analysis of recent works. Expert Systems with Applications, 41(4), 1432-1462.
doi:10.1016/j.eswa.2013.08.042
Peter S., Judith M., & Robyn J. (2009). Tasks and pedagogies that facilitate mathematical problem solving. In B. Kaur, Y. B. Har, & M. Kapur (Eds.), Mathematical problem solving: Yearbook 2009, Association of Mathematics Educators (pp. 17-42). Singapore: World Scientific.
Poropat, A. E. (2009). A meta-analysis of the five-factor model of personality and academic performance. Psychological Bulletin, 135(2), 322-338.
Romero, C., & Ventura, S. (2007). Educational data mining: A survey from 1995 to 2005. Expert Systems with Applications, 33(1), 135-146.
doi:10.1016/j.eswa.2006.04.005
Romero, C., & Ventura, S. (2010). Educational data mining: A review of the state of the art. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 40(6), 601-618. doi:10.1109/TSMCC.2010.2053532 Romero, C., & Ventura, S. (2013). Data mining in education. Wiley Interdisciplinary
Reviews Data Mining and Knowledge Discovery, 3(1), 12-27.
doi:10.1002/widm.1075
Sachin, R. B., & Vijay, M. S. (2012). A survey and future vision of data mining in educational field. Proceedings of the 2012 Second International Conference on Advanced Computing & Communication Technologies, 96-100.
doi:10.1109/ACCT.2012.14
Sanzana, M. B., Garrido, S. S., & Poblete, C. M. (2015). Profiles of Chilean students according to academic performance in mathematics: An exploratory study using classification trees and random forests. Studies in Educational Evaluation, 44, 50-59. doi:10.1016/j.stueduc.2015.01.002
Scherer, R., & Gustafsson, J. E. (2015). The relations among openness, perseverance, and performance in creative problem solving: A substantive-methodological approach. Thinking Skills and Creativity, 18, 4-17. doi:10.1016/j.tsc.2015.04.004 Schunk, D. H. (1991). Self-efficacy and academic motivation. Education Psychology,
26, 207-231. doi:10.1080/00461520.1991.9653133
Slack, S. J., & Stewart, J. (1990). High school students’ problem‐solving performance on realistic genetics problems. Journal of Research in Science Teaching, 27(1), 55-67. doi:10.1002/tea.3660270106
Smith, M. U., & Good, R. (1984). Problem solving and classical genetics: Successful versus unsuccessful performance. Journal of Research in Science Teaching, 21(9), 895-912. doi:10.1002/tea.3660210905
Strobl, C., Boulesteix, A. L., & Augustin, T. (2007). Unbiased split selection for classification trees based on the Gini index. Computational Statistics & Data Analysis, 52(1), 483-501. doi:10.1016/j.csda.2006.12.030
Strobl, C., Malley, J., & Tutz, G. (2009). An introduction to recursive partitioning:
Rationale, application, and characteristics of classification and regression trees, bagging, and random forests. Psychological Methods, 14(4), 323-348.
doi:10.1037/a0016973
Trilling, B., & Fadel, C. (2009). 21st century skills: Learning for life in our times. San Francisco, CA: Jossey-Bass.