THE ANALYSIS OF COGNITIVE DEMAND AND MATHEMATICAL COMPETENCIES: A CASE STUDY OF THE PYTHAGOREAN THEOREM

全文

(1)THE ANALYSIS OF COGNITIVE DEMAND AND MATHEMATICAL COMPETENCIES: A CASE STUDY OF THE PYTHAGOREAN THEOREM. Weverton Ataide Pinheiro. 106. 4.

(2)

(3) To my family.

(4) THE ANALYSIS OF COGNITIVE DEMAND AND MATHEMATICAL COMPETENCIES: A CASE STUDY OF THE PYTHAGOREAN THEOREM. Department of Mathematics of National Taiwan Normal University by Weverton Ataide Pinheiro. In Partial of the Requirements for the Degree of Master of Science In Mathematics Education National Taiwan Normal University Taipei. Abril 2017.

(5) ACKNOWLEDGEMENTS. I would like to express my deepest appreciation to my supervisor, Professor Ph.D. Tso Tai-Yih for all his encouragement, support and guidance for this study. He kindly always helped me to make the objectives of this research clearer, and helped me to keep moving forward throughout the distant and long way. I would like to express my deepest thanks for the Ph.D. student Lei Kin-Hang for all the help provided during the very beginning of this research until the very end with plenty of suggestions and corrections. I would like to express my thanks to the Department of Mathematics from NTNU, and also the Office of International Affairs for all the scholarships provided, which helped me to be able to finish this program. I would like to express my thanks to all the students who volunteered for answering the PISAbased exam making an enormous contribution to this study. I am also grateful for the help received from all my friends from the beginning to the end of this study, in exceptional for Agda, Alexia, Cheng-Chun, Kai-Wen, Lenise, Ming-Lun, WenHao and Williman. Finally, I would like to thank my mother for all her support and encouragement that she has given me throughout my life.. i.

(6) ABSTRACT THE ANALYSIS OF COGNITIVE DEMAND, MATHEMATICAL COMPETENCIES, AND PISA-BASED EXAMS: A CASE STUDY OF THE PYTHAGOREAN THEOREM Weverton Ataide Pinheiro MSc, Mathematics Education Supervisor: Prof. Ph.D. Tso Tai-Yih December 2016 The purpose of this study was to analyze and compare different perspectives of the way the Pythagorean theorem questions appeared in three mathematics textbooks. Tudo é Matemática (Brazilian textbook), Nani (Taiwanese book), and New Syllabus Mathematics (Singaporean textbook) regarding number of exercises and worked examples, cognitive demand and mathematical competencies. This research used content analysis as research methods and exams based on the PISA exam were applied to students from Brazil and Taiwan to check their difference in performance and mathematical reasoning. The analysis of the number of exercises and worked examples showed that both Taiwanese and Singaporean textbooks have a superior amount of exercises and worked examples compared with the Brazilian one which makes Taiwanese and Singaporean textbooks advantageous over the Brazilian book. From the perspectives of cognitive demand, results showed that Taiwanese and Singaporean textbooks are more different from each other because Taiwanese textbook opts to focus more on lower-level demand, while Singaporean textbook has a strong focus on higher-level demand questions. Regarding mathematical competencies results also showed that the Taiwanese textbook is more different than Singaporean textbook. Overall Singaporean textbook is more different from the other two textbooks because many of its questions use level 2 and 3, while the other two textbooks are more used to test students in between levels 0 and 1. Finally, the exam showed that students from Taiwan have higher performance in a PISA-based exam than Brazilian students. Through a qualitative research, the results also revealed that the cultural aspects of students from each country also influenced the way students answered to the questions. Keywords: Textbook analysis, cognitive demand, mathematical competencies, Pythagorean theorem, PISA. ii.

(7) TABLE OF CONTENTS. CHAPTER 1: INTRODUCTION .............................................................................................. 1 Introduction ......................................................................................................................... 1 Background .......................................................................................................................... 3 Problem ................................................................................................................................ 5 Purpose ................................................................................................................................ 7 Research Questions ............................................................................................................. 9 Significance ........................................................................................................................ 10 Definition of key terms ...................................................................................................... 10 CHAPTER 2: LITERATURE REVIEW .................................................................................... 12 Introduction ....................................................................................................................... 12 The features and analysis of textbooks ............................................................................ 12 Importance of textbooks ................................................................................................. 12 Cross-country textbook analysis ..................................................................................... 13 The coverage of Pythagorean theorem in mathematics curricula ................................... 17 The importance of the Pythagorean theorem .................................................................. 17 When and why the Pythagorean theorem is learned ....................................................... 19 Cognitive Demand of Mathematical Task ......................................................................... 20 Mathematical Task ......................................................................................................... 20 Cognitive Demand .......................................................................................................... 21 Mathematical Competence ............................................................................................. 23 The Programme for International Student Assessment (PISA) ........................................ 26 General Information about PISA .................................................................................... 26. iii.

(8) Conclusion .......................................................................................................................... 27 CHAPTER 3: METHODOLOGY ........................................................................................... 28 Introduction ....................................................................................................................... 28 Research Design ................................................................................................................. 29 Procedure ........................................................................................................................... 29 Textbook Sampling ........................................................................................................ 29 Cognitive demand .......................................................................................................... 31 Mathematical Competencies .......................................................................................... 32 PISA-based Exam .......................................................................................................... 35 Participants ........................................................................................................................ 36 Textbooks ....................................................................................................................... 36 Students .......................................................................................................................... 38 Measures ........................................................................................................................... 39 Framework of Textbook Analysis .................................................................................. 39 Connection of PISA-based exam and mathematical competencies ................................ 42 PISA-based Exam .......................................................................................................... 43 Data Analysis ..................................................................................................................... 49 Textbook Analysis .......................................................................................................... 49 PISA-based Exam .......................................................................................................... 50 Reliability of this study .................................................................................................. 53 Limitation ........................................................................................................................... 54 CHAPTER 4: RESULTS ....................................................................................................... 55 Introduction ....................................................................................................................... 55 Exercises ............................................................................................................................. 55. iv.

(9) Worked examples ........................................................................................................... 57 Exercises ........................................................................................................................ 59 Cognitive Demand ............................................................................................................. 60 Mathematical Competencies ............................................................................................ 63 Statistical Methods ............................................................................................................ 66 PISA-based exam ............................................................................................................... 82 Brazilian students’ responses ......................................................................................... 82 Taiwanese students’ responses ....................................................................................... 86 Chapter 5: DISCUSSION ................................................................................................... 93 Introduction ....................................................................................................................... 93 Discussion of the findings .................................................................................................. 94 Number of exercises and worked examples ................................................................... 94 Cognitive demand .......................................................................................................... 98 Mathematical Competencies .......................................................................................... 99 PISA-based exams ........................................................................................................ 100 Relationship between textbook analyses and PISA-based exam .................................. 102 Conclusion ........................................................................................................................ 103 Implications for practice .................................................................................................. 104 Implications for further research .................................................................................... 104 Limitations ....................................................................................................................... 105 References ....................................................................................................................... 107 APPENDICES .................................................................................................................. 111 APPENDIX A: Mathematical Competencies and its levels .............................................. 111 APPENDIX B: PISA-based exams ...................................................................................... 115. v.

(10) PISA-based exam in Portuguese ................................................................................... 115 PISA-based exam in Chinese ....................................................................................... 121. vi.

(11) LIST OF TABLES. Table 1: Level of Cognitive Demands ..................................................................................... 32 Table 2: Relationship of Quasar project cognitive demands and PISA framework levels. ..... 34 Table 3: Information about the three textbooks ....................................................................... 37 Table 4: Analysis of mathematical competencies of the PISA-based exam questions............ 43 Table 5: Description of the specificity of the questions in the PISA-based exam ................... 51 Table 6: Number of Exercises and Application area of the Pythagorean theorem .................. 60 Table 7: K-S test for Brazilian, Taiwanese and Singaporean textbooks for the Communication competency. ..................................................................................................................... 77 Table 8: K-S test for Brazilian, Taiwanese and Singaporean textbooks for the Devising Strategies competency. .................................................................................................... 77 Table 9: K-S test for Brazilian, Taiwanese and Singaporean textbooks for the Mathematizing competency. ..................................................................................................................... 77 Table 10: K-S test for Brazilian, Taiwanese and Singaporean textbooks for the Representation competency. ..................................................................................................................... 78 Table 11: K-S test for Brazilian, Taiwanese and Singaporean textbooks for the Using Symbols competency. ..................................................................................................................... 78 Table 12: K-S test for Brazilian, Taiwanese and Singaporean textbooks for the Reasoning and Argument competency. .................................................................................................... 78 Table 13: K-S test for Brazilian, Taiwanese and Singaporean textbooks for the Aids and Tools competency. ..................................................................................................................... 79 Table 14: K-S test for Brazilian, Taiwanese and Singaporean textbooks for all the competencies looking at level 0.............................................................................................................. 80. vii.

(12) Table 15: K-S test for Brazilian, Taiwanese and Singaporean textbooks for all the competencies looking at level 1.............................................................................................................. 80 Table 16: K-S test for Brazilian, Taiwanese and Singaporean textbooks for all the competencies looking at level 2.............................................................................................................. 80 Table 17: K-S test for Brazilian, Taiwanese and Singaporean textbooks for all the competencies looking at level 3.............................................................................................................. 80 Table 18: Results of the proportional test. ............................................................................... 81. viii.

(13) LIST OF FIGURES. Figure 1: The 'competency flower' from the KOM project (Niss, 2015, p.41) ........................ 25 Figure 2: Number of students vs. age in Brazil ....................................................................... 38 Figure 3: Number of students vs. age in Taiwan .................................................................... 39 Figure 4: Exercise in New Syllabus Mathematics 7th edition (Singaporean textbook, page 217). .......................................................................................................................................... 42 Figure 5: Example of multiple choice item in the PISA-based exam ...................................... 45 Figure 6: Example of numeric entry in the PISA-based exam. ............................................... 46 Figure 7: Example of a question that student can get a partial grade in the PISA-based exam. .......................................................................................................................................... 47 Figure 8: Example of the argumentative question in the PISA-based exam. .......................... 48 Figure 9: Question developed by the author in the PISA-based exam. ................................... 49 Figure 10: Example of the worked example in the Brazilian textbook ................................... 57 Figure 11: Example of a worked example in the Taiwanese textbook .................................... 58 Figure 12: Example of worked example in the Singaporean textbook .................................... 59 Figure 13: Percentage of higher/lower level of Cognitive demand in the Brazilian textbook 61 Figure 14: Percentage of higher/lower level of Cognitive demand in the Taiwanese textbook .......................................................................................................................................... 62 Figure 15: Percentage of higher/lower level of Cognitive demand in the Singaporean textbook .......................................................................................................................................... 62 Figure 16: Distribution of the percentage of the mathematical competencies within the four levels of demand in the Brazilian textbook exercises ...................................................... 64 Figure 17: Distribution of the percentage of the mathematical competencies within the four levels of demand in the Taiwanese textbook exercises .................................................... 65. ix.

(14) Figure 18: Distribution of the percentage of the mathematical competencies within the four levels of demand in the Singaporean textbook exercises................................................. 66 Figure 19: Bonferroni analysis for Brazilian (1), Taiwanese (2), and Singaporean (3) textbooks for the Communication competency. ............................................................................... 69 Figure 20: Bonferroni analysis for Brazilian (1), Taiwanese (2), and Singaporean (3) textbooks for the Devising Strategies competency. ......................................................................... 69 Figure 21: Bonferroni analysis for Brazilian (1), Taiwanese (2), and Singaporean (3) textbooks for the Mathematizing competency. ................................................................................ 70 Figure 22: Bonferroni analysis for Brazilian (1), Taiwanese (2), and Singaporean (3) textbooks for the Representation competency. ................................................................................. 70 Figure 23: Bonferroni analysis for Brazilian (1), Taiwanese (2), and Singaporean (3) textbooks for the Using Symbols competency. ................................................................................ 71 Figure 24: Bonferroni analysis for Brazilian (1), Taiwanese (2), and Singaporean (3) textbooks for the Reasoning competency. ........................................................................................ 71 Figure 25: Bonferroni analysis for Brazilian (1), Taiwanese (2), and Singaporean (3) textbooks for the Aids and Tools competency. ................................................................................ 72 Figure 26: Bonferroni analysis for Brazilian (1), Taiwanese (2), and Singaporean (3) textbooks for all competencies looking at level 0. ........................................................................... 74 Figure 27: Bonferroni analysis for Brazilian (1), Taiwanese (2), and Singaporean (3) textbooks for all competencies looking at level 1. ........................................................................... 74 Figure 28: Bonferroni analysis for Brazilian (1), Taiwanese (2), and Singaporean (3) textbooks for all competencies looking at level 2. ........................................................................... 75 Figure 29: Bonferroni analysis for Brazilian (1), Taiwanese (2), and Singaporean (3) textbooks for all competencies looking at level 3. ........................................................................... 75 Figure 32: Common Brazilian students' responses for question 1 ........................................... 83. x.

(15) Figure 33: Common Brazilian students' responses for question 2 ........................................... 84 Figure 34: Kind of reasoning in most of the Brazilian students' responses for question 3 ...... 84 Figure 35: Common answers for question 4 ............................................................................ 85 Figure 36: Sample of reasoning for question 5 ........................................................................ 86 Figure 37: Most common responses for question 1. ................................................................ 87 Figure 38: Most common responses for question 2 ................................................................. 88 Figure 39: Common answers for question 3 ............................................................................ 89 Figure 40: Common answers for question 4 ............................................................................ 90 Figure 41: Common answers for question 5 ............................................................................ 91 Figure 42: Worked example and practice in the Taiwanese textbook applied to real-world context .............................................................................................................................. 96 Figure 43: Problem proposed by the Brazilian textbook. ........................................................ 97 Figure 44: Solution of the problem proposed by Brazilian textbook....................................... 97. xi.

(16) . CHAPTER 1: INTRODUCTION Introduction One of the main concerns of mathematics educators is finding effective ways to teach mathematics providing students a better learning opportunity. The learning process of mathematics involves many variables such as students, teachers, textbooks, computers, etc. We have no doubt that worldwide textbooks are crucial tools for educators (Pehkonen, 2004). Textbooks are key elements in the learning process of mathematics, and most of the teachers of mathematics classroom classes are based on this tool to transmit knowledge to students (Alajmi, 2009). In students' lives textbooks serve as a guide that can help them to prepare in advance for classes, it acts as a reference for students during the course, and also work as a reference after school, helping students to practice and revise what they have learned in the mathematics classroom. Because of this extremely importance for students, and knowing that textbooks have played a significant role in students’ learning. It is relevant to research how textbooks in different countries have led students to active learning of mathematics, helping them to be able to succeed in the learning of mathematics, and being able to apply the content learned in diverse situations, such as math daily problems, or even in assessments such as PISA. However, textbooks are not just a useful tool for students, but it is also crucial for teachers since this instrument make it possible for teachers to have a guide to help them to plan their daily classes. It serves as a reference while teachers are teaching mathematics in the classroom, and it also suggests teachers extra activities to work with students inside and outside of the mathematics classroom. Tanner and Tanner (1980), defined textbooks as useful guides for teachers and stable orientation for the students. One cannot ignore the substantial importance that textbooks have in the learning and teaching process of mathematics. Thus, it is important. 1.

(17) to have researchers that evaluate the way these tools are written, and also the opportunities textbooks offer to students to learn a determined topic. For example the Pythagorean theorem, an important theorem that has been studied for centuries and provide students with basic mathematics formula to solve abstract questions in many fields of mathematics such as geometry, algebra, etc. Firstly, this study was though to make a comparison of a Brazilian and a Taiwanese textbook’s approach of the Pythagorean theorem, to understand why students from Brazil and Taiwan performed in such different way on the PISA exam. In other words, why students from these two countries have different mathematical competence, mainly focusing on why Brazilian students had such a low-level performance in the 2015 Math PISA assessment. After the analyses between the Brazilian and the Taiwanese textbooks had been made, the authors compared these two textbooks with a Singaporean textbook. The reason for adding a Singaporean textbook to the research is due to Singaporean students also performed very well on PISA 2015 assessment, so it could be possible to draw some conclusions on whether Singaporean textbook and Taiwanese textbook are similar to each other and that is the reason why students from these both countries are excelling in the performance of international exams. It could also provide us outcomes that shows that even though two textbooks are different from each other, students from the countries that used such textbooks are performing well due to other variables involved in the process of learning mathematics. To make a bridge between the textbooks analyzed and PISA exam results, this research also investigates students’ performance from Brazil and Taiwan when tested with PISA-based exams that only includes as a topic the Pythagorean theorem. The idea of the exams is about narrowing down the subject of the PISA exam making it the same as the topic that this research analyzes on the textbooks. The PISA-based exam also gave this study the opportunity to take. 2.

(18) a close look at different kinds of reasoning from Brazilian and Taiwanese students on the PISA exam questions. Background Mathematics textbooks have been one of the most regarding tools in the teaching and learning of mathematics. This valuable tool defines the way mathematics is taught and the way it is learned (Kajander & Lovric, 2009). Therefore, textbooks analyses have been used by many researchers to try to identify what are the differences presented by textbooks in different fields of studies, including mathematics. More specifically, in the area of mathematics, there are a significant number of textbook analyses. For example, Xenofontos and Papadopoulos (2015) have analyzed the ways the history of mathematics is integrated into the national textbooks of Cyprus and Greece. In another study, Charalambous, Delaney, Hsu, and Mesa (2010) investigated the treatment of addition and subtraction of fractions in primary mathematics textbooks used in Cyprus, Ireland, and Taiwan. This study examines the learning opportunities afforded by these textbooks, especially concerning the presentation of the content and the textbook expectations as manifested in the associated tasks. The work of Charalambous et al. (2010) has a significant contribution to this research since they defined three different kinds of cross-country textbook analysis, and this analysis is going to use one of their methods to conduct the study. In another study, the analysis of the standards in primary school mathematics in England and Cyprus were made, drawing upon national curricula, the content of textbooks and date from international comparisons of attainment (Campbell & Kyriakides, 2000). Campbell and Kyriakides (2000) research has a contribution to this investigation since it gives particular attention to the evaluation of how problematic learning through textbook has influence in pupils’ outcomes-realised standards.. 3.

(19) Rezat (2006) focused on the different structural elements about characteristics regarding content; linguistic features, visual characteristics, their pedagogical functions within the learning process, and situative conditions in German mathematics textbooks. Rezat and Straesser (2014) research focused on the core of other four papers to provide particular insights into different perspectives within the field of research on mathematics textbooks, the comparison of math textbooks and the use of math textbooks by teachers in three European countries. For reference look at (Pepin & Haggarty, 2001; Remillard, 2005; Rezat, 2008; and Valverde et al., 2002). The study of Tam and Wang (2012) analyzes how the Pythagorean theorem was stated and proved in four versions of textbooks in Taiwan. In this study, the authors focused in how textbooks covered the Pythagorean theorem, what are the reasons introduced to substantiate its truth and the applications each textbook shows to present the use of the theorem. However, this study was not focused on the learning opportunity of the topic when using the different textbooks analyzed. For this study, it is important to have an understanding of PISA assessment. The OECD Programme for International Student Assessment (PISA), is a collaborative effort among OECD member countries to measure how well 15-years-old young adults approaching the end of compulsory schooling are prepared to meet challenges of today's knowledge societies (Ray & Margaret, 2003). According to Törnroos (2005), the opportunity to learn is an important contributing factor in learning outcomes, and some of the latest international comparative studies of mathematics achievement (SIMS and TIMSS) have been making painstaking efforts to find out what the participating students' opportunity to learn mathematics had been.. 4.

(20) Morgan Niss (2004), defined mathematical competence as the ability to understand, judge, do, and use mathematics in a variety of contexts. PISA (OECD, 2003) states that mathematical competence deals with the capacity of students to analyze, reason and communicate efficiently, as they pose, formulate, solve and interpret mathematical problems in a variety of situations. For this study, it is necessary to have an understanding of mathematical competencies, since it is one of the analysis unit used for the analyses of the textbooks. This research focuses on the comparison of three textbooks, and the application of PISA-based exams in term of the understanding of student's performance analyzed through mathematical competencies and students’ reasoning. Problem In Brazil, Taiwan, and Singapore, mathematics textbooks are the primary tool used by junior high school teachers inside the classroom to teach math. Adequate textbooks offer a better opportunity for the effective teaching of mathematics (Rezat, 2010). There are not many available kinds of literature that can explain the difference in cognitive demand of exercises in an analysis of a cross-country textbook analysis. Furthermore, literature has shown that textbooks have influenced the way students perform in mathematics (Monaghan, 2013; Törnroos, 2005). Therefore, cross-country textbook analysis has a rich contribution to help teachers to understand better the way textbooks are used inside the classroom, and also this kind of study collaborate to the improvement of future textbooks. Furthermore, it seems that students from some countries around the world learn more mathematics than students from other nations. If they do not learn more, at least it appears that they can better apply what they have learned. These assumptions come to reality when. 5.

(21) we take a look at the results of countries' performance in the mathematics part of the PISA assessment. For example, In PISA 2015 survey, Singapore and Taiwan obtained one of the highest performances in Mathematics among all countries around the world. Respectively 1st and 4th position according to PISA, Organisation for Economic Cooperation and Development (OECD, 2016). From these results, we could see that Singaporean and Taiwanese students have high performance in mathematics exams. On the contrary, Brazil was one of the countries that had one of the lowest rank achievements in mathematics within all the countries around the world, out of 70 countries, Brazil appeared in the 63rd position, showing that Brazilian students have a very low performance in mathematics. Their poorly achievement in mathematics can be found in the same article (OECD, 2016). PISA is an exam created by the OECD with the intention to better understand how students are learning reading, science, and mathematics, and been able to apply the knowledge learned in situations students might face in their future based on their daily life challenges. Students representing more than 70 countries already took this exam. This assessment happens every three years, and 15 years old students all around the world assess their literacy in mathematics and the other subjects (reading and science). The first edition of the exam happened in 2000, and so far we already had six editions. The reason for the exam application for 15 years old students is because in most of the countries 15 years old students are in the last year of the compulsory school. It makes the exam feasible for the organizers to understand how students leave school and how students are prepared to apply the knowledge learned during their school years in real-world problems. The key term to define the PISA exam in the mathematics part is summarized as mathematical literacy, which is discussed later in the literature review.. 6.

(22) The main problem is to understand through analyses of textbooks from three countries how the different mathematical competencies and levels of cognitive demand required in exercises of these textbooks have influenced students’ performance on PISA. Besides, this research applies mathematical exams to students from Brazil and Taiwan and checks the reasoning of students of these countries in a wide variety of mathematical cognitive demand questions. Purpose Since students from Brazil have been achieving low scores in all PISA exam editions from 2000 to 2015, and on the contrary students from Taiwan and Singapore have been doing greatly, this study compares these countries textbooks. Also, this study applies exams for Brazilian and Taiwanese students to better provide answers of how the cognitive demand and the mathematical competencies, and the number of the exercises in those textbooks have influenced Taiwanese and Brazilian students’ performances. Summarizing, what are the learning opportunity provided by these textbooks? The Pythagorean theorem was chosen as the only, and main topic for analyses in the textbooks of these three countries. Some reasons for this choice are: It is a very familiar theme for most of the people who ever studied geometry or algebra in their lives. Thus the theorem is fundamental, and we should pay attention and see how it has been taught in different countries. Since we are considering the PISA exam as a base to our studies to find out why the Brazilian students have a little proficiency in mathematics literacy, this research focused on a topic that could be found on the PISA assessment. Not only a topic that could be in the exam but also a topic that could be broadly found on the exam. The Pythagorean theorem was one of the most found mathematics topics on the 2012 PISA survey, what gives the authors of this research a concise motivation to pick up this topic for the analyses. One might ask why the authors do not make an analysis of the whole textbooks? The answer is that: firstly, not all the topics in the. 7.

(23) textbooks have application to real world context problems (PISA questions type). Thus we could have found findings that would not be relevant to the research. And secondly, the authors do not have too much time to run the analyses of both entire three countries' textbook. The authors also considered the Pythagorean theorem as the main topic, because of the ease of applying the Pythagorean theorem to many areas of knowledge. This theorem discovered thousands of years ago can be applied to geometry, algebra, and several real world problems. Since of its broad application, it is a topic that not from today have been given a deep importance to people, and it is considered to be the famous and mostly known theorem in mathematics science. School's curriculum from all over the world inserts the Pythagorean theorem as a topic that should be learned in the 8th grade or equivalent (13-14 years old students). So once again, the Pythagorean theorem is important for our analysis since it is taught to students in the previous years of the PISA assessment PISA (15 years old students), thus opening many other possibilities to apply additional exams or surveys for future studies and drawing conclusions. It is also relevant here to mention what is the importance given to the Pythagorean theorem by the mathematics curriculum in these three countries to state the importance of the usage and how the theorem has been applied in the mathematics classroom. Starting with Singaporean National Curriculum (Secondary Mathematics Syllabus), established by the Singaporean Ministry of Education in the year of 2007, the Pythagorean theorem should be taught to students in the in Secondary Two (typical age 13-14 years old) under the topic of geometry. The National Taiwanese curriculum established by the Taiwanese Ministry of Education states that the Pythagorean theorem must be taught in Taiwan for 8th graders (14 years old), and it must be taught with an application for both geometrical and algebraic approach. In Brazil there is no National Curriculum, so we are going to look at the curriculum of the capital of the country (Brasilia). In Brazil, the Pythagorean theorem must be taught in the 9th year of the secondary school (14 years old), and it is applied to the geometrical. 8.

(24) knowledge. So from everything mentioned and considered above, the authors confirmed the importance of using the Pythagorean theorem as the main topic for the analyses of the textbooks. This research uses a mixed method which includes quantitative and qualitative research. The quantitative research includes content analysis of the textbooks and two statistical methods were used for measuring the significant difference on the textbooks. The qualitative part of this research was made throughout the analysis of the different answers given by Brazilian and Taiwanese students that solve the PISA-based exams. As a result of this research, explanations for a different level of mathematical competencies and cognitive demand of the exercises of the three textbooks are given. This result is going to help researchers to understand better the differences in the cognitive demand and mathematical competencies on the textbooks and whether mathematical textbooks are a critical point or not in determining students’ performance in studies such as the PISA exam. Research Questions 1). What is the difference between cognitive demand and mathematical competencies in. textbooks from Brazil, Taiwan, and Singapore in an analysis of exercises and working examples? 2). Are there any statistically significant differences between Brazilian and Taiwanese. students’ performance and reasoning in a PISA-based exam, which only contains questions related to the Pythagorean theorem? 3). What is the relationship between the results of the PISA-based exams and the analyses. of textbooks?. 9.

(25) Significance Since textbooks play a significant role in the teaching and learning of mathematics, it is crucial to make an analysis of textbooks to understand better the quality of the material that has been used around the world to instruct students. The first reason why this study is significant is that once the results are presented teachers and researchers are going to understand what kind of mathematical competency plays a stronger influence in the students learning of mathematics, making it possible for students to score better in international assessment such as PISA. Secondly, since the analysis of textbooks always brings new knowledge and a better understanding of many aspects related to the learning and teaching of mathematics, researchers and teachers are going to have a new framework to analyze textbooks according to its mathematical competencies presented in its questions. And thirdly, this research is going to clarify why Brazilians students have a poor achievement on PISA when compared to Singapore and Taiwan. Definition of key terms Textbook analysis: As defined on the website of the University of Kansas, a textbook analysis is the systematic analysis of the text materials including the structure, the focus, and special learning assists (problems). Cognitive demand: Defined by Stein & Smith (1998), the cognitive demand of mathematical tasks is related to the processes that are required to solve a mathematical task. "Tasks that ask students to perform a memorized procedure in a routine manner lead to one type of opportunity for student thinking; tasks that require students to think conceptually and that stimulate students. to. make. connections. of opportunities for student thinking.". 10. lead. to. a. different. set.

(26) Mathematical competencies: Morgan Niss (2002) defined mathematical competencies like the ability to understand, judge, do, and use mathematics in a variety of contexts. PISA-based exam: The PISA-based exam is an exam which contains five questions. The first four questions were taken from PISA exams and the last question was written by the author. The last question also follows the same style of the other four questions and it was added in order to have more questions on the exam, which facilitates the reliability of the results.. 11.

(27) . CHAPTER 2: LITERATURE REVIEW Introduction The aim of this study is to analyze textbooks’ mathematical competencies and cognitive demand of their exercises and apply a PISA-based exam for students from Brazil and Taiwan. This chapter discusses the various kinds of literature for cross-country textbook analysis, the importance of the Pythagorean theorem in teaching and learning, and also mathematical literacy been extended to the descriptions of mathematical competencies. The features and analysis of textbooks Importance of textbooks. Textbooks are the most used tool inside the classroom to provide student opportunity to learn. The role played by textbooks in teaching and learning of a subject has an extreme importance, since textbook does not only serve as an excellent tool for students but also plays an important paper in teacher’s role (Gak, 2011). Textbooks are curricula written in another way. It is the best connection or bridge to lead students to learn what curricula state. Textbooks are the primary guide that students can have to orientate their studies. Students use textbooks during the classes, at home while they are preparing for next classes, revising what they have learned or practicing exercises. Textbook provide security for students just like a road map, letting students have the exact comprehension of what they have learned and what they are going to learn. (Gak, 2011). 12.

(28) Cross-country textbook analysis In recent years, there are many researchers making studies on the analysis of textbooks (Xenofontos & Papadopoulos, 2015; Charalambous et al., 2010; Rezat & Straesser, 2014; Campbell & Kyriakides, 2000; Cai, Lo, & Watanabe, 2002; Tam & Wang, 2012). There are many different ways to make an analysis of a textbook. Usually, analyses have various purposes such as finding the relationship of textbooks to the curricula of countries or investigate how textbooks are used by teachers and students. For example, the research made by Xenofontos and Papadopoulos (2015), examined the ways the history of mathematics is integrated into the national textbooks of Cyprus and Greece for lower secondary education (grades 7, 8, and 9). Their study was divided into two stages. Firstly, they identified four categories for analyses: biographical references about mathematicians or historical references regarding the origins of a mathematical concept. References to the history of a mathematical method or formula containing a solution or proof. Mathematical tasks of purely cognitive elements that require a solution, explanation or evidence, and tasks that encourage discussion or the production of a project that would connect the history of mathematics with life outside mathematics. Secondly, the authors also employed a framework for the levels of cognitive demand developed by the QUASAR project team (Stein & Lane, 1996; Smith & Stein, 1998; Stein et al., 2000). In conclusion for the study, the authors found out that both countries value the history of mathematics in their textbooks. Other analyses show the interest to discover how different textbooks present a particular topic in mathematics field, or what are the objectives of some textbooks looking at the way the textbooks addresses questions in mathematics. One example of this kind of analysis can be found in Charalambous et al., 2010, where the authors have reported on a comparison of the treatment of addition and subtraction of fractions in primary mathematics textbooks used in. 13.

(29) Cyprus, Ireland, and Taiwan. To carry the research, the authors used a framework that could investigate the learning opportunities afforded by the textbooks. The authors considered three different categories to classify cross-national textbooks analysis, classified as horizontal, vertical and contextual. Their study focused on six criteria (two from the horizontal and four from the vertical dimension), which are topics, sequencing, worked examples, constructs, latent cognitive demand, type of response. In conclusion, the authors found similarities and differences among the textbooks regarding topics included and their sequencing, the constructs of fractions, the worked examples, the cognitive demands of the tasks, and the types of responses required of students. The findings emphasized the need to examine textbooks to understand differences in instruction and achievement across countries. Rezat and Straesser (2014) analyzed different textbooks framework based on four core types of research about textbook analysis, developed by Pepin and Haggarty (2011), Remillard (2005), Rezat (2008), and Valverde et al. (2002). For the development of their research, they took a close look at three most important aspects of the way textbooks can be analyzed. They first defined analyses of textbooks as curriculum materials, noticing that in this case, textbooks would only offer an auxiliary idea for teaching. The second way they considered was the analyses of textbooks reviewing textbooks as artifacts, in this case, textbooks were seen as an opportunity tool used by teachers, students and other agents in the process of teaching and learning of mathematics. Textbooks as artifacts were also divided into micro, meso, and macro structures by the authors. From these structures, they tried to find the unity of analysis of textbooks, which means what has been analyzed in that particular textbook. The last way took into consideration in their research; textbooks were considered as instruments. In this case, they looked at the actual use of textbooks for teaching and learning of mathematics. The authors concluded the research using the reference to a ‘tetrahedron model of textbook use.' Where four constituents are identified – textbook, student, teacher and mathematical knowledge.. 14.

(30) In another study, Campbell and Kyriakides (2000) made a comparison of the standards in primary school mathematics in England and Cyprus. Their research was drawn upon national curricula, the content of textbooks and data from international comparisons attainment. The standards were conceptualized in three ways: expected, planned, and realized. Their research has demonstrated that high expected and high designed standards, as set in national curricula and textbooks, are not associated with high achieved standards. The studies of Cai, Lo, and Watanabe (2002) examined how selected U.S. and Asian mathematics curricula are designed to facilitate student’s understanding of the arithmetic average. The results of their studies were divided into four section, the grade levels, the kinds of learning goals, then how the concept is introduced in each curriculum, and worked-out examples. This study concluded that textbooks from the U.S. and Asian countries have different focuses, and the reason might be the difference in the language used in those countries. Tam and Wang (2012), presented a paper to show how geometry is covered in Taiwan’s textbooks and discussed the relative merits on those presented in textbooks from China and Singapore. In their analyses of the four approved textbooks in Taiwan, the authors chose as variables for analysis the proofs furnished, knowledge requires, caveat, tacit geometry knowledge, and applications of the theorem. When comparing the Taiwanese four versions of textbooks with textbooks from Singapore and China, the authors considered as the layout for analysis, when the Pythagorean theorem was covered in the syllabus, coverage of prior knowledge, approach, style of proof and application. Their studies concluded that the Pythagorean theorem is a good topic for encouraging students to come up with different kind of proofs for the same theorem, and also when geometric activities are carefully arranged, they can be a good use in nurturing student’s creativity in secondary schools.. 15.

(31) The study of Alajmi (2012), has shown the similarities and differences in the presentation of fractions in the elementary mathematics textbooks in the USA, Japan, and Kuwait. For this study was analyzed the physical characteristics of those textbooks, the structure of the lessons on fractions, and the characteristics of this text, the structure, and sequence of the lessons on fractions, and the features of the problems presented. This research concluded that the textbooks from the USA are larger than textbooks from Japan and Kuwait, it was also discovered that the placement of the lesson of fractions appeared in different part of the textbooks, what influenced in the way teachers focused on these topics in these various countries. Another study made by Mayer and Tajika (1995), compared how mathematical problem solving is taught in mathematics textbooks used in Japan and the United States. In particular, the authors examined the hypothesis that a typical Japanese textbook is more oriented toward teaching conceptual understanding and problem-solving skills whereas standard U.S. textbooks are more oriented toward teaching isolated facts and rote computation. This analysis was conducted through a quantitative procedure. The raters broke the lesson into four parts denominated as – exercises, irrelevant illustrations, relevant examples, and explanation. The authors concluded that in Japan, the major use of space is to explain mathematical procedures and concepts in words, symbols, and graphics, with an emphasis on worked-out examples and accurate analogies. In U.S. books, where the use of page space for explanation is minimized about Japanese textbooks, the major use of page space is to present unexplained exercises in symbolic form for the students to solve on their own.. 16.

(32) . The coverage of Pythagorean theorem in mathematics curricula The importance of the Pythagorean theorem The Pythagorean theorem is a compulsory topic for kids in the 7th /8th grade of junior high schools all around the world. Everyone who has ever attended to this grades might remember of the old and famous Pythagorean theorem. As written by Veljan (2000), in his article The 2500-Year-Old Pythagorean Theorem: This is probably the only nontrivial theorem in mathematics that most people know by heart. A good many might even know how to prove it more or less correctly. This "Methuselah" among theorems is one of the most quoted theorems in the history of mathematics, particularly in elementary geometry. But this "folklore" theorem remains eternally youthful, as many people continue to find new interpretations, generalizations, analogs, proofs, and applications. Great mathematicians came up with great results in mathematics using applications of the Pythagorean theorem. For example in the Fermat’s Last Theorem, the Diophantine equation (a" + b" = c " , n ≥ 3, abc ≠ 0) was said by Fermat to have no solution in 1650. But in 1995, the mathematician A. Wiles proved that actually, this equation has a solution for. The Pythagorean theorem has also been used by the second most prolific mathematicians of all time, the Hungarian Pál Erdõs (Euler remains the most prolific mathematician of all times). Erdõs already published over 1500 papers in applied areas of mathematics such as discrete, combinatorial, and computational geometry. In most of his research, he relies on the Pythagorean theorem to solve mathematical problems.. 17.

(33) The Pythagorean theorem if not, might be one of the most proved theorems of the history of mathematics, and it is also considered the most famous theorem of mathematics, as written by Jacob Bronowski in his book, The Ascent of Man, he stated that the Pythagorean theorem is undoubtedly the most important theorem in the field of mathematics (Bronowski, 2011). The Pythagoras’s theorem is also part of one of the most translated and copied book (except the Holy Bible) in the history of humanity, the Elements of Euclid. In this book, Euclides has presented two proofs of the theorem. The first proof is shown as Proposition 47 in Book 1, and the second proof is shown as Proposition 31 in Book 6. A long time ago, Mr. Colburn made a study about the number of proof of the Pythagorean theorem, and he has found a great number of new proofs, having as base those two ones that appeared on the elements of Euclid, he found about seventy-three more proofs of the Pythagorean theorem (EDITORIAL, 1911). Recently studies of Eli Maor has presented that this great theorem has already more than four hundred proofs, and the number of proofs keeping getting higher (Maor, 2007). The Pythagorean theorem is a familiar topic for most of the people who ever studied geometry or algebra in their lives, what characterizes this theorem as a very important theorem. It can be introduced to the student during the middle school years, and it is extremely important for students during the high school years. As Habibi, 2010 wrote in this article Short Proofs for Pythagorean Theorem: The Pythagorean Theorem is a crucial concept for students to learn and to understand. It cannot be stressed enough that students need to understand the geometric concepts behind the theorem as well as its algebraic representation. This can be accomplished through the use. 18.

(34) of technology, manipulatives, and proofs. Students who are taught the Pythagorean Theorem using these methods will see the connections, and thus, benefit greatly. When and why the Pythagorean theorem is learned According to the National 2012 curriculum planning and development division published by the Ministry of Education Singapore, the Pythagorean theorem must be learned in the secondary two (students 13-14 years old), on the subject of geometry. The curriculum determines that secondary two students need to learn how to use the Pythagorean theorem. Also, have the opportunity to use a string with the length of 12 units to form a right-angled triangle with sides of whole units measures. Finally, find out if there is any relationship between the parties, or use cut-out pieces of two squares with sides of 3 units and four units respectively to form a square of sides five units (Ministry of Education, 2012). Published in 2005 by the Ministry of Education, the Taiwanese national curriculum establishes that student in the junior high school (13-14 years old) must learn the Pythagorean theorem. The theorem is introduced under the topics of geometry and algebra. Students must understand what a right angle triangle is, need to have the knowledge of its history; they need to derivate Pythagorean theorem from a simple calculation of the area, and also be able to apply it in various activities in class and daily life (Ministry of Education, 2005). In Brazil there isn’t a national curriculum; instead, there are several curricula. The Federal District’s curriculum states that the Pythagorean theorem must be learned in the 9th grade of junior high school (14-15 years old). The objectives of learning the theorem are stimulating the logical thinking and the capacity for abstraction of mathematics to real-life problems solution, recognizing situations that can be transformed into mathematical language and being able to. 19.

(35) apply them, and also establish the relation in between different fields of mathematics and knowledges of other disciplines (Secretaria de Educação do Distrito Federal, 2014). Cognitive Demand of Mathematical Task. Mathematical Task. The mathematical task is an important topic to be discussed since this study is an analysis of exercises and worked examples on textbooks. Defined by Stein, Grover and Henningsen (1996) in their article “Building Student Capacity for Mathematical Thinking and Reasoning: An Analysis of Mathematical Tasks Used in Reform Classrooms.”, The mathematical task is a classroom activity, the purpose of which is to focus student’s attention on a particular mathematical idea. The study of Stein, Grover and Henningsen gives a great contribution for this thesis, since it clarifies the understanding of the cognitive demand of mathematical tasks in its primary studies. It also supports this research with the claim that the tasks used in mathematics classroom profoundly influence the kinds of thinking processes in which students engage, which, in turn, impacts students learning outcomes (Stein, Grover, & Henningsen, 1996), such as success on PISA exam. Stein and Smith (2011) defined the mathematical task as a segment of classroom activity that is devoted to the development of a particular mathematical idea. They also determined that a task can involve several related problems or extended work, up to an entire class period, on a single complex problem. In this study, tasks are the exercises found on three textbooks that engaged students in mathematical thinking without presenting a giving answer, which means the students needed to think by themselves a way to offer a solution for the given problem related to the Pythagorean theorem.. 20.

(36) Cognitive Demand The core literature for this study is the article written by Stein and Smith (2011). In their study, they have developed a new framework to help teachers to reflect on their work as teachers giving mathematical tasks to students inside the mathematics classroom. One of the parts discussed in this article and that contribute to the development of this thesis is about the assumptions made by Stein and Smith that different kind of task places a different kind of cognitive demand on students. Their article introduces four different types of cognitive demand on students, defined as lower-level demands (Memorization and Procedures without connections) and Higher-level Demands (Procedures with connections and. Doing. mathematics). The contribution of Stain and Smith is used as the framework in this thesis. The authors of this thesis analyzed the exercises in the three textbooks according to the level of cognitive demand each exercise involves, but instead of using the four different categories (Memorization, Procedures without connections, Procedures with connections and Doing mathematics), the authors had a new interpretation of the four levels and rewrote a new framework to develop their analysis. The new framework is going to be presented in the Methodology chapter.. Mathematical Literacy and Mathematical Competencies. Mathematical Literacy The PISA framework used to elaborate the PISA exam is mostly worried about how students learn mathematics and how the mathematics learned can be applied by students in their everyday life’s problems. To answer questions such as: What is the best way to access student’s knowledge in mathematics, PISA assessment writers have decided to think of what would be. 21.

(37) the best topics to look at, and also the way it would be possible to best present questions for students. The answer to this question can be summarized as Mathematical Literacy. The key element of PISA assessment is Mathematical Literacy, and it was defined by OECD in 2012 as Mathematical literacy is an individual’s capacity to formulate, employ, and interpret mathematics in a variety of contexts. It includes reasoning mathematically and using mathematical concepts, procedures, facts, and tools to describe, explain, and predict phenomena. It assists individuals to recognize the role that mathematics plays in the world and to make the well-founded judgments and decisions needed by constructive, engaged and reflective citizens. (OECD 2013, p. 25) The definition of mathematics literacy has also been shown in other authors work such as the article written by Bobby Ojose (2011) where he defines it as Mathematics literacy does not imply detailed knowledge of calculus, differential equations, topology, analysis, linear algebra, abstract algebra, and complex, sophisticated mathematical formulas, but rather a broad understanding and appreciation of what mathematics is capable of achieving (p. 89). He also pointed out that: Mathematical literacy involves more that executing procedures. It implies a knowledge base and the competence and confidence to apply this understanding in the practical world. A mathematically literate person can estimate, interpret data, solve day-to-day problems, reason in numerical, graphical, and geometric situations, and communicate using mathematics. As knowledge expands and the economy evolves, more people are working with technologies or working in settings where mathematics is a cornerstone. Problem-solving, the processing of information and communication are becoming routine job requirements. Mathematics literacy is necessary both at work and in daily life. It is one of the keys to coping with a changing society. Mathematics literacy is as important as proficiency in reading and writing (Ojose, 2011, p. 91).. 22.

(38) Another definition of mathematics literacy was given by the Educational Testing Service (ETS) in 2001, which state that literacy involves “using printed and written information to function in society, to achieve one’s goals and to develop one’s potential.” ETS's Research defines three kinds of literacies: Prose literacy, Document Literacy and Quantitative Literacy (Mathematics literacy). The Prose literacy is the ability to apply your knowledge and skills for the use of new information such as in medicine labels, editorials, new stories, etc. Document literacy stands for the knowledge and skill needed to use information in a variety of records such as maps, schedules, entry forms, etc. And, Quantitative literacy means the skills and knowledge necessary to apply mathematical operations to numbers in printed formats, complete an order form, etc.. Mathematical Competence Mathematical Competence is stated by Morgan Niss (2002) as to master mathematics means to possess mathematical competence. But then, what is that? To possess a competence (to be competent) in some domain of personal, professional or social life is to master (to a fair degree, modulo the conditions and circumstances) essential aspects of life in that area. Mathematical competence then means the ability to understand, judge, do and use mathematics in a variety of intra and extra-mathematical contexts and situations in which mathematics plays or could play a role. Necessary, but certainly not sufficient, prerequisites for mathematical competence are lots of factual knowledge and technical skills, in the same way as vocabulary, orthography, and grammar is necessary but not sufficient requirements for literacy. (p. 5, 6) From the definition above given by Morgan Niss, the author of this study concluded that mathematical competencies are one of the key points for mathematics literacy, motivating its use in this research. Morgan Niss, 2002 also defined the elements of Mathematical Competence, that are the Mathematical Competencies, they were defined in the Kom project and used as a. 23.

(39) framework for many other research studies. For Morgan, Mathematical Competency is “a clearly recognizable and distinct, major constituent of mathematical competence.” Bobby Ojose (2011), in his studies of Mathematical Literacy, defined Mathematical competencies as one of the essential element of Mathematical Literacy. He also stated the mathematical competencies drawn by the Kom Project and rewritten by OECD: The competencies needed for mathematics literacy are described in the work of Program for International Students Assessment (PISA) under the auspices of OECD and are in line with the description by Steen (2001): •Mathematics Thinking and Reasoning: Posing questions characteristic of mathematics; knowing the kind of answers that mathematics offers; distinguishing among different kinds of statements; understanding and handling the extent and limits of mathematical concepts. •Mathematical Argumentation: Knowing what proofs are; knowing how proofs differ from other forms of mathematical reasoning; following and assessing chains of arguments; having a feel for heuristics; creating and expressing mathematical arguments. •Mathematical Communication: Expressing oneself in a variety of ways in oral, written, and another visual form; understanding someone else’s work. •Modeling: Structuring the field to be modeled; translating reality into mathematical structures; interpreting mathematical models regarding context or reality; working with models; validating models; reflecting, analyzing, and offering critiques of models or solutions; reflecting on the modeling process. •Problem Posing and Solving: Posing, formulating, defining, and solving problems in a variety of ways.. 24.

(40) •Representation: Decoding, Encoding, translating, distinguishing between, and interpreting different forms of representations of mathematical objects and situations as well as understanding the relationship between different representations. •Symbols: Using symbolic, formal, and technical language and operations. •Tools and Technology: Using aids and tools, including technology when appropriate. To be mathematically literate, individuals need all these competencies to varying degrees, but they also lack confidence in their ability to use mathematics and comfort with quantitative ideas. An appropriation of mathematics from historical, philosophical, and societal points of view is also desirable. (p. 98,99). Figure 1: The 'competency flower' from the KOM project (Niss, 2015, p.41). 25.

(41) . The Programme for International Student Assessment (PISA) General Information about PISA According to the Web Page of the Organization for Economic Cooperation and Development (“What is PISA”, 2016), the PISA exam is a triennial international survey which aims to evaluate education systems worldwide by testing the skills and knowledge of 15-year-old students. To date, students representing more than 80 economies have participated in the assessment. The most recently published results are from the evaluation in 2015. Around 510,000 students in 65 economies took part in the PISA 2012 assessment of reading, mathematics, and science representing about 28 million 15-year-olds globally. Of those economies, 44 participated in an evaluation of creative problem solving and 18 in an assessment of financial literacy. Seventy-two economies took part in the assessment in 2015 which focused on science. The 1st set of results from the 2015 round will be published on 6 December 2016. PISA is unique because it develops tests which are not directly linked to the school curriculum. The tests are designed to assess to what extent students at the end of compulsory education, can apply their knowledge to real-life situations and be equipped for full participation in society. The information collected through background questionnaires also provides context which can help analysts interpret the results. Besides, given PISA is an ongoing triennial survey, countries, and economies participating in successive surveys can compare their students' performance over time and assess the impact of education policy decisions. Since the year 2000, every three years, fifteen-year-old students from randomly selected schools worldwide take tests in the key subjects: reading, mathematics,. 26.

(42) and science, with a focus on one subject in each year of assessment. In 2012, some economies also participated in the optional assessments of Problem Solving and Financial Literacy. Students take a test that lasts 2 hours. The tests are a mixture of open-ended and multiplechoice questions that are organized in groups based on a passage setting out a real-life situation. A total of about 390 minutes of test items is covered. Students take different combinations of different tests. The students and their school principals also answer questionnaires to provide information about the students' backgrounds, schools and learning experiences and the broader education system and learning environment. Conclusion The textbook analysis is a valuable tool used by mathematics educators to find out the importance of textbooks, how textbooks contribute to the processes of learning and teaching, and to judge how textbooks from different countries present proper contents able to help the better students to succeed and understand mathematics. This literature review shows that textbooks are crucial in the learning process of students to help them to develop their mathematical literacy. We also read that the only way to be mathematical literate is being able to use the mathematical competencies correctly to be able to solve real-life problems and make mathematical a useful tool in every individual's lives. But no study has developed a framework to analyze textbooks according to their proposed exercises mathematical competencies and cognitive demand to determine whether those practices lead or not students to a better achievement in mathematics. Therefore, these studies aim to do so.. 27.

(43) . CHAPTER 3: METHODOLOGY Introduction. The purpose of this study is to understand through analyses of textbooks from three countries how the different mathematical competencies and level of cognitive demand required in exercises of these textbooks have influenced or not students’ performance on PISA. For that was needed the review of literature in textbook analysis and the importance of textbooks, cognitive demand, and mathematical competencies. Then the author followed with the comparison of textbooks from Brazil, Taiwan and Singapore, and application of PISA-based exams about exercises competencies and level of cognitive demand. This study was conducted to address the following research questions: 1) What is the difference between cognitive demand and mathematical competencies in textbooks from Brazil, Taiwan, and Singapore in an analysis of exercises and working examples? 2) Are there any significant differences between Brazilian and Taiwanese students’ performance and reasoning in a PISA-based exam, which only contains questions related to the Pythagorean theorem? 3) What is the relationship between the results of the PISA-based exams and the analyses of textbooks? This chapter describes the way the research was carried out through the analysis of Pythagorean theorem exercises in textbooks, and the application of a PISA-based exam.. 28.

(44) Research Design. This research study is designed as a quantitative and qualitative research which content analysis, statistical methods, and analysis of answers to exams are used as the research method. The study was divided into two main parts. For the first part, textbooks from Brazil, Taiwan, and Singapore were analyzed. The number of exercises and worked examples were counted. The level of cognitive demand was checked in the three textbooks, and finally, the level of mathematical competency needed to solve each question in the textbooks was labeled within levels varying from 0 to 3. Intercoder reliability was used in this study. The second part of this research consisted in applying a PISA-based exam for students from Brazil and Taiwan. The reliability of the results of the exams was also checked using intercoder reliability.. Procedure. Textbook Sampling. The variety of textbooks in Brazil, Singapore, and Taiwan is very different. On the one hand, Brazil does not have a national curriculum, and there are many textbooks series on the market, so schools have freedom to choose what textbooks they want to use. However, students from public schools in Brazil are going to receive free textbooks from the Brazilian government to use during the school year. In Singapore, the Ministry of Education has a list of approved textbooks including five different texts granted approval by math teachers. Taiwan does have. 29.

(45) a national curriculum, and schools are free to choose the textbooks they are going to use during the academic year. The Brazilian textbook selected for this research is from the series Tudo é Matemática (Brazilian textbook) by Luiz Roberto Dante. This textbook was chosen because it is one of the textbooks that is provided for free from the Brazilian government to several public schools in Brazil. Brazilian textbook is also one of the textbooks that are part of the Brazilian program called National Program of Textbooks (PNLD, Programa Nacional do Livro Didático). PNDL chose the series of the Brazilian textbook as one of the textbook series used in public juniorhigh schools in Brazil from the year of 2011 until 2014. Textbooks selected as part of PNLD are good textbooks that need to pass through a strict process of selection and fulfill in all characteristics the curriculum that each administrative region (states and Federal District) in Brazil follows. In Taiwan, textbooks may only be used if they are approved by the National Academy for Educational Research(. ). Nowadays there are only four textbooks series. approved, which are: Nani (. ), Kang Hsuan (. ), Han Lin (. ), and National Institute. of Compilation and Translation (NICT). According to a 2013 report of newspaper publisher China Times, the five best junior high school in Taiwan are: Taipei Municipal Zhongzheng Junior High School(. ), Taipei Municipal Dunhua Junior High School (. Municipal Jinhua Junior High School (. ), Taipei. ), Taipei Long Men Junior High School (. and Taipei Municipal Jieshou Junior High School ( junior high school are using the textbook Nani (. ),. ). Currently, three of the best five. )(Taiwanese textbook ), in the second year. of junior high school. Thus, the author decided to choose this series of textbooks to conduct the analysis.. 30.