Connecting Principled Information and Worked
Examples: Effects of Content Abstractness and
Solution Complexity
Hsinmei Liao
Department of Human Development and Psychology Tzu Chi University
Two experiments were conducted to investigate the effect of content abstractness (Experiment 1) and solution complexity (Experiment 2) for connecting principled information and worked examples. College students who had not learned the selected statistical distribution studied the principled information and two examples. In Experiment 1, the participants completed tasks for principle-example mapping and example-example mapping. Overall, their performance in principle-example mapping was lower than that in example-example mapping. They also committed more one-to-many errors and mapped fewer parallel contents and more nonparallel contents in principle-example mapping in comparison with those in example-example mapping. However, these effects were weaker when one example in the example-example pair was relatively complex. In Experiment 2, the participants completed tasks for principle-simple example mapping and principle-complex example mapping. Their performance in principle–complex example mapping was lower than that in principle-simple example mapping. In addition, the participants made more one-to-many errors and mapped fewer parallel elements and slightly more nonparallel elements. This study adds direct evidence to the research literature concerning the process of connecting a principle to its example. Abstractness causes principle-example mapping difficulty but not example-example mapping difficulty. Because a general element in the principle may be mapped onto several possible candidates in the example, mapping of corresponding elements between the principle and example is less accurate. However, principle-example mapping provides learners an opportunity to examine elements not demonstrated in the examples, and such learning tends to be more accurate in comparison to example-example mapping. Furthermore, pairing a simple example with the principle appears to facilitate learning of the parallel content, whereas pairing a complex example with the principle tends to promote learning of the nonparallel content. The implications of these findings are discussed.
Principled information and illustrative examples are two of the most crucial types of content for learning and instruction. Principled information conveys general types of information such as a definition, rule, theorem, principle, and formula. Illustrative examples can be used to demonstrate a concept or principle (e.g., Klausmeier & Feldman, 1975) or how a principle or rule can be used to solve a problem (e.g., Sweller & Cooper, 1985). Illustrative examples for the latter usage are commonly called worked examples (e.g., Atkinson, Derry, Renkl, & Wortham, 2000). This study is concerned with acquiring and applying knowledge to solve problems. As such, it focuses on general principles, which may be presented in different forms such as rules or theorems and worked examples that demonstrate their applications. Building connections between principled information and worked example is critical. Worked examples demonstrate the types of situations for which a principle may apply. They also demonstrate how the variables of the principle and the relations among the variables can be instantiated. Moreover, they can be used to induce the principle that underlies them. From another perspective, a principle provides a structure that categorizes examples into the same type. It also defines and constrains the elements and relations that are valid in an example. In addition, a principle provides rationales and goals underlying the solution procedures of related problems.
However, research has shown that many learners do not relate examples to the underlying principle or they do it ineffectively when they make an attempt (e.g., Chi, Bassok, Lewis, Reimann, & Glaser, 1989; Renkl, 1997). This suggests that external assistance is required to help learners build interconnections between principles and examples, particularly for poor learners. Many methods have been proposed to enhance connections between principles and examples (e.g., Catrambone & Holyoak, 1990; Gentner, Loewenstein, & Thompson, 2003; Huang & Yu, 2014; Renkl, 2002; Tu, 2011). Nevertheless, this research has been mostly concerning effectively implementing instruction with worked examples. Principled information as the learning material is typically not the target of the investigation, even though recent research has been trying to bridge the gap (e.g., De Bock, Deprez, Dooren, Roelens, & Verschaffel, 2011; Eiriksdottir & Catrambone, 2015; Kaminski, Sloutsky, & Heckler, 2008; McNeil & Fyfe, 2012). However, principled information is crucial in and of itself. Furthermore, scarce knowledge about learners’ difficulties in connecting a principle and its examples is available. Research on this topic may facilitate designing instructions that directly address students’ learning problems. Conducting such research may also provide theoretical accounts for some of the instructional practices. In this study, a rarely investigated topic was explored, that is, learning structured knowledge by mapping principled information and worked examples. The primary goal was to investigate two factors in the process that might cause the difficulty: the abstractness of the principle and the complexity of the example.
Connecting as Analogical Mapping
In analogical problem solving, the individual refers to a source problem such as the example in Appendix Part 3, which is often already solved, to gain assistance for solving the target problem, say, the problem in Appendix Part 4. For the source to be useful, it must share a common conceptual structure with the target, although the two may be depicted in different contexts. Cognitive psychologists have discovered a sequence of processes that are essential to the successful solving of analogical problems (e.g., Gentner, 1989; Gick & Holyoak, 1980). First, an appropriate analogue or source must be retrieved. Second, correct correspondences in entities and relations between the source and the target must be aligned or mapped. Third, accurate inferences about the target must be generated on the basis of the source; then, the solution (procedure) is generalized or transferred from the source to the target. Among these steps, mapping is considered the cornerstone of analogical reasoning because the congruence in structure is thereby established.
I proposed that connecting a principle and its worked examples is similar to analogical reasoning. Although the principle is general rather than specific, to understand how the example instantiates the principle, the learner must correctly conduct mapping between the principle and the example. Therefore, as a form of analogical reasoning (Gentner, 1989), mapping is the key process that enables the learner to perceive the isomorphism between the principle and the example. In this study, connecting a principle
with its example was regarded as a mapping process, and how this process was affected by the principle and the example were investigated.
Abstractness in Learning: The Principle
A major reason for the difficulty in connecting the principle with its example may be the abstractness of the principle. Relative to concrete representation, abstract representation is unfamiliar, concise, and domain-general (Goldstone & Son, 2005; Koedinger, Alibali, & Nathan, 2008). These properties may exert both positive and negative influences on the process involved in initial learning. The to-be-learned principle is not familiar to the learner (Koedinger et al., 2008); thus, he or she possesses scarce real-world knowledge that can be used to comprehend the content. Although the principled information is concise and accurate, its generality provides little information to constrain potential user actions; therefore, the search space for appropriate actions can be large (Catrambone, 1995). Consequently, learning may be inefficient because the learner must spend more time searching for appropriate instances. However, because the principle is not tied to any specific context, it is more likely to be treated as a conceptual entity than as a specific scenario, and such knowledge can be easily transferred across situations once it is acquired (e.g., Goldstone & Son, 2005).
More direct evidence regarding the effect of abstractness on connecting the principle and its example has been derived from research in which participants were given either the principle or the example(s) to learn, and they later used the principle or the example(s) to solve other problems (e.g., Reed & Bolstad, 1991; Rittle-Johnson & Alibali, 1999). For instance, in two experiments (Reed & Bolstad, 1991), the participants learned how to solve the work problem (Rate1 × Time1 + Rate2 × Time2 = Tasks Completed) using different instructional materials. Afterwards, they were tested on eight problems that differed from the work problem in terms of number of transformations (0, 1, 2, or 3) to be carried out before the above equation could be used. In the first experiment, the effects of three types of instructional material problem solution, general procedure, and the combination were compared. The problem solution consisted of the problem statement, the solution procedure, and a detailed explanation. The general procedure was close to a typical schema which included the problem type, the target principle and equation, the definitions of relevant variables, the relations between the variables, and the abstract descriptions for finding the values for the variables. Consequently, the combination and problem-solution conditions performed much better than the general-procedure condition on the 0-transformation problems and somewhat better on the 1-transformation problems. A mathematical model verified that successful problem solving was affected by the instructional material and transformation demand. In Experiment 2, more conditions were included; however, the results were consistent with the first experiment.
Other findings from this research suggest that principled information requires more time to process and poses greater cognitive demands than worked examples during initial learning, but it leads to opposite pattern of performance on transfer problems later (e.g., Catrambone, 1995). In addition, training with abstract principles typically results in more conceptual knowledge, whereas training with worked examples typically results in more procedural knowledge (Reed & Bolstad, 1991; Rittle-Johnson & Alibali, 1999). For principle training, although its effects may vary according to the implementation method, the knowledge acquired pertains to various aspects of the principle (Rittle-Johnson & Alibali, 1999). For example training, the primary improvement is on the problem category and formula. Additionally, the procedural knowledge or problem solving skills acquired by either training method is typically not sufficient for farther transfer, particularly within concept-rich domains (Reed & Bolstad, 1991). For simple concepts or principles, principle training may result in more favorable conceptual attainment and more flexible problem solving ability (Perry, 1991; Rittle-Johnson & Alibali, 1999).
Although an impression exists that relating a principle to a problem is more difficult than relating an example to a problem, the evidence does not consistently support this conclusion. Rather, evidence indicates that principles and examples may lead learners to learn about different aspects of a principle. This suggests that a combination of these two types of information may support simultaneously learning about them. To gain further understanding about this process, the first experiment examined how the
abstractness of the principle affects the difficulty and behavior of learning by mapping or comparing the principle to its example.
Complexity in Learning: The Worked Example
Except for familiarity, redundancy, and domain-specificity, which are the opposite of the common traits of principled information, worked examples may possess other features that can change their effects (e.g., Atkinson et al., 2000). The focus of this investigation is solution complexity, which is crucial for flexible problem solving in many subject domains (e.g., Rittle-Johnson & Star, 2007). Although in real situations, problem solving may require more than one principle, this research dealt with initial learning of a principle and thus problems concerning only that principle.
Knowing a principle’s complete essence typically requires studying multiple examples constructed with different contexts and highlighting the different aspects of the principle (e.g., Renkl, Stark, Gruber, & Mandl, 1998). Varying complexity of a solution procedure across worked examples is commonly adopted to meet these requirements. For instance, the problem in Appendix Part 3 is structurally similar to the part of the principle in Appendix Part 2. One can map the elements between the two structures and apply the formula directly. The problem in Appendix Part 4 first states the condition under 1000 kilometers. However, the problem asks for the probability under the condition of 5000 kilometers. To solve this problem, one must understand the part of the principle in Appendix Part 1 and restructure the relation between the length and average number; that is, find the average number under 5000-kilometer condition. Afterwards, the formula can be applied. This example is more complex solution-wise than the former example, but it reveals additional aspects of the principle. In this study, solution complexity is conceptualized as a structural factor because the original structure that connects the elements in an example must be transformed for applying the principle or formula. Although the principle-wise structure is still the same, local structural reconstruction increases the complexity of the problem and thus the solution.
Indirect evidence is available regarding the issue of solution complexity in mapping a principle with its example. In particular, research on analogical problem solving and studying multiple worked examples may facilitate an understanding of the topic. For example, Paas and Van Merriënboer (1994) investigated the effect of worked examples and example variability on transferring the acquired problem-solving skills in geometry. All the participants received general instruction on determining line length and plotting coordinates, given line length, in two-dimensional space. Next, the participants received training in one of the four conditions created by crossing training type (solving problems or studying examples) and problem variability (solving for same goal or two different goals). Finally, the participants solved six transfer problems that were different in subgoals of a hierarchical goal structure. The results showed that, during training, studying worked examples was more efficient and less demanding in mental effort than solving practice problems. During transfer test, the former solved more problems than the latter with less mental effort. More importantly, training with a variety of examples had a positive effect for the former but not for the latter.
Additional findings indicate that, if different types of problems are related in a way that their solution procedures form a hierarchy, and the complex ones provide information on how to reach the subgoals in the problems, then they facilitate forming a more detailed schema and transferring the knowledge to even less similar problems (e.g., Paas & Van Merriënboer, 1994; Reed, Stebick, Comey, & Carroll, 2012). Furthermore, the same problem type-different solution method design may be more effective than the same problem type–same solution method design in increasing learners’ conceptual knowledge and procedural flexibility (e.g., Große & Renkl, 2006). For learners without knowledge about the content domain, studying sets of examples that all highlight the same aspect, such as a problem type or solution method, or which each highlights a different aspect is beneficial for acquiring different types of knowledge (e.g., Rittle-Johnson, Star, & Durkin, 2009). For learners possessing some knowledge about the content domain, comparing methods seems to be more effective in advancing conceptual knowledge (e.g., Rittle-Johnson et al., 2009).
As suggested by the review, organizing worked examples to highlight different aspects of a principle typically leads to different learning outcomes. Furthermore, the review indicates that during the learning process, variation may be required to be provided with some prioritization. Through this, learners may expand their knowledge about the topic domain from the basic to the advanced and from a simple form to an elaborated form. Experiment 2 extended this research to principle and example mapping by investigating how the solution complexity of the worked example affects learners’ difficulty and behavior.
Experiment 1
The purpose of Experiment 1 was to investigate the effect of principle abstractness on connecting the principle with its example. Thus, tasks that required the learner to determine the connection between a principle and its example were used. The process involving the completion of the tasks was called PE mapping for the convenience of discussion. The first question of this study was whether the abstract feature of the principle makes mapping more difficult. According to the review, no clear conclusion exists regarding whether relating principled information to worked examples is more difficult than relating worked examples to worked examples (e.g., Reed & Bolstad, 1991; Rittle-Johnson & Alibali, 1999). However, Catrambone (1995) and other researchers have indicated that the generality of the entities in general information poses some uncertainty about the roles of the elements, which may in turn result in uncertainty for the relationships between or among the elements. Such uncertainty might lead to the one-to-many problem during the PE mapping process; that is, one element in the principle (e.g., a certain section in Appendix Part 2) may be mapped onto many candidates in the example (e.g., the shooting-star zone or one hour in Appendix Part 3) or vice versa. Consequently, PE mapping is made difficult.
If abstractness causes the difficulty in PE mapping, then decreasing the abstractness may make mapping easier. To achieve this goal, the principle in the PE pair can be replaced with an example. Therefore, the instance of PE mapping becomes an instance of EE mapping and EE mapping will be easier than PE mapping. However, the replacement example may not make PE mapping easier despite it being more concrete. As research has demonstrated, when the problem to be solved is similar to the worked example studied, the problem solving rate is high. However, when the problem is different from the example, surface- or solution-wise, the rate of successful problem solving decreases. Similarly, if the example has similar entities and relations and the same solution as another example in the pair, this may make EE mapping easier than PE mapping (Hypothesis A1). If the example illustrates the principle more fully, which may result in a dissimilar content and/or a dissimilar solution than the other example, making an instance of PE mapping more concrete may not make mapping easier (Hypothesis A2). For the first question, these conjectures were tested.
The next question of inquiry was how PE mapping is affected by the abstractness of the principle. Two possible outcomes were examined in this experiment. First, PE mapping may differ from EE mapping regarding the number of one-to-many errors. As discussed, the one-to-many problem is expected for PE mapping. However, for EE mapping, this problem may be trivial because the concreteness of the example may increase the clarity of the role of each entity and their relationships, thus making mapping easier. Therefore, it was expected that learners performing PE mapping would make more one-to-many errors (Hypothesis B).
The second possible outcome, which is applicable to some principles, is that PE mapping may differ from EE mapping regarding the number of parallel and nonparallel elements. An example of mapping parallel elements from Appendix Part 2 and Part 3 is mapping “a certain section” onto “one hour.” An example of nonparallel mapping from Appendix Part 1 and Part 4 is mapping “the expected value of events occurring in a continuous section is proportional to the size of the section” onto “the expected value for 5000 kilometers is 10 accidents” that is inferred from “2 events per 100 kilometers.” Due to one-to-many errors, PE mapping may lead to fewer correct alignments of the corresponding elements than EE mapping (Hypothesis C1). Although this factor may also influence the mapping of the noncorresponding elements, the effect may be overcome for the following reason. For some principles, such as the Poisson probability distribution, their content may not be fully or explicitly illustrated by
corresponding examples, or they may not have exact and clear correspondences to the examples. Therefore, noticing such facts and trying to relate the nonparallel content to the example may be easier for PE mapping than for EE mapping. For EE mapping, no such content exists to be noticed or studied. Such content may be presented in the examples by adding or changing some structural information. Nevertheless, the learner must figure out the underlying structural elements on his or her own, which may be difficult. Thus, PE mapping may yield more mapping on nonparallel elements than EE mapping (Hypothesis C2). These hypotheses were tested for the second outcome.
Methods
Design
To investigate the effect of principle abstractness, the participants performed an instance of PE mapping and an instance of EE mapping (mapping type, within-subject variable). For examining whether the abstractness effect would be reduced by the complexity of worked examples, half of the participants were randomly assigned to the simple condition and the other half to the complex condition (example complexity, between-subject variable). In the simple condition, the two examples for EE mapping were simple and similar solution-wise and domain-wise, whereas in the complex condition, the example pair differed in complexity by the solution and domain. In each condition, the order of the two types of mapping was counterbalanced (mapping order, between-subject variable).
Two mapping tasks were used, one was to measure the participants’ natural behavior of mapping (the comparison task) and another measure their mapping under some guidance (the matching task). For the comparison task, all the content, parallel or nonparallel, of the two targets were free to be mapped onto each other without any hints. However, the questions on the matching task were constructed only from the corresponding part of the text because this was the common conceptual content accessible for both PE and EE mapping. For Hypothesis A1, A2, and B, the data from the matching task were used for testing because the participants must be compared on the same set of questions to evaluate their level of performances and the amount of mistakes they made. For Hypothesis C1 and C2, the data from the comparison task were used because the participants' mapping tendencies under different conditions must be gauged with minimum constraints.
Participants
Undergraduate students of rural universities were recruited for the study. Only volunteers having no prior lessons on the selected topic were allowed to participate. The qualified volunteers were randomly assigned to one of the four groupings with 30 volunteers in each group.
Materials
The materials were provided in a booklet. The first page contained some rudimental concepts of probability. This part provided the definitions of some of the terms that would help the participants to understand the materials more favorably. The selected principle, Poisson distribution, and two examples were presented on the next two pages. Though the information was presented holistically, Example 1 was paired with the principle and Example 2, respectively, for later PE mapping and EE mapping. In other words, Example 2 was the concretized version of the principle.
The principle consisted of two parts, a Poisson random experiment and Poisson distribution (see Appendix Part 1 and Part 2). Because the experiment part did not directly correspond to the examples, its content was designated as nonparallel. In addition, its ideas were crucial albeit supplemental to the second part, thus each idea was illustrated with a concrete instance. The distribution part included content corresponding to the examples and therefore was designated as parallel. These parts were prepared according to some of the published statistics textbooks.
Each example consisted of a problem statement and the solution procedure. The problem statement and the solution procedure were written as similarly as possible to the content in the distribution part. The first example was the same for all conditions (see Appendix Part 3). It was delineated with time-domain, and its solution was obtained by simply entering the numbers into the formula. The second example was similar to the first example, domain- and solution-wise, for the simple condition and was dissimilar to the
first example for the complex condition (see Appendix Part 4). The dissimilar example was presented with a space-domain and required determining the value for one variable before applying the formula.
The next page contained survey questions for checkup purposes, and the results were not to be reported. On the next four pages, two sets of tasks were presented. One set was for PE mapping, and the other set was for EE mapping. The order of the two sets of tasks was counterbalanced, and different booklets were created for each order. The two sets of tasks were exactly the same except for the targets to be mapped. Each set comprised a comparison task and a matching task, with the comparison task always being presented before the matching task. This arrangement would prevent the influence of the guidance in the matching task on the unguided comparison task. The comparison task had two open-ended questions that each inquired into the similarities and differences between the two targets. The matching task had four short-answer questions that each inquired into a part of the target that matched with the selected part of the other target. For example, given “event A” from the principle, the participants were required to provide an answer (“seeing a shooting star”) based on the example that corresponded to the specified part of the principle.
Procedure
Experiments were conducted in groups of six or fewer. The experimenter first gave an overall introduction to the experiment. Next, the participants read the consent form and signed it if they agreed to participate. The participants were then given the booklet, followed the instructions on each page, and completed the task as required. The participants studied and completed the materials following the order of the pages. Instructions regarding whether to view or not to view the materials were given in written form and printed on the relevant pages. For the checkup questions, the participants were not allowed to turn to other pages when answering them. For the remaining questions, the participants were free to go back to the text to prevent any differential influences of memory. The participants completed the materials at their own pace. Those who completed the materials were asked to remain quiet and seated until all the other participants had finished. Throughout the experiment, the experimenter was available for assistance. At the end of the experiment, each participant received a monetary reward for his or her participation.
Analysis
In this and the next experiment, data analyses were performed using the following protocol. The planned contrast tests were conducted under the model specified in the design of the experiment. Because the order variable was not of interest and did not influence the study findings, it was not included in the analyses. PROC MIXED of SAS was used to perform the analyses, with the Kenward-Roger method for correction (Littell, Milliken, Stroup, Wolfinger, & Schabenberger, 2006) and unstructured option for the covariance structure. Throughout the analyses, Type I error was set at .05, and a two-tailed test was used for planned tests. The effect sizes and their confidence intervals were calculated using BootES developed by Kirby and Gerlanc (2013). The effect sizes were set to Hedges’ g statistics, and their 95% confidence intervals were obtained using the bias-corrected-and-accelerated bootstrap method performed for 2000 resamples.
Results
Coding and scoring
A participant’s responses for the comparison task were first segmented and then coded. A unit of response was determined by the wholeness or complexity of an idea. Some responses were simple, such as those of an “event” or a “type of event,” and some were more complex, such as those of an “event probability in a certain range” or “probability of certain number of events occurring in a certain range.” A category for “other responses” was also created, which comprised incomprehensible statements, subjective opinions, answers indicating that the volunteer did not know the answer, or no responses. Except for the responses in this category, each segmented response was also scored as being correct or incorrect.
Intercoder agreement was assessed by drawing 36 samples from the respective simple and complex conditions. Overall, the percentages of agreement for PE mapping and EE mapping for both comparison
questions ranged from 91.3% to 100.0%. The coding of the whole sample was in addition checked by the researcher. The incongruent codes were resolved through discussion among the coders and the researcher or based on the researcher’s decision.
To examine how the two types of mapping differed in terms of the type of elements mapped, the original codes were further combined into parallel and nonparallel categories, excluding the “other responses” category. In the parallel category, the mapped contents were presented in both the items. In the nonparallel category, the mapped elements were not directly mentioned in either one of the items.
For the matching task, each correctly answered question was credited with 1 point. This number-correct score was used for analyzing mapping difficulty. Another measure was created for the purpose of error analysis. For a question that wasincorrectly answered and for which the answer clearly involved some content from the other pair, the response was recorded as a mismatch. The total number represented the number of one-to-many errors committed.
Mapping difficulty
The analyses were conducted on the number-correct score of the matching task. As listed in Table 1, PE mapping yielded a lower mean than EE mapping did for the simple condition, t(118) = -3.86, p < .001, Hedges’ g = -0.51, CI [-0.75, -0.14], and for the complex condition, t(118) = -2.68, p < .01, Hedges’ g = -0.33, CI [-0.58, -0.06]. However, no difference was observed between simple and complex EE mapping, t(118) = 1.25, p > .10, Hedges’ g = 0.23, CI [-0.14, 0.58].
Table 1
Number of correct responses and one-to-many errors on the matching task in Experiment 1
Correct Error Complexity Mapping M SD M SD Simple PE 2.95 1.06 0.62 0.76 EE 3.55 0.96 0.25 0.60 Complex PE 2.90 1.05 0.77 0.74 EE 3.32 1.08 0.48 0.79
Note. n = 60 for each row.
The preceding analysis considered only the corresponding part of the pair of targets. To compare PE and EE mapping according to the whole content, a second set of analyses was conducted on the comparison task (see Table 2). Planned analysis was conducted on the total responses and the proportion of correct responses generated by the participants. Because an almost identical pattern of results was obtained for the total responses, only the results for the proportion of correct responses were reported. The tests indicated that PE mapping was less correct than EE mapping for the simple condition, t(118) = -4.02, p = .0001, Hedges’ g = -0.44, CI [-0.69, -0.18], and for the complex condition, t(118) = -2.35, p = .02, Hedges’ g = -0.37, CI [-0.62, -0.11]. Although simple EE mapping was less correct than complex EE mapping, this effect might not be reliable, t(118) = -1.78, p = .08, Hedges’ g = -0.32, CI [-0.66, 0.05]. The results for both the constrained and free mapping tasks supported the general expectation that PE mapping is more difficult than EE mapping due to the abstractness of the principle. However, they did not support the assumption that, when the example is complex, PE mapping is not more difficult than EE mapping. Therefore, Hypothesis A1 was supported but A2 was not. The rejection of Hypothesis A2 was probably because the complex EE mapping was not more difficult than the simple EE mapping, even though the simple and complex EE tasks did induce different mapping behaviors as shown in the following analysis.
Table 2
Number of total and proportion of correct responses on the comparison task in Experiment 1
Similarity Difference Total Complexity Mapping M SD M SD M SD Total response Simple PE 1.85 0.90 1.17 0.42 3.02 1.10 EE 1.85 0.82 1.62 0.90 3.47 1.28 Complex PE 2.08 0.93 1.28 0.56 3.37 1.18 EE 2.15 1.18 1.80 0.95 3.95 1.59 Correct response Simple PE 0.79 0.37 0.39 0.48 0.64 0.31 EE 0.83 0.31 0.79 0.38 0.81 0.24 Complex PE 0.88 0.21 0.64 0.47 0.78 0.24 EE 0.88 0.26 0.86 0.31 0.88 0.18
Note. n = 60 for each row.
Mapping behavior
The analyses of one-to-many errors were performed on the error scores for the matching task (see Table 1). Planned tests indicated that more errors were committed for PE mapping than for EE mapping for both the simple condition, t(118) = 3.18, p < .01, Hedges’ g = 0.42, CI [0.18, 0.66], and the complex condition, t(118) = 2.46, p =.02, Hedges’ g = 0.30, CI [0.02, 0.55]. Furthermore, the results indicated that fewer errors tended to be committed for simple EE mapping than for complex EE mapping, t(118) = -1.82, p = .07, Hedges’ g = -0.33, CI [-0.67, 0.05]. Overall, the evidence indicated that PE mapping induced more such errors than EE mapping, F(1, 118) = 15.89, p = .0001. Hypothesis B was thus supported.
The analyses for parallel and nonparallel mapping were conducted on the comparison task. Planned tests showed similar outcomes for both the number of and correct rate of responses, as shown in Table 3. Thus, only the results for the correct rates were reported. One notable result was that parallel mapping dominated nonparallel mapping across the conditions. However, for the two conditions, the patterns of outcome were different. For the simple condition, lower performance in the parallel category, t(118) = -6.01, p < .0001, Hedges’ g = -0.71, CI [-0.96, -0.45], but greater performance in the nonparallel category, t(118) = 3.35, p = .001, Hedges’ g = 0.46, CI [0.33, 0.61], was obtained for PE mapping than for EE mapping. For the complex condition, lower performance in the parallel category, t(118) = -2.01, p = .05, Hedges’ g = -0.28, CI [-0.54, -0.02], and marginally lower performance in the nonparallel category, t(118) = -1.72, p = .09, Hedges’ g = -0.21, CI [-0.47, 0.05], was obtained for PE mapping than for EE mapping. Compared with complex EE mapping, similar performance in the parallel category, t(118) = 0.12, p > .10, Hedges’ g = 0.02, CI [-0.35, 0.37], but lower performance in the nonparallel category, t(118) = -3.53, p < .001, Hedges’ g = -0.64, CI [-0.86, -0.37], was obtained for simple EE mapping.The results indicated that, with the corresponding content, PE mapping was more difficult than EE mapping for both the simple and complex conditions. These supported Hypothesis C1. With the noncorresponding content, PE mapping was slightly easier than EE mapping for the simple condition but an opposite result tended to emerge for the complex condition. Thus, Hypothesis C2 was partly supported.
Experiment 2
In this experiment, the influence of the other component, namely the worked example, on PE mapping was investigated. The first question studied was whether the solution complexity of a worked example affects the difficulty in PE mapping. As demonstrated by the reviewed studies, for novice learners, procedure complexity inconsistently contributed to the difficulty in studying worked examples (e.g., Rittle-Johnson et al., 2009). Factors such as content domains or designs of the instructional material were possibly the cause of these inconsistencies. Nevertheless, many studies have indicated that applying the acquired solution procedures to problems whose solution procedures are not identical strongly affects
the success of problem solving (e.g. Siler & Willows, 2014). This suggests that solution complexity is likely to have some effect on PE mapping.
Table 3
Number of total and proportion of correct responses in parallel and nonparallel categories on the comparison task in Experiment 1
Parallel Nonparallel Complexity Mapping M SD M SD Total response Simple PE 1.97 1.35 0.55 0.96 EE 3.00 1.38 0.12 0.49 Complex PE 2.60 1.26 0.30 0.72 EE 3.35 1.68 0.30 0.59 Correct response Simple PE 0.47 0.35 0.08 0.17 EE 0.76 0.28 0.01 0.04 Complex PE 0.66 0.26 0.04 0.12 EE 0.75 0.24 0.08 0.16
Note. n = 60 for each row.
It is apparent that a principle does not have a solution procedure but may be accompanied by a formula in its general form. When the solution procedure of a related worked example is a direct application of the formula, the example and principle have the same structure, and their “solutions” look the same. In this situation, PE mapping is expected to be relatively easy. When additional steps are required to reach a solution, such as calculating other variable values, the example and principle do not have an identical structure, and their solutions look different. In this situation, the learner has to first transform some of the entities into different entities. How the new entities are created and which of these entities are to be mapped onto the entities in the principle, or vice versa, likely create uncertainty for the learner. Consequently, the difficulty level is likely to increase for PE mapping (Hypothesis D). For the first question, this hypothesis was tested.
The second question of the investigation was how solution complexity affects PE mapping. One likely outcome was one-to-many errors. As discussed, as the problem becomes more complex, which element in the principle that should be typically mapped onto an element of the problem becomes more uncertain. Thus, more one-to-many errors were expected to occur in the complex condition than in the simple condition (Hypothesis E).
Another outcome that might be affected by solution complexity was nonparallel mapping. Experiment 1 showed that higher performance was obtained for complex EE mapping in the nonparallel category than for simple EE mapping in the same category, whereas the performance for the two conditions did not differ in the parallel category. The reason for this effect on the nonparallel content was that, in the complex EE pair, principled information that was not directly presented in the examples was implicitly displayed in the complex example through a slightly different problem structure. This prompted the participants to notice the differences between the two items and therefore increased their chances to learn about those contents. For the parallel content, because aligning correspondences for the information presented in the two examples was easy, participants’ performance was the same for the two conditions. Likely for similar reasons, equivalent performance would be yielded for the two conditions in the parallel category (Hypothesis F1). However, higher performance would be obtained for complex PE mapping in the nonparallel content than for simple PE mapping in the same category (Hypothesis F2).
Finally, a minor question was included to examine the effect of the problem context. Research has demonstrated that, when other factors are equal, a change in the problem context affects the likelihood of whether a problem will be solved (e.g., Bassok & Holyoak, 1989; Novick, 1988). Research has also demonstrated that the arrangement of the content may be a factor contributing to such an effect (e.g., Ross, 1987). In one study it was observed that participants’ notions about the application of the principle might be biased by earlier presented examples (Liao, 2014). For instance, given the example“...when the
earth passes by the shooting-star zone, visible shooting stars show up at a rate of seven stars in an hour...,” some participants mapped the element “section” in the principle onto “the shooting-star zone” instead “an hour.” This observation suggests that the context might affect the rate of correct mapping through guiding the content to be mapped. Therefore, this aspect was also examined.
Methods
Design
To investigate the effect of solution complexity, the participants completed both the simple and complex PE mapping tasks (solution complexity, within-subject variable). For examining the context effect, the participants were randomly assigned to a condition in which the context for the simple and complex pairs was time and space or space and time, respectively (example context, between-subject variable). This context pairing was crossed with the two orders of presentation: simple mapping or complex mapping first (mapping order, between-subject variable). Two tasks, one for parallel mapping (the matching task) and one for nonparallel mapping (the identification task), similar to the matching task in Experiment 1, were used because such constrained tasks provided the same basis for comparison. Both tasks were used for the analysis of mapping difficulty and parallel vs. nonparallel mapping. For the analysis of one-to-many errors, the nonparallel-mapping task was not used because there were no direct correspondences between the principle and the example.
Participants
The participants were recruited from a small university located in a rural region. The inclusion criteria were the same as those in Experiment 1. Volunteers were randomly assigned to one of the four groupings with 30 participants in each.
Materials
The materials contained in the booklet were similar to those used in Experiment 1. On the first page, the fundamental concepts of probability were presented. The next page contained a Poisson random experiment. On the third page, a Poisson distribution was presented with one of the two examples. The next two pages contained the matching and identification tasks for mapping the principle onto the example. The last three pages presented the same content as the previous three pages, except that the Poisson distribution was accompanied by the second example.
The two examples used in the booklet were constructed in the following manner. Two context domains, time and space, were each used to construct simple and complex examples. These four examples were divided into two sets, each containing one simple and one complex example with the two examples being of different contexts. If a context was used for the simple example in one set, the same context was used for the complex example in the second set. For each pairing, the order of the examples was counterbalanced.
The matching and identification tasks were used to measure mappings of the corresponding and noncorresponding content between the principle and example, respectively. Six matching questions, which were similar to those in Experiment 1, were adopted from the distribution part of the text. Five identification questions were constructed from the text for the Poisson random experiment. They were similar to the matching questions, in which the critical characteristics were individually specified and the participants were required to determine whether the example contained any of these characteristics. However, each of these characteristics involved more than one element; thus, the task measured mapping of more complex elements. To facilitate responses, two questions were posed for each characteristic. The first question simply asked whether the example endorsed the specified property (two-choice questions). The second requested for an explanation for the choice (explanation questions).
Procedure
Results
Coding and scoring
For the matching and identification tasks, a question was deemed correctly answered if the response was correct and complete. In particular, the responses for the two-choice and the explanation questions on the identification task were used to determine the score. The proportion-correct score for each task was calculated due to unequal number of problems in the two tasks and for the purpose of cross-task comparison; the score of the matching task was the score of parallel mapping, and the score of the identification task was the score of nonparallel mapping. A sum score was calculated by combining the scores of the two tasks to represent the overall mapping performance. In addition, a measure for one-to-many errors was created as in Experiment 1.
Mapping difficulty
As the sum scores in Table 4 show, simple PE mapping was easier than complex PE mapping for the space domain, t(172.2) = 1.97, p = .05, Hedges’ g = 0.37, CI [0.02, 0.72], but not for the time domain, t(172.2) = 0.77, p > .10, Hedges’ g = 0.13, CI [-0.23, 0.51]. Furthermore, the analysis did not reveal any effect of problem context, whether for the simple example, t(118) = -0.30, p > .10, Hedges’ g = -0.05, CI [-0.42, 0.30], or for the complex example, t(118) = 0.89, p > .10, Hedges’ g = 0.16, CI [-0.18, 0.54]. Further test averaging over the two contexts indicated that the effect of procedure complexity was significant, F(1, 118) = 9.55, p < .01. These results supported Hypothesis D, but did not reveal any effect of context.
Mapping behavior
A set of analyses was conducted on the number of one-to-many errors for the matching task (see Table 4). The results indicated that fewer one-to-many errors were committed for simple PE mapping than for complex PE mapping for the time context, t(228.4) = -2.95, p < .01, Hedges’ g = -0.55, CI [-0.93, -0.18], but not for the space context, t(228.4) = -0.96, p > .10, Hedges’ g = -0.17, CI [-0.55, 0.19]. Furthermore, for either mapping condition, the two example contexts did not differ (ps > .10). Overall, the two example contexts did not differ in the number of one-to-many errors committed, F(1, 118) = 0.03, p > .10. However, fewer errors were committed for simple PE mapping than for complex PE mapping, F(1, 118) = 9.34, p < .01. These results supported Hypothesis E, but did not reveal any effect of context.
The next set of analyses was performed for the mapping performance on parallel and nonparallel content (see Table 4). The results confirmed that for both simple and complex PE mapping, matching parallel content was superior to matching nonparallel content, t(118) = 19.74, p < .0001, Hedges’ g = 1.80, CI [1.39, 2.14] for the simple condition and t(118) = 13.38, p < .0001, Hedges’ g = 1.21, CI [0.88, 1.52] for the complex condition. Furthermore, a higher correct rate was achieved for simple PE mapping than for complex PE mapping for aligning the parallel content, t(118) = 5.67, p < .0001, Hedges’ g = 0.46, CI [0.20, 0.72]. However, the performance was marginally lower for simple PE mapping than for complex PE mapping for aligning the nonparallel content, t(118) = -2.31, p = .02, Hedges’ g = -0.21, CI [-0.47, 0.04]. The last two findings rejected Hypothesis F1 but tended to favor Hypothesis F2. Finally, the effect of the context on parallel and nonparallel mapping was not confirmed with the tests across all complexity levels of mapping (ps > .10).
Discussion
Summary and Discussion
The purpose of this study was to examine the role of the abstractness of a principle and the solution complexity of an example in connecting the principle with its worked example. The mapping concept in analogical problem solving was also applied to facilitate the understanding of the connecting process. Several interesting findings were obtained across the two experiments. In Experiment 1, the abstract
nature of the principle appeared to affect the mapping process. Mapping became easier when the principle was replaced with a concrete example. This was true whether the replacement example was simple or complex. These findings support the notion that concretizing the principle with an example in PE mapping might reduce mapping difficulty; however, it does not support the notion that the decrease in difficulty may be small or nonexistent when the replacement is complex (Hypothesis A1 and A2). The failure to obtain supporting evidence for the second notion might be, in part, because the complex example was not complex enough. However, the findings are consistent with those of Reed and Bolstad (1991). In that study, the complex example required three transformations in the solution. However, higher performance was obtained for the simple-and-complex example condition than for the other conditions, including the two principle-and-example conditions. Therefore, abstractness can be a powerful factor that hinders mapping and thus learning.
Table 4
Proportion of correct responses on the matching and identification tasks and number of one-to-many errors on the matching task in Experiment 2
Matching Identification Sum Error Complexity Context M SD M SD M SD M SD
Simple Time 0.65 0.28 0.14 0.18 0.42 0.18 1.22 1.12 Space 0.67 0.21 0.14 0.19 0.43 0.16 1.45 1.14 Complex Time 0.55 0.23 0.21 0.21 0.40 0.18 1.83 1.11 Space 0.55 0.27 0.15 0.21 0.37 0.17 1.65 1.20
Note. n = 60 for each row.
How abstractness affects PE mapping was demonstrated through two possible methods. First, when PE mapping was concretized to EE mapping, the number of one-to-many errors decreased substantially (Hypothesis B). This finding supports Catrambone’s (1995) argument about the generality of principled information, which was discussed earlier. Additionally, when the replacement example demonstrated more aspects of the principle, one-to-many errors tended to increase. Consequently, concretizing PE mapping reduced fewer one-to-many errors when the EE pair was complex than when the pair was simple. These results imply that, although PE mapping suffers from abstractness, and ambiguity in aligning elements is one effect, alignment problem is not restricted to PE mapping. For EE mapping, as the disparity between the two examples increases, the likelihood of mismatching increases, too.
Abstractness not only resulted in more one-to-many errors but also affected which elements were mapped. When the PE pair was concretized to the simple EE pair, mapping of corresponding elements increased, but mapping of noncorresponding elements decreased (Hypothesis C1 and C2). As discussed earlier, the disadvantage observed for parallel mapping was partly due to the generality of the principle. Generality or abstractness led to more one-to-many errors and thus resulted in fewer correct parallel mappings. However, the advantage observed for nonparallel mapping was due to the explicit presentation of the to-be-learned content by the principle. This information may not be demonstrated through the worked example in some cases and must be learned directly from the principle. Furthermore, the findings indicated that when the PE pair was concretized to the complex EE pair, the disadvantage observed for parallel mapping remained (Hypothesis C1). However, the advantage observed for nonparallel mapping dissipated, contrary to the expectation (Hypothesis C2). One possible cause for this was that the complex example in the EE pair prompted the participants to explain the differences between the two examples. Another cause was that the participants might have consulted the noncorresponding content in the principle for the explanation because this information was available during the task. If the principle had been restricted for referencing, the advantage with noncorresponding content observed for PE mapping might have been sustained. Followup research should address this issue. Together, these findings indicate that aligning corresponding content is a challenge in PE mapping, but nonetheless, the noncorresponding content has a good chance to be noted and attended and correctly learned.
In Experiment 2, the solution complexity of the worked example was found to influence PE mapping. First, increasing the solution complexity of the example was demonstrated to generally increase the difficulty in PE mapping (Hypothesis D). This finding is similar to that for analogical problem
solving, in which using a simple base to solve a complex target was less successful than when it was used to solve a simple target. Similarly, this variation was demonstrated to generally affect one-to-many errors in PE mapping (Hypothesis E). These findings support the justification that, in complex examples, more elements of the principle are involved; therefore, more mismatches between the principle and example are likely to occur.
The contents that were mapped were also influenced. For both simple and complex PE mapping, more parallel contents were correctly aligned than nonparallel contents were. This was expected because the mapping of corresponding information is relatively straightforward. Contrary to the expectation (Hypothesis F1), the simple PE pair was more powerful in facilitating parallel mapping than was the complex pair. However, the complex PE pair was slightly more powerful than the simple PE pair for promoting the mapping of noncorresponding elements (Hypothesis F2). The advantage with corresponding content observed for the simple PE pair was not the same as the finding for the simple EE pair in Experiment 1. Because correspondences within the concrete information were easy to detect, no differences were observed between the simple and complex EE pairs. For simple PE mapping, more direct correspondences existed between the items, which resulted in a higher correct rate. However, for complex PE mapping, some transformations were required in the example, which required the participants to consider the noncorresponding elements of the principle. This led to lower number of parallel elements between the items. Such complexity decreased the possibility of correct mapping. The advantage observed for complex PE mapping in the nonparallel category was consistent with the result for complex EE mapping, though the effect was less strong. The reason why this effect was not as strong might be the general nature of the principle. As proposed earlier, the content of the complex example that demonstrated the nonmutual part of the principle could be related to more than one element in the principle. This may have reduced the chance of success.
Finally, the context factor investigated in Experiment 2 did not show any effect. Although early research has found a content or context effect in problem solving, additional studies (e.g., Ross & Kilbane, 1997) have suggested that this effect might be due to the different arrangements of the problem contents. In a recent study (Fisher, Borchert, & Bassok, 2011), the researchers found that even though participants possessed the relevant domain knowledge, they constructed the equations following the semantic cues afforded by the problems. These researchers explained that the participants were compelled to use the direct translation heuristic to exploit the correspondences between the standard form of algebraic models and the linear form of verbal statements because using the heuristic to solve problems was much faster than performing real modeling. Given that in the present study, the problem statements and solution procedures were written as similarly as possible to the distribution text, the participants might also use such a heuristic. Therefore, little or no context effect was observed.
As this study demonstrates, connecting the principle and example can be a good strategy to patch the gaps in the learner’s knowledge because the principle exhibits the complete structure without unnecessary details. Other evidence suggests that this is an action that has to be demanded (Eiriksdottir & Catrambone, 2015). Further support comes from a meta-analytic study (Alfieri, Nokes-Malach, & Schunn, 2013), which reported that providing the principle after compared two worked examples further increases the learning outcome. Finally, it is expected that varying the complexity of the example will highlight different components of the principle when directly relating the principle and example in instruction. Although the effect in this study was not as strong as expected, it is consistent with the findings from the worked example research (Paas & Van Merriënboer, 1994; Reed et al., 2012; Rittle-Johnson et al., 2009). Future research is prompted to gain further insight on this issue.
Study Limitations
The principle of Poisson distribution was selected as the learning content. As described earlier, a part of the principle did not have a corresponding part explicitly expressed in the example. The comparison between parallel and nonparallel mapping using specifically formulated questions is desirable because participants’ responses for the open-ended questions may be biased to some learning content or unexpected perspectives. However, in Experiment 1, no direct mapping task was used for mapping the
noncorresponding elements. To fully understand the difference between PE mapping and EE mapping, further research must apply an appropriate measure.
Principles and their examples differ in many aspects. In this study, only one factor of each was investigated. The principle was relatively complex, involving several conceptual entities and meaningful relations connecting them. The worked example was a relatively realistic word problem. The results obtained in this study may only be generalizable to principles and examples that are similar to the principle and examples used in this study. Whether the results can be generalized to other types of material requires further research.
Theoretical Implications
Learning abstract information is challenging for many people (e.g., Tso et al., 2011). Therefore, educators and psychologists have proposed that abstract information should be learned from cases or examples. This study suggested that learning can involve abstract information. In particular, a general mechanism of analogical mapping was proposed as a method to describe the learning process. Nevertheless, in this type of learning, an abstract principle and concrete example were involved. Whether mapping the principle onto the example is the same as mapping in the reverse order is not clear at the present stage. Mapping the principle onto the example may involve deductive thinking, whereas mapping in the reverse order may involve inductive thinking. Formulation of theories of any type should await further, supporting evidence.
Practical Implications
The study results indicated that PE mapping appears to facilitate learning content for which the principle and worked example do not directly correspond. Although numerous questions await to be answered before any meaningful instructional implications can be proposed, some premature guesses may be made. The use of worked examples as an instructional approach has its own strengths and weaknesses (Atkinson et al., 2000; Atkinson & Renkl, 2007; Renkl, 2014). Similarly, the use of PE mapping also has its own strengths and weaknesses. Therefore, for this approach to fully exert its strengths and avoid its weaknesses, collaboration with other approaches is likely necessary. Furthermore, which and how other approaches are used are also crucial factors to be determined. For example, mapping a pair of simple examples may proceed mapping the principle onto a simple example so that the learner can observe in a concrete situation what the common elements and relations are and then later formulize and formalize the underlying principle with a PE mapping. In this implementation, the instructor utilizes the strength of EE mapping in its concreteness and then the strength of PE mapping in its formalism. Later, a more complex EE mapping can be introduced before a PE mapping with a complex example so that the learner will be guided to learn the other aspects of the principle in a concrete or meaningful way.
Future Research Directions
Only the outcome of PE mapping was investigated in this study. The effectiveness of PE mapping in learning the principle is still unclear. Additionally, how its effect differs from that of other types of learning approaches was not determined. Furthermore, which factors and how they mediate the effect of learning were not researched. For PE mapping to be promoted as an approach for learning or instruction, these questions must be answered.
References
Alfieri, L., Nokes-Malach, T. J., & Schunn, C. D. (2013). Learning through case comparisons: A meta-analytic review. Educational Psychologist, 48(2), 87-113. DOI: 10.1080/00461520.2013.77 5712
Atkinson, R. K., Derry, S. J., Renkl, A., & Wortham, D. (2000). Learning from examples: Instructional principles from the worked examples research. Review of Educational Research, 70(2), 181. DOI: 10.3102/00346543070002181
Atkinson, R. K., & Renkl, A. (2007). Interactive example-based learning environments: Using interactive elements to encourage effective processing of worked examples. Educational Psychology Review, 19(3), 375-386. DOI: 10.1007/s10648-007-9055-2
Bassok, M., & Holyoak, K. J. (1989). Interdomain transfer between isomorphic topics in algebra and physics. Journal of Experimental Psychology: Learning, Memory, and Cognition, 15(1), 153-166. DOI: 10.1037/0278-7393.15.1.153
Catrambone, R. (1995). Following instructions: Effects of principles and examples. Journal of Experimental Psychology: Applied, 1(3), 227-244. DOI: 10.1037/1076-898x.1.3.227
Catrambone, R., & Holyoak, K. J. (1990). Learning subgoals and methods for solving probability problems. Memory & Cognition, 18(6), 593-603. DOI: 10.3758/BF03197102
Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Self-explanations: How students study and use examples in learning to solve problems. Cognitive Science, 13(2), 145-182. DOI: 10.1207/s15516709cog1302_1
De Bock, D., Deprez, J., Dooren, W. V., Roelens, M., & Verschaffel, L. (2011). Abstract or concrete examples in learning mathematics? A replication and elaboration of Kaminski, Sloutsky, and Heckler’s study. Journal for Research in Mathematics Education, 42(2), 109-126.
Eiriksdottir, E., & Catrambone, R. (2015). The effects of timing of exposure to principles and procedural instruction specificity on learning an electrical troubleshooting skill. Journal of Experimental Psychology: Applied, 21(4), 383-394. DOI: 10.1037/xap0000065
Fisher, K. J., Borchert, K., & Bassok, M. (2011). Following the standard form: Effects of equation format on algebraic modeling. Memory & Cognition, 39(3), 502-515. DOI: 10.3758/s13421-010-0031-6 Gentner, D. (1989). The mechanisms of analogical learning. In S. Vosniadou & A. Ortony (Eds.),
Gentner, D., Loewenstein, J., & Thompson, L. (2003). Learning and transfer: A general role for analogical encoding. Journal of Educational Psychology, 95(2), 393-408. DOI: 10.1037/0022-066 3.95.2.393
Gick, M. L., & Holyoak, K. J. (1980). Analogical problem solving. Cognitive psychology, 12(3), 306-355. DOI: 10.1016/0010-0285(80)90013-4
Goldstone, R. L., & Son, J. Y. (2005). The transfer of scientific principles using concrete and idealized simulations. Journal of the Learning Sciences, 14(1), 69-110. DOI: 10.1207/s15327809jls1401_4 Große, C. S., & Renkl, A. (2006). Effects of multiple solution methods in mathematics learning. Learning
and Instruction, 16(2), 122-138. DOI: 10.1016/j.learninstruc.2006.02.001
Huang, Y. H., & Yu, X. (2014). 不同範例與解題組合對初學者在學習上的影響。[The effect of different combinations of examples and problems on novices’ learning]. Bulletin of Educational Psychology, 45(4), 497-515. DOI: 10.6251/BEP.20131002
Kaminski, J. A., Sloutsky, V. M., & Heckler, A. F. (2008). The Advantage of Abstract Examples in Learning Math. Science, 320(5875), 454-455.
Kirby, K. N., & Gerlanc, D. (2013). BootES: an R package for bootstrap confidence intervals on effect sizes. Behavior Research Methods, 45(4), 905-927. DOI: 10.3758/s13428-013-0330-5
Klausmeier, H. J., & Feldman, K. V. (1975). Effects of a definition and a varying number of examples and nonexamples on concept attainment. Journal of Educational Psychology, 67(2), 174-178. DOI: 10.1037/h0076998
Koedinger, K. R., Alibali, M. W., & Nathan, M. J. (2008). Trade-offs between grounded and abstract representations: evidence from algebra problem solving. Cognitive Science, 32(2), 366-397. DOI: 10.1080/03640210701863933
Liao, H. (2014). Learners’ attempt in connecting a probability principle and its worked example. Journal of the Humanities and Social Science, 15, 1-36.
Littell, R., Milliken, G., Stroup, W., Wolfinger, R., & Schabenberger, O. (2006). SAS for mixed models (2nd ed.). Cary, NC: SAS Institute Inc.
McNeil, N. M., & Fyfe, E. R. (2012). “Concreteness fading” promotes transfer of mathematical knowledge. Learning and Instruction, 22(6), 440-448. DOI: 10.1016/j.learninstruc.2012.05.001 Novick, L. R. (1988). Analogical transfer, problem similarity, and expertise. Journal of Experimental
Psychology: Learning, Memory, and Cognition, 14(3), 510-520. DOI: 10.1037/0278-7393.14.3.510 Paas, F. G. W. C., & Van Merriënboer, J. J. G. (1994). Variability of worked examples and transfer of
geometrical problem-solving skills: A cognitive-load approach. Journal of Educational Psychology, 86(1), 122-133. DOI: 10.1037/0022-0663.86.1.122
Perry, M. (1991). Learning and transfer: Instructional conditions and conceptual change. Cognitive Development, 6(4), 449-468. DOI: 10.1016/0885-2014(91)90049-j
Reed, S. K., & Bolstad, C. A. (1991). Use of examples and procedures in problem solving. Journal of Experimental Psychology: Learning, Memory, and Cognition, 17(4), 753-766. DOI: 10.1037/0278 -7393.17.4.753
Reed, S. K., Stebick, S., Comey, B., & Carroll, D. (2012). Finding similarities and differences in the solutions of word problems. Journal of Educational Psychology, 104(3), 636-646. DOI: 10.1037/ a0027181
Renkl, A. (1997). Learning from worked-out examples: A study on individual differences. Cognitive Science, 21(1), 1-29. DOI: 10.1207/s15516709cog2101_1
Renkl, A. (2002). Worked-out examples: instructional explanations support learning by self-explanations. Learning and Instruction, 12(5), 529-556. DOI: 10.1016/s0959-4752(01)00030-5
Renkl, A. (2014). Toward an instructionally oriented theory of example-based learning. Cognitive Science, 38(1), 1-37. DOI: 10.1111/cogs.12086
Renkl, A., Stark, R., Gruber, H., & Mandl, H. (1998). Learning from worked-out examples: The effects of example variability and elicited self-explanations. Contemporary Educational Psychology, 23(1), 90-108. DOI: 10.1006/ceps.1997.0959
Rittle-Johnson, B., & Alibali, M. W. (1999). Conceptual and procedural knowledge of mathematics: Does one lead to the other? Journal of Educational Psychology, 91(1), 175-189. DOI: 10.1037/0022-06 63.91.1.175
Rittle-Johnson, B., & Star, J. R. (2007). Does comparing solution methods facilitate conceptual and procedural knowledge? An experimental study on learning to solve equations. Journal of Educational Psychology, 99(3), 561-574. DOI: 10.1037/0022-0663.99.3.561
Rittle-Johnson, B., Star, J. R., & Durkin, K. (2009). The importance of prior knowledge when comparing examples: Influences on conceptual and procedural knowledge of equation solving. Journal of Educational Psychology, 101(4), 836-852. DOI: 10.1037/a0016026
Ross, B. H. (1987). This is like that: The use of earlier problems and the separation of similarity effects. Journal of Experimental Psychology: Learning, Memory, and Cognition, 13(4), 629-639. DOI: 10.1037/0278-7393.13.4.629
Ross, B. H., & Kilbane, M. C. (1997). Effects of principle explanation and superficial similarity on analogical mapping in problem solving. Journal of Experimental Psychology: Learning, Memory, and Cognition, 23(2), 427-440. DOI: 10.1037/0278-7393.23.2.427
