The Experts below are selected from a list of 1161 Experts worldwide ranked by ideXlab platform

Kenneth R Koedinger - One of the best experts on this subject based on the ideXlab platform.

  • Toward a Rapid Development Environment for Cognitive Tutors
    2020
    Co-Authors: Kenneth R Koedinger
    Abstract:

    We are developing a suite of Cognitive Tutor Authoring Tools (CTAT) intended to make Tutor development both easier and faster for experienced modelers and possible for potential modelers who are not experts in Cognitive psychology or artificial intelligence programming. Our goal is to demonstrate a reduction in development time by a factor of three. We employ Human-Computer Interaction (HCI) methods and Cognitive Science principles to design development tools that are both useful and useable. Our preliminary analytic and empirical analyses compare use of CTAT with use of our current develop environment and indicate a potential reduction in development time by a factor of about two. Toward a Rapid Development Environment for Cognitive Tutors Kenneth R. KOEDINGER Vincent A.W.M.M. ALEVEN Human-Computer Interaction Institute Carnegie Mellon University Neil HEFFERNAN Computer Science Department Worcester Polytechnic University Abstract. We are developing a suite of Cognitive Tutor Authoring Tools (CTAT) intended to make Tutor development both easier and faster for experienced modelers and possible for potential modelers who are not experts in Cognitive psychology or artificial intelligence programming. Our goal is to demonstrate a reduction in development time by a factor of three. We employ Human-Computer Interaction (HCI) methods and Cognitive Science principles to design development tools that are both useful and useable. Our preliminary analytic and empirical analyses compare use of CTAT with use of our current develop environment and indicate a potential reduction in development time by a factor of about two. We are developing a suite of Cognitive Tutor Authoring Tools (CTAT) intended to make Tutor development both easier and faster for experienced modelers and possible for potential modelers who are not experts in Cognitive psychology or artificial intelligence programming. Our goal is to demonstrate a reduction in development time by a factor of three. We employ Human-Computer Interaction (HCI) methods and Cognitive Science principles to design development tools that are both useful and useable. Our preliminary analytic and empirical analyses compare use of CTAT with use of our current develop environment and indicate a potential reduction in development time by a factor of about two. Cognitive Tutors have been demonstrated to yield dramatic improvements in student learning. For example, evaluations of the Algebra Cognitive Tutor have demonstrated that students in Tutor classes outperform students in comparison classes [4]. This Tutor is being marketed and in use in over 1000 schools across the US (see www.carnegielearning.com). Despite the great potential of Cognitive Tutors to improve student learning in other areas, development of such systems is currently costly and has been limited to just a few research teams. Such teams currently require PhD level expertise in Cognitive task analysis and advanced AI programming to create the Cognitive models that drive Cognitive Tutors. We have begun to create a development environment that addresses these difficulties. Our goal is to make Tutor development both easier and faster for current developers and possible for researchers, trainers, and educators who are not experts in Cognitive psychology or AI. We are designing, implementing, and evaluating Cognitive Tutor Authoring Tools (CTAT) that will support all phases of design and development. Creating an effective development environment is as much about getting the HCI details right as it is about innovation in algorithms. We use both empirical HCI methods, like Think Aloud user studies [3], as well as analytical methods, like Keystroke Level Modeling [2], to guide interface design. We employed such methods in an earlier project [5] involving the redesign of part of RIDES [6], an authoring environment for simulation-based intelligent Tutoring systems, and were able to reduce programmer time by a factor of 2.6. 1. The Cognitive Tutor Authoring Tools Our rapid development environment, illustrated in Figure 1, consists of the following tools: An Intelligent GUI Builder, whose windows are shown in the top-left of Figure 1, can be used to create a graphical user interface (GUI) to be used in the Tutor. The modeler can use the interface to demonstrate how to carry out the task to be modeled. A Behavior Recorder (top right), which records solution paths through a given problem scenario, as the modeler demonstrates these paths in the GUI. Figure 1: The prototype Cognitive Tutor Authoring Tools A WME Editor and a Production Rule Editor, dedicated editors used to implement the production rules that model the demonstrated paths (bottom left and bottom middle). A debugging tool called the Cognitive Model Visualizer, which has two windows shown in the bottom right (“Conflict Tree” and “Rule Instantiation”). 2. Preliminary Empirical and Analytic Evaluations to Guide Design In order to get an initial impression of the savings afforded by CTAT and of the ways in which the tools might be improved, we conducted preliminary analytical and empirical evaluations. Our preliminary analysis of CTAT used a method called the Keystroke Level Model (KLM) [2]. KLM is a way of estimating the time required for expert performance on routine tasks in a computer interface. The analyst creates a detailed specification of the task at the level of keystrokes and mouse clicks and uses it to estimate the time the task will take. Time estimates derived from KLM correlate well with expert performance times [2]. Using this method, we compared our existing modeling tools, an environment called TDK [1], which has been used for over a decade to create many large-scale Cognitive Tutors to both 1) the initial CTAT environment created in four months and 2) a preliminary redesign of CTAT that was not implemented at the time of the initial evaluation. We created simplified KLM models for three common modeling tasks, namely (1) creating the initial configuration in working memory for a problem scenario, (2) writing a production rule of medium complexity, and (3) debugging why a rule that was expected to fire did not. As shown in Table 1, the KLM analysis predicts that the current CTAT will reduce the time needed to create an initial working memory configuration by a factor of 2.3. It predicts lesser savings for the other tasks. However, the results in the future CTAT column indicate significant future savings in debugging, where programmers spend much of their time. We also conducted a preliminary empirical analysis comparing the amount of time it takes to complete a modeling task with the existing TDK and the current preliminary version of CTAT. The task was to implement, test, and debug a working memory Dr r ag & dr r op i i nt t er r f f ac c e des s i i gn & i i mpl l ement t at t i i on Us s er r gui i ded gener r al l i i z z at t i i on f f or r model l des s i i gn & i i mpl l ement t at t i i on Aut t omat t ed t t es s t t i i ng Debuggi i ng v v i i s s ual l i i z z at t i i ons s C C ogni i t t i i v v e t t as s k k anal l y y s s i i s s by y demons s t t r r at t i i on Table 1: Results of a preliminary evaluation of CTAT using Keystroke Level Models: estimates of the time (in seconds) spent at the keystroke level for three commonly-occurring modeling tasks TDK Current CTAT Future CTAT Initialization 209.3 90.6 (90.6) Writing rule 203.1 207.3 130.1 Debugging 39.3 30.8 12.5 representation of a problem scenario and a single production rule. One of the authors completed this task in 50 minutes using TDK. He then did the same task using CTAT, this time taking only 15 minutes. Finally, to address the possibility of confounding effects due to learning, the same author re-did the task using the TDK tools. This time he needed 30 minutes. Thus the time savings due to CTAT were considerable. As shown in Table 2, the majority of savings occurred at the debugging stage. We anticipate that there will be further savings as we continue to develop the tools and scale up to larger evaluation studies. Table 2: Minutes spent on various sub-tasks for each trial in a preliminary empirical evaluation of CTAT. 1. TDK 2. CTAT 3. Redo TDK Initialization 10 5 8 Writing Rule 12 5 5 Testing & Debugging 28 5 17 Total 50 15 30

  • Logic-Based Natural Language Understanding for Cognitive Tutors
    2020
    Co-Authors: Octav Popescu, Vincent Aleven, Kenneth R Koedinger
    Abstract:

    High-precision Natural Language Understanding is needed in Geometry Tutoring to accurately determine the semantic content of students’ explanations. The paper presents an NLU system developed in the context of the Geometry Cognitive Tutor. The system combines unification-based syntactic processing with description logics based semantics to achieve the necessary accuracy level. The paper describes the compositional process of building the syntactic structure and the semantic interpretation of NLU explanations. It also discusses results of an evaluation of classification performance on data collected during a classroom study.

  • CELDA - ALPS: Bringing Active Inquiry into Active Problem Solving.
    2020
    Co-Authors: Scott Stevens, Kenneth R Koedinger, Angela Z Wagner, Albert T Corbett, Chih-yu Chao, Harry Ulrich, Sharon Lesgold
    Abstract:

    ABSTRACT The ALPS project (Active Learning in Problem Solving) is building and evaluating an educational technology that combines Cognitive Tutors with a novel interactive questioning environment called Synthetic Interviews. Our goal is to develop an “active learning” environment that rivals the effectiveness of human Tutors. The first version combines an existing Cognitive algebra Tutor with a virtual algebra teacher that students can query for additional help. Beta versions of the system have been used by over one hundred middle school (six, seventh, and eighth grade) students. This demonstration will allow participants to work through a simple algebra problem with the Cognitive Tutor while they ask deeper questions of the Synthetic Interview. KEYWORDS Cognitive Tutor Virtual Agent Active Learning 1. INTRODUCTION K-12 mathematics and science education is shifting from teacher-centered “learning-by-listening” to student-centered “learning-by-doing.” However, the individual attention that students require to learn with maximum efficiency is necessarily still lacking in classrooms of 20-30 students. While the peer support afforded by small-group problem solving is effective, it is less effective than the individualized support provided by the best human Tutors. The ALPS project (Active Learning in Problem Solving) is building and evaluating an educational technology that addresses this classroom need. In developing this environment, the project is building on a growing body of research regarding effective learning and Tutoring strategies. It integrates a state-of-the-art educational technology called Cognitive Tutors with a novel interactive question-answering environment called Synthetic Interviews to develop an active learning environment that rivals the effectiveness of human Tutors. Over the past 15 years, Cognitive Tutors has been highly effective in supporting learning-by-doing problem solving. Cognitive Tutors are constructed around detailed Cognitive models of the ways that students solve problems. They provide the help that students need in problem solving, both feedback on problem-solving actions and in-context problem-solving advice on request. Cognitive Tutors speed problem-based learning by as much as a factor of three (Corbett & Anderson 2001) and yield an achievement effect size of about one-standard deviation, compared with conventional instruction (Anderson et al. 1995, Corbett 2001). Student motivation and achievement in Cognitive Tutor classrooms outstrip that observed in traditional

  • Towards Tutorial Dialog to Support Self- Explanation: Adding Natural Language Understanding to a Cognitive Tutor *
    2020
    Co-Authors: Vincent Aleven, Octav Popescu, Kenneth R Koedinger
    Abstract:

    Self-explanation is an effective metaCognitive strategy, as a number of Cognitive science studies have shown. In a previous study we showed that self-explanation can be supported effectively in a Cognitive Tutor for geometry problem solving. In that study, students explained their own problem-solving steps by selecting from a menu the name of a problem-solving principle that justifies the step. They learned with greater understanding, as compared to students who did not explain their reasoning. Currently, we are working toward testing the hypothesis that students will learn even better when they provide explanations in their own words rather than selecting them from a menu. We have implemented a prototype of a Cognitive Tutor that understands students' explanations and provides feedback. The Tutor uses a knowledge-based approach to natural language understanding. We are entering a phase of pilot testing, both for the purpose of assessing the coverage of the natural language understanding component and for gaining insight into the kinds of dialog strategies that are needed.

  • User Modeling - Evaluating a Simulated Student using Real Students Data for Training and Testing
    2020
    Co-Authors: Noboru Matsuda, Jonathan Sewall, William W. Cohen, Gustavo Lacerda, Kenneth R Koedinger
    Abstract:

    SimStudent is a machine-learning agent that learns Cognitive skills by demonstration. It was originally developed as a building block of the Cognitive Tutor Authoring Tools (CTAT), so that the authors do not have to build a Cognitive model by hand, but instead simply demonstrate solutions for SimStudent to automatically generate a Cognitive model. The SimStudent technology could then be used to model human students' performance as well. To evaluate the applicability of SimStudent as a tool for modeling real students, we applied SimStudent to a genuine learning log gathered from classroom experiments with the Algebra I Cognitive Tutor. Such data can be seen as the human students' "demonstrations" of how to solve problems. The results from an empirical study show that SimStudent can indeed model human students' performance. After training on 20 problems solved by a group of human students, a Cognitive model generated by SimStudent explained 82% of the problem-solving steps performed correctly by another group of human students.

Vincent Aleven - One of the best experts on this subject based on the ideXlab platform.

  • Advances in Intelligent Tutoring Systems - Rule-Based Cognitive Modeling for Intelligent Tutoring Systems
    Studies in Computational Intelligence, 2020
    Co-Authors: Vincent Aleven
    Abstract:

    Rule-based Cognitive models serve many roles in intelligent Tutoring systems (ITS) development. They help understand student thinking and problem solving, help guide many aspects of the design of a Tutor, and can function as the “smarts” of a system. Cognitive Tutors using rule-based Cognitive models have been proven to be successful in improving student learning in a range of learning domain. The chapter focuses on key practical aspects of model development for this type of Tutors and describes two models in significant detail. First, a simple rule-based model built for fraction addition, created with the Cognitive Tutor Authoring Tools, illustrates the importance of a model’s flexibility and its Cognitive fidelity. It also illustrates the model-tracing algorithm in greater detail than many previous publications. Second, a rule-based model used in the Geometry Cognitive Tutor illustrates how ease of engineering is a second important concern shaping a model used in a large-scale Tutor. Although Cognitive fidelity and ease of engineering are sometimes at odds, overall the model used in the Geometry Cognitive Tutor meets both concerns to a significant degree. On-going work in educational data mining may lead to novel techniques for improving the Cognitive fidelity of models and thereby the effectiveness of Tutors.

  • Towards Tutorial Dialog to Support Self- Explanation: Adding Natural Language Understanding to a Cognitive Tutor *
    2020
    Co-Authors: Vincent Aleven, Octav Popescu, Kenneth R Koedinger
    Abstract:

    Self-explanation is an effective metaCognitive strategy, as a number of Cognitive science studies have shown. In a previous study we showed that self-explanation can be supported effectively in a Cognitive Tutor for geometry problem solving. In that study, students explained their own problem-solving steps by selecting from a menu the name of a problem-solving principle that justifies the step. They learned with greater understanding, as compared to students who did not explain their reasoning. Currently, we are working toward testing the hypothesis that students will learn even better when they provide explanations in their own words rather than selecting them from a menu. We have implemented a prototype of a Cognitive Tutor that understands students' explanations and provides feedback. The Tutor uses a knowledge-based approach to natural language understanding. We are entering a phase of pilot testing, both for the purpose of assessing the coverage of the natural language understanding component and for gaining insight into the kinds of dialog strategies that are needed.

  • Logic-Based Natural Language Understanding for Cognitive Tutors
    2020
    Co-Authors: Octav Popescu, Vincent Aleven, Kenneth R Koedinger
    Abstract:

    High-precision Natural Language Understanding is needed in Geometry Tutoring to accurately determine the semantic content of students’ explanations. The paper presents an NLU system developed in the context of the Geometry Cognitive Tutor. The system combines unification-based syntactic processing with description logics based semantics to achieve the necessary accuracy level. The paper describes the compositional process of building the syntactic structure and the semantic interpretation of NLU explanations. It also discusses results of an evaluation of classification performance on data collected during a classroom study.

  • Can Tutored Problem Solving Be Improved By Learning from Examples
    2020
    Co-Authors: Ron Salden, Vincent Aleven, Alexander Renkl
    Abstract:

    Can Tutored Problem Solving Be Improved By Learning from Examples? Ron Salden (rons@cs.cmu.edu) Human-Computer Interaction Institute, School of Computer Science, Carnegie Mellon University 5000 Forbes Ave, Pittsburgh, PA 15213 USA Vincent Aleven (aleven@cs.cmu.edu) Human-Computer Interaction Institute, School of Computer Science, Carnegie Mellon University 5000 Forbes Ave, Pittsburgh, PA 15213 USA Alexander Renkl (renkl@psychologie.uni-freiburg.de) Department of Psychology, University of Freiburg Engelbergerstr. 41, 79085 Freiburg, Germany the principles when they reach an impasse. Cognitive Tutors can then help to repair the knowledge gaps. Keywords: Cognitive Tutor; Worked-Out Examples Theoretical Background Experiments One very successful instructional approach which tries to optimize Cognitive skill acquisition is the use of Cognitive Tutors (e.g., Koedinger, Anderson, Hadley, & Mark, 1997). These computer-based Tutors provide individualized support for learning by doing by selecting appropriate problems to- be-solved, by providing feedback and problem-solving hints, and by on-line assessment of the student’s learning progress. While problem solving supported by Cognitive Tutors has been shown to be successful in fostering initial acquisition of Cognitive skills, it seems sub-optimal when focusing the learner on the domain principles to be learned. One instructional idea to further improve the focus on principles can be taken from the instructional model of example-based learning by Renkl (2005). The basic idea is to reduce problem-solving demands by providing worked-out solutions in the intermediate stage, when the primary instructional goal is to gain understanding. Thereby, more of the learners’ limited processing capacity can be devoted to understanding the domain principles and their application in problem solving, especially when worked-out examples are combined with self-explanation prompts. It is expected that the effectiveness of a Cognitive Tutor unit will be further enhanced when it presents faded worked-out examples to learners in the beginning of each curricular section. When studying worked-out examples, more of the learners’ limited processing capacity can be devoted to an effort to understand solution steps in terms of the application of domain principles. Assuring that learners have a basic understanding before they start to solve problems should help them to deal with the problem-solving demands by referring to already-understood principles instead of shallow strategies. The use of principles during problem solving not only enables learners to deepen their knowledge, by successfully applying it to new problems, but will also cause them to notice gaps in their understanding of A pilot study showed that both the regular Cognitive Tutor and the example-enriched version yielded the same learning and transfer effects. Interestingly, other recent studies investigating the use of examples in the Cognitive Tutor environment showed similar outcomes (e.g., Ringenberg & VanLehn, 2006). A possible explanation could be that the Cognitive Tutor itself contains a high amount of help which diminishes the possible benefits of examples to be found. We are currently running a study which compares the regular Tutor and the example-enriched version with an “unTutored” version of each. This study might reveal that when using a Cognitive Tutor which resembles problem solving as used in the literature on examples might replicate the findings in that literature. References Koedinger, K. R., Anderson, J. R., Hadley, W. H., & Mark, M. A. (1997). Intelligent Tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education, 8, 30-43. Renkl, A. (2005). The worked-out-example principle in multimedia learning. In R. Mayer (Ed.), Cambridge Handbook of Multimedia Learning. Cambridge, UK: Cambridge University Press. Ringenberg, Michael A. & VanLehn, Kurt (2006). Scaffolding Problem Solving with Annotated, Worked-Out Examples to Promote Deep Learning. Paper presented at the ITS 2006, Taiwan. Acknowledgements This work was supported in part by the Pittsburgh Science of Learning Center which is funded by the National Science Foundation award number SBE-0354420.

  • towards sensor free affect detection in Cognitive Tutor algebra
    Educational Data Mining, 2012
    Co-Authors: Ryan S Baker, Vincent Aleven, Angela Z Wagner, Sujith M Gowda, Michael Wixon, Jessica Kalka, Aatish Salvi, Gail W Kusbit, Jaclyn Ocumpaugh, Lisa M Rossi
    Abstract:

    In recent years, the usefulness of affect detection for educational software has become clear. Accurate detection of student affect can support a wide range of interventions with the potential to improve student affect, increase engagement, and improve learning. In addition, accurate detection of student affect could play an essential role in research attempting to understand the root causes and impacts of different forms of affect. However, current approaches to affect detection have largely relied upon sensor systems, which are expensive and typically not physically robust to classroom conditions, reducing their potential real-world impact. Work towards sensor-free affect detection has produced detectors that are better than chance, but not substantially better— especially when subject to stringent cross-validation processes. In this paper we present models which can detect student engaged concentration, confusion, frustration, and boredom solely from students' interactions within a Cognitive Tutor for Algebra. These detectors are designed to operate solely on the information available through students’ semantic actions within the interface, making these detectors applicable both for driving interventions and for labeling existing log files in the PSLC DataShop, facilitating future discovery with models analyses at scale.

Stephen Fancsali - One of the best experts on this subject based on the ideXlab platform.

  • EDM - Variable Construction and Causal Discovery for Cognitive Tutor Log Data: Initial Results.
    2020
    Co-Authors: Stephen Fancsali
    Abstract:

    We present a method to simultaneously search for student-level variables constructed from Cognitive Tutor log data and graphical causal models. We seek causal explanations of behavior in Cognitive Tutors, including “gaming the system” and off-task behavior, selecting variables by their contribution to causal structure and strength learning.

  • EDM - Predicting Standardized Test Scores from Cognitive Tutor Interactions.
    2020
    Co-Authors: Steven Ritter, Stephen Fancsali, Ambarish Joshi, Tristan Nixon
    Abstract:

    Cognitive Tutors are primarily developed as instructional systems, with the goal of helping students learn. However, the systems are inherently also data collection and assessment systems. In this paper, we analyze data from over 3,000 students in a school district using Carnegie Learning’s Middle School Mathematics Tutors and model performance on standardized tests. Combining a standardized pretest score with interaction data from Cognitive Tutor predicts outcomes of standardized tests better than the pretest alone. In addition, a model built using only 7th grade data and a single standardized test outcome (Virginia’s SOL) generalizes to additional grade levels (6 and 8) and standardized test outcomes (NWEA’s MAP).

  • EDM - Causal Discovery with Models: Behavior, Affect, and Learning in Cognitive Tutor Algebra
    2020
    Co-Authors: Stephen Fancsali
    Abstract:

    Non-Cognitive and behavioral phenomena, including gaming the system, off-task behavior, and affect, have proven to be important for understanding student learning outcomes. The nature of these phenomena requires investigations into their causal structure. For example, given that gaming the system has been associated with poorer learning outcomes, would reducing such behavior improve outcomes? Answering this question requires an understanding of whether gaming the system is a cause of poor outcomes, rather than, for example, only sharing a common cause with factors influencing learning. Because controlled experiments to settle such causal questions are often costly or impractical, we employ algorithmic search for the structure of graphical causal models from non-experimental data. Using sensor-free, data-driven detectors of behavior and affect, this work extends Baker and Yacef’s notion of “discovery with models” to incorporate causal discovery and reasoning, resulting in an approach we call “causal discovery with models.” We explore a case study of this approach using data from Carnegie Learning’s Cognitive Tutor for Algebra and raise questions for future research.

  • AIED Workshops - Toward "Hyper-Personalized" Cognitive Tutors: Non-Cognitive Personalization in the Generalized Intelligent Framework for Tutoring.
    2020
    Co-Authors: Stephen Fancsali, Steven Ritter, John C Stamper, Tristan Nixon
    Abstract:

    We are starting to integrate Carnegie Learning’s Cognitive Tutor (CT) into the Army Research Laboratory’s Generalized Intelligent Framework for Tutoring (GIFT), with the aim of extending the Tutoring systems to understand the impact of integrating non-Cognitive factors into our Tutoring. As part of this integration, we focus on ways in which non-Cognitive factors can be assessed, measured, and/or “detected.” This research provides the groundwork for an Office of the Secretary of Defense (OSD) Advanced Distributed Learning (ADL)-funded project on developing a “Hyper-Personalized” Intelligent Tutor (HPIT). We discuss the integration of the HPIT project with GIFT, highlighting several important questions that such integration raises for the GIFT architecture and explore several possible resolutions.

  • generalizing and extending a predictive model for standardized test scores based on Cognitive Tutor interactions
    Educational Data Mining, 2014
    Co-Authors: Ambarish Joshi, Steven Ritter, Stephen Fancsali, Tristan Nixon, Susan R Berman
    Abstract:

    Recent work demonstrates that process data from intelligent Tutoring systems (ITSs) can be used to predict student outcomes on high-stakes, standardized tests. Such models are important if ITSs are to be used for formative assessment and as replacements for external assessments. Recent work used various measures of learning efficiency and performance from problem-level, aggregate data from Carnegie Learning’s Cognitive Tutor to predict standardized test scores on the state of Virginia’s Standards of Learning exam. We generalize this model to a different school district, state, and standardized test and examine extending the model using finer-grained data.

Ryan S Baker - One of the best experts on this subject based on the ideXlab platform.

  • CogSci - Preparing Students for Effective Explaining of Worked Examples in the Genetics Cognitive Tutor
    Cognitive Science, 2020
    Co-Authors: Albert T Corbett, Angela Z Wagner, Ryan S Baker, Benjamin A Maclaren, Linda R Kauffman, Aaron P Mitchell, Sujith M Gowda
    Abstract:

    Preparing Students for Effective Explaining of Worked Examples in the Genetics Cognitive Tutor Albert Corbett (corbett@cmu.edu) Ben MacLaren (maclaren@andrew.cmu.edu) Angela Wagner (awagner@cmu.edu) Human-Computer Interaction Institute, Carnegie Mellon University Pittsburgh, PA 15213 USA Linda Kauffman (lk01@andrew.cmu.edu) Aaron Mitchell (apm1@andrew.cmu.edu) Department of Biological Sciences, Carnegie Mellon University Pittsburgh, PA 15213 USA Ryan S. J. d. Baker (rsbaker@wpi.edu) Sujith M. Gowda (sujithmg@wpi.edu) Department of Social Science and Policy Studies, Worcester Polytechnic Institute Worcester, MA 01609 USA Abstract This study examines the impact of integrating worked examples into a Cognitive Tutor for genetics problem solving, and whether a genetics process modeling task can help prepare students for explaining worked examples and solving problems. Students participated in one of four conditions in which they engaged in either: (1) process modeling followed by interleaved worked examples and problem solving; (2) process modeling followed by problem solving without worked examples; (3) interleaved worked examples and problems without process modeling; or (4) problem solving alone. Tutor data analyses reveal that process modeling led to faster reasoning and greater accuracy in explaining problem solutions. Process modeling and worked examples together led to faster reasoning in problem solving than did any of the other three conditions. Students in all conditions achieved equivalent problem-solving knowledge, as measured by posttest accuracy, although the Tutor results suggest reasoning speed may be a more sensitive measure of learning. Keywords: Education; Problem solving; Learning; Intelligent Tutors; Worked Examples. Introduction It is well-documented that integrating worked examples with problem solving, either by interleaving full problem solutions with problems to be solved (Pashler, Bain, Bottge, Graesser, Koedinger, McDaniel & Metcalfe, 2007; Sweller & Cooper, 1985) or by gradually fading the number of solved steps that are provided (Renkl & Atkinson, 2003), serves to decrease total learning time and yields improved learning outcomes. Recently, several studies have examined the benefits of incorporating worked examples into intelligent Tutoring systems (ITSs) for problem solving in a variety of domains: stoichiometry (Mclaren, Lim & Koedinger, 2008) algebra (Anthony, 2008; Corbett, Reed, Hoffman, MacLaren & Wagner, 2010b); geometry (Salden, Aleven, Schwonke & Renkl, 2010; Schwonke, Renkl, Krieg, Wittwer, Aleven & Salden, 2009; Schwonke, Renkl, Salden & Aleven, 2011) and statistics (Weitz, Salden, Kim & Heffernan, 2010). In these ITS studies, the chief benefit of incorporating worked examples has been to increase learning efficiency. The studies that report learning time universally find that interleaving worked examples (Corbett, et al, 2010b; McLaren, et al, 2008; Weitz, et al, 2010) or fading solution steps (Schwonke, et al, 2009) reduces learning time for a fixed set of activities compared to pure problem solving, primarily because students process worked solutions more rapidly than they can solve corresponding problems. But unlike the classic worked-example literature, these ITS studies generally do not find that incorporating worked examples leads to more accurate posttest problem-solving than problem solving alone (Anthony, 2008; Corbett et al, 2010b; McLaren, et al, 2008; Schwonke, et al, 2009, 2011; Weitz, et al, 2010). The exception is Salden, et al (2010), who found that adaptively fading examples based on a model of each student’s knowledge led to some relative improvement on posttest problem solving. Similarly, the evidence that students learn more deeply when worked examples are integrated into ITSs is mixed at best, although two papers report better retention of problem solving knowledge (Anthony, 2008; Salden, et al, 2010) and Schwonke, et al (2009) found evidence of greater conceptual transfer in one of two studies. The present study examines the hypothesis that • integrating worked examples and problem solving in an ITS will yield better learning outcomes when preceded by ITS learning activities that focus on domain knowledge relevant to the student explanations This study examines worked examples and problem solving in the domain of genetics. The study employs an existing Cognitive Tutor for genetics problem solving, which has

  • CogSci - Enhancing Robust Learning Through Problem Solving in the Genetics Cognitive Tutor
    Cognitive Science, 2020
    Co-Authors: Albert T Corbett, Angela Z Wagner, Benjamin A Maclaren, Linda R Kauffman, Aaron P Mitchell, Ryan S Baker
    Abstract:

    Enhancing Robust Learning Through Problem Solving in the Genetics Cognitive Tutor Albert Corbett (corbett@cmu.edu) Ben MacLaren (maclaren@andrew.cmu.edu) Angela Wagner (awagner@cmu.edu) Human-Computer Interaction Institute, Carnegie Mellon University Pittsburgh, PA 15213 USA Linda Kauffman (lk01@andrew.cmu.edu) Aaron Mitchell (apm1@andrew.cmu.edu) Department of Biological Sciences, Carnegie Mellon University Pittsburgh, PA 15213 USA Ryan S. J. d. Baker (baker2@exchange.tc.columbia.edu) Department of Human Development, Columbia University Teachers College, New York, NY 10027 Abstract In this paper, we examine the impact of three learning ac- tivities designed to foster more robust learning in a Ge- netics Cognitive Tutor module on pedigree analysis prob- lem solving, in an experimental study. The three activi- ties are (1) interleaved worked examples with student ex- planations; (2) enhanced feedback with Tutor-provided explanations of problem solving steps; and (3) explicit scaffolding of the reasoning steps in this abductive proc- ess-of-elimination reasoning task. The study included four between-subject conditions, a baseline condition in which students exclusively solved standard problems, and three conditions in which students engaged in one of the new learning activities along with standard problem solving. The scaffolded-reasoning condition was most successful in fostering robust learning, as measured by transfer, retention, and preparation for future learning tests. The enhanced feedback condition, in contrast, yielded the poorest performance on the robust learning measures. Keywords: Education; Problem solving; Robust Learn- ing; Intelligent Tutors. Introduction Problem solving is an essential learning activity across STEM domains. Successful problem solving results in “robust” knowledge: knowledge that is well-grounded in domain knowledge, and as a consequence, is well- retained by students, transfers more readily to related problem situations and prepares students for more suc- cessful future learning (Koedinger, Corbett & Perfetti, 2012). One of the well-documented risks in problem solving, across STEM domains, is that students can develop superficial knowledge that fails these tests of robust learning. In particular, when students are not well-prepared for problem solving, they can develop problem solving knowledge which focuses on surface elements in problem situations, formal representations, and features of the learning environment itself (Chang, Koedinger & Lovett, 2003; Chi, Feltovich & Glaser, 1981; Rittle-Johnson & Siegler, 1998). In this paper we examine how to structure problem solving in an intelligent Tutoring system to support ro- bust learning in the domain of genetics. Because of its foundational place in the biological sciences, genetics is a large and growing component of high school biology courses, but it is also viewed as one of the hardest top- ics in biology by both students and instructors, at the secondary and at the post-secondary level (Tsui & Treagust, 2006). Genetics problem solving is character- ized by abductive reasoning. In contrast with deductive hypothesis testing, abductive reasoning starts with a set of observations and reasons backwards to infer proper- ties of the genetic processes that produced the data (e.g., whether a trait is dominant or recessive). In this paper, we study these issues within a Tutor les- son for pedigree analysis in the Genetics Cognitive Tutor (Corbett, Kauffman, MacLaren, Wagner & Jones, 2010), which has been successfully piloted in both high school and college classrooms. Pedigree analysis relies on a complex reasoning process, which nonetheless lends itself to straightforward natural language descrip- tion. This study examines whether robust learning is supported by a scaffolded reasoning activity prior to conventional problem solving, or by incorporating ex- plicit explanations during problem solving. The Domain: Pedigree Analysis Basic pedigree analysis problems pose an interesting challenge both for students and for an intelligent Tutor- ing system. Figure 1 displays a typical pedigree analysis problem, in the Genetics Cognitive Tutor (GCT). This pedigree chart displays four generations in a small fam- ily. Females are represented as circles and males as squares. In this family, the founding parents have a daughter affected by a rare genetic trait, as represented by the dark circle. No other family members are af- fected. The student’s task is to determine whether this genetic trait is dominant or recessive, and whether it is X-linked, or transmitted on one of the twenty-two auto- somal chromosomes in humans.

  • EDM - Automatically Detecting a Student's Preparation for Future Learning: Help Use is Key
    2020
    Co-Authors: Ryan S Baker, Sujith M Gowda, Albert T Corbett
    Abstract:

    We present an automated detector that can predict a student’s later performance on a paper test of preparation for future learning, a post-test involving learning new material to solve problems involving skills that are related but different than the skills studied in the Tutoring system. This automated detector operates on features of student learning and behavior within a Cognitive Tutor for College Genetics. We show that this detector predicts preparation for future learning better than Bayesian Knowledge Tracing, a widely-used measure of student learning in Cognitive Tutors. We also find that this detector only needs limited amounts of student data (the first 20% of a student’s data from a Tutor lesson) in order to achieve a substantial proportion of its asymptotic predictive power.

  • EDM - Less is More: Improving the Speed and Prediction Power of Knowledge Tracing by Using Less Data.
    2020
    Co-Authors: Bahador B. Nooraei, Zachary A. Pardos, Neil T. Heffernan, Ryan S Baker
    Abstract:

    Knowledge Tracing is perhaps the most widely used student model in the field of educational data mining. In this paper we report on the effects of using only a subset of data in training the Bayesian Network that represents this student model. The standard practice is to use all of the students’ data for a given skill to fit the model. We analyze two datasets; one from the Algebra Cognitive Tutor and the other from the Genetics Cognitive Tutor. We found that in both datasets, the difference in accuracy between using all the students' data versus only the most recent 15 data points of each student was not significantly different. Using only 15 responses however, resulted in an EM training time which was 15 times faster than using all data. This result suggests that the Knowledge Tracing model needs only a small range of data in order to learn reliable parameters. The implications of this result is a substantial savings in model training time that allows for more complex models to be fit or individualized models to be trained online.

  • CogSci - The Antecedents of Moments of Learning.
    Cognitive Science, 2020
    Co-Authors: Gregory R. Moore, Ryan S Baker, Sujith M Gowda
    Abstract:

    In this paper, we study the antecedents of moments of particularly successful learning while students use a Cognitive Tutor for geometry. Students used the Cognitive Tutor as part of their regular classroom activities and data was collected automatically. Learning moments were operationalized as when the probability that the student just learned was extremely high, as determined by a probabilistic model: the moment-by-moment learning model. The results indicate that while self-explanation is weakly predictive of learning moments, contextual guessing and several other factors are even better predictors of learning moments. These results suggest that unexpected events in student behavior may be good predictors of changes in knowledge.

Jonathan Sewall - One of the best experts on this subject based on the ideXlab platform.

  • User Modeling - Evaluating a Simulated Student using Real Students Data for Training and Testing
    2020
    Co-Authors: Noboru Matsuda, Jonathan Sewall, William W. Cohen, Gustavo Lacerda, Kenneth R Koedinger
    Abstract:

    SimStudent is a machine-learning agent that learns Cognitive skills by demonstration. It was originally developed as a building block of the Cognitive Tutor Authoring Tools (CTAT), so that the authors do not have to build a Cognitive model by hand, but instead simply demonstrate solutions for SimStudent to automatically generate a Cognitive model. The SimStudent technology could then be used to model human students' performance as well. To evaluate the applicability of SimStudent as a tool for modeling real students, we applied SimStudent to a genuine learning log gathered from classroom experiments with the Algebra I Cognitive Tutor. Such data can be seen as the human students' "demonstrations" of how to solve problems. The results from an empirical study show that SimStudent can indeed model human students' performance. After training on 20 problems solved by a group of human students, a Cognitive model generated by SimStudent explained 82% of the problem-solving steps performed correctly by another group of human students.

  • hands on introduction to creating intelligent Tutoring systems without programming using the Cognitive Tutor authoring tools ctat
    International Conference of Learning Sciences, 2010
    Co-Authors: Vincent Aleven, Jonathan Sewall
    Abstract:

    Intelligent Tutoring systems guide learners as they practice a complex Cognitive skill. They have been shown to enhance learning in a range of domains, and are increasingly being used as platforms for learning science experiments. This workshop provides a hands-on Tutorial introduction to building Tutors using the freely available Cognitive Tutor Authoring Tools (CTAT). Using CTAT, authors can create a new type of Tutors, example-tracing Tutors, without programming. No background in computer science is required.

  • ICLS - Hands-on introduction to creating intelligent Tutoring systems without programming using the Cognitive Tutor authoring tools (CTAT)
    2010
    Co-Authors: Vincent Aleven, Jonathan Sewall
    Abstract:

    Intelligent Tutoring systems guide learners as they practice a complex Cognitive skill. They have been shown to enhance learning in a range of domains, and are increasingly being used as platforms for learning science experiments. This workshop provides a hands-on Tutorial introduction to building Tutors using the freely available Cognitive Tutor Authoring Tools (CTAT). Using CTAT, authors can create a new type of Tutors, example-tracing Tutors, without programming. No background in computer science is required.

  • Evaluating a Simulated Student Using Real Students Data for Training and Testing$^{\thanks{The research presented in this paper is supported by National Science Foundation Award No. REC-0537198.}}$
    User Modeling 2007, 2007
    Co-Authors: Noboru Matsuda, Jonathan Sewall, William W. Cohen, Gustavo Lacerda, Kenneth R Koedinger
    Abstract:

    SimStudent is a machine-learning agent that learns Cognitive skills by demonstration. It was originally developed as a building block of the Cognitive Tutor Authoring Tools (CTAT), so that the authors do not have to build a Cognitive model by hand, but instead simply demonstrate solutions for SimStudent to automatically generate a Cognitive model. The SimStudent technology could then be used to model human students' performance as well. To evaluate the applicability of SimStudent as a tool for modeling real students, we applied SimStudent to a genuine learning log gathered from classroom experiments with the Algebra I Cognitive Tutor. Such data can be seen as the human students' "demonstrations" of how to solve problems. The results from an empirical study show that SimStudent can indeed model human students' performance. After training on 20 problems solved by a group of human students, a Cognitive model generated by SimStudent explained 82% of the problem-solving steps performed correctly by another group of human students.

  • Tutorial on rapid development of intelligent Tutors using the Cognitive Tutor authoring tools ctat
    International Conference on Advanced Learning Technologies, 2006
    Co-Authors: Vincent Aleven, Bruce M Mclaren, Jonathan Sewall
    Abstract:

    Intelligent Tutoring Systems (ITS) can both help improve student learning and serve as useful platforms for experiments in learning science [1,2]. But the difficulty of building or customizing ITSs has hindered their acceptance among educators and researchers [3]. The Cognitive Tutor Authoring Tools (CTAT) project aims to provide a suite of authoring tools that make Tutor development more affordable by leveraging human-computer interaction and artificial intelligence techniques. Previous efforts on CTAT added the capability for nonprogrammers to create exampletracing Tutors via a programming-by-demonstration technique that requires no coding [4]. While exampletracing Tutors provide a student experience similar to that of the more general Cognitive Tutors, they also require that an author demonstrate and fully annotate each individual problem to be presented.