CourseKata was founded in 2017 by a group of learning scientists at UCLA and Cal State LA with a shared interest in understanding and improving how students learn things that are hard to learn. Our initial focus is on the teaching and learning of introductory statistics and data science. But our longer-term agenda is to make research on teaching and learning more relevant and more effective for improving important learning outcomes for all students.
The CourseKata Approach to Education R&D
Though much research on teaching and learning is conducted in labs over short periods of time, the development of transferable knowledge in complex domains such as mathematics, science, and statistics takes place over long periods of time – weeks, months, or even years – and in complex cultural contexts. Theories and findings from lab-based research often don't apply in the complex contexts in which students learn.
Because of this, we are developing a new approach to R&D (the Better Book approach ) that is grounded in authentic educational settings. Leveraging the power of modern web technologies, we start by developing Version 1.0 of a set of online instructional materials. Embedded in the materials are interactive exercises and formative assessment questions that generate detailed data related to students' learning over time.
We then distribute the materials widely to teachers and students, usually in the context of a college or high school course, and study implementation and learning, applying systems improvement methodologies to the continuous improvement of the online materials. By working in the context of a course, with real students and teachers and a common set of materials, we can be assured that what researchers learn will not only be published in articles, but will also be captured as incremental improvements to the online materials.
Our first implementation of this approach is in the domain of statistics and data science in colleges, universities, and high schools. Our introductory online textbook, Statistics and Data Science: A Modeling Approach (see preview here), has been used at over 50 institutions worldwide, and adoption is growing. As the number of students grows, so do opportunities for research. So far, more than 1,000 incremental improvements have been made in the textbook and in supplemental materials.
ABOUT THE BETTER BOOK PROJECT
A Team Approach to R&D
Most people who write an introductory textbook breathe a huge sigh of relief when it is done. Our approach is different: we see the book we have written as just a starting point! Our real interest is in building a community in which researchers, curriculum designers and developers, instructors, and students work together - in cycles of understanding, innovating, and improving - to continuously improve the textbook.
Most education R&D up to this point has been heavily siloed (see the animated figure below). Researchers develop theories and findings, and write them up in journal articles. Curriculum designers and developers are assumed to read these articles, and use the research to make their materials better. But most often things don't work out this way. Often, the research, because it originates in the lab, is just not easily adapted to the real world in which students learn. In the end, instructors are left to fend for themselves, figuring out how best to implement materials to achieve important learning goals.
CourseKata is working to overcome these silos. Our vision is to work in a network of collaborative teams that include researchers, designers/developers, and practitioners. Each leg of this three-legged stool is critical for success, and each brings considerable talents to the table. Yet, unless all legs work together, progress will be limited. We are working to develop a strategy, technologies, and protocols to guide and support this collaborative work.
Applying a Systems Improvement Methodology
The CourseKata approach is heavily influenced by the field of improvement science. In fact, we stole the word kata (from the Japanese, meaning routine) from Mike Rother's 2009 book, Toyota Kata, where he does a deep dive into the continuous improvement kata as practiced at Toyota. Because educational environments are complex systems, methods developed for systems improvement have a lot to contribute to the work of improving student learning.
A first step in improving systems is to understand how the system functions now (what Rother calls grasping the current condition). As such, we are less interested in decomposing the context into isolated variables, and more interested in understanding how the system works and how it could be improved. Then, with a clear goal in mind (which for us is the development of flexible and transferable knowledge), we can begin the iterative work of designing and testing improvements – in context – to help us get there.
Understanding how the system currently works requires theories of teaching and learning. From these theories we generate hypotheses, which guide experimentation toward a next (improved) condition. Our current working theory of how transferable knowledge develops – which we call the practicing connections hypothesis – is set forth in this paper. Because our work is guided by theories, some of the things we learn in the course of our improvement work can be generalized to other learning contexts and published to the field. But everything we learn can result in improvements to the online materials.
CourseKata Technology Platform
The CourseKata R&D process is supported by a robust and innovative technology platform we have been developing since 2017. This platform manages a number of functions:
- Authoring process and distributed version control using git
- Delivery of course content through common learning management systems (e.g., Canvas, Moodle, Blackboard)
- Real time feedback of student data to instructors
- Deidentified data access for researchers
- Random assignment experiments within classes of students
Applying Learning Science to the Teaching of Introductory Statistics