Interactive Science through Technology Enhanced Play (iSTEP)

The project will develop and research a new Mixed Reality environment (MR), called GEM-STEP, that leverages play and embodiment as resources for integrating computational modeling into the modeling cycle as part of science instruction for elementary students. GEM stands for Generalized Embodied Modeling. Through these embodied, play-as-modeling activities, students will learn the core concepts of science, and the conceptual skills of modeling and systematic measurement. MR environments use new sensing technologies to help transform young children's physical actions during pretend play into a set of symbolic representations and parameters in a science simulation. As students physically move around the classroom, the computer will track their motion and interactions with selected objects and translate their physical activity into a shared display. For example, students pretend they are water particles and work together to model different states of matter. The children see their activity projected onto a computer simulation where a model of a water particle is displayed over the video of themselves. As students collectively reflect upon the nature of a water molecule, they refine their understanding of water as ice, a liquid or a gas. The proposed innovation allows the students to program and revise their own mixed reality simulations as part of their modeling cycle. Embodied and computational modeling will help students to reflect on their models in a unique way that will make their models more computationally accurate and enhance their understanding of the underlying concepts.

The project will research how using the body as a component of the modeling cycle differs from and interacts with the articulation of a scientific model through more structured computational means. The project will investigate the benefits of combining embodiment with computational elements in GEM:STEP by studying the range of concepts that students can learn in this manner. Lessons will be developed to address different disciplinary core ideas, such as states of matter, pollination as a complex system, or decomposition, as well as cross-cutting concepts of systems thinking, and energy/matter flow, all of which link directly to upper elementary science curriculum. Project research will gather data to understand what kinds of models students develop, what learning processes are supported using GEM:STEP, and what learning results. The data will include: (1) documenting and analyzing what students modeled and how accurate the models are; (2) recording student activity using audio and voice to code their activity to document learning processes and to look at how different forms of modeling interact with one another to promote learning; and (3) pre-post content measures to assess learning. All of the software that is developed for GEM:STEP will be made available as Open Source projects, allowing other researchers to build upon and extend this work. The results of the research will be disseminated in academic conferences and peer reviewed journals. The motion tracking software is already available on Github, a popular open-source repository. Once developed, the aim is to implement GEM:STEP in a wide range of classroom contexts, supported by a user-friendly interface, teacher guides, and professional development.

More information