LEARN to Scale Toolkit: Invent

Overview	Identify a Need	Define Indicators of Success	Imagine a Solution	Test a Prototype

Test a Prototype

After identifying a need, defining indicators of success, and imagining a solution, you are likely ready to build an early version of your educational product to test the key ideas. Use this early version to gather authentic feedback from potential end users and then consider engaging in testing and improving your prototype, using an approach such as iterative plan-do-study-act (PDSA) cycles. These cycles are like mini-experiments that will help you to quickly learn what aspects of your prototype are working as intended and how you might improve the solution.

A goal of these mini-experiments is to start understanding the performance, reliability, convenience, and cost (PRCC) of your educational product. PRCC can guide the process of refining and adapting your product so it is ready for further development and testing. PRCC analysis is an ongoing process that bridges to the Apply stage, in which you will continue to test and adapt the product for scale.

Performance	Reliability	Convenience	Cost
refers to what kind of impact the innovation has on educational outcomes. Researchers should consider the innovation’s efficacy and how this compares with the efficacy of existing solutions.	refers to how well the innovation can be implemented in a variety of contexts and whether the innovation will be able to achieve its intended outcomes for all intended users.	refers to how easy the innovation is to implement and how smoothly it fits within the existing educational contexts for which it is intended.	refers to the financial and labor investment needed to implement the innovation.

Researchers and developers often invest most in understanding and strengthening the performance or efficacy of an educational product. However, aspects of reliability, convenience, and cost—such as ease of implementation and adaptation, seamless fit with the local context, and availability of resources to support implementation—are crucial for successful scaling of products.

Stories of Scaling Highlight

InqITS, an AI-based science program that has engaged students in 1.5 million labs to date, leveraged user engagement to identify and test the critical features of their product before building a complete scalable solution.

At first, the Inq-ITS team did not have a product that could assess and scaffold the full gamut of science practices. Sao Pedro explained, “There’s all this non-research work that needs to be done to get [the system] ready to use and scale in classrooms. We didn’t have all the necessary components. Our first customers had to be interested enough in Inq-ITS to use what was ready and be willing to overcome some of the things that weren’t ready.”

The team had to be “very scrappy” to set priorities and build the software. It’s important to distinguish what’s critical vs. what’s nice to have and can be added in the future. Based on teachers’ use of the early versions of the system, they identified the need for an LMS that allows teachers to find and assign labs, assess students’ competencies at doing science, and manage their workflows.

From Stories of Scaling: Inq-ITS

Voices from the field
“It had the right ingredients, it had early-stage traction, it showed a good effect size (though not as good as others), but it only took 4 minutes whereas others took 15 so it was the right combination of convenience and efficacy. Schools are pragmatic, and if it’s good enough and convenient and usable, they’ll use it.”
—Expert Panel Member, From Research to Market. Read more

Notice and Reflect

During the activities to test a prototype, take time to pause to notice and reflect. Consider the following questions:

Who am I testing the prototype with and what is their context? How might this shape what I learn?
What assumptions do I have about these end users? How might I test these assumptions?
What are the tensions between the efficacy the prototype and end-user accessibility and feasibility of implementation? How can I improve the prototype to address these tensions?
How will I make sure the data and feedback I gather is addressed and share this back with end users?