| Overview | Identify a Need | Define Indicators of Success | Imagine a Solution | Test a Prototype |
|---|
Define Indicators of Success
After formulating a needs hypothesis, you can begin to define how you will know whether you have addressed the identified problem of practice or need. Key performance indicators, or KPIs, define measurable outcomes that align with the identified need and goals, indicators of implementation integrity, and anticipated success timelines that are linked to implementation integrity.
Like other key activities in the Invent stage, developing KPIs is an iterative process. They should be grounded in prior research and theory, informed by end-user and stakeholder needs discovery, refined when imagining a solution, and updated through prototype development and testing.
As you develop KPIs, there are three important components to define.
| Measurable outcomes |
|---|
| What outcomes would the educational product improve if effective? Make sure these outcomes are aligned with your needs hypothesis. For example:
|
| Indicators of implementation integrity |
| What measures would suggest the educational product is likely to be efficacious once adopted? For example:
|
| Anticipated success timeline |
| If adopted, when could end users see evidence of “success”? Make sure to consider the expectations and targets of end users. For example:
|
Note, that before imagining a solution, you may not be able to identify indicators of implementation integrity. On first pass, think about what inputs are needed for users to make progress toward outcomes, for example, students spend time on a reading intervention. In your next iteration, you might update these indicators to describe what use of the product would look like in ideal settings or with unlimited capacity. As you progress through the Invent stage and test your prototype, pay close attention to what level of implementation is feasible for users given their context, capacity, and resources, and update these indicators accordingly.
While developing the KPIs, it is important to think about how your educational product will address inequities in access and outcomes in the education system. Consider:
- how your intended outcomes account for existing opportunity gaps and systemic inequities and reflect a commitment to narrowing those gaps or addressing systemic barriers;
- how feasible your indicators of implementation fidelity might be amid resource constraints or in underserved contexts; and
- how your anticipated success timeline is both ambitious and accounts for varying levels of readiness and starting capacities.
Also, continued engagement of end users and other stakeholders is essential while developing the KPIs. If the KPIs are misaligned with the outcomes that schools, districts, and state education agencies care about and their contexts, the educational product bears the risk of not achieving product-user fit, which then creates challenges with subsequent product adoption and scale.
Notice and Reflect
While defining KPIs, take time to pause to notice and reflect. Consider the following questions:
- How have my personal experiences informed what is being valued as an outcome? What assumptions have I made in developing the KPIs?
- What outcomes do my end users care about? Do these outcomes differ by user? How do these outcomes differ for different groups of students?
- How do the KPIs align with the stated goals of schools and districts?
- How feasible are the indicators of implementation fidelity in different school environments, with end users of varying capacity levels, and with different student groups?
Spotlight Resource
KPI Worksheet. This resource provides a template and examples to help you draft your KPIs.