Invent: Iterating Toward User-Product Fit
As a graduate student at New York University (NYU), Tammy Kwan was interested in exploring ways to improve learning in early childhood education (ECE). Her first idea was to develop a physical toy to help students learn language. After spending time in classrooms and talking to teachers, administrators, children, and families, she realized that the unmet need wasn’t for objects to support learning, but for better ways to monitor children’s progress.
Kwan then developed Cognitive ToyBox, a play-based early childhood assessment tool that provides both observational and direct assessment formative feedback for educators. Today, the system is used in about half of U.S. states, and there have been two state-wide adoptions.
According to Kwan, early childhood assessment had been static for decades. Typically, teachers spend 4 to 6 hours per week observing children, taking notes on their development, and then entering their notes into a system. These data are often subjective, and the results aren’t action-oriented. Moreover, teachers typically review the aggregated data just two to three times a year.
Kwan saw an opportunity to develop a combination observational and game-based assessment platform that could be integrated into regular classroom play—providing more timely, objective, and actionable feedback on children’s learning without disrupting their day.
In addition to providing insight into children’s progress, Cognitive ToyBox is easy for teachers to learn and use. It requires only a couple hours of training and can reduce the time teachers spend on assessment paperwork. Because of these benefits, many ECE providers have adopted the system, even in states where the product has not yet been reviewed and certified to meet assessment requirements (meaning they still require a parallel assessment system for compliance).
However, as many entrepreneurs know, a great product does not guarantee success in terms of widespread adoption. Cognitive ToyBox has had to contend with common start-up questions over its 8-year journey.
Apply: Designing for Scale with Research
Kwan had her eye on scale from the beginning. While at NYU, she sought advice from industry mentors and took entrepreneurship classes to build out her business plan. She secured a series of grants and seed money and used those funds to build a product-focused team that could respond to needs they learned about from prospective users. After completing her degree at NYU, Kwan dedicated herself full-time to getting Cognitive ToyBox off the ground.
A pivotal step for the project was winning a National Science Foundation Innovation Corps (I-Corps) grant, which provided support to productize and commercialize the research. With these funds, Kwan spoke to over 100 early childhood educators, program administrators, families, and children, developing a deep understanding of end user needs and use cases.
The Cognitive ToyBox team used local convenience samples to learn how children interacted with their product. For example, the team set up a demo in the lobby of the nearby Brooklyn Children’s Museum so kids and families could play with an initial prototype. Most of the I-Corps-funded interviews also took place in the New York City area.
— Tammy Kwan
Cognitive ToyBox has since contracted out research to third parties to get more diverse samples, including in urban and rural areas. Kwan noted that assessments need to account for rural versus urban localities in terms of culture and language. Thus, the team examined the psychometric properties of their assessment instruments to make sure they would work effectively across localities. The team also partnered with researchers to investigate how use of game-based assessments could address equity gaps.
Kwan cautioned against jumping to efficacy research too early. “It is important to validate the market need before conducting efficacy research, which can be quite costly.” She advised that early stage organizations start with rapid-cycle testing for formative purposes in-house. The Cognitive ToyBox team ultimately raised more funding to work with external research partners for the efficacy research.
Transition: Overcoming Barriers to Market Entry
The early childhood market is highly fragmented among public and private providers at district-based programs, Head Start programs, licensed childcare homes or centers, and other types of facilities. Initially, Cognitive ToyBox primarily targeted publicly funded early childhood programs, which tend to be larger school districts or community-based organizations.
More recently, Cognitive ToyBox began supporting smaller providers, helping them satisfy licensing requirements to qualify for certain subsidies. To meet these requirements, the company formed relationships at the state level to understand the landscape and identify key decision-makers for ECE. These decision-makers could be housed in the education, health and human services, or workforce departments, or across multiple agencies.
For Kwan, state policies are the biggest barriers to entry in the market. Every state has requirements or guidelines for ECE, and each state’s policies are different. This variation presents a range of challenges that often favor the status quo and large, established providers.
— Tammy Kwan
When asked what she sees as the company’s primary competition, Kwan cited “the status quo of early childhood assessment: observational assessment.” Because Cognitive ToyBox provides timely formative feedback from a combination of direct observation and assessment, it entails a change in philosophy and practice for many early childhood educators. “We are in the business of changing hearts and minds around assessment approaches.”
With Cognitive ToyBox, teachers only need to designate 5 minutes per child to do an assessment, changing how they plan their day. Instead of two to three times a year, they can look at the data weekly, which can help them respond to student needs much more quickly.
Finding Transition and Partnership Strategies
To grow Cognitive ToyBox, the team expanded the product to cover math as well as early language and literacy and to provide holistic assessment. Yet offering an assessment system as a stand-alone product is not enough to satisfy early childhood educators.
“We know assessment goes hand-in-hand with curriculum,” Kwan said. Cognitive ToyBox pursued several strategies to address this need. One was to obtain curriculum goals from districts and to align the product based on those goals. For example, in Texas, the team was able to provide an alignment between the leading early childhood curriculum and Cognitive ToyBox assessments. Providing this complete solution was key to the company’s product growth.
Another challenge is large-scale sales and distribution. Licensing deals can seem appealing for a start-up company without a large sales force, but there are some key factors to consider.
Kwan had been actively looking for the right partner for several years, seeking a way to grow beyond direct sales. She prioritized synergies between assessment and curriculum and wasn’t interested in a distribution partner for sales capacity alone.
Recently, Cognitive ToyBox announced a partnership with their first curriculum provider. The curriculum has been fully integrated into their assessment system, and the partner company will be responsible for selling the bundled product.
“We took time to find the right fit,” Kwan said.