PrintBy Rebecca Griffiths
Cognitive ToyBox helps early childhood educators efficiently and equitably assess children through research-based technology tools. The tool, which is composed of observation and game-based assessment tools, is aligned to the 50 state early learning standards and is compliant with federal Head Start requirements on formative assessment. The company was spun out of research from NYU, and it has been working with birth to six programs for over four years to support school readiness and subsequent third grade reading success.
Invent: Iterating Toward User-Product Fit
Tammy Kwan,
Co-Founder and CEO of Cognitive Toybox.
As a graduate student at New York University (NYU), Tammy Kwan was interested in exploring ways to improve learning in early childhood education (ECE). Her first idea was to develop a physical toy to help students learn language. After spending time in classrooms and talking to teachers, administrators, children, and families, she realized that the unmet need wasn’t for objects to support learning, but for better ways to monitor children’s progress.
Kwan then developed Cognitive ToyBox, a play-based early childhood assessment tool that provides both observational and direct assessment formative feedback for educators. Today, the system is used in about half of U.S. states, and there have been two state-wide adoptions for PreK and Kindergarten Entry Assessment.
According to Kwan, early childhood assessment had been static for decades. Typically, teachers spend 4 to 6 hours per week observing children, taking notes on their development, and then entering their notes into a system. These data can be subjective, and the results aren’t action-oriented. Moreover, teachers typically review the aggregated data just two to three times a year.
Common questions facing new projects:
- How should we focus use of our limited resources in building an organization to provide the product or service?
- How should we meet the market’s demand for a comprehensive solution—integrating assessment with curriculum—given resource limitations?
- What strategies should we pursue to expand reach nationally with a small, direct sales force and policy barriers in many states?
Kwan saw an opportunity to develop a combination observational and game-based assessment platform that could be integrated into regular classroom play—providing more timely, objective, and actionable feedback on children’s learning without disrupting their day.
In addition to providing insight into children’s progress, Cognitive ToyBox is easy for teachers to learn and use. It requires only a couple hours of training and can reduce the time teachers spend on assessment paperwork. Because of these benefits, many ECE providers have adopted the system, even in states where the product has not yet been reviewed and certified to meet assessment requirements (meaning they still require a parallel assessment system for compliance).
However, as many entrepreneurs know, a great product does not guarantee success in terms of widespread adoption. Cognitive ToyBox has had to contend with common start-up questions over its 8-year journey.
Apply: Designing for Scale with Research
Using Research to Address Equity
Cognitive ToyBox created a game-based version of an executive function measure originally developed by researchers from the University at Buffalo and Oregon State University. Executive function refers to mental skills we use to regulate our emotions, plan ahead, manage multiple tasks, and work toward goals. The researchers found that when teachers relied solely on observational assessment, they rated Black children lower on self-regulation/executive function compared with non-Black students, while the game-based assessment found no difference. Their research indicates that use of Cognitive ToyBox can potentially reduce the risk of implicit bias associated with observation-only assessment tools.
Kwan had her eye on scale from the beginning. While at NYU, she sought advice from industry mentors and took entrepreneurship classes to build out her business plan. She secured a series of grants and seed money and used those funds to build a product-focused team that could respond to needs they learned about from prospective users. After completing her degree at NYU, Kwan dedicated herself full-time to getting Cognitive ToyBox off the ground.
A pivotal step for the project was winning a National Science Foundation Innovation Corps (I-Corps) grant, which provided support to productize and commercialize the research. With these funds, Kwan spoke to over 100 early childhood educators, program administrators, families, and children, developing a deep understanding of end user needs and use cases.
The Cognitive ToyBox team used local convenience samples to learn how children interacted with their product. For example, the team set up a demo in the lobby of the nearby Brooklyn Children’s Museum so kids and families could play with an initial prototype. Most of the I-Corps-funded interviews also took place in the New York City area.
— Tammy Kwan
Cognitive ToyBox has since contracted out research to third parties to get more diverse samples, including in urban and rural areas. Kwan noted that assessments need to account for rural versus urban localities in terms of culture and language. Thus, the team examined the psychometric properties of their assessment instruments to make sure they would work effectively across localities. The team also partnered with researchers to investigate how use of game-based assessments could address equity gaps.
Kwan cautioned against jumping to efficacy research too early. “It is important to validate the market need before conducting efficacy research, which can be quite costly.” She advised that early stage organizations start with rapid-cycle testing for formative purposes in-house. The Cognitive ToyBox team ultimately raised more funding to work with external research partners for the efficacy research.
Transition: Overcoming Barriers to Market Entry
Responding to Market Challenges with a “Land and Expand” Approach
In Kwan’s experience, some states only review new assessments every 7 years and provide minimal (if any) notice when these windows open up. In response, Cognitive ToyBox has employed a “land and expand” approach. The team brings on early customers who are excited enough about the product to use it without a compliance benefit. Then, these providers advocate on behalf of the company to encourage state agencies to conduct a review.
The early childhood market is highly fragmented among public and private providers at district-based programs, Head Start programs, licensed childcare homes or centers, and other types of facilities. Initially, Cognitive ToyBox primarily targeted publicly funded early childhood programs, which tend to be larger school districts or community-based organizations.
More recently, Cognitive ToyBox began supporting smaller providers, helping them satisfy licensing requirements to qualify for certain subsidies. To meet these requirements, the company formed relationships at the state level to understand the landscape and identify key decision-makers for ECE. These decision-makers could be housed in the education, health and human services, or workforce departments, or across multiple agencies.
For Kwan, state policies are the biggest barriers to entry in the market. Every state has requirements or guidelines for ECE, and each state’s policies are different. This variation presents a range of challenges that often favor the status quo and large, established providers.
— Tammy Kwan
When asked what she sees as the company’s primary competition, Kwan cited “the status quo of early childhood assessment: observation-only assessment.” Because Cognitive ToyBox provides timely formative feedback from a combination of direct observation and assessment, it entails a change in philosophy and practice for many early childhood educators. “We are in the business of changing hearts and minds around assessment approaches.”
With Cognitive ToyBox, teachers only need to designate 5 minutes per child per week to do an assessment, changing how they plan their day. Instead of two to three times a year, they can look at the data weekly, which can help them respond to student needs much more quickly.
Finding Transition and Partnership Strategies
Responding to Two Key Factors in Licensing Deals
Sales staff typically work on commission, so they tend to focus their energy on selling larger dollar, more established products. Entrepreneurs need to understand these incentives and consider ways to increase incentives for selling their products.
Mission fit is an important factor. Instead of finding a distribution partner for sales capacity alone, prioritize synergies between assessment and curriculum. Take the time to find the right fit.
To grow Cognitive ToyBox, the team expanded the product to cover math as well as early language and literacy and to provide holistic assessment. Yet offering an assessment system as a stand-alone product is not enough to satisfy early childhood educators.
“We know assessment goes hand-in-hand with curriculum,” Kwan said. Cognitive ToyBox pursued several strategies to address this need. One was to obtain curriculum goals from districts and to align the product based on those goals. For example, in Texas, the team was able to provide an alignment between the leading early childhood curriculum and Cognitive ToyBox assessments. Providing this complete solution was key to the company’s product growth.
Another challenge is large-scale sales and distribution. Licensing deals can seem appealing for a start-up company without a large sales force, but there are some key factors to consider.
Kwan had been actively looking for the right partner for several years, seeking a way to grow beyond direct sales. She prioritized synergies between assessment and curriculum and wasn’t interested in a distribution partner for sales capacity alone.
Recently, Cognitive ToyBox announced a partnership with their first curriculum provider. The curriculum has been fully integrated into their assessment system, and the partner company will be responsible for selling the bundled product.
“We took time to find the right fit,” Kwan said.
Key Takeaways
A student working with the Cognitive ToyBox.
- Spend time in educational settings to identify unmet needs. Through observing early childhood classrooms, Cognitive ToyBox founder Tammy Kwan shifted her focus to address the limitations of observation-only assessments with a combined observational and game-based assessment platform.
- Plan for scale from the start. Early on, Kwan sought advice from an industry mentor and secured an NSF Innovation Corps grant to refine the product offering and develop a commercialization strategy.
- Take a strategic approach to research. In-house rapid-cycle testing can be useful for formative purposes, while efficacy studies are more cost effective once the market need is validated. Cognitive ToyBox opted to contract research to third parties due to the distinct capabilities and timeline involved.
- Advance equity by including diverse populations in product development and research. Cognitive ToyBox conducted user testing of early prototypes in natural settings that attracted children from diverse backgrounds. They later used third parties to validate the tool in different locations.
- Understand how your solution should align with other widely used products. Cognitive ToyBox recognized the need to offer an integrated curriculum and assessment bundle, leading to a strategic partnership.
- Recruit customers to advocate with local policymakers. Enthusiastic customers helped Cognitive ToyBox get on the review list with their state agencies.
Published March 2024.