Cognitive ToyBox – Apply: Designing for Scale with Research

Invent Icon

Using Research to Address Equity
Cognitive ToyBox created a game-based version of an executive function measure originally developed by researchers from the University of Buffalo and University of Oregon. Executive function refers to mental skills we use to regulate our emotions, plan ahead, manage multiple tasks, and work toward goals. The researchers found that when teachers relied solely on observational assessment, they exhibited biases by rating Black children lower on executive function compared with non-Black students, while the game-based assessment found no difference. Their research indicates that use of Cognitive ToyBox can reduce bias in the way teachers evaluate Black children.

 

Kwan had her eye on scale from the beginning. While at NYU, she sought advice from industry mentors and took entrepreneurship classes to build out her business plan. She secured a series of grants and seed money and used those funds to build a product-focused team that could respond to needs they learned about from prospective users. After completing her degree at NYU, Kwan dedicated herself full-time to getting Cognitive ToyBox off the ground.

A pivotal step for the project was winning a National Science Foundation Innovation Corps (I-Corps) grant, which provided support to productize and commercialize the research. With these funds, Kwan spoke to over 100 early childhood educators, program administrators, families, and children, developing a deep understanding of end user needs and use cases.

The Cognitive ToyBox team used local convenience samples to learn how children interacted with their product. For example, the team set up a demo in the lobby of the nearby Brooklyn Children’s Museum so kids and families could play with an initial prototype. Most of the I-Corps-funded interviews also took place in the New York City area.

“It is important to validate the market need before conducting efficacy research, which can be quite costly.”
— Tammy Kwan

Cognitive ToyBox has since contracted out research to third parties to get more diverse samples, including in urban and rural areas. Kwan noted that assessments need to account for rural versus urban localities in terms of culture and language. Thus, the team examined the psychometric properties of their assessment instruments to make sure they would work effectively across localities. The team also partnered with researchers to investigate how use of game-based assessments could address equity gaps.

Kwan cautioned against jumping to efficacy research too early. “It is important to validate the market need before conducting efficacy research, which can be quite costly.” She advised that early stage organizations start with rapid-cycle testing for formative purposes in-house. The Cognitive ToyBox team ultimately raised more funding to work with external research partners for the efficacy research.