Stellenbosch University & Trackosaurus

Learning by Doing: Pilot project


Stellenbosch University partnered with Trackosaurus to develop AI models to enable spoken interaction in maths and language assessment games for young children in South Africa.

Through this subgrant, Stellenbosch University will develop AI voice technology and Trackosaurus with provide the platform for the product and carry out field testing. The games will be developed in Afrikaans, isiXhosa and siSwati.


The games aim to at assess children’s foundational learning skills and to enable teachers to track skills such as counting and number recognition. As such, the project aims to reduce the burden on teachers, thus freeing up their time to provide additional support to children.

Why is this important?

Speech recognition can help automation of assessments and supporting children who are pre-reading. Using AI, assessments related to voice can be conducted and transcribed automatically. As such, the project can help provide an open-source model that can be used by others for integration into low-stakes early grade assessments or other play-based learning models.

Games that will be developed

The two types of games are:

  1. Maths game(s) with simple spoken interaction (e.g. out-loud counting, number-name recognition, spatial orientation)
  2. A Storytelling game that examines oral narrative skills.


The team will be sharing:
• Codes for the AI models
• Insights on game development processes

In addition to this, they will make the games available (where) and aim to publish results in peer-reviewed publications.

Project timeline

7 months

Key activities from Stellenbosch University:
1. Develop foundational self-supervised speech models.
2. Fine-tune the models for keyword spotting (maths games) and speech recognition (language games).
3. Develop methods for measuring language development metrics.
Key activities from Trackosaurus:
4. Develop maths game(s) to work alongside the voice technology.
5. Develop a new oral narrative game.
6. Collect preliminary game speech data to test the feasibility of implementation at scale.

Additional resources

Meet the team

Herman Kamper is an Associate Professor in Electrical and Electronic Engineering at Stellenbosch University. He is working on problems in speech and language processing, particularly for low- and zero-resource languages.

Luke Crowley is the founder of Trackosaurus. Luke had led the development of the Trackosaurus tool, establishing connections with government entities and teachers.

What is the biggest challenge you are currently facing?

There is one question that is often not asked, “Is AI really the best solution for this particular problem?” Sometimes a simpler intervention would work just as well or better. If we try to apply AI everywhere and under deliver, this could hamper AI implementation in cases where it is right approach.

Specific to this pilot, we’re operating in noisy classrooms with low-cost hardware. So, we’ll need to come up with strategies for producing “good” child speech data.

What are you hoping to get from the community?

For our project, existing child speech datasets would be a massive benefit. We also hope to get advice about navigating ethical clearances for working with child speech data. Additionally, it would be great if the community can support with identifying strategies for pushing relatively large audio files to the cloud over our middling to poor cellular networks. It’s the practical elements like this that are more daunting than the machine learning components.

AI models to enable spoken interaction in maths and language assessment games.

Pilot information
Audio: ASR
Audio: Classification

Based in South Africa


Developing products for South Africa

Contact – Stellenbosch

Dr. Herman Kamper


Associate Professor

Contact – Trackosaurus

Luke Crawley



Learning By Doing

We are providing small grants to support the development of AI products & components in LMICs. We know that innovation investment is high-risk. Our aim is that our community can benefit from the lessons learned in these pilots – what works and what doesn’t.

Learn more about our pilot projects here. We will be following each project and reporting on key learnings.

Sign Up

Join our mailing list to keep up to date with news and events.

Community          Knowledge          FAQ
          Privacy Policy was set up by Fab Inc. in partnership with Team4Tech. We are grateful to the Bill & Melinda Gates Foundation and the Jacobs Foundation for their support.

Powered by FabData.IO