Microsoft Research has today unveiled Project Gecko, an initiative designed to build AI systems from the ground up. In a way that prioritizes local knowledge, language, and accessibility. The project is launching its initial phase in Kenya and India, with a sharp focus on the agriculture sector.
While existing generative AI boosts productivity across many regions, the systems are predominantly trained on data originating from online communities. This oftens resulting in poor performance in local languages and a failure to reflect the social and cultural realities of populations across Africa and South Asia.
“Building AI systems from the ground up shaped by the knowledge, languages, and modalities of the global majority yields more innovative, useful solutions for a great number of people,” explains Ashley Llorens, Corporate Vice President and Managing Director of the Microsoft Research Accelerator.
Project Gecko aims to deliver a tailorable AI that incorporates culturally relevant knowledge and engages through text, speech, and video, effectively closing this digital divide.
Microsoft has chosen agriculture as the starting point for Project Gecko, recognizing its role as a powerful economic catalyst. In Kenya, the sector contributes significantly to the national GDP and employs millions, predominantly smallholder farmers working with limited resources.
However, these farmers face major digital barriers:
- Linguistic Diversity: Farmers frequently switch between English, Kiswahili, Kikuyu, Kalenjin, Dholuo, and Maa.
- Modality: They often prefer oral instruction and visual demonstrations over text.
- Infrastructure: Low bandwidth and limited device capabilities impact the accessibility of digital tools.
Existing farming apps, largely trained in English, often fail to grasp the domain-specific nuances that change between languages and even districts, leading to incomplete or inaccurate advice.
Central to Project Gecko is the MultiModal Critical Thinking Agent (MMCTAgent). This innovative AI system enhances frontier models by allowing them to reason across audio, visuals, and text. MMCTAgent is designed to:
- Break complex agricultural questions into smaller components.
- Verify its own answers.
- Ground responses in community-generated videos and transcripts, ensuring local accuracy.
This work aligns directly with the needs of farmers, who strongly prefer voice interactions. To make this possible, Project Gecko’s team, including experts from the Microsoft Research Africa lab in Nairobi, has developed Automatic Speech Recognition (ASR) and Text-to-Speech (TTS) tools from scratch for several local languages.
The project has already expanded support to Swahili, Kikuyu, Kalenjin, Dholuo, Maa, and Somali, using a dataset of 3,000 hours of crowd-sourced Kenyan speech. Furthermore, the team uses Small Language Models (SLMs) to ensure the tools run efficiently on the low-cost devices common in rural areas.
Project Gecko builds on Digital Green’s FarmerChat, a speech-first assistant. By unlocking this knowledge base with MMCTAgent, a farmer in Nyeri County, Kenya, can now verbally ask a question in Kikuyu and receive answers in text, audio, and video.