Developed for the Murf AI "Voice for Bharat Challenge 2026," Sydney is a Hinglish-speaking AI/ML mentor designed to help Indian learners master technical concepts like RAG, embeddings, and backpropagation through natural voice dialogue. The system utilizes a real-time voice loop consisting of Deepgram's nova-3 STT (configured for multi-language code-switching), Google's Gemini-3.5-flash-lite LLM for low-latency reasoning, and Murf Falcon TTS (using the "Anisha" voice) for rapid response delivery. The architecture is orchestrated via LiveKit Agents, employing Silero VAD and multilingual turn detection to handle the linguistic nuances of Hinglish. Key technical features include persistent learner memory via SQLite and UUIDs passed through localStorage to maintain identity across WebRTC sessions, outbound calling capabilities, and a specialist handoff mechanism that transfers conversation context using LiveKit's chat_ctx.copy method. The platform also includes a human escalation system that triggers Discord webhooks with PII-scrubbed summaries when learners require assistance. To ensure meaningful metrics, the developer implemented a custom analytics dashboard that defines a "successful call" based on active engagement with the exercise-fetch tool rather than simple session duration.
Read original
dev.to