Vero AI - Real-Time Voice Agent Platform

A live customer-operations platform: businesses connect their phone numbers, and callers reach a voice agent that handles support queries in real time - on phone calls, WhatsApp calls and messages, and email, in five languages. Admins configure knowledge bases, voices, and model sources from their own console.

I was the AI engineer who made the voice chain real. Three critical early stakeholder demos ran on my machine.

1. The problem

Customer support calls at business scale, answered by an agent you can actually speak with - not a phone tree, not a chatbot with a delay. That means a full streaming chain: speech-to-text, reasoning, text-to-speech, with voice activity detection and tone handling, at conversational latency. Then the part most demos never reach: making a real phone number ring into it.

The tech stack was complex, the documentation thin, and no one on the team had made the full chain work.

2. What I built

• The full voice chain, end to end. STT (Whisper locally, OpenAI API), reasoning (Qwen3 with tool calling locally, API models where the task fit), TTS (Kokoro as the strongest open-source option, ElevenLabs and OpenAI voices as alternatives). Both the local path and the API path work; a customer can choose.
• Telephony. Inbound and outbound calls through Twilio, SIP trunking, LiveKit Agents, and Egress for recording. This is the part that turns a demo into a product: a landline dials a number and the agent picks up.
• Multimodal RAG. Document ingestion across formats with OpenAI, Cohere, and local embedding models; PostgreSQL with VectorChord and Voyager for storage and retrieval.
• Agent tools. Database access as callable tools, so the agent answers from the business’s actual systems, not from vibes.
• Model serving. LLM and embedding models deployed on an A100 with SGLang and vLLM, tuned for VRAM and concurrent inference.
• Dockerized services for LiveKit Server, SIP, the agents, and Egress - handed to DevOps for deployment.

3. The two genuinely hard problems

The chain itself. Voice activity detection and tone handling break in ways text pipelines never do, and differently on local models than on APIs. The system half-worked, then broke, for weeks. What fixed it was not a clever trick. It was method: pen and paper, the pipeline drawn and redrawn, every piece tested alone, then connected one layer at a time, tested again at every join. That method is now what I teach.

The telephone. Making the agent reachable from a real number meant learning how telecommunication actually routes: SIP trunking, Twilio configuration, environment by environment, command by command. Nobody glamorous lives in that layer. The product does.

One more, on the human side: the approach that ended up working was not the approved one at the time. I built it anyway, on my own laptop with its own GPU, and the working demo settled the argument. I was less confident then than I am now. The demo helped with that too.

4. What I’d highlight

• First on the team to get the agent speaking end to end, and first to make it answer a real phone call.
• Three successful stakeholder demos, all running locally on my machine, before the cloud deployment existed.
• Both local and API model paths shipped - the customer’s privacy and cost posture decides, not the architecture.
• Five languages: English, French, Spanish, Italian, Arabic.
• The platform is live at veroai.com. I left for Strategy& in Oct 2025; the product has kept evolving since - the cloud infrastructure was never my part, and I say so.

5. Skills demonstrated

Real-time streaming pipelines · voice activity detection · SIP trunking and telephony · LiveKit and WebRTC · local model serving (SGLang, vLLM) · multimodal RAG · tool-calling agents · Docker service architecture