Sesame, the AI startup behind the widely discussed virtual assistant Maya, has released the base model that powers its advanced voice technology. The companyโs new AI model, CSM-1B, is now available under an Apache 2.0 licence, meaning it can be used commercially with minimal restrictions.
The model is built with 1 billion parameters, referring to the individual components that help process and generate responses. According to Sesame, CSM-1B produces โRVQ audio codesโ from text and audio inputs. This process, known as Residual Vector Quantisation (RVQ), converts audio into digital tokens called codes. RVQ is commonly used in AI-powered audio tools, including Googleโs SoundStream and Metaโs Encodec.
How Sesameโs AI model works
CSM-1B combines Metaโs Llama language model with an audio decoder, creating a system capable of generating realistic speech. While the base model can produce various voices, Sesame notes that it is not specifically fine-tuned for any particular voice. However, the company has developed a refined version that powers its virtual assistant, Maya.
Sesame acknowledges that the model has some capacity to understand and generate non-English languages, but this is limited due to the nature of its training data. The company has not disclosed details about the datasets used to train CSM-1B.
Despite this AI’s impressive capabilities, Sesame has implemented very few safeguards. Developers and users are urged to follow an honour system, refraining from using the model to replicate voices without consent, spread misinformation, or engage in harmful activities. However, there are no built-in restrictions to prevent misuse.
Concerns over voice cloning risks
A hands-on test of CSM-1Bโs Hugging Face demo on the AI platform revealed how quickly it can replicate a personโs voice. The cloning process took less than a minute, and from there, generating speech on various topics, including politically sensitive issues like elections and Russian propaganda, was effortless.
Consumer Reports recently raised concerns about the growing number of AI-powered voice cloning tools, warning that many lack meaningful safeguards to prevent fraud or abuse. The rapid development of these technologies has sparked discussions about the potential risks of deepfake audio and misinformation.
Sesame was co-founded by Brendan Iribe, best known as the co-creator of Oculus. The company gained widespread attention in February when Maya and its other AI assistant, Miles, were unveiled. Unlike traditional virtual assistants, these AI voices take breaths, pause naturally, and can even be interrupted mid-sentenceโfeatures similar to OpenAIโs Voice Mode, which aims to make AI interactions more human-like.
Sesame has secured funding from major investors, including Andreessen Horowitz, Spark Capital, and Matrix Partners. In addition to developing AI voice assistants, the company is also working on AI-powered smart glasses. These wearable devices, designed for all-day use, will integrate Sesameโs custom AI models to enhance user interactions.
As AI voice technology evolves, concerns over ethical use and security risks remain. With CSM-1B now open to the public, it is yet to be seen how developers will use itโand whether safeguards will eventually be put in place to prevent misuse.
