AI audioCAN I VIBECODE
Resemble AI?
BUILD FIRST
- A non-sensitive or read-only version of the core experience
- A manual review step wherever correctness or trust matters
- A narrow companion tool that removes a real piece of work
LEAVE FOR LATER
- Regulated decisions, payments, or irreversible actions
- Claims of correctness, compliance, or uptime that the MVP cannot support
- Sensitive integrations until security, support, and auditability are ready
RESEARCH NOTES
What this tool actually does.
Resemble AI is an AI audio platform offering voice cloning, text-to-speech (TTS), speech-to-speech, audio enhancement, WebSocket streaming, and transcript intelligence. Its core workflow: upload 10 seconds–3 minutes of audio (via dataset file or individual recordings) to train a voice clone in under 1 minute; training can notify via callback webhook and reports quality metrics (STOI, PESQ, SI-SDR). Two tiers exist: Rapid Clone (10s, ~1 min training) and Professional Clone (10–25+ min audio, ~40 min training, fuller emotional range). Cloned voices support 23 languages without retraining, custom pronunciation rules, and natural-language per-use-case variants (conversational, commercial, phone agent). It provides cloud API, open-source self-hosted Chatterbox (MIT-licensed) with PerTh watermarking, and on-prem Docker/Kubernetes options. The platform includes Voices, Recordings, Clips, and Pr
RESEARCHED BUILD PROMPT
Give this to your coding agent.
Build a web application that demonstrates Resemble AI's core voice-cloning-to-narration workflow: a user uploads a 10-second to 3-minute clear audio sample, the system trains a per-voice synthetic voice model, and the user then renders new text into speech with that cloned voice. Do not copy Resemble AI's brand, trademark, UI design, or proprietary code; build an original clean interface.
User-facing requirements:
- Voice creation page: upload one or more audio files, optional voice name/description, mandatory consent checkbox confirming the uploader owns or has rights to the voice.
- Training progress indicator: show queued/training/ready states via polling or Server-Sent Events; simulate or actually run model training behind a job queue.
- Text-to-speech page: select a ready cloned voice, type text, submit, and play/download the generated audio.
- Voice management: list cloned voices with status, delete, and rename.
- Optional: pronunciation override per voice (simple word-to-phoneme mapping).
Technical requirements:
- Use any stack (Python/Node backend + React/Vue frontend is fine).
- Integrate an open-source pretrained voice cloning model such as Coqui XTTS-v2 or OpenVoice for actual inference; do not require users to record voice samples per synthesis.
- Backend exposes REST endpoints: create voice, list voices, get voice status, synthesize speech, delete voice. Use unique voice IDs and store audio files and metadata separately.
- Use a job queue (e.g., Celery/Redis or Node BullMQ) for training and synthesis jobs so requests don't block.
- Store optional per-voice pronunciation dictionary and apply it during text preprocessing.
- Add basic error handling for unsupported audio formats, file size limits, and empty text.
- Provide a README covering setup, model weight download, environment variables, API examples, known limitations (e.g., cloned-voice quality depends on input clarity, model license terms, ethical use, possible latency), and disclaimer that the tool is an independent learning project inspired by Resemble AI's workflow, not affiliated with it.
Out of scope (do not build): multilingual generation, professional-grade emotional control, real-time WebSocket streaming, audio enhancement, transcript intelligence, billing/consent management beyond the simple checkbox, on-prem deployment, watermarking, and production-grade authentication. Keep the scope to a single-user local demo with sensible documentation.