API¶

This is the supported integrator contract for AbstractVoice.

Start with README.md and docs/getting-started.md for setup. Use docs/faq.md for cache/history reset and troubleshooting, docs/repl_guide.md for the interactive REPL, and docs/architecture.md for implementation details.

Implementation map: - abstractvoice/voice_manager.py → abstractvoice/vm/manager.py (constructor + wiring) - abstractvoice/vm/tts_mixin.py (TTS + cloning methods) - abstractvoice/vm/stt_mixin.py (STT + listening methods) - abstractvoice/vm/core.py (voice-mode behavior during playback)

Primary entry point¶

abstractvoice.VoiceManager
abstractvoice.VoiceProfile (data type; used by the voice-profile APIs)

from abstractvoice import VoiceManager

vm = VoiceManager(language="en", remote_api_key="sk-...", allow_downloads=True)

Constructor (most-used knobs)¶

The source of truth is abstractvoice/vm/manager.py:

VoiceManager(
    language: str = "en",
    tts_model: str | None = None,
    whisper_model: str = "base",
    debug_mode: bool = False,
    tts_engine: str = "openai",
    stt_engine: str = "openai",
    allow_downloads: bool = True,
    cloned_tts_streaming: bool = True,
    cloning_engine: str = "omnivoice",
    tts_delivery_mode: str | None = None,  # buffered|streamed (override)
    stt_model: str | None = None,
    remote_base_url: str | None = None,
    remote_api_key: str | None = None,
    remote_timeout_s: float | None = None,
    cloning_model: str | None = None,  # Qwen3-TTS Base checkpoint for new clones
)

Notes: - VoiceManager() and library auto are remote-first. Hosted OpenAI audio requires OPENAI_API_KEY or remote_api_key=.... - The CLI and web examples deliberately wrap this with an interactive TTS resolver: installed Supertonic, installed Piper, then OpenAI remote. Plain abstractvoice starts on OpenAI; abstractvoice[all-apple] and abstractvoice[all-gpu] start on Supertonic. That behavior lives in abstractvoice/examples/tts_defaults.py and does not change the library constructor. - allow_downloads gates implicit local model downloads in adapters. The REPL sets False (offline-first). - whisper_model controls the faster-whisper model size used by local listen() / transcribe_*() paths. - tts_engine selects the TTS provider and supports: - openai (default; remote OpenAI /v1/audio/speech; requires OPENAI_API_KEY) - auto (deterministic default: resolves to openai) - piper (local TTS; requires abstractvoice[piper], abstractvoice[apple], or abstractvoice[gpu]) - supertonic (local fixed-profile ONNX TTS; requires abstractvoice[supertonic], abstractvoice[apple], or abstractvoice[gpu]) - openai-compatible (remote compatible /v1/audio/speech; configure remote_base_url or OPENAI_BASE_URL) - audiodit (LongCat-AudioDiT; requires abstractvoice[audiodit]; upstream focuses on EN/ZH; direct/base TTS has a known quality caveat in 0.8.1) - omnivoice (OmniVoice; requires abstractvoice[omnivoice]; upstream supports 600+ languages) - none (speech-to-text only: no TTS adapter is built, so the manager needs no TTS runtime or credentials) - qwen3-tts (Qwen3-TTS 12Hz; requires abstractvoice[qwen3-tts], Python 3.10+; tts_model selects the checkpoint — the default Qwen/Qwen3-TTS-12Hz-0.6B-CustomVoice exposes 9 preset speakers as profiles, the 1.7B VoiceDesign checkpoint builds voices from the instructions selector and refuses an empty description; Base checkpoints are cloning-only and are not offered as TTS models) - stt_engine selects the STT provider and supports openai|auto|faster_whisper|openai-compatible|transformers-asr. auto resolves to openai. - faster_whisper requires abstractvoice[stt], abstractvoice[apple], or abstractvoice[gpu], and uses whisper_model/--whisper for tiny|base|small|medium|large-v2|large-v3|large. - transformers-asr requires abstractvoice[stt-hf], abstractvoice[apple], or abstractvoice[gpu], and uses stt_model as the Hugging Face model id (for example openai/whisper-large-v3, openai/whisper-large-v3-turbo, or Qwen/Qwen3-ASR-1.7B). Missing credentials or missing explicit local dependencies raise actionable errors; the legacy OpenAI Whisper fallback was removed. - tts_model is reserved/back-compat for local Piper (selection is language-driven today); for remote TTS it maps to the request model. - For remote STT, stt_model maps to the transcription request model. - For stt_engine="transformers-asr", stt_model is the Hugging Face model id to load. - Remote configuration can be passed in the constructor or via env vars: - OpenAI: OPENAI_API_KEY, optional ABSTRACTVOICE_OPENAI_TTS_MODEL, ABSTRACTVOICE_OPENAI_STT_MODEL - Compatible endpoints: OPENAI_BASE_URL, optional OPENAI_API_KEY, ABSTRACTVOICE_REMOTE_TTS_MODEL, ABSTRACTVOICE_REMOTE_STT_MODEL - OpenAI-compatible profile discovery: GET /audio/voices is tried by default for compatible endpoints; override with ABSTRACTVOICE_REMOTE_VOICE_PROFILE_PATH or ABSTRACTVOICE_REMOTE_VOICE_PROFILE_PATHS. Static voice/profile ids can also be supplied with ABSTRACTVOICE_REMOTE_TTS_VOICES. - tts_delivery_mode is an optional override that applies consistently to both base TTS and cloned voices: - buffered: synthesize full audio first (one payload) - streamed: deliver audio in chunks when available (lower time-to-first-audio) - cloning_engine defaults to omnivoice for new local clones. Install abstractvoice[omnivoice] or a platform/full profile for that default, or pass f5_tts, chroma, audiodit, qwen3-tts, openai, or openai-compatible explicitly. Qwen3-TTS cloning uses the Base checkpoints and the stored reference transcript (auto-filled once via STT when missing). cloning_model selects the default Qwen Base checkpoint; model=... on clone_voice(...) / clone_voice_from_wav_bytes(...) overrides it per clone. The checkpoint is persisted in the voice metadata and restored on replay. Existing Qwen clones without checkpoint metadata use the original 0.6B Base.

Supported language codes for the Piper mapping: en, fr, de, es, ru, zh (see abstractvoice/config/voice_catalog.py and abstractvoice/adapters/tts_piper.py). Supertonic supports fixed-style local TTS for ar, bg, cs, da, de, el, en, es, et, fi, fr, hi, hr, hu, id, it, ja, ko, lt, lv, nl, pl, pt, ro, ru, sk, sl, sv, tr, uk, vi. For other non-Piper providers (e.g. OmniVoice or remote OpenAI-compatible providers), language is treated as a pass-through hint and the provider decides what it supports.

TTS (text → audio)¶

CLI one-shot TTS uses the same VoiceManager synthesis path and exits without starting the REPL:

abstractvoice --provider openai --model tts-1 --voice alloy --prompt "Hello" --output hello.wav

Use --provider openai-compatible --api <base-url> for compatible remote audio servers, or --tts-engine <local-provider> for installed local engines.

speak(text: str, speed: float = 1.0, callback=None, voice: str | None = None, *, sanitize_syntax: bool = True) -> bool
Plays audio locally (non-blocking playback; synthesis time depends on backend).
If voice is provided, it is treated as a cloned voice_id (requires a cloning backend extra such as abstractvoice[omnivoice]; abstractvoice[cloning] is the explicit OpenF5 backend).
By default, common Markdown syntax is stripped from spoken output (headers + emphasis). Set sanitize_syntax=False to speak raw text.
set_speed(speed: float) -> bool, get_speed() -> float
Adjusts the default speaking speed used by speak_to_*() and the REPL.
set_tts_quality_preset(preset: str) -> bool, get_tts_quality_preset() -> str | None
Provider-agnostic speed/quality knob (low|standard|high). Back-compat aliases: fast→low, balanced→standard.
Providers that don’t support quality tuning may return False / None (Piper is typically a no-op).
For AudioDiT this primarily maps to diffusion steps (and a small guidance-strength tweak).
For Supertonic this maps to ONNX vector-estimator steps: low 5, standard 8, high 12.
get_profiles(*, kind: str = "tts") -> list[VoiceProfile]
set_profile(profile_id: str, *, kind: str = "tts") -> bool
get_active_profile(*, kind: str = "tts") -> VoiceProfile | None
set_tts_engine(engine: str, *, tts_model: str | None = None) -> str
Cross-provider voice profile abstraction (preset packs).
Profiles are provider-local: select tts_engine (provider) first, then apply a profile id for that provider.
Runtime provider switching should use set_tts_engine(...); it rebuilds the base TTS adapter, rewires playback callbacks, and resets the active profile to the provider/language default (for example Piper en -> amy, Supertonic -> M1, OpenAI -> alloy).
Providers without profiles return an empty list / False / None.
Concurrency note: profile selection mutates provider state. For servers, prefer one VoiceManager per session (or guard profile changes with a lock).
Remote OpenAI note: hosted built-in voices are always exposed as profiles (for example vm.set_profile("alloy")), and the adapter also tries OpenAI voice discovery for account/org-specific voices such as voice_.... tts_engine="openai" defaults to https://api.openai.com/v1 and reads OPENAI_API_KEY.
Remote compatible note: compatible endpoints may expose GET /v1/audio/voices (adapter path: GET /audio/voices) returning profiles, voices, cloned_voices, or OpenAI-style data. Returned ids are exposed as VoiceProfiles and used as the request voice for /audio/speech.
The voice= argument on speak_to_bytes(...) remains the cloned-voice handle path for backward compatibility; select base-provider voices with set_profile(...).
Supertonic note: tts_engine="supertonic" exposes fixed local profiles M1-M5 and F1-F5. Listing or selecting profiles does not download the model; synthesis requires cached artifacts or allow_downloads=True. By default Supertonic runs on ONNX Runtime's CPU provider with 4 intra-op threads (fewer on smaller machines), which keeps a busy host responsive; on Apple Silicon the CoreML provider measured slower than the CPU for this model. SupertonicRuntime(intra_op_num_threads=...) sets another thread count, and the adapter's execution_device() (also in get_info()) reports device, reason, providers and threads.
OmniVoice notes:
- Some profiles may enable persistent prompt caching (a tokenized voice_clone_prompt). The first set_profile(...) can pay a one-time build cost; later synthesis reuses cached tokens for stable voice identity. Prompt-conditioned synthesis can be heavier than pure voice design; use /tts quality low|standard|high (or VoiceManager.set_tts_quality_preset(...)) to tune the trade-off.
- On macOS / Apple Silicon, OmniVoice uses MPS (Metal) by default when device="auto".
pause_speaking() -> bool, resume_speaking() -> bool, stop_speaking() -> bool
Playback control.
is_speaking() -> bool, is_paused() -> bool
Playback state helpers.
set_tts_delivery_mode(mode: str | None) -> bool, get_tts_delivery_mode() -> str, get_tts_delivery_modes() -> dict
Toggle buffered vs streamed delivery (applies to both base TTS and cloned voices).
Behavior note: streamed delivery is implemented as a pipeline:
- text is chunked into short segments (sentence-first): the first segment is one short sentence or clause of at most 60 characters, so the first audio arrives quickly; later segments pack whole sentences up to 240 characters (or the engine's own limit when smaller),
- then each segment is synthesized and enqueued as soon as possible, so later segments are synthesized while the first one plays.
- Engines that can stream audio natively may further reduce TTFB by yielding multiple audio chunks per segment.
speak_to_bytes(text: str, format: str = "wav", voice: str | None = None, *, instructions: str | None = None, sanitize_syntax: bool = True) -> bytes
Headless/server‑friendly: returns encoded audio bytes.
speak_to_audio_chunks(text: str, *, voice: str | None = None, instructions: str | None = None, sanitize_syntax: bool = True) -> Iterator[tuple[np.ndarray, int]]
Headless/server‑friendly: yields (audio_chunk, sample_rate) tuples for incremental delivery.
Qwen3-TTS 1.7B CustomVoice and VoiceDesign accept per-call instructions without changing session state. Unsupported models and cloned speech reject streaming instructions explicitly.
open_tts_text_stream(*, voice: str | None = None, callback=None, sanitize_syntax: bool = True, max_chars: int | None = None, min_chars: int | None = None) -> TextToSpeechStream
Push-based streaming bridge for LLM streaming → TTS streaming pipelining.
Returned object supports: .push(delta), .close(), .cancel(), .join(timeout=...).
speak_to_file(text: str, output_path: str, format: str | None = None, voice: str | None = None, *, instructions: str | None = None, sanitize_syntax: bool = True) -> str
Writes an audio file and returns the path.

Qwen3-TTS checkpoints¶

Install abstractvoice[qwen3-tts] (Python 3.10+) and explicitly download each checkpoint you intend to use. Discovery lists cached checkpoints without loading their weights; synthesis loads the selected checkpoint on demand. It runs on Transformers 5.9 or newer (the extra's floor); with an older Transformers, loading refuses with a plain error before any download or weight load.

Checkpoint suffix	Sizes	Selection
`CustomVoice`	0.6B, 1.7B	`tts_model=...`, preset via `set_profile("aiden")`; 1.7B also accepts instructions
`VoiceDesign`	1.7B	`tts_model=...`, required natural-language `instructions`
`Base`	0.6B, 1.7B	`cloning_engine="qwen3-tts"`, `cloning_model=...` or per-clone `model=...`

Full ids follow Qwen/Qwen3-TTS-12Hz-<size>-<suffix>. TTS defaults to Qwen/Qwen3-TTS-12Hz-0.6B-CustomVoice, cloning to the corresponding Base. Supported language codes are en, zh, ja, ko, de, fr, ru, pt, es, and it. Long input is split into bounded segments in both buffered and streamed delivery, including text without spaces. Streamed output is sentence-segmented, not native codec-frame streaming.

python -m abstractvoice download --qwen3-tts Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign
abstractvoice --tts-engine qwen3-tts --tts-model Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign \
  --instructions "A warm, clear narrator." --prompt "Hello from Qwen." --output hello.wav
abstractvoice web --tts-engine qwen3-tts --cloning-engine qwen3-tts \
  --cloning-model Qwen/Qwen3-TTS-12Hz-1.7B-Base

The web example exposes the TTS model, speech instructions, and Qwen cloning model beside their provider selectors. The REPL supports /tts engine qwen3-tts, /tts_download qwen3-tts <model-id> and /cloning_download qwen3-tts <Base-model-id>; use startup --tts-model and --cloning-model to select checkpoints. Switching providers resets the model to the target provider's default; switching within a provider retains its configured model unless an override is supplied.

Through AbstractCore, use core.voice.tts(...), core.voice.tts_stream(...) or core.voice.clone(...) with provider="qwen3-tts" and model=<checkpoint>. core.voice.voice_catalog(provider="qwen3-tts", model=<checkpoint>) filters profiles and stored clones by checkpoint. Plugin defaults can be configured with the voice_cloning_model setting, separately from voice_tts_model, and the codebook predictor/sampler with voice_qwen3_tts_predictor / voice_qwen3_tts_sampler (see below).

Qwen performance controls¶

Qwen uses the shared automatic GPU/device policy and a specialized codebook predictor by default, retaining PyTorch's multinomial sampler. No additional vendor SDK is required. Unsupported generation settings use the original Transformers path with a #FALLBACK warning.

Every surface can choose both settings; the runtime reads them when it loads the model:

Surface	How to set them
CLI, REPL, web example	`--qwen3-tts-predictor auto\\|reference`, `--qwen3-tts-sampler multinomial\\|exponential`
Web example (running)	Qwen3-TTS predictor / sampler selects, or `POST /api/qwen3-tts/codebook` with `{"predictor": ..., "sampler": ...}`
`VoiceManager`	`VoiceManager(qwen3_tts_predictor=..., qwen3_tts_sampler=...)`; change later with `vm.set_qwen3_tts_codebook_generation(predictor=..., sampler=...)`
AbstractCore plugin	settings `voice_qwen3_tts_predictor`, `voice_qwen3_tts_sampler`
Runtime	`Qwen3TTSSettings(predictor=..., sampler=...)`

The VoiceManager choice applies to Qwen3-TTS synthesis and to Qwen3-TTS voice cloning. Changing it at runtime unloads a resident Qwen3-TTS model; the next synthesis reloads it with the new choice.

Setting	Default	Options
`predictor`	`auto`	`auto`: guarded specialized loop; `reference`: original Transformers generation
`sampler`	`multinomial`	`multinomial`: standard sampler; `exponential`: opt-in exponential-race sampler with frame-level validation

from abstractvoice.adapters.tts_qwen3_tts import Qwen3TTSAdapter
from abstractvoice.qwen3_tts.runtime import Qwen3TTSRuntime, Qwen3TTSSettings

runtime = Qwen3TTSRuntime(
    "Qwen/Qwen3-TTS-12Hz-0.6B-CustomVoice",
    settings=Qwen3TTSSettings(sampler="exponential"),  # or predictor="reference"
)
adapter = Qwen3TTSAdapter(runtime=runtime)

An invalid value is rejected before any weights load: the launch flags refuse it, the Python surfaces and plugin settings raise ValueError, and the web endpoint answers 400.

reference takes precedence over the sampler choice. Automatic fallback also uses the original sampler. Exponential sampling validates all distributions before returning a complete codebook frame; invalid probabilities raise without emitting that frame. Failed requests may consume more random numbers than the standard sampler because validation happens at the frame boundary. Reproducibility across different hardware or PyTorch versions is not guaranteed.

Model loading adds cold-start latency; keep the model resident for repeated requests. PyTorch version also matters on Apple Silicon: M5 Max measurements with Torch 2.12.0 / Transformers 5.8.1 were substantially faster than Torch 2.8.0 / Transformers 5.8.0. Compare warm generation time against audio duration in the Python environment running your CLI or plugin; an optimized sampler alone does not guarantee realtime throughput.

Faster inference does not imply native audio streaming: /tts delivery buffered waits for complete synthesis, while streamed delivers independently synthesized text segments progressively. Neither setting enables native Qwen codec-frame streaming.

Language & voice selection¶

set_language(language: str) -> bool
Switches the active language.
For explicit Piper, validation uses the curated Piper mapping in abstractvoice/config/voice_catalog.py.
For Supertonic, validation uses the adapter's 31-language ONNX text frontend list.
For non-Piper providers such as OmniVoice, the language code is passed through to the adapter and the provider decides what it supports.
If microphone listening is active, the recognizer is recreated on the next listen(...) call so STT receives the updated language.
get_language() -> str, get_language_name(language_code: str | None = None) -> str
get_supported_languages() -> list[str]
list_available_models(language: str | None = None) -> dict
Lists voice/model catalog entries for CLI/web display (see abstractvoice/vm/tts_mixin.py).
Piper returns local voice cache status by language.
Supertonic returns the 10 fixed styles for the active language plus shared cache status.
OpenAI/OpenAI-compatible TTS returns remote voice profiles plus configured/discovered TTS model ids when the active adapter supports model listing.
Back-compat alias: list_voices().
set_voice(language: str, voice_id: str) -> bool
Backward-compatible method; Piper voice selection is currently best-effort.

Residency (local engine preload/unload)¶

AbstractVoice is remote-first by default, but local engines benefit from explicit process-startup warmup. These methods are best-effort and are meaningful only for local providers (remote OpenAI/OpenAI-compatible are treated as configured, not resident).

preload_tts_engine(*, warmup: bool = True, warmup_text: str | None = None, warmup_format: str = "wav") -> dict
Forces the active local base TTS runtime to load weights into memory.
When warmup=True, also synthesizes a short sample to prime kernels/caches.
unload_tts_engine() -> dict
Best-effort release of loaded local base TTS weights/sessions.
preload_stt_engine(*, warmup: bool = False, warmup_audio_path: str | None = None, language: str | None = None) -> dict
Forces the configured local STT runtime to load weights into memory.
When warmup=True, also runs one transcription on warmup_audio_path when provided.
unload_stt_engine() -> dict
Best-effort release of loaded local STT weights/pipelines (drops adapter refs so next use reloads cleanly).
list_resident_components() -> list[dict]
Process-local introspection used by the AbstractCore plugin residency surface.
Returns clone engines plus base TTS/STT residency (when determinable).

STT (audio → text)¶

transcribe_file(audio_path: str, language: str | None = None) -> str
Transcribes audio from a file.
transcribe_from_bytes(audio_bytes: bytes, language: str | None = None) -> str
Transcribes audio sent over the network.

Qwen3-ASR (Transformers ASR)¶

stt_engine="transformers-asr" with a Qwen3-ASR checkpoint (for example Qwen/Qwen3-ASR-0.6B or Qwen/Qwen3-ASR-1.7B) runs AbstractVoice's bundled Qwen3-ASR model code, without trust_remote_code. It needs abstractvoice[stt-hf] and runs on Transformers 5.4 or newer, so Python 3.10+ (tested in CI on the extra's floor and the latest release). With an older Transformers (what the extra resolves on Python 3.9) loading refuses with a plain error, which is also the adapter's unavailable reason. Prefetch with abstractvoice-prefetch --stt-hf Qwen/Qwen3-ASR-0.6B.

STT configuration¶

set_whisper(model_name: str) -> None | bool
Updates the faster‑whisper model size used for subsequent operations.
get_whisper() -> str

Microphone capture (local assistant mode)¶

listen(on_transcription, on_stop=None) -> bool
Starts microphone capture + VAD + STT in-process (abstractvoice/recognition.py).
Stop phrase(s): "ok stop", "okay stop", and (conservatively) "stop"; see abstractvoice/recognition.py and abstractvoice/stop_phrase.py.
stop_listening() -> bool
Stops microphone capture.
pause_listening() -> bool, resume_listening() -> bool
Pauses/resumes audio processing while keeping the listening thread alive.
is_listening() -> bool
Whether the background recognizer thread is running.
cleanup() -> bool
Best-effort cleanup for long-lived apps (stop listening, stop speaking, release audio resources).

Advanced tuning (best-effort)¶

change_vad_aggressiveness(aggressiveness: int) -> bool
For advanced mic/VAD tuning; see abstractvoice/recognition.py.

Voice modes (behavior while speaking)¶

Voice modes control what the microphone loop does while TTS is playing. Set via:

set_voice_mode(mode: str) -> bool where mode ∈ {"full","wait","stop","ptt"}

Mode semantics (implemented in abstractvoice/vm/core.py):

full: keep listening and allow barge‑in (interrupt TTS on detected speech). Best with AEC or headset; speakers can cause self-interruption (mitigations exist; see echo gating in abstractvoice/recognition.py).
wait: pause microphone processing while speaking. No barge‑in and no stop‑phrase detection during TTS. Good for strict turn‑taking.
stop: keep listening, but suppress normal transcriptions during TTS and disable “interrupt on any speech”; a rolling stop‑phrase detector stays active so users can say “ok stop” to cut playback.
ptt: push‑to‑talk profile (thresholds tuned for short utterances). During TTS it behaves like stop mode; the integrator controls when to start/stop capture.

The REPL defaults to mic input off, and recommends --voice-mode stop for hands‑free usage; see docs/repl_guide.md.

Acoustic echo cancellation (optional)¶

enable_aec(enabled: bool = True, stream_delay_ms: int = 0) -> bool
Opt‑in AEC support for true barge‑in (requires abstractvoice[aec]).
Playback audio chunks are fed to the recognizer via abstractvoice/vm/core.py → VoiceRecognizer.feed_far_end_audio() in abstractvoice/recognition.py.

Voice cloning (optional; heavy)¶

Requires installing at least one cloning backend extra (and explicit artifact downloads; see docs/installation.md). The recommended/default local backend is OmniVoice:

abstractvoice[omnivoice] → omnivoice (default for new clones)
abstractvoice[cloning] → f5_tts
abstractvoice[chroma] → chroma
abstractvoice[audiodit] → audiodit
abstractvoice[qwen3-tts] → qwen3-tts (Base checkpoints)

Supertonic is not listed here because it is fixed-profile base TTS, not a voice-cloning provider.

Remote clone-compatible endpoints can also be used without local cloning model weights by selecting cloning_engine="openai-compatible" (or engine="openai-compatible" per call). Configure remote_base_url or OPENAI_BASE_URL; the default clone endpoint is POST /voice/clone and must return a remote voice id (voice_id or id). The local clone store keeps a handle and routes later speak_to_bytes(..., voice=<local_id>) calls to remote /audio/speech with that remote voice id.

cloning_engine="openai" targets OpenAI's hosted API by default. Custom voice creation is provider/org gated and requires explicit consent configuration such as ABSTRACTVOICE_OPENAI_VOICE_CONSENT_ID; otherwise the adapter raises an actionable error instead of silently pretending cloning is standardized.

Core cloning calls:

clone_voice(reference_audio_path: str, name: str | None = None, *, reference_text: str | None = None, engine: str | None = None, model: str | None = None) -> str
clone_voice_from_wav_bytes(wav_bytes: bytes, name: str | None = None, *, reference_text: str | None = None, engine: str | None = None, model: str | None = None) -> str
speak(..., voice="<voice_id>") / speak_to_bytes(..., voice="<voice_id>") / speak_to_file(..., voice="<voice_id>")
list_cloned_voices(), get_cloned_voice(voice_id: str) -> dict

Clone management helpers:

set_cloned_voice_reference_text(voice_id: str, reference_text: str) -> bool
rename_cloned_voice(voice_id: str, new_name: str) -> bool
delete_cloned_voice(voice_id: str) -> bool
export_voice(voice_id: str, path: str) -> str, import_voice(path: str) -> str
set_cloned_tts_quality(preset: str) -> bool (low|standard|high; aliases: fast, balanced)
get_cloning_runtime_info() -> dict
unload_cloning_engines(*, keep_engine: str | None = None) -> int (best-effort memory relief)
unload_piper_voice() -> bool (best-effort memory relief)

For the user-facing workflow and commands, see docs/repl_guide.md.

Provider caveats that affect release choice are tracked in docs/known-issues.md.

Metrics (optional)¶

pop_last_tts_metrics() -> dict | None
Best-effort last-utterance stats used by the REPL verbose mode.
Streamed synthesis (speak_to_audio_chunks(...)) records engine, ttfb_s, synth_s, audio_s, rtf, chunks, segments, segment_max_chars, first_segment_max_chars and, for engines that report it, device (cpu or coreml for Supertonic) with device_reason when synthesis runs on the CPU.

Callbacks & hooks¶

Per-utterance callback: speak(..., callback=...) (invoked after playback drains).
TTS lifecycle callbacks: vm.tts_engine.on_playback_start / vm.tts_engine.on_playback_end (synthesis/queue lifecycle).
Audio lifecycle callbacks (actual output): vm.on_audio_start / vm.on_audio_end / vm.on_audio_pause / vm.on_audio_resume (wired in abstractvoice/vm/core.py).

Explicit downloads (offline-first)¶

For offline deployments, prefetch explicitly (cross-platform):

python -m abstractvoice download --stt small
python -m abstractvoice download --stt-hf openai/whisper-large-v3
python -m abstractvoice download --piper en
python -m abstractvoice download --supertonic # optional; requires abstractvoice[supertonic]
python -m abstractvoice download --openf5   # optional; requires abstractvoice[cloning]
python -m abstractvoice download --chroma   # optional; requires abstractvoice[chroma] (GPU-heavy)
python -m abstractvoice download --audiodit # optional; requires abstractvoice[audiodit]
python -m abstractvoice download --omnivoice # optional; requires abstractvoice[omnivoice]

Or use the convenience entrypoint:

abstractvoice-prefetch --stt small
abstractvoice-prefetch --stt-hf openai/whisper-large-v3
abstractvoice-prefetch --piper en
abstractvoice-prefetch --supertonic         # optional; requires abstractvoice[supertonic]
abstractvoice-prefetch --openf5            # optional; requires abstractvoice[cloning]
abstractvoice-prefetch --chroma            # optional; requires abstractvoice[chroma] (GPU-heavy)
abstractvoice-prefetch --audiodit          # optional; requires abstractvoice[audiodit]
abstractvoice-prefetch --omnivoice         # optional; requires abstractvoice[omnivoice]

Notes: - --chroma artifacts may require Hugging Face access to download.

See also: docs/installation.md, docs/model-management.md, and docs/voices-and-licenses.md.

Performance note: prefetch vs preload (important for servers)¶

Prefetch (download to disk): python -m abstractvoice download ... / abstractvoice-prefetch ...
Preload (load into memory): create a long-lived VoiceManager (or adapter) and reuse it, optionally calling preload_tts_engine() / preload_stt_engine() once at process startup.

If you construct a new VoiceManager for every request, heavy engines (AudioDiT/OmniVoice) will pay a large one-time cost repeatedly (imports + weight load + accelerator kernel compilation).

Recommended pattern (server/process startup):

from abstractvoice import VoiceManager

# Load once, reuse for all requests.
vm = VoiceManager(
    language="en",
    tts_engine="omnivoice",
    stt_engine="openai",
    remote_api_key="sk-...",
    allow_downloads=False,
)

# Optional explicit warmup for local providers.
vm.preload_tts_engine(warmup=True, warmup_text="Hello.")
# vm.preload_stt_engine(warmup=True, warmup_audio_path="hello.wav", language="en")

Integrations (AbstractFramework ecosystem)¶

AbstractVoice is designed to work standalone, and also integrate cleanly into the AbstractFramework ecosystem (AbstractCore + AbstractRuntime). Overview and links: README.md.

Boundary note: - AbstractVoice owns the in-process voice backend (VoiceManager, adapters, model/cache policy). - AbstractCore owns agent orchestration, provider routing, capability selection, and OpenAI-compatible HTTP endpoints. - When both are installed, AbstractCore can expose AbstractVoice-backed audio endpoints such as POST /v1/audio/speech and POST /v1/audio/transcriptions.

AbstractCore capability plugin (auto-discovery)¶

AbstractVoice exposes an AbstractCore capability plugin entry point:

Entry point declaration: pyproject.toml → [project.entry-points."abstractcore.capabilities_plugins"]
Implementation: abstractvoice/integrations/abstractcore_plugin.py

The plugin registers: - a voice backend (backend_id="abstractvoice:default") for TTS+STT - an audio backend (backend_id="abstractvoice:stt") for STT-only

Audio outputs can optionally be stored into an AbstractRuntime-like artifact_store via the duck-typed adapter in abstractvoice/artifacts.py.

Discovery note (generic capability discovery): - The voice backend exposes rich discovery helpers (available_providers(), list_models(kind=...), voice_catalog(), etc). - The audio backend implements the minimal generic discovery contract used by llm.capabilities.* for STT-only routing: - available_providers(task=None) - list_models(task=None, provider=...|provider_id=...)

The voice backend also exposes thin catalog discovery methods for Core/Gateway integration code: - list_profiles(kind="tts") -> list[dict] - list_models(kind="tts"|"stt"|"cloning", provider: str | None = None) -> list[str] - list_tts_models(provider: str | None = None) -> list[str] - list_stt_models(provider: str | None = None) -> list[str] - list_cloning_models(provider: str | None = None) -> list[str] - list_tts_voices(provider: str | None = None, model: str | None = None, include_clones: bool = True) -> list[dict] - list_cloned_voices(provider: str | None = None, model: str | None = None) -> list[dict] - list_voices(...) -> list[dict] (alias of list_tts_voices(...)) - available_providers() -> {tts, stt, cloning, providers, details, unavailable} - compatibility_catalog() -> {version, providers} - get_capability_support(kind, feature, provider, model=None, surface="default") -> dict | None - find_compatible_models(kind, feature, surface="default", support_in=("native","emulated","conditional")) -> list[dict] - surface="default" resolves to the kind's primary surface (tts→bytes, stt→transcribe, cloning→create); surface names are case-insensitive. Surfaces can genuinely differ — for example qwen3-tts reports instructions support on bytes but not playback — so live-playback consumers should query surface="playback" explicitly. - clone(audio, *, name=None, reference_text=None, provider=None, model=None, artifact_store=None, metadata=None, **kwargs) -> voice_id | dict - clone_voice(...) -> ... (compatibility alias of clone(...)) - voice_catalog() -> {kind, provider_id (alias engine_id), active_profile, active_model, voices (profiles + clones), tts_models, stt_models, tts_models_by_provider, stt_models_by_provider, tts_model_variants, stt_engine_variants, tts_catalog_by_provider, stt_catalog_by_provider, available_providers, catalog} - voice_catalog() also includes additive capability metadata: - controls: legacy control map kept backward-compatible for existing clients - tts_capabilities: richer per-field support truth (native|emulated|conditional|unsupported) for newer clients - speech_request_contract: current request contract id (currently "speech_request_v1") - compatibility_catalog: package-owned provider/model feature matrix across tts, stt, and cloning - Residency / warmup (local engines only; remote providers remain “configured”): - load_resident_model(request: dict) -> dict - voice backend: clone engines (provider="cloned") and local base TTS (provider="piper"|"supertonic"|"omnivoice"|"audiodit"|...) - audio backend: local STT (provider="faster-whisper"|"transformers-asr"|...) - list_resident_models(filters: dict | None = None) -> list[dict] - unload_resident_model(request: dict) -> dict - tts(text, *, provider=None, model=None, voice=None, format="wav", ...) -> bytes | artifact_ref - stt(audio, *, provider=None, model=None, language=None, ...) -> str - transcribe(audio, *, provider=None, model=None, language=None, ...) -> str

These methods delegate to the active VoiceManager and keep voice/profile/model semantics in AbstractVoice. AbstractCore still owns HTTP routing, auth, and browser/security policy.

available_providers() is intentionally lightweight: it lists currently usable provider ids without constructing local model runtimes. Remote OpenAI appears when OPENAI_API_KEY is configured or an active/injected manager is already using it; OpenAI-compatible appears when a compatible base URL is configured. Local providers appear when their runtime dependency is installed and any required cached model artifacts are present. The same payload also includes known_tts_providers, known_stt_providers, and known_cloning_providers so UI/provider selectors can distinguish installed availability from the broader capability catalog.

A provider left out of those lists is explained, never silently missing. available_providers()["unavailable"] maps each kind (tts, stt, cloning) to one record per known provider that is not available:

`code`	Meaning
`runtime_missing`	The engine's Python packages are not installed; `runtime` carries the `engine_runtime_status` record (missing modules, extra, install command).
`model_not_downloaded`	The runtime is installed but no model for it is on this machine (TTS).
`not_configured`	A remote provider without its API key (`openai`) or base URL (`openai-compatible`).

Each record has a plain-words reason. voice_catalog() carries the same map as unavailable_providers (without providers the listing does include) plus unavailable_reason: None when the listing has something to offer, otherwise one sentence saying why — for a provider filter, that provider's reason; unfiltered, the configured provider's reason first. A provider that cannot answer — the configured one or a filter such as provider="openai" without a key — and an unknown provider id never raise: the listing comes back empty with the reason ("'bogus' is not a known text-to-speech provider (known: openai, ...)" for an unknown id), without building an engine. list_tts_models(provider) and list_models(kind="tts", provider=...) return [] for the same providers, also engine-free; a list cannot carry the reason, so read it from voice_catalog.

catalog = core.voice.voice_catalog(provider="supertonic")
catalog["unavailable_reason"]
# 'Supertonic is not installed: the Python package onnxruntime is missing.
#  Install it with: pip install "abstractvoice[supertonic]"'

Engine runtime status¶

abstractvoice.engine_runtime answers whether an engine's Python runtime is installed, for any known engine, without importing the engine or any ML framework (it uses importlib.util.find_spec only):

from abstractvoice.engine_runtime import engine_runtime_status, engine_runtime_installed, known_engines

status = engine_runtime_status("supertonic")        # or "faster-whisper", "qwen3-tts", ...
status.installed          # bool
status.missing_modules    # ("onnxruntime",) when missing; groups read "piper or piper_phonemize"
status.extra              # "supertonic"
status.install_command    # 'pip install "abstractvoice[supertonic]"'
status.reason             # plain-words sentence, None when installed
status.to_dict()          # JSON-safe record
engine_runtime_installed("faster-whisper", kind="stt")
known_engines("tts")      # engine ids this module answers for

Engine ids: openai, openai-compatible, supertonic, piper, audiodit, qwen3-tts, omnivoice, f5_tts, chroma, faster-whisper, transformers-asr (aliases such as faster_whisper or F5-TTS are accepted). An unknown id, or a kind the engine does not serve, raises ValueError. Remote engines always report installed=True; whether they are configured is a separate question answered by available_providers()["unavailable"].

Discovery has a cost model, and it is worth knowing which side of it you are on:

Local providers cost a stat(). "Is this model here?" is answered from the model cache by abstractvoice.local_models — never by importing an engine or building an adapter. available_providers(), list_models(), and voice_catalog(providers_only=True) or voice_catalog(provider=<local>) never load torch, transformers, or ONNX Runtime. Selecting an engine that has nothing downloaded still works and still downloads on demand; it just is not listed until its weights are on the machine.
Remote providers share one 5 second budget. Every provider is probed at once, one task each. A provider that does not answer inside the budget gets tts_catalog_by_provider[provider]["unreachable"] = True and no entry in catalogs, because "we could not reach it in time" and "it has no models" are different facts and its empty models/voices are ours, not its. The key is only ever set when a probe was attempted and failed — its absence is no claim. Override the budget with ABSTRACTVOICE_DISCOVERY_TIMEOUT_S; the per-request socket timeout is the smaller of the budget and ABSTRACTVOICE_REMOTE_TIMEOUT_S. A list-returning helper such as list_tts_models(provider) cannot express "unknown", so read the flag when the difference matters. voice_catalog() also carries unreachable_tts_providers. It is absent on paths that contacted nobody (providers_only=True, a local provider filter), because an empty list there would assert that every provider listed — including remote ones never asked — is reachable. Absent means "not checked"; empty means "checked, all reachable".
Remote discovery answers age out after five minutes. A remote model or voice list is remembered so repeated listings do not re-probe the provider, and re-probed once the answer is older than the TTL — so a model loaded into a local server, or a machine that regains internet access, appears without a restart. refresh_profiles() is the immediate override. ABSTRACTVOICE_REMOTE_DISCOVERY_TTL_S sets the window; values below the 300-second floor, non-numeric values, and non-finite values are clamped back to 300, since a never-expiring memo would freeze the first probe's answer (including an empty one from a failed probe) for the life of the process.
Engine-free: available_providers(), list_models() for every kind and filter, list_tts_voices(provider=...), compatibility_catalog(), capability_support(), find_compatible_models(), and voice_catalog(providers_only=True) / voice_catalog(provider=<local>).
Builds the active engine: the unfiltered voice_catalog() and voice_catalog(provider=<remote>) when that provider is available, list_tts_voices(), list_cloned_voices(), and list_profiles(). These report the active engine's live state — its profiles, its cloned voices, its STT side — and for a local engine that loads its weights. It is deliberate: the light path reads cloned voices from the clone store while the full one also accepts clones the manager reports, so switching would silently drop the latter. Prefer the engine-free calls above for discovery.

The provider/model/voice abstraction used by the plugin is: - provider: backend/engine id such as openai, openai-compatible, piper, supertonic, audiodit, omnivoice, qwen3-tts, faster-whisper, or transformers-asr - model: provider-specific selectable model id - voice: base voice/profile id or cloned voice id available for the selected provider + model

voice_catalog() is meant to be the single rich discovery payload for AbstractCore/Gateway: - tts_catalog_by_provider[provider] contains models, model_variants, voices, profiles, cloned_voices, voices_by_model, and formats - stt_catalog_by_provider[provider] contains models, model_variants, and formats - controls remains the legacy compatibility surface for control discovery and still uses simple supported booleans for fields such as speed, quality_preset, instructions, profile, and voice_clone - tts_capabilities is the newer truthful capability map for engine-aware clients and reports support plus optional reason per field - support values may be native, emulated, conditional, or unsupported - conditional means AbstractVoice forwards or orchestrates the feature, but actual backend support can still vary by provider or endpoint - speech_request_contract versions the richer package-owned request shape without forcing older clients to understand it - compatibility_catalog is the central matrix for provider/model capability queries across tts, stt, and cloning - clone(...) accepts a local file path, raw audio bytes, an artifact-ref dict, or a direct audio payload dict with content/bytes plus optional filename / content_type. File-path and filename-bearing payloads preserve the original audio suffix for remote cloning providers. - each provider entry can define provider-wide defaults plus explicit model records - feature support is surface-specific, for example tts.bytes vs tts.playback

The lighter helper methods are convenience views over that same abstraction: - available_providers() for provider selectors - list_models(...) / list_tts_models(...) / list_stt_models(...) / list_cloning_models(...) for model selectors - list_tts_voices(...) / list_cloned_voices(...) for voice selectors - get_capability_support(...) / find_compatible_models(...) for provider-model-surface capability filtering across tts, stt, and cloning

For both TTS and STT, the plugin accepts either split fields or combined provider:model selectors. AbstractCore callers may pass separate arguments such as provider="openai", model="tts-1" or provider="transformers-asr", model="Qwen/Qwen3-ASR-1.7B", or a single combined provider value such as provider="openai:tts-1", provider="faster-whisper:large", or provider="transformers-asr:Qwen/Qwen3-ASR-1.7B".

For TTS execution, the plugin accepts either a simple voice="alloy" string or the selected voice/cloned-voice record from the catalog. When a voice dict is passed, the plugin derives provider and model from that record when needed.

Plugin configuration (owner config dict, best-effort). In AbstractCore integrations, the env/default path uses OpenAI remote TTS/STT (OPENAI_API_KEY, optional ABSTRACTVOICE_OPENAI_* overrides) unless owner config or ABSTRACTVOICE_TTS_ENGINE / ABSTRACTVOICE_STT_ENGINE selects a different provider: - voice_language: default language (e.g. "en") - voice_allow_downloads: allow on-demand downloads (bool) - voice_tts_engine: base TTS provider ("auto"|"piper"|"supertonic"|"openai"|"openai-compatible"|"audiodit"|"omnivoice") - voice_stt_engine: STT provider ("auto"|"faster_whisper"|"transformers-asr"|"openai"|"openai-compatible") - voice_tts_model: model id for remote TTS providers - voice_stt_model: model id for remote STT providers, or a Hugging Face model id when voice_stt_engine="transformers-asr" - voice_remote_base_url: base URL for OpenAI-compatible remote audio endpoints - voice_remote_api_key: optional bearer key for remote audio endpoints - voice_remote_timeout_s: request timeout for remote audio endpoints - voice_openai_api_key: OpenAI API key handed over by the host (for example a key saved in a gateway's Providers screen). Used for provider openai only (TTS, STT, cloning) and never sent to voice_remote_base_url; it also makes openai available in discovery. - voice_openai_base_url: optional OpenAI base URL used with voice_openai_api_key (default: the OpenAI API, https://api.openai.com/v1; OPENAI_BASE_URL is not used for this key, because it names the OpenAI-compatible server) - voice_qwen3_tts_predictor / voice_qwen3_tts_sampler: Qwen3-TTS codebook predictor and sampler (see Qwen performance controls) - voice_whisper_model: faster-whisper model size (e.g. "base", "small") - voice_cloning_engine: default cloning provider ("omnivoice" by default; also "f5_tts"|"chroma"|"audiodit"|"openai"|"openai-compatible") - voice_cloned_tts_streaming: stream cloned-voice chunks for faster time-to-first-audio (bool). Used when voice_tts_delivery_mode is unset. - voice_tts_delivery_mode: unified audio delivery mode for base + cloned voices ("buffered"|"streamed"). Takes precedence over voice_cloned_tts_streaming. - voice_tts_streaming: bool alias for voice_tts_delivery_mode (true → "streamed", false → "buffered"). - voice_debug_mode: enable debug prints (bool)

Hosts pass provider credentials through this config dict, per plugin instance: AbstractCore puts create_llm(...) keyword arguments there, so create_llm(provider, model=..., voice_openai_api_key=key) is enough in-process. Without voice_openai_api_key, provider openai uses voice_remote_api_key / voice_remote_base_url as before, then OPENAI_API_KEY. The same credentials are available on the library API as VoiceManager(openai_api_key=..., openai_base_url=...).

Boolean owner config/env values accept common strings such as true, false, on, off, 1, and 0; string values like "false" are not treated as truthy.

Performance note: - The capability plugin caches VoiceManager instances in-process (keyed by the config above) so provider runtimes are not reloaded per request.

TTS metrics: - After synthesis, the plugin stores best-effort stats in artifact metadata under abstractvoice_tts (when artifact_store is used).

AbstractCore OpenAI-compatible audio endpoints¶

The production OpenAI-compatible HTTP server lives in AbstractCore. AbstractVoice also ships a local FastAPI web example (abstractvoice web) for package-level smoke testing, but the supported API server path is AbstractCore Server.

With abstractcore[server] and abstractvoice installed in the same environment, AbstractCore delegates its audio endpoints to the discovered capability plugin:

POST /v1/audio/speech -> core.voice.tts(...) -> VoiceManager.speak_to_bytes(...)
POST /v1/audio/transcriptions -> core.audio.transcribe(...) -> VoiceManager.transcribe_*()
GET /v1/audio/voices -> core.voice.voice_catalog() / list_tts_voices(...) for provider, model, profile, and cloned-voice discovery
GET /v1/audio/speech/models -> core.voice.list_tts_models(...) or voice_catalog()["tts_catalog_by_provider"]
GET /v1/audio/transcriptions/models -> core.voice.list_stt_models(...) or voice_catalog()["stt_catalog_by_provider"]

Recommended AbstractCore mapping:

providers = core.voice.available_providers()
catalog = core.voice.voice_catalog()

tts_audio = core.voice.tts(
    "Hello.",
    provider="openai",
    model="tts-1",
    voice="alloy",
    format="wav",
)

text = core.audio.transcribe(
    wav_bytes,
    provider="faster-whisper",
    model="large",
    language="en",
)

That same shape also supports local or remote cloned voices: the selected catalog item from tts_catalog_by_provider[provider]["cloned_voices"] can be passed back as the TTS voice, or its voice_id can be passed directly.

Example:

OPENAI_API_KEY=... python -m abstractcore.server.app

curl -X POST http://localhost:8000/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{"input":"Hello.","format":"wav"}' \
  --output hello.wav

curl -X POST http://localhost:8000/v1/audio/transcriptions \
  -F "file=@hello.wav" \
  -F "language=en"

If AbstractCore Server is configured with ABSTRACTCORE_API_KEY, include the standard Authorization: Bearer <key> header. ABSTRACTCORE_API_KEY is AbstractCore’s master key: when set, AbstractCore can use any preconfigured provider API keys, while provider-specific env vars (for example OPENAI_API_KEY) can still override.

If the plugin is unavailable, AbstractCore returns 501 with install/config guidance instead of silently falling back.

For openai-compatible plugin configuration, do not set voice_remote_base_url to the same AbstractCore Server instance that is currently routing the /v1/audio/* request. That configuration recurses through the plugin path; use an upstream compatible provider/gateway, or select local providers explicitly.

The local abstractvoice web example exposes these smoke-test routes. They are example routes, not a replacement for AbstractCore Server, and they do not inherit AbstractCore/Gateway authentication or browser-origin policy:

abstractvoice web
abstractvoice web --tts-engine openai --stt-engine openai
abstractvoice web --tts-engine openai-compatible --stt-engine openai-compatible --remote-base-url http://localhost:8000/v1

GET /api/status -> lightweight server/config status
GET /api/voices -> VoiceManager.get_profiles(), list_available_models(), list_cloned_voices()
GET /v1/audio/voices -> compatible extension for remote profile/voice discovery (VoiceManager.get_profiles() + list_cloned_voices())
POST /api/tts/provider (alias: /api/tts/engine) -> switch the browser example's base TTS provider through VoiceManager.set_tts_engine(...); resets the base profile/voice to the provider/language default and clears local role clone selections
POST /api/voices/select -> select base TTS, a cloned voice, or a TTS profile; optional local role="assistant"|"user" stores browser-example defaults; optional preload=true warms a cloned voice by calling a tiny VoiceManager.speak_to_bytes(...)
POST /api/voices/clone -> example-only multipart upload for browser voice cloning; stores the uploaded/recorded reference with VoiceManager.clone_voice(...) and validates by synthesizing a short sample by default (validate=false skips validation)
POST /v1/voice/clone -> compatible extension for remote clone creation; returns voice_id/id for later /v1/audio/speech voice
POST /api/tts -> VoiceManager.speak_to_bytes(...); accepts input/text, voice, role, language, speed, format/response_format, and sanitize_syntax
POST /api/stt/transcriptions -> VoiceManager.transcribe_file(...)
POST /api/stt/transcribe -> compatibility alias for /api/stt/transcriptions
GET /api/llm/models -> example-only model listing for an OpenAI-compatible local provider such as Ollama
POST /api/chat -> example-only non-streaming chat completion proxy; the browser owns history and sends the full short message list
POST /v1/audio/speech and POST /v1/audio/transcriptions -> local aliases for quick AbstractCore-compatible smoke tests. In the web example, voice may be either a cloned voice id/name or an active-provider profile id.

Local web example payload sketches:

# Browser-example role default; still resolves to a cloned voice_id.
curl -X POST http://127.0.0.1:5000/api/voices/select \
  -H "Content-Type: application/json" \
  -d '{"role":"assistant","kind":"clone","voice":"my_voice","preload":true}'

# Browser-example cloned voice creation.
curl -X POST http://127.0.0.1:5000/api/voices/clone \
  -F "name=my_voice" \
  -F "provider=omnivoice" \
  -F "reference_text=Exact transcript of the reference audio." \
  -F "file=@reference.wav"

# TTS still maps to VoiceManager.speak_to_bytes(...).
curl -X POST http://127.0.0.1:5000/api/tts \
  -H "Content-Type: application/json" \
  -d '{"input":"Hello.","role":"assistant","response_format":"wav"}' \
  --output hello.wav

# Switch the browser example to Supertonic and its default profile.
curl -X POST http://127.0.0.1:5000/api/tts/provider \
  -H "Content-Type: application/json" \
  -d '{"provider":"supertonic"}'

# Compatible extension: discover profiles/cloned voices from another
# AbstractVoice client configured with remote_base_url=http://127.0.0.1:5000/v1.
curl http://127.0.0.1:5000/v1/audio/voices

# Compatible extension: create a remote cloned voice handle.
curl -X POST http://127.0.0.1:5000/v1/voice/clone \
  -F "name=my_remote_voice" \
  -F "reference_text=Exact transcript of the reference audio." \
  -F "file=@reference.wav"

# Example dialogue call via a local OpenAI-compatible provider.
curl -X POST http://127.0.0.1:5000/api/chat \
  -H "Content-Type: application/json" \
  -d '{"provider":"ollama","model":"gemma3:1b","messages":[{"role":"user","content":"Say hi in one sentence."}]}'

The local role field is only a convenience for the example UI. AbstractCore compatibility remains the capability/plugin contract: clients may pass a voice cloned-voice id to TTS, and AbstractCore routes that to VoiceManager.speak_to_bytes(..., voice=...).

AbstractCore tool helpers (manual wiring)¶

If you prefer to wire tools explicitly, abstractvoice/integrations/abstractcore.py provides:

make_voice_tools(voice_manager, store) -> list[callable]
Requires abstractcore at runtime (it imports abstractcore.tool).
store can be a MediaStore-like object, or an AbstractRuntime-like ArtifactStore (adapted via RuntimeArtifactStoreAdapter in abstractvoice/artifacts.py).

Tools exposed by make_voice_tools(...) (current): - voice_tts(text, voice=None, format="wav", run_id=None) -> artifact_ref - voice_profile_list(kind="tts") -> {profiles, active_profile} - voice_profile_set(profile_id, kind="tts") -> {ok, active_profile} - audio_transcribe(audio_artifact|audio_b64, ...) -> {text, transcript_artifact}

Minimal sketch:

from abstractvoice import VoiceManager
from abstractvoice.integrations.abstractcore import make_voice_tools

vm = VoiceManager(remote_api_key="sk-...")
tools = make_voice_tools(voice_manager=vm, store=artifact_store)

Example (provider-agnostic profile selection):

vm = VoiceManager(tts_engine="omnivoice", allow_downloads=False)
vm.set_profile("female_01", kind="tts")
wav_bytes = vm.speak_to_bytes("Hello.", format="wav")

TTS metrics (library-level): - VoiceManager.speak_to_bytes(...) / VoiceManager.speak_to_file(...) record best-effort stats for the last synthesis. - Call vm.pop_last_tts_metrics() to retrieve and clear them (dict with fields like engine, synth_s, audio_s, rtf, sample_rate).

Non-contract surface (may change without notice)¶

CLI behavior (abstractvoice/examples/*)
Internal adapter details and model catalogs beyond the documented defaults