Skip to content

FAQ

See also: - Getting started: docs/getting-started.md - API reference: docs/api.md - Architecture: docs/architecture.md - Backends: docs/reference/backends.md - Configuration: docs/reference/configuration.md

What is AbstractVision?

AbstractVision is a small, model-agnostic API for generative vision outputs (images, optional video) with: - a small orchestrator (VisionManager) - pluggable execution engines (“backends”) in ../src/abstractvision/backends/ - a packaged capability registry (vision_model_capabilities.json) - optional artifact-ref outputs via stores (../src/abstractvision/artifacts.py)

How does AbstractVision fit into AbstractFramework?

AbstractVision is part of the AbstractFramework ecosystem:

Where AbstractVision fits: - It standardizes generative vision outputs behind VisionManager (library mode). - AbstractCore can discover and use AbstractVision via the capability plugin (see ../src/abstractvision/integrations/abstractcore_plugin.py and the entry point in ../pyproject.toml). - Artifact refs are designed to cross process boundaries; RuntimeArtifactStoreAdapter bridges to an AbstractRuntime-style artifact store (see ../src/abstractvision/artifacts.py).

What does AbstractVision support today?

  • Built-in backends implement images: text_to_image and image_to_image.
  • Video (text_to_video, image_to_video) works only via the OpenAI-compatible backend when video endpoints are configured.
  • multi_view_image exists in the public API (VisionManager.generate_angles) but no built-in backend implements it yet (they raise CapabilityNotSupportedError).

Details: docs/reference/backends.md.

Which backend should I use?

What model should I start with (local)?

If you’re running locally via the Diffusers backend and want a reliable starting point, we recommend:

  • Default / ≤16GB VRAM (cross-platform): runwayml/stable-diffusion-v1-5

Quickstart:

huggingface-cli download runwayml/stable-diffusion-v1-5
export ABSTRACTVISION_BACKEND=diffusers
export ABSTRACTVISION_MODEL_ID=runwayml/stable-diffusion-v1-5
export ABSTRACTVISION_DIFFUSERS_DEVICE=auto
abstractvision repl

More model recommendations (by VRAM tier) are in docs/getting-started.md.

After that works, black-forest-labs/FLUX.2-klein-4B is the recommended next local test for a newer non-gated model (it currently requires Diffusers from source).

Does abstractvision t2i run locally?

abstractvision t2i / abstractvision i2i are one-shot helpers for the OpenAI-compatible HTTP backend (../src/abstractvision/cli.py).

For local generation, use abstractvision repl with /backend diffusers ... or /backend sdcpp ....

Where do generated outputs go?

It depends on whether you configured a store:

What is an “artifact ref”?

An artifact ref is a small JSON dict that points to a stored blob. Minimal shape:

{"$artifact":"<id>"}

Helpers: is_artifact_ref() / make_media_ref() in ../src/abstractvision/artifacts.py.

How do I allow or block Diffusers downloads?

Why do I get “missing pipeline class” errors (e.g. GlmImagePipeline)?

Some newer pipelines may only exist on Diffusers GitHub main. Install:

  • pip install -U "abstractvision[diffusers-dev]" (compatible dependency versions)
  • pip install -U "git+https://github.com/huggingface/diffusers@main" (Diffusers main)

See: docs/getting-started.md.

macOS (MPS): why do I get black images / dtype errors?

The Diffusers backend includes MPS-specific mitigations (e.g. VAE upcast and optional fp32 retry) in ../src/abstractvision/backends/huggingface_diffusers.py.

Common fixes: - set ABSTRACTVISION_DIFFUSERS_TORCH_DTYPE=float32 (more stable, higher memory) - disable retry if memory is tight: ABSTRACTVISION_DIFFUSERS_AUTO_RETRY_FP32=0 - consider using the stable-diffusion.cpp backend for GGUF diffusion models (docs/getting-started.md)

Windows/Linux (CUDA): why is torch.cuda.is_available() false?

On Windows/Linux, pip install torch (and packages that depend on torch) may install a CPU-only PyTorch build by default.

If you have an NVIDIA GPU and want CUDA acceleration:

1) Install a CUDA-enabled PyTorch wheel using the official selector: https://pytorch.org/get-started/locally/
2) Verify:

python -c "import torch; print('cuda', torch.cuda.is_available())"

How do I pass advanced flags / parameters?

AbstractVision exposes an extra dict on requests (../src/abstractvision/types.py), and the REPL forwards unknown --flags into request.extra (../src/abstractvision/cli.py).

Examples: - Diffusers backend: accepts extra keys like loras_json and rapid_aio_repo (used by Qwen Image Edit flows; see docs/getting-started.md and ../src/abstractvision/backends/huggingface_diffusers.py). - stable-diffusion.cpp backend: - CLI mode forwards flags to sd-cli - python-binding mode maps supported keys to binding kwargs and ignores unsupported keys (../src/abstractvision/backends/stable_diffusion_cpp.py)

What does the capability registry mean (and what does it not mean)?

The registry answers “what a model claims to support” (task keys/params) and can be used for optional gating:

It does not guarantee your configured backend can execute the task; backend support is a separate constraint (docs/reference/backends.md).

I only need the HTTP backend. Do I have to install Torch/Diffusers?

No. The base install is lightweight and includes the stdlib OpenAI-compatible HTTP backend without Torch/Diffusers (see ../pyproject.toml). Heavy local backend modules are still imported lazily (../src/abstractvision/backends/__init__.py).

Install abstractvision[diffusers] only when you want local Diffusers generation. Use abstractvision[sdcpp] or an external sd-cli only when you need stable-diffusion.cpp.

How do I integrate with AbstractCore?

Two options (details in docs/reference/abstractcore-integration.md):

AbstractCore is the host package; AbstractVision does not install it as a dependency.

How do I run tests?

From the repo root:

python -m unittest discover -s tests -p "test_*.py" -q