Skip to content

MLX-Gen Documentation

MLX-Gen is an MLX-native generative image and video runtime forked from mflux. It can be used directly from the command line or embedded in Python applications that need local Apple Silicon generation.

Guides

  • Getting started: install MLX-Gen, inspect the CLI, prepare or download model files, run image generation/editing, restore or upscale with SeedVR2, and start Wan or Bernini video workflows.
  • Architecture: system overview and model-file lifecycle diagrams, package and factored-source shapes, role-aware routing, command boundaries, runtime failure contract, and Python integration boundary.
  • API and CLI: public mlxgen command surface, generation router behavior, image/video/reference input roles, image editing, the --outpaint-fill and --outpaint-passes conditioning-canvas options, negative-prompt support per route, SeedVR2, Wan video controls, Bernini R2V/RV2V/V2V, metadata, and capability schema 16.
  • Image edit modes: plain-language guide to latent img2img, edit-reference, masked edit/inpaint, base-Qwen control-inpaint, Qwen structured control, multi-reference, generative reframe, and outpaint including its selectable conditioning canvas, covering what each mode is good at and what to expect from the output.
  • Masked editing: the canonical masked-edit page — request contract, per-model route matrix with proof grades, per-family behavior (Qwen edit and base routes with the tunable --mask-strength warm start, Z-Image Turbo, FLUX.2 Klein, video masks), and route selection advice.
  • Qwen route matrix: current MLX-Gen route truth for Qwen Image and Qwen Image Edit, mapping public mlxgen capability ids to the upstream Diffusers Qwen pipelines and the exact proof surfaces that already exist.
  • Qwen localized editing: plain-language explanation of Qwen masked edit, Qwen structured control, and shipped base-Qwen control-inpaint, including definitions of ControlNet and “sidecar”, the exact proof rows, and when each route is the right tool.
  • Wan video: practical Wan2.2 T2V/I2V sizing, plain and masked prompt-guided A14B video-to-video, the natively ported Wan2.1-VACE-1.3B route (wan-vace: reference-image object injection and learned mask conditioning with --vace-masked-region), the measured motion-fidelity ladder (strength vs gesture preservation, with a motion-preserving restyle recipe), the fps-resampling and audio copy-through contract with a playable proof, broader A14B target size families, full example commands, and included MP4/frame-strip assets.
  • MiniMax-H3 video with audio: native MiniMax-H3 text-to-video with a synchronized stereo AAC soundtrack in one MP4, the minimax-h3, minimax-h3-turbo and minimax-h3-turbo-544p catalog entries with lightx2v 8-step Turbo adapters, community LoRA adapters in either key layout stacked on them, the structured prompt format (--soundscape, --music), frame/canvas rules, measured M5 Max runtime and memory at 544p and 768p, the prepared-package workflow (mlxgen prepare, --base-model), the generated-audio metadata contract, Python usage, first-frame image-to-video (--image-path) with motion-prompt guidance, current limits, and the included contact sheets and clips.
  • Bernini-R 1.3B: experimental reference-to-video and video-edit routes, pinned BF16 runtime setup, measured memory, the committed official public parity bundle (full prompts and contact sheets per upstream example row), and task-specific recipes for many-reference binding and structure-changing edits.
  • Spaceship snow workflow: reproducible model-backed T2I, I2I edit, multi-reference I2I, T2V A14B, and I2V A14B commands with included assets.
  • Generation previews: step-wise previews rendered with published tiny autoencoders (taef1, taef2), the --preview-decoder auto|tiny|full contract, the per-latent-space availability table, measured agreement against the full VAE, and the guarantee that final outputs are always full-VAE decoded.
  • Image upscaling: SeedVR2 command usage for image and video restoration, published 3B/7B q8/q4 package handles, shortest-edge and scale-factor sizing, the conservative safe-video profile, the validated June 21 five-second Eiffel 1x and 2x 3B/7B proof bundles plus timings/memory data, and real 5x image comparisons from a 133x113 source.
  • Image edit capabilities: image-edit plus generative reframe and outpaint contact sheets, exact model/package status, the Qwen Image Edit 2511 q8 masked-edit proof, the exact base Qwen q8 structured-control and control-inpaint proofs, the exact Z-Image Turbo q8 native-inpaint proof, the FLUX.2 Klein base source-model starship proof, the published outpaint capability fields including outpaint_preservation and the per-route validated envelope, and command logs for Qwen Image Edit, Qwen Image Edit 2509/2511, FLUX.2 Klein, Qwen control routes, Z-Image native inpaint, and latent I2I rows. FIBO Edit is unsupported through unified mlxgen generate.
  • Reframe and outpaint: --reframe-padding and --outpaint-padding workflows, per-side padding and how to extend one side, one axis, or all four in a single call, the --outpaint-fill conditioning-canvas modes and how auto resolves them, padding guidance, supported models, strict FLUX.2 Klein outpaint on distilled and base weights, the cross-route outpaint matrix with per-route timings and source drift on one source, the axis-coverage sheets across three source aspect ratios, and the validation profile ids for canvas expansion workflows.
  • LoRA: route-specific capability fields, explicit adapter download, strict scale matching, model-card base-model compatibility, source/no-LoRA/with-LoRA validation, exact public proof rows for the current Qwen, Z-Image, FLUX.2, ERNIE, and Wan routes, MiniMax-H3 adapters in the diffusers and original-checkpoint key layouts, and the current guidance that MLX-Gen q8 packages are the validated Lightning target rather than arbitrary external FP8 checkpoints.
  • Model management: explicit download and prepare workflows, generation from local files, Bernini's pinned factored download and disk preflight, local package resolution, and Depth Pro downloads.
  • Model recommendations: conservative starting picks for 18 GB, 24 GB, 32 GB, 64 GB, and 128+ GB Macs, using published MLX-Gen memory measurements rather than package size alone.
  • Quantization: current low-bit compatibility by model family, including the published package matrix, Bernini's BF16-only boundary, Bonsai ternary 2-bit support, SeedVR2 packages, Qwen/ERNIE policies, and Wan storage/runtime behavior.
  • Hugging Face publishing: generated model cards, source license/access wording, default AbstractFramework/<repo-name> usage, upload flow, and optional collection membership.
  • Python integration: route-resolved runtime planning/loading, Bernini's typed reference inputs, serial multi-output reuse, the model-agnostic outpaint and reframe pipeline (run_outpaint, prepare_outpaint, prepare_reframe) and the post_process before-save hook, SeedVR2's direct-model boundary, and shared progress callbacks.
  • Release: GitHub Release and PyPI trusted publishing workflow.
  • FAQ: common questions about setup, memory versus disk, quantization, image/edit modes, SeedVR2, Wan, and Bernini.
  • Troubleshooting: common setup, cache, local-path, routing, quantization, outpaint conditioning-canvas, Bernini reference-fidelity, and Wan video problems with concrete fixes.

The top-level README remains the starting point for installation, model families, and project relationship details.

Project Documents