Z-Image-Turbo-Fun-Controlnet-Union: Precise Control on a Lightweight Model

Z-Image-Turbo-Fun-Controlnet-Union is a lightweight but powerful combo of Z-Image-Turbo and a multi-condition ControlNet, designed to give creators precise structural control without sacrificing speed or image quality. It targets real production workflows: fast iteration, predictable control, and minimal GPU footprint, which is especially attractive for indie developers and small studios.

What is Z-Image-Turbo-Fun-Controlnet-Union?

Z-Image-Turbo-Fun-Controlnet-Union is a ControlNet model family built specifically for the Z-Image-Turbo text-to-image model released by Alibaba Cloud PAI on Hugging Face. Instead of shipping many separate ControlNet models (one for pose, one for edges, one for depth, etc.), it adopts a "Union" design so that multiple control types can be supported under a unified framework.

The base version adds control blocks to 6 layers, while the newer 2.x series extends control to 15 main layer blocks plus refiner blocks for stronger and more detailed conditioning. Both generations are trained from scratch on 1 million high-quality images at 1328 resolution with bfloat16 precision and a batch size of 64, balancing quality and efficiency.

Why a "Union" ControlNet?

Traditional ControlNet workflows often require you to download and manage several separate models: one for Canny edges, one for HED, one for depth, one for pose, and so on. The Union approach consolidates these into a more unified control model, reducing model-switching overhead and making it easier to build streamlined pipelines.

With Z-Image-Turbo-Fun-Controlnet-Union, you can drive Z-Image-Turbo using various structural hints such as edges, depth, line segments, or pose while keeping the same generation engine and a consistent configuration style. For production use, this simplifies deployment, monitoring, and scaling, because you standardize on one coherent control ecosystem instead of a zoo of loosely-related checkpoints.

Core Features and Capabilities

Multi-condition structural control

The model supports common control conditions including Canny, HED, Depth, Pose, and MLSD, which can be used like a standard ControlNet input. This allows workflows based on sketches, 3D depth maps, or extracted edges while still letting the text prompt define style, lighting, and semantics.

High-quality training and inpaint support (2.x)

The 2.0 model is trained for 70,000 steps on the same 1M-image dataset, with more control blocks and added inpainting support for editing existing images. Version 2.1 further fixes a speed regression by adjusting how refiner latents are processed, bringing inference performance back to normal while preserving quality.

Tunable control strength

A key hyperparameter is control_context_scale, which controls how strongly the structural map influences the final result. The recommended range is roughly 0.65–0.80 for the first generation and 0.65–0.90 for 2.0+, and higher control strength often benefits from more sampling steps to maintain realism.

Practical Use Cases

Creative ideation and concept art

Artists can sketch a rough pose or layout, feed it as a control map (e.g., Canny or pose), and let the model fill in details according to a descriptive prompt, greatly accelerating iteration. Because the structure is anchored by the control input, changing style prompts won't randomly break composition or character proportions.

Product and e‑commerce imagery

By using edge or depth maps derived from 3D renders or existing product shots, teams can generate stylistic variants, seasonal backgrounds, or marketing layouts while preserving core product geometry. The relatively compact model size makes it easier to run on-prem or in a small cloud setup without enterprise-scale hardware.

Character sheets, comics, and game assets

Pose control combined with fixed prompts and seeds can yield multiple consistent views or actions of the same character for storyboards, comics, or 2D game assets. Since the Union model is compatible with tools like ComfyUI workflows, it can slot into existing node-based pipelines with minimal glue code.

Getting Started and Integration Tips

Use the official Hugging Face models

The original Z-Image-Turbo-Fun-Controlnet-Union checkpoint and the enhanced 2.0/2.1 versions are hosted under the Alibaba-PAI namespace on Hugging Face. For most new projects, starting directly with the 2.x line is recommended, as it brings more control blocks, inpainting support, and better performance.

Integrate via VideoX-Fun

Official inference examples and scripts are provided in the VideoX-Fun repository, which shows how to set up directories, download weights, and run text-to-image or inpainting with control inputs. This repo is currently the reference implementation for using Z-Image-Turbo and its Fun ControlNet Union variants together in Python-based pipelines.

Try ComfyUI workflows

There are example ComfyUI workflows that demonstrate using Z-Image-Turbo plus the Fun Union ControlNet model with Canny edge guidance. These workflows illustrate how to wire text encoders, diffusion models, VAE, and the Union ControlNet checkpoint in a node graph so you can visually tweak prompts, strengths, and image post-processing.

Official Links

For anyone who wants to experiment or integrate the model into a pipeline, here are the official links:

Z-Image-Turbo-Fun-Controlnet-Union (first release, 6 control blocks): https://huggingface.co/alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union

Z-Image-Turbo-Fun-Controlnet-Union-2.0 (2.x series, more control blocks + inpainting, includes 2.1 weights): https://huggingface.co/alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union-2.0

Official inference and workflow code (VideoX-Fun repository): https://github.com/aigc-apps/VideoX-Fun