Level: advanced · Reading time: 35 minutes · Updated:

Как запустить MiniMax H3 на игровой видеокарте: int8, offload и честный замер скорости

Как запустить MiniMax H3 на игровой видеокарте: int8, offload и честный замер скорости
Temporary fallback cover; replace in editorial pass.

Open weights do not mean "runs on your PC". With a large video-generation model such as MiniMax H3, the hard part is not downloading the checkpoint. It is the unstable inference stack, the checkpoint conversion step, a system-RAM requirement that is easy to underestimate, and latency figures that nobody publishes in a form you can compare. This guide builds a bench that answers one narrow question with numbers you produce yourself: on this specific gaming GPU, what does int8 save compared with bf16, what does each offload mode cost, and how do resolution and step count change latency and memory use?

Why a "честный замер" needs a protocol

Most published speed claims for local video models skip at least one of these:

If any of these is missing, two numbers are not comparable. The bench below records all of them for every run, so that a row in your table means the same thing as a row in someone else's.

The concrete case

The scenario is a single desktop machine with one consumer card, which we call "the gaming GPU". Typical examples are cards with 16 or 24 GB of VRAM. The goal is a short text-to-video clip at the lowest settings that still produce usable output, followed by a scale-up. You compare:

Expect some combinations to fail with out-of-memory errors. A failure is a valid result. Record it as OOM with the stage at which it happened, and do not drop the row.

Prerequisites and what to verify first

  1. Read the model card. Record the license, the officially supported inference path (the repository's own scripts, a diffusers pipeline, or both), the declared minimum hardware, and whether an official quantized checkpoint exists. Do not rely on third-party summaries.
  2. Check disk space. You need room for the original checkpoint, a converted copy if conversion is required, and the Hugging Face cache. Conversion can briefly need roughly twice the checkpoint size.
  3. Check system RAM and swap. Offload moves weights into system RAM. If RAM plus swap is smaller than the bf16 weights, offload modes will thrash or be killed by the OOM killer. Look at the actual safetensors sizes in the repository and plan from those, not from a parameter count.
  4. Check the driver and CUDA. Use nvidia-smi to confirm that the driver supports the CUDA version your PyTorch wheel was built for.
# Record the machine once; store it next to the results
nvidia-smi --query-gpu=name,memory.total,driver_version --format=csv
free -g
df -h ~/.cache/huggingface
python -c "import torch;print(torch.__version__, torch.version.cuda, torch.cuda.get_device_name(0))"

Step 1. Pin an isolated environment

The stack around new video models moves quickly. Install a pinned environment, so that a run you repeat next week uses the same code. Use the exact versions the model repository specifies. The values below are placeholders and must be replaced.

python -m venv ~/venvs/h3-bench
source ~/venvs/h3-bench/bin/activate
pip install --upgrade pip

# Replace with the versions pinned in the model repository's requirements
pip install "torch==<PINNED>" --index-url https://download.pytorch.org/whl/<CUDA_TAG>
pip install "diffusers==<PINNED>" "transformers==<PINNED>" "accelerate==<PINNED>" \
            "torchao==<PINNED>" safetensors psutil

pip freeze > bench/env.lock.txt

If the model requires installing its repository from source, clone it, pin it to a commit, and record that commit hash:

git clone <OFFICIAL_REPO_URL> h3-src
cd h3-src && git checkout <COMMIT> && git rev-parse HEAD > ../bench/model_repo_commit.txt
pip install -e .

Step 2. Download and, if required, convert the checkpoint

Download into an explicit directory rather than relying on the cache layout. Use huggingface-cli login interactively or set HF_TOKEN in your shell. Never write the token into scripts or into this bench's result files.

huggingface-cli download <MODEL_REPO_ID> --local-dir ./weights/h3-original
sha256sum ./weights/h3-original/*.safetensors > bench/weights.sha256

Some open video models ship weights in a layout that the diffusers loader cannot read directly, and provide a conversion script for that purpose. If the H3 repository includes one, run it exactly as documented:

# Placeholder: use the script name and flags from the official repository
python h3-src/<convert_script>.py \
  --input ./weights/h3-original \
  --output ./weights/h3-diffusers \
  --dtype bfloat16

Run the conversion to bf16, not to int8. Quantize at load time (Step 4). That keeps one source of truth on disk and makes the int8 against bf16 comparison clean: same weights, different runtime precision.

How to verify the conversion: load the converted model on CPU, count the parameters and compare the count with the original's sum of tensor element counts. Then generate a 2–4-step smoke clip. A conversion bug usually shows up as pure noise or a solid colour, not as a subtle quality drop.

Step 3. The measurement harness

The harness does not depend on how the model is invoked. It wraps any command and records wall time, peak VRAM (sampled from nvidia-smi), peak RSS of the process tree, and the exit status. That way you can measure the official CLI script and your own Python pipeline with the same tool.

# bench/measure.py
import json, subprocess, sys, threading, time, platform
import psutil

def sample_gpu(stop, peak):
    while not stop.is_set():
        out = subprocess.run(
            ["nvidia-smi", "--query-gpu=memory.used", "--format=csv,noheader,nounits", "-i", "0"],
            capture_output=True, text=True).stdout.strip()
        if out:
            peak["vram_mib"] = max(peak["vram_mib"], int(out.splitlines()[0]))
        time.sleep(0.2)

def tree_rss(proc):
    try:
        procs = [proc] + proc.children(recursive=True)
        return sum(p.memory_info().rss for p in procs if p.is_running())
    except psutil.Error:
        return 0

def main():
    label, cmd = sys.argv[1], sys.argv[2:]
    peak = {"vram_mib": 0, "rss_bytes": 0}
    stop = threading.Event()
    t = threading.Thread(target=sample_gpu, args=(stop, peak), daemon=True)
    t.start()
    t0 = time.perf_counter()
    p = subprocess.Popen(cmd)
    ps = psutil.Process(p.pid)
    while p.poll() is None:
        peak["rss_bytes"] = max(peak["rss_bytes"], tree_rss(ps))
        time.sleep(0.2)
    wall = time.perf_counter() - t0
    stop.set(); t.join()
    rec = {
        "label": label, "cmd": cmd, "exit_code": p.returncode,
        "wall_s": round(wall, 2),
        "peak_vram_mib": peak["vram_mib"],
        "peak_rss_gib": round(peak["rss_bytes"] / 2**30, 2),
        "host": platform.node(), "ts": time.strftime("%Y-%m-%dT%H:%M:%S"),
    }
    with open("bench/results.jsonl", "a") as f:
        f.write(json.dumps(rec) + "\n")
    print(json.dumps(rec, indent=2))

if __name__ == "__main__":
    main()

Note the limits of this approach. nvidia-smi reports memory used on the whole device, including other processes and the CUDA context. Close the browser and other GPU applications, and record the idle baseline before the run. Sampling every 200 ms can miss very short peaks. For exact allocator peaks, also log torch.cuda.max_memory_reserved() from inside the pipeline script (Step 4).

Step 4. One run script with switchable precision and offload

This script assumes that a diffusers-compatible pipeline exists for the converted weights. If the model only ships its own inference script, apply the same three switches through that script's options, if it has them, and still wrap it with measure.py. The diffusers and torchao calls below are real APIs of those libraries. Whether H3's pipeline class supports them without changes is something you have to confirm yourself.

# bench/run_h3.py
import argparse, json, time, torch
from diffusers import DiffusionPipeline

ap = argparse.ArgumentParser()
ap.add_argument("--weights", default="./weights/h3-diffusers")
ap.add_argument("--precision", choices=["bf16", "int8"], default="bf16")
ap.add_argument("--offload", choices=["none", "model", "sequential"], default="none")
ap.add_argument("--width", type=int, required=True)
ap.add_argument("--height", type=int, required=True)
ap.add_argument("--frames", type=int, required=True)
ap.add_argument("--steps", type=int, required=True)
ap.add_argument("--seed", type=int, default=1234)
ap.add_argument("--prompt", default="A red paper boat drifting on a calm pond, morning light")
ap.add_argument("--out", default="bench/out.mp4")
a = ap.parse_args()

t_load = time.perf_counter()
pipe = DiffusionPipeline.from_pretrained(a.weights, torch_dtype=torch.bfloat16)

if a.precision == "int8":
    # Weight-only int8 on the denoiser only; text encoder and VAE stay bf16.
    from torchao.quantization import quantize_, int8_weight_only
    denoiser = getattr(pipe, "transformer", None) or getattr(pipe, "unet")
    quantize_(denoiser, int8_weight_only())

if a.offload == "none":
    pipe.to("cuda")
elif a.offload == "model":
    pipe.enable_model_cpu_offload()
else:
    pipe.enable_sequential_cpu_offload()
load_s = time.perf_counter() - t_load

torch.cuda.reset_peak_memory_stats()
g = torch.Generator(device="cpu").manual_seed(a.seed)
torch.cuda.synchronize()
t_gen = time.perf_counter()
result = pipe(prompt=a.prompt, width=a.width, height=a.height,
              num_frames=a.frames, num_inference_steps=a.steps, generator=g)
torch.cuda.synchronize()
gen_s = time.perf_counter() - t_gen

from diffusers.utils import export_to_video
export_to_video(result.frames[0], a.out, fps=16)

print(json.dumps({
    "precision": a.precision, "offload": a.offload,
    "w": a.width, "h": a.height, "frames": a.frames, "steps": a.steps, "seed": a.seed,
    "load_s": round(load_s, 2), "gen_s": round(gen_s, 2),
    "s_per_step": round(gen_s / a.steps, 3),
    "torch_peak_reserved_gib": round(torch.cuda.max_memory_reserved() / 2**30, 2),
}))

Design choices worth keeping:

Step 5. The run matrix

Do not run every combination at once. Start from the cheapest configuration that has a chance of fitting and expand outward. If a configuration hits OOM, every heavier configuration on the same axis will too, so record it and skip them.

# bench/matrix.sh
set -u
FR=<SUPPORTED_FRAME_COUNT>          # fix one frame count for the whole matrix
RES=("<W_LOW>x<H_LOW>" "<W_MID>x<H_MID>")
STEPS=(10 30)
for prec in int8 bf16; do
  for off in sequential model none; do
    for r in "${RES[@]}"; do
      W=${r%x*}; H=${r#*x}
      for s in "${STEPS[@]}"; do
        L="$prec-$off-${W}x${H}-f$FR-s$s"
        # warm-up run, discarded
        python bench/run_h3.py --precision $prec --offload $off \
          --width $W --height $H --frames $FR --steps 2 --out /tmp/warm.mp4 >/dev/null 2>&1
        for rep in 1 2 3; do
          python bench/measure.py "$L-r$rep" \
            python bench/run_h3.py --precision $prec --offload $off \
              --width $W --height $H --frames $FR --steps $s \
              --out "bench/clips/$L-r$rep.mp4" \
            | tee -a "bench/log-$L.txt"
        done
      done
    done
  done
done

Three repetitions per cell is the minimum for reporting a median and seeing variance. Every command starts a new Python process, so each repetition includes load time in wall_s. The gen_s value printed by run_h3.py is the warm generation figure you report as speed. Keep both: for someone generating one clip per session, wall_s is the honest number.

Step 6. The results table

Aggregate with a short script that takes the median of the three repetitions per cell. Then fill in the table. The cells are deliberately empty here: these are your measurements to make. A published version of this table should state the GPU model, VRAM, system RAM, driver, the PyTorch/diffusers/torchao versions and the model repository commit.

PrecisionOffloadResolutionSteps Load, sGen, s (median)s/step Peak VRAM, GiBPeak RAM, GiBStatus
bf16nonelow10—————not measured
bf16modellow10—————not measured
bf16sequentiallow10—————not measured
int8nonelow10—————not measured
int8modellow10—————not measured
int8sequentiallow10—————not measured
… repeat for mid resolution and 30 steps
 
# bench/aggregate.py — reads per-run JSON lines printed by run_h3.py from the logs
import glob, json, statistics, collections
rows = collections.defaultdict(list)
for path in glob.glob("bench/log-*.txt"):
    for line in open(path):
        line = line.strip()
        if line.startswith("{") and '"gen_s"' in line:
            r = json.loads(line)
            key = (r["precision"], r["offload"], f'{r["w"]}x{r["h"]}', r["steps"])
            rows[key].append(r)
for k, rs in sorted(rows.items()):
    med = lambda f: statistics.median(x[f] for x in rs)
    print(*k, med("load_s"), med("gen_s"), med("s_per_step"),
          med("torch_peak_reserved_gib"), f"n={len(rs)}", sep=" | ")

How to read the table without fooling yourself

Step 7. Verify output quality, not just speed

A faster configuration that produces worse video is not a win. With identical seeds and prompts, do at least the following:

  1. Put the bf16 and int8 clips side by side for the same cell and inspect them frame by frame for texture loss, colour shift and temporal flicker.
  2. Compute a simple numeric difference between corresponding frames, such as mean absolute difference or PSNR against the bf16 clip. This is not a perceptual quality score. It flags when int8 diverges sharply.
  3. Use three or four different prompts, including one with fine detail such as text, hands or thin lines, where quantization artifacts appear first.
# Quick frame-level PSNR between two clips (requires ffmpeg)
ffmpeg -i bench/clips/int8-model-LOWxLOW-fFR-s30-r1.mp4 \
       -i bench/clips/bf16-model-LOWxLOW-fFR-s30-r1.mp4 \
       -lavfi psnr -f null - 2>&1 | tail -n 1

Failure cases and what they usually mean

CUDA out of memory during pipe.to("cuda")
The weights alone do not fit. Switch to int8 or to an offload mode. This happens before generation, so record "OOM at load".
OOM in the middle of denoising or during VAE decoding
Activations or the decoder are the limit. Lower the resolution or frame count, or check whether the pipeline exposes VAE tiling or slicing options. Record which stage failed.
The process is killed with no Python traceback
The Linux OOM killer stopped it because system RAM ran out, which is common with offload or during checkpoint conversion. Check dmesg | tail. Add swap or reduce concurrent memory use. This is the "large RAM" requirement in practice.
Output is noise or a flat colour
Usually a conversion or dtype mismatch, such as weights converted with the wrong key mapping or a component running in fp16 where it needs bf16. Re-check Step 2 before blaming quantization.
quantize_ fails or has no effect
The torchao version does not match the installed PyTorch, or the denoiser uses layer types that the weight-only config does not target. Check the torchao release notes for your PyTorch version. Newer torchao releases expose config-object APIs alongside or instead of int8_weight_only().
enable_*_cpu_offload raises an error or does nothing
The pipeline may not declare an offload order, or the model's own code may move tensors to the device manually. Fall back to the official script's options, if it has any, and note it in the table.
Large run-to-run variance
Look for thermal throttling (nvidia-smi -q -d PERFORMANCE), background GPU applications, or a power limit. Lock conditions down and re-run before reporting.

Limitations of this bench

Reproducibility checklist

For related walkthroughs on local models and agent testing, see the guides index. Definitions of terms such as quantization, offload and VRAM are in the glossary.

We publish what works for us—and implement the same solutions for your business. We design AI automation, Telegram bots, chats, and AI agents for real-world processes. Discuss your project →