Python with PEFT

Published evaluations run in transformers with PEFT, so this is the tested way to use a patch in your own code. Install the patch with the CLI first, so the signature and every file hash are checked, then point PEFT at the install directory.

Requirements

• The exact base revision named in the patch manifest. Other revisions, sizes and quantized variants are not supported.
• The library versions the patch was tested with, recorded in the manifest under runtimeCompatibility.testedVersions and shown on the patch page. pip install 'retoque[runtime]' installs torch, transformers, peft and safetensors.
• About 3.1 GB of disk for the Qwen2.5-1.5B weights and about 7 GB of free memory in float32. CPU works but is slow.

Load the base and the adapter

Keep remote code off and load weights from safetensors only.

import os

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

BASE = "Qwen/Qwen2.5-1.5B-Instruct"
REVISION = "989aa7980e4cf806f80c7fef2b1adb7bc71aa306"  # the full commit from the manifest
PATCH_DIR = os.path.expanduser("~/.retoque/patches/<slug>/<version>")

tok = AutoTokenizer.from_pretrained(BASE, revision=REVISION, trust_remote_code=False)
tok.padding_side = "left"
base = AutoModelForCausalLM.from_pretrained(
    BASE,
    revision=REVISION,
    torch_dtype=torch.float32,
    trust_remote_code=False,
    use_safetensors=True,
)
model = PeftModel.from_pretrained(base, PATCH_DIR, is_trainable=False).eval()

The install directory only holds the eight package files, so PEFT can only find adapter_model.safetensors. There is no pickle file to fall back to.

Generate with and without the patch

Use the same system prompt and decoding settings the patch was evaluated with. Published results use greedy decoding with a 128-token limit. Sampled decoding is untested.

messages = [
    {"role": "system", "content": SYSTEM_PROMPT},  # the prompt the patch was evaluated with
    {"role": "user", "content": "Today is 2026-03-02 (Monday).\nPay Halcyon Realty $4,200.00 for the office rent, due next Friday."},
]
prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tok(prompt, return_tensors="pt").to(model.device)

with torch.inference_mode():
    patched = model.generate(**inputs, do_sample=False, max_new_tokens=128, pad_token_id=tok.pad_token_id)
    with model.disable_adapter():
        base_only = model.generate(**inputs, do_sample=False, max_new_tokens=128, pad_token_id=tok.pad_token_id)

new_tokens = inputs["input_ids"].shape[1]
print(tok.decode(patched[0, new_tokens:], skip_special_tokens=True))

Check what you loaded

retoque status prints the active adapter sha256, and retoque verify <slug>@<version> re-checks the installed files. To confirm in Python that the adapter on disk is the one the manifest names:

import hashlib, json, os

manifest = json.load(open(os.path.join(PATCH_DIR, "retoque.manifest.json")))
digest = hashlib.sha256(open(os.path.join(PATCH_DIR, "adapter_model.safetensors"), "rb").read()).hexdigest()
assert digest == manifest["adapter"]["sha256"]

What not to do

• Don't call merge_and_unload() and redistribute the result. Merged weights are not a supported or tested form, and a merge can't be undone with retoque disable.
• Don't load an adapter from a directory you haven't verified. Run retoque verify <slug>@<version> if the files may have changed.
• Don't assume a quantized base behaves the same. It hasn't been evaluated.
• Don't set trust_remote_code=True for a patch. Nothing in a Retoque package needs it.