AI model provenance is becoming a release-control system

A production incident often starts with a deceptively simple question: which model was running? The answer “Model X” is usually too vague. The same name may point to different weight revisions, a new tokenizer, a changed adapter, a fresh quantization, altered inference code or a provider alias that moved to another snapshot.

Artifact identity needs content-addressed references for every component that can change behavior. A digest answers which bytes were supplied. Provenance explains where those bytes came from and how they were produced. A signature identifies who made a claim about them. Approval records show which policy allowed the release. None of those controls replaces the others.

The practical standard is whether an independent operator can reconstruct the deployed release without trusting a mutable registry tag or someone’s memory. If the release cannot be rebuilt or at least verified from retained evidence, the model name is only a convenience label.

A modern model release is rarely one file. It may contain sharded weights, configuration, tokenizer vocabulary, generation defaults, custom preprocessing, adapters, safety classifiers, quantization tables, kernels, serving code and a container image. A retrieval or agent system can also depend on prompt packages, tool schemas, policy bundles and embedding models.

Treat these pieces as a graph with explicit relationships. The base model was adapted by this training run. The adapter expects this tokenizer. The quantized artifact was derived from this source digest using this converter and calibration data. The serving image contains this runtime and these native libraries. The policy bundle approved only this combination.

A flat registry entry hides transitive substitutions. An attacker or careless operator may preserve the top-level name while replacing a converter, adapter or code path. The graph makes those changes visible and lets policy distinguish harmless metadata edits from behavior-changing transformations.

A cryptographic digest provides a stable reference to content. It prevents a tag or filename from silently resolving to different bytes after approval. Large models require a canonical rule for which paths, metadata and ordering are included, otherwise two tools may calculate different identities for the same logical model directory.

The release process should produce a manifest that lists every object and digest, plus a top-level digest for the manifest or OCI artifact. OCI registries can store non-container artifacts and attach signatures, attestations and bills of materials through subject and referrer relationships. Blob-signing tools can cover model directories when a registry is not used.

Hashing does not say whether an artifact is safe, licensed or useful. It makes later claims unambiguous. Once evaluation, approval and deployment refer to the same digest, evidence cannot drift quietly to a different candidate.

A valid signature can prove that a trusted identity signed a digest. It does not prove that the signer trained the model correctly, owned the data, evaluated the final bytes or intended the artifact for your use. Provenance can describe the builder, inputs and process, but the builder may be compromised or the process may violate local rules.

Approval is the consumer’s decision. It verifies signatures and attestations against expectations: permitted builders, source repositories, branch protection, training environment, model license, data restrictions, security scans, evaluation thresholds and reviewer separation. High-risk deployments should fail closed when required evidence is missing or unverifiable.

Transparency logs or SCITT-style receipts add accountability by making signed statements discoverable and auditable. They reduce silent equivocation, but they still do not turn a false statement into a true one. Consumers must evaluate the issuer and the claim.

Model lineage should begin before training and continue after it. The training statement can identify code revision, framework, hyperparameters, data references, random seeds, hardware class and checkpoint outputs. Fine-tuning, adapter merging, pruning, conversion, quantization and packaging should each create a new artifact with its own transformation record.

This matters because the final deployed candidate may differ materially from the checkpoint that passed evaluation. Quantization can change quality. A converter can alter tensor layout. A merge can use the wrong base revision. Custom code can change preprocessing or output interpretation without touching weights.

SLSA-style provenance and in-toto attestations provide a useful general structure: subjects, materials, builders and execution details. AI-specific profiles in SPDX and CycloneDX add models, datasets and lifecycle metadata. The strongest systems connect those formats rather than forcing one document to carry every fact.

Training-data transparency is difficult because datasets can be enormous, licensed, private, filtered dynamically or assembled from many sources. A useful record may combine dataset identifiers, versions, custodians, license and policy categories, collection windows, filtering code, deduplication settings and privacy controls without exposing the underlying records.

Model cards and bills of materials are declarations. They help users understand intended use, limitations, datasets and evaluation, but they can be incomplete or stale. Bind the card to a model digest and identify who produced it. Separate verified facts from provider claims and local observations.

When exact data cannot be disclosed, retain enough protected lineage for authorized audit and incident response. A vague sentence such as “trained on public and licensed data” is not equivalent to verifiable lineage, but a public system may legitimately expose less detail than an internal compliance record.

Model registries provide versioning, metadata, lineage, evaluation links, approval status and promotion workflows. They are valuable control points, but their names and aliases are usually mutable. “Champion” may point to version 12 today and version 13 tomorrow.

A deployment pipeline can accept an alias as an operator-friendly input, resolve it once, verify the underlying immutable artifact and record the resolution. Runtime configuration should pin the approved digest or version. Rollback should select a previously admitted identity rather than whatever an old tag currently resolves to.

Registry permissions deserve the same scrutiny as source control and build systems. Whoever can overwrite artifacts, move aliases, edit approval metadata or bypass policy can change production behavior. Audit the control plane as well as the model files.

Evidence generated in CI is useful only if the deployed environment enforces it. Admission policy should verify the artifact digest, signer identity, provenance, builder expectations, required BOMs, evaluation results, license rules, vulnerability status and approval record before the release can become active.

A policy engine can express different requirements by use case. A local summarizer may allow a community model after malware scanning and license review. A credit, medical or safety workflow may require controlled builders, independent evaluation, formal approval and a protected rollback path.

Verification should occur again when artifacts cross trust boundaries. Download caches, mirrors and model gateways can corrupt or substitute content. Runtime startup should log the exact admitted graph, and serving health should fail clearly if verification material expires or becomes unavailable.

With a hosted API, customers usually cannot hash the weights or inspect the serving image. Identity depends on the provider’s model identifier, snapshot policy, region or inference profile, API version and contractual commitments. Stable version strings are stronger than aliases such as “latest,” but they are still provider assertions.

The application should record the requested identifier, the identifier returned by the service when available, provider, endpoint, API version, region, deployment or profile ARN, and relevant feature flags. Provider changes, deprecations and routing rules should trigger re-evaluation even when the application code is unchanged.

For high-assurance use, ask what the provider can attest: model snapshot stability, change notification, data handling, regional routing, safety configuration and incident disclosure. Hosted provenance is therefore partly technical and partly contractual. Do not present it as equivalent to self-hosted byte-level verification.

A verified model package can still run in the wrong environment. Driver, kernel, serving framework, quantization backend, generation defaults and safety configuration can change outputs. Runtime identity should include the container or environment digest, accelerator class, inference engine version and loaded model graph.

Traces and inference receipts should reference that runtime identity along with the application release and provider identifier. The aim is not to sign every token. It is to make consequential outputs attributable to a known admitted release and configuration.

Confidential-computing attestation can strengthen claims about what loaded inside protected hardware, while ordinary deployments may rely on signed images, startup verification and controlled configuration. Choose evidence proportional to risk and be explicit about what remains outside the trust boundary.

A signed malicious artifact remains malicious. Model formats and custom code can create execution risk before inference begins. Prefer non-executable weight formats such as safetensors where supported, disable remote code by default, pin repository revisions and scan files before load.

Signatures help detect tampering and establish origin. Malware, pickle and dependency scanning examine different risks. Sandboxed conversion can reduce exposure when legacy formats must be accepted. Security policy should identify which formats, code paths and dependencies are permitted in each environment.

The verifier itself is part of the trusted computing base. Protect roots of trust, certificate policies, transparency-log verification and canonicalization code. Test malformed manifests, path traversal, duplicate filenames, huge metadata, unsupported algorithms and signature-confusion cases.

Provenance adds storage, signing, verification, policy maintenance and review work. Model artifacts are large, but attestations should remain compact references rather than copies of weights or datasets. Store large objects once and attach small signed statements by digest.

The economic comparison is not provenance versus zero cost. The alternative includes manual release spreadsheets, ambiguous incidents, repeated evaluation, unsafe downloads, delayed audits and rollbacks that cannot reproduce the former state. Measure the cost per admitted release and per verified investigation.

Governance should define who may build, sign, approve, promote, revoke and override. Exceptions need expiry and visible risk acceptance. Revocation or newly discovered defects should identify every deployment and derived artifact affected by the compromised component.