Cisco Releases Open Source Tool for AI Model Provenance

Cisco has open-sourced a new toolkit aimed at solving a growing challenge in enterprise AI deployments: knowing exactly where an AI model came from, what happened to it during training and fine-tuning, and whether it has been tampered with before reaching production. The release targets a set of risks that have become increasingly urgent as organizations integrate third-party foundation models, pre-trained weights, and externally sourced ML pipelines into critical workflows.

What the Tool Does

The toolkit provides a framework for capturing and verifying AI model provenance — a chain of custody record that documents a model's origin, training data lineage, fine-tuning steps, and any modifications made throughout its lifecycle. Key capabilities include:

• Cryptographic signing of model artifacts — generates verifiable signatures at each stage of the model lifecycle (pre-training, fine-tuning, adapter merging, quantization) so that tampering can be detected downstream
• SBOM-style model manifests — produces machine-readable bills of materials describing the model's components, training datasets, and dependencies in a format compatible with emerging AI supply chain transparency standards
• Regulatory reporting support — generates audit-ready documentation aligned with frameworks such as the EU AI Act's transparency and risk documentation requirements
• Incident response integration — when a model is flagged as potentially compromised, the provenance record gives responders a structured starting point for root-cause analysis without needing to reconstruct the training pipeline from scratch

Why AI Model Provenance Matters Now

The risks the tool is designed to address are not hypothetical. Over the past 18 months, security researchers have demonstrated multiple practical attack vectors against the AI model supply chain:

Model Poisoning

Adversaries can introduce subtle backdoors into model weights during training or fine-tuning. A poisoned model behaves normally under standard evaluation conditions but produces attacker-controlled outputs when a specific trigger is present in the input. Without provenance tracking, detecting whether a deployed model has been poisoned requires re-running full evaluation suites — which may not catch well-designed backdoors — or retraining from scratch.

Supply chain provenance records can narrow the investigation window dramatically. If an organization can verify that a model's weights match a known-good signed artifact from a trusted source, they can rule out tampering at the artifact level (though not at the training data level).

Third-Party Model Risk

Enterprises are increasingly adopting pre-trained foundation models from public repositories (Hugging Face, GitHub, model zoos) and fine-tuning them internally. The security posture of these upstream sources varies significantly. Without a provenance framework, organizations have limited ability to verify what they are deploying or detect when a previously trusted model has been updated with malicious content.

This risk materialized concretely in early 2026 with multiple incidents involving trojanized model weights distributed via public repositories, targeting organizations in the financial services and healthcare sectors.

Regulatory Pressure

The EU AI Act, which entered into force in 2024, imposes documentation and transparency requirements on high-risk AI systems. Article 11 requires providers to maintain technical documentation sufficient to assess compliance — documentation that, in practice, requires provenance-level tracking of models, training data, and development processes. Similar requirements are emerging in US federal procurement contexts under draft AI assurance frameworks.

Organizations lacking model provenance records face both compliance exposure and the practical risk of being unable to demonstrate due diligence following an incident.

Technical Architecture

Cisco's toolkit is designed to integrate into existing MLOps pipelines rather than replace them. It operates as a layer of attestation tooling that hooks into common ML workflow steps:

from cisco_model_provenance import ProvenanceTracker
 
tracker = ProvenanceTracker(
    signing_key="path/to/private.pem",
    registry="your-registry.example.com"
)
 
# Attest a trained model artifact
attestation = tracker.attest(
    model_path="./models/classifier_v2.pt",
    training_run_id="run-abc123",
    dataset_hash="sha256:...",
    metadata={"framework": "pytorch", "version": "2.6.0"}
)
 
# Verify a model before deployment
result = tracker.verify(
    model_path="./models/classifier_v2.pt",
    expected_attestation=attestation
)

The attestation format is based on the in-toto framework — a supply chain security standard originally developed for software packages at NYU — extended with ML-specific fields for tracking training hyperparameters, dataset provenance, and hardware environment attestations.

Verification can be integrated into deployment gates:

# Example GitHub Actions gate
- name: Verify model provenance
  run: |
    cisco-model-provenance verify \
      --model ./models/prod_model.pt \
      --attestation ./attestations/prod_model.json \
      --trusted-keys keys/production.pub

Relation to Existing Standards

The release arrives as several parallel standardization efforts are attempting to address the same problem space:

• NIST AI RMF (AI Risk Management Framework) — Cisco's tool generates artifacts that map to NIST AI RMF Govern and Map function requirements
• MITRE ATLAS — The ML attack taxonomy provides the threat model that motivated the tool's design
• ML SBOM efforts — Multiple organizations including the Linux Foundation and CNCF are developing ML-specific SBOM formats; Cisco's manifest format is designed to be compatible with these emerging standards

Availability

The toolkit is available as an open source release on GitHub. Cisco intends to contribute the core attestation format specification to an industry working group to encourage cross-vendor adoption. Enterprise integrations with Cisco's Panoptica cloud security platform are expected to follow the open source release.

Organizations evaluating the tool should note that provenance tracking addresses the artifact integrity dimension of AI supply chain risk but does not, by itself, address training data poisoning (which would require dataset provenance tracking at a much finer level of granularity) or model evaluation security gaps.

Recommendations

For security and ML engineering teams evaluating AI supply chain controls:

1. Adopt model signing now — even without a full provenance framework, cryptographic signing of model artifacts creates a baseline integrity check that blocks the simplest substitution attacks
2. Inventory your model dependencies — identify all pre-trained models and adapters in your production systems and their upstream sources
3. Integrate verification into deployment pipelines — treat model artifact verification as a deployment gate, similar to container image scanning or code signing checks in software CI/CD
4. Track regulatory requirements — EU AI Act compliance documentation for high-risk systems likely requires provenance-level records; start capturing this data before it becomes an emergency audit requirement