AI Model Supply Chain Security: Risks and Controls

AI model supply chain security means verifying where every model, adapter, dataset, and package in your AI system came from, and proving it has not been tampered with before it runs. The short version: treat a downloaded model like executable code, not like a data file.

Most teams spend their security effort on prompts and guardrails. Meanwhile the model file itself, pulled from a public hub with one line of code, often gets more trust than any npm package would. This guide explains how the AI supply chain is attacked, what OWASP says about it, and the controls a cloud security engineer should put in place.

Why the AI supply chain is different

A traditional software supply chain has source code, dependencies, build systems, and container images. An AI system has all of that plus several new layers:

• Pre-trained base models downloaded from public hubs
• Fine-tuning adapters such as LoRA and PEFT adapters
• Training and fine-tuning datasets
• Model conversion and merge services
• Inference platforms and serving frameworks
• Third-party model APIs and their terms of use

OWASP lists this as LLM03:2025 Supply Chain in its Top 10 for LLM Applications. It notes that supply chains can be compromised at the level of training data, models, and deployment platforms, and that open-access models and fine-tuning methods like LoRA widen the attack surface. For the full list, see our guide to the OWASP LLM Top 10.

The core difficulty is that a model is hard to inspect. You cannot read a set of weights the way you read source code, so a backdoor or bias can hide in a file that looks perfectly normal.

The main attack paths

1. Malicious model files

Some model file formats can contain more than numbers. Python's pickle format, which has long been used to save machine learning models, can run code when a file is loaded. If you load an untrusted pickle-based model, you may be running an attacker's code on your machine or cloud workload.

Safer formats store only tensor data and no executable logic. Safetensors is the best-known example. Moving to a data-only format removes one entire class of risk, though it does not tell you whether the weights themselves are trustworthy.

2. Weak provenance and look-alike repositories

OWASP points out that published models often lack strong origin guarantees, and that model cards do not verify where a model came from. Attackers can compromise a repository account or publish a look-alike repository with a near-identical name. A typo in a model identifier is enough to pull the wrong artifact.

3. Poisoned or backdoored models

A model can be trained or tampered with so it behaves normally in testing but misbehaves on a specific trigger. Benchmarks do not reliably catch this, because OWASP notes that benchmark results can be gamed through fine-tuning.

4. Vulnerable LoRA adapters

A malicious adapter can compromise a base model when the two are merged. This matters most on platforms that let users download and apply adapters to a deployed model.

5. Compromised conversion and merge services

Shared services that convert or merge models can introduce vulnerabilities, and OWASP notes they may be used to bypass reviews. Treat any output from a shared conversion pipeline as untrusted until verified.

6. Vulnerable packages and outdated models

The libraries around the model matter as much as the model. Outdated frameworks, unmaintained models, and unpatched dependencies in your training or serving stack are the same old risks in a new place.

7. Licensing and data-use terms

Models and datasets carry different licenses, and some restrict commercial use. Some model operators also reserve the right to use your application data for training. That is a legal and a data-leak risk at the same time.

Controls that work

OWASP's mitigation list gives a solid foundation. Here is how it translates into engineering work.

Risk	Control	Where it lives
Untrusted model files	Use data-only formats, load only from approved sources	Model loading code, CI checks
Weak provenance	Pin exact versions, verify file hashes and signatures	Artifact registry, deployment pipeline
Backdoored models	AI red teaming of third-party models for your use case	Pre-production testing
Vulnerable dependencies	Scan and patch packages, including in dev environments	CI/CD, container scanning
Unknown components	Keep a signed SBOM and AI BOM inventory	Build pipeline
Rogue adapters	Allow-list adapters, review before merging	MLOps approval workflow
Shared conversion services	Audit and restrict who can convert and publish	Platform governance
Data-use terms	Review supplier terms and privacy policies	Procurement and legal review

Build an internal model registry

The single most effective step is to stop pulling models directly from public hubs into production. Instead:

1. Download the model into an isolated environment with no access to production data or credentials.
2. Scan the files and convert to a data-only format where possible.
3. Test the model, including with AI red teaming.
4. Record the hash, source, version, and license in your registry.
5. Promote the approved artifact to a private registry that production workloads read from.

This mirrors how mature teams handle container images, and the same cloud primitives apply: private storage, least-privilege IAM, and network egress controls.

Keep an AI BOM

A software bill of materials lists what is inside your application. An AI BOM extends that idea to models, datasets, and adapters. OWASP calls AI BOMs and ML SBOMs an emerging area and points to CycloneDX as a starting point. Even a simple, signed inventory lets you answer the question that matters during an incident: which of our systems use this component?

Lock down the runtime

Assume something will slip through and limit the damage:

• Run model loading and inference with minimal IAM permissions
• Block outbound network access from inference workloads unless required
• Keep secrets out of the environment that loads untrusted artifacts
• Log model loads and artifact changes for audit

These are the same fundamentals covered in our guide to AI platform security on Bedrock, Azure AI, and Vertex AI.

Governance: where NIST fits

Technical controls need a policy wrapper. The NIST AI Risk Management Framework gives a structure of Govern, Map, Measure, and Manage that you can use to document who approves third-party models, what testing is required, and how incidents are handled. NIST also publishes a Generative AI Profile that applies the framework to generative AI risks. Start with the NIST AI RMF overview and use the OWASP LLM03 Supply Chain entry as your technical checklist.

What this means for your career

Employers hiring cloud and AI security engineers want people who can secure the whole pipeline, not only the chat interface. Being able to design a model approval workflow, write the IAM policies behind a private registry, and explain provenance risk in an interview sets you apart. These are practical skills you can demonstrate with a portfolio project, such as building a model intake pipeline that scans, tests, hashes, and promotes artifacts.

Frequently asked questions

What is AI model supply chain security?

It is the practice of verifying the origin and integrity of every component in an AI system, including pre-trained models, adapters, datasets, and software packages, so that tampered or malicious components do not reach production.

Is it safe to download models from public hubs?

It can be, but only with controls. Verify the source, pin versions, check hashes, prefer data-only file formats, and scan and test the model in an isolated environment before approving it for production use.

Why are pickle files considered risky?

Python's pickle format can execute code during loading. If a model is saved in that format and comes from an untrusted source, loading it may run an attacker's code. Data-only formats such as safetensors avoid that specific risk.

What is an AI BOM?

An AI bill of materials is an inventory of the models, datasets, adapters, and software components in an AI system. It works like a software SBOM and helps you respond quickly when a component is found to be vulnerable or compromised.

How is this different from prompt injection?

Prompt injection attacks a running application through its inputs. Supply chain attacks compromise the components before the application runs. You need defenses against both.

Do I need AI skills to work in cloud security?

Increasingly, yes. Cloud security teams are being asked to secure AI workloads, so combining cloud fundamentals with AI security knowledge makes you much more employable.

Build these skills hands-on

PrimeSec Academy's 20-week program covers AWS, Azure, GCP, and AI/LLM security through labs, 36 projects, and a defended capstone. Review the full curriculum or enroll today to start building a portfolio that proves you can secure the cloud and AI stack.