Security of the AI Supply Chain: Models, Weights, Training Data

Software supply-chain security is now a mature discipline — firms know to worry about the libraries and dependencies they build on. The AI supply chain is the same problem, one layer over, and most organisations have not extended their thinking to it. When your product depends on a third-party model, a set of pretrained weights, or a training dataset you did not curate, you have inherited a supply chain whose integrity and provenance you probably cannot vouch for — and that is a security and compliance exposure that a software SBOM does not cover.

The new links in the chain

The AI supply chain adds components that behave differently from ordinary software dependencies.

• Models and weights. A pretrained model is a binary artefact you did not build and usually cannot fully inspect. Where did it come from, was it tampered with, does it contain a backdoor or a poisoned behaviour triggered by specific inputs? These are real questions, and “we downloaded it from a popular repository” is not reassurance.
• Training data. The data a model was trained on shapes its behaviour, its biases and its legal status. Data you cannot account for is a provenance and rights problem that transfers into your product, and poisoned training data is an attack vector that plants behaviour before you ever deploy.
• Fine-tuning and adapters. Layers you or others add on top of a base model are another link, each with its own provenance question.

Why this is not just the software supply chain

The instinct is to treat models as another dependency and apply existing controls. That helps, but it misses what is distinctive. A poisoned library usually fails visibly; a poisoned model may work perfectly until a specific trigger, making the compromise far harder to detect. A model’s “source” is not just code but data and training process, which are rarely transparent. And the legal dimension — rights to the training data — has no clean equivalent in software dependencies. So the AI supply chain needs its own assessment, not just an extension of the software one.

What to actually do

• Know your model provenance. Where each model and weight set came from, who produced it, and what assurance you have about its integrity — the AI equivalent of vetting a dependency.
• Prefer sources you can trust and verify. Integrity checks, signatures where available, and reputable provenance matter more here because tampering is harder to spot after the fact.
• Assess training-data provenance where you can. For models you fine-tune or train, the data’s rights and integrity are your responsibility; for models you consume, understand what you can and cannot know.
• Build an AI bill of materials. Extend the SBOM idea to models, weights, datasets and adapters, so you can answer what your AI systems are actually built from when a supply-chain issue emerges.

The AI supply chain is where a lot of AI risk quietly lives, because it is inherited rather than built and therefore easy to overlook. Firms that extend their supply-chain security to models, weights and data — treating provenance and integrity as first-class questions — close an exposure that their software controls, however good, were never designed to cover.

Who this is for

This reading is for:

• CISOs and CTOs whose products depend on third-party models
• Security teams whose supply-chain thinking stops at software
• Compliance leads assessing AI provenance and integrity
• Firms fine-tuning or deploying open-weight models

Sixteen Pillars extends your supply-chain security to models, weights and data, treating provenance and integrity as first-class questions your software controls never covered. Pricing is published at /pricing/. If this is live for your organisation and you would like an independent reading, the place to start is a conversation.

Sixteen Pillars is a technology governance consultancy based in Cyprus. Engagements run remote across the EU, UK, and Middle East, with on-site time where the engagement requires it.