Choose open-weight or closed AI models on control

You must choose between open-weight and closed AI models based on which control and audit chain you can genuinely operate and defend, not on the label or benchmark score alone. Open-weight models give you inspection rights but place provenance, safeguards, monitoring and incident response with your organisation. Closed models centralise those controls with the supplier, limiting your independent verification but reducing your direct management burden.

The prompt is an analysis of 10 November 2026 of the control and audit chain you can operate across open-weight versus closed AI model deployments, which argues that the choice between architectures should turn on demonstrable control of confidentiality, security and audit evidence rather than on capability rankings or openness labels. The Australian Cyber Security Centre distinguishes between closed models, where weights and safeguards remain with the supplier; open-weight models, where you host and inspect the weights yourself; and genuinely open-source deployments, which may also provide code, documentation and training information. In our assessment, this distinction matters because it reshapes where your organisation's security and compliance obligations actually sit. A CISO or director who selects an open-weight model takes on permanent responsibility for version management, monitoring and incident response; one who selects a closed model must enforce contractual evidence of the supplier's controls instead. Neither choice is safer in the abstract; both demand different forms of proof.

Which responsibilities does self-hosting place with your organisation?

Whoever deploys an open-weight model gains the right to inspect model weights and configuration, but assumes responsibility for the full deployment chain. This includes:

• Provenance and supply-chain security — establishing where the model weights came from and whether they have been altered or compromised.
• Protective measures and hardening — configuring the hosting environment, access controls, sandboxing and runtime enforcement.
• Monitoring and telemetry — detecting anomalous behaviour, drift, poisoning or unauthorised access in real time.
• Maintenance and version management — tracking which model version is in use, when it was updated and what changed.
• Incident response — investigating and remediating security events without waiting for supplier support.
• Audit and evidence retention — recording decisions, access logs, model configurations and test results for compliance and investigation.

These are not optional tasks. The National Institute of Standards and Technology's assessment of a leading open-weight model found it to be the most cyber-capable publicly released model at the time, yet it performed below current frontier models on four cyber benchmarks. That test measures one dimension only. A peer-reviewed benchmark of open-source and commercial models in financial scenarios found no single model dominant across all dimensions, and measured financial capability did not correlate reliably with security robustness. Capability and safety are separate assessment dimensions. You must test each model separately on task performance, security and confidentiality before deployment.

What concrete controls must you be able to demonstrate?

Before you commit to an open-weight model, establish that your organisation can operate and evidence these controls:

1. Document the model, its purpose and the lawful basis — record which model each workflow uses, why it was selected, what data it processes and the legal ground for that processing.
2. Verify provenance and establish a supply-chain audit trail — confirm the source of the model weights, check for tampering or poisoning, and maintain records of every version deployed.
3. Implement and test protective measures — configure access controls, sandboxing, input validation and output filtering; test these measures before the model handles sensitive data.
4. Establish continuous monitoring and alerting — detect drift, anomalous outputs, unauthorised access or signs of compromise; log all access and model decisions.
5. Define and rehearse incident response procedures — establish who investigates security events, how quickly you can isolate or roll back a model, and what evidence you preserve.
6. Retain audit evidence and demonstrate compliance — keep logs, configuration records, test results and decisions for the period required by your regulators and your own risk appetite.

If your organisation cannot staff and resource these tasks, an open-weight model is not the right choice, regardless of its capability or cost. Record that decision and the reason for it.

What must you enforce in a closed-model contract?

Where you select a closed model, you lose the right to inspect the weights but must gain contractual evidence of the supplier's controls. This means you require clauses on data retention, training use, subprocessors, model versions, hosting region, incident notification timelines and your right to audit. The contract must specify what you can inspect, when, and under what conditions. Without these clauses, you have no way to verify that the supplier's controls match your obligations. In our assessment, a closed model is not a shortcut to security; it is a shift of responsibility that demands stronger contractual evidence than most organisations currently secure.

How do you make the choice?

The decision procedure is straightforward. First, establish which controls your organisation can genuinely operate. If you can staff version management, monitoring and incident response for model files, and you can retain and defend the audit evidence, an open-weight model is viable. If you cannot, or if you lack the technical capacity to test and harden the deployment, a closed model may reduce your direct burden—but only if you enforce the contract. Second, test the model separately on task performance, security and confidentiality for your specific use case and data. Do not rely on published benchmarks; they measure specific conditions and do not predict production behaviour. Third, record the decision, the evidence that supports it, and the controls you will operate. This record is your defence if a regulator or auditor asks why you chose this architecture.

Automated tools can help you track model versions, log access and alert on anomalies, but they cannot replace your own professional judgement about which controls your organisation can truly operate. The choice between open-weight and closed is not a technical question; it is a governance question about where your accountability sits and what you can prove.