Tool Use by AI Systems as Its Own Risk Layer

You must now document which AI agents in your systems may call which functions, log every call that occurs, and maintain independent verification of requests that touch sensitive data or high-trust workflows. This is not optional infrastructure; it is the control layer that separates reliable tool use from exploitable attack surface.

The prompt is an analysis of 16 August 2026 of tool-use benchmarks and function-calling security, which argues that AI function calls form both the reliability mechanism and the primary attack surface of agentic systems, and that evaluation frameworks now measure agentic behaviour rather than simple parameter accuracy. The Berkeley Function Calling Leaderboard version 4 tests multi-turn interactions, tool chaining and the ability to refuse unsuitable functions; a separate academic study demonstrates that models with function-call interfaces can be driven to unauthorised calls through specialised attack techniques even when standard prompt-injection mitigations are present. In our assessment, this means you can no longer treat tool use as a single undivided capability or assume that moderation inside the model is sufficient control; you must instead implement a verifiable chain of custody over which agents call which functions and under what conditions.

What has changed in how tool use is now evaluated?

Function calls have moved from a peripheral feature to the core of how AI systems execute work: calling search functions, reading calendars, sending emails, making bookings. The shift in evaluation frameworks reflects a shift in what matters. Earlier benchmarks measured whether a model could produce correctly structured JSON and call the right function with the right parameters. Current benchmarks now test broader agentic behaviour: whether a model can handle interactions across multiple turns, chain several tools in sequence, maintain memory across sessions and — critically — recognise when no tool is suitable and refuse to call one. Performance on simple single-call scenarios diverges sharply from performance on complex multi-turn and multi-tool chains. This means you cannot rely on a single headline score to tell you whether a model is reliable for your specific use case.

Which failure modes does tool use introduce?

• Unauthorised function calls — models driven to call functions outside their intended scope through specialised attack techniques, even when standard mitigations are present.
• Parameter injection and manipulation — adversarial input crafted to alter function arguments or route calls to unintended backends.
• Moderation bypass — the model's internal moderation layer deceived into approving risky or out-of-policy calls.
• Tool chaining errors — failures in multi-step sequences where one faulty call propagates across multiple systems or backends.
• Anomalous call patterns — calls that deviate from normal usage but are not caught by static rules or schema validation alone.
• Loss of audit trail — function calls executed without sufficient logging to reconstruct what happened, by whom and under which conditions.

What concrete controls must you be able to demonstrate?

1. Record which agent may call which function — document the identity of each non-human actor (agent or service) and the specific functions it is authorised to invoke.
2. Log every function call with full context — capture the agent identity, function name, parameters, timestamp, result and any error state in a tamper-evident format.
3. Implement independent validation for sensitive calls — route requests that touch confidential or high-trust data through a verification layer separate from the model's own moderation.
4. Monitor for anomalous call patterns — establish runtime detection of calls that deviate from baseline behaviour, including unusual parameter values or sequences.
5. Enforce fail-closed behaviour on verification failure — if a call cannot be validated or a privacy check fails, the request must not proceed to execution.
6. Maintain a current inventory of available functions — keep a versioned record of which functions exist, their parameters, their backends and their risk classification.

How does standardisation of tool protocols change the risk picture?

Standardisation through protocols such as the Model Context Protocol makes tool interfaces more uniform across different models and systems. This uniformity is operationally valuable: it reduces the work of integrating new models and makes tool definitions portable. But uniformity also means that a single faulty function call, once it propagates through a standardised interface, can affect multiple backends simultaneously. A compromised or misdirected call is no longer isolated to one system; it becomes a vector across your entire tool ecosystem. This is why the shift from evaluation of individual model capability to evaluation of agentic reliability matters in practice: you need to know not just whether a model can call functions, but whether it can be trusted to call them correctly and safely in a multi-tool, multi-backend environment.

What tooling can help, and what remains your responsibility?

Verification layers designed for confidential workflows can make the chain of custody visible and testable. A verification console can route tasks through selected models, make verification steps and sources visible for inspection, and support human review of anomalous calls. Privacy-focused architectures can replace sensitive values with synthetic equivalents before processing, ensuring that only anonymised content reaches the model. These tools support oversight; they do not eliminate the need for it. Hallucinations are not ruled out by verification infrastructure. The final judgement over whether a call should proceed, and the accountability for that decision, remains with the human professional who operates the system. Tooling can make that judgement better informed and more defensible; it cannot replace it.