Tool Calling

When a Model Can Choose a Capability

Tool calling can look like a small feature in a model API: present some tools, receive a structured request, run an operation, and return the result.

The shape is familiar. The control problem is different.

An ordinary application reaches a capability because deterministic code, configuration, or a user flow selected that path. A tool-using application can ask a model to interpret a goal, choose among described capabilities, and propose the arguments.

The model does not directly execute the operation. Depending on the system, the application, a provider, or a protocol host may run it. In every case, an execution boundary still has to validate, authorize, execute, and observe the operation.

That makes tool calling a useful boundary concept. The wire shape resembles an API call, but the caller behaves more like a probabilistic participant in control flow.

This is Post 04 in The Abstraction Shift: How Software Keeps Moving Up.

Why This Exists

Language models are useful for interpreting open-ended requests. Enterprise systems are designed to enforce precise operations. Tool calling connects those strengths.

A user does not naturally ask for get_order_status(order_id="A-1842"). They ask, “Where is my order?” A model can map the request to a capability and infer that the order identifier is needed. The application can then execute a known operation against an order system and return the result as context for the next model turn.

Without tools, a model can describe what an application might do, but it cannot reliably access current private data or cause a business operation to happen. Without a model in the loop, the path from a goal to a capability has to be selected by code, configuration, or user interaction.

Tool calling occupies the space between those extremes. It lets software expose a controlled capability surface while allowing the model to participate in the choice of the next operation.

The capability surface can include read operations, such as retrieving an order or searching a knowledge base. It can also include consequential operations, such as creating a ticket, issuing a refund, changing an address, or deploying a service. The wire shape may be similar in both cases. The risk and required controls are not.

What We Did Before

The lineage is familiar.

Procedure and function calls

Within one process, a caller invokes a function with a name and arguments. The compiler, type system, and program structure constrain which calls are possible. The developer chooses the path.

RPC and service APIs

Remote procedure call systems make a function-like operation available across a process or network boundary. A client sends a structured request to a service and receives a result or error. Serialization, transport, timeouts, retries, and versioning make the remote boundary explicit.

An API client still normally knows which operation to call because that decision is encoded in application logic, configuration, or user interface flow.

Plugin systems and extension points

Plugin architectures let a host discover or load capabilities supplied by other components. The host may use metadata, registration, or conventions to decide which extension to invoke.

This moved capability selection away from a single compiled program, but the selection logic was still usually deterministic. A plugin registry did not normally infer the user’s goal and construct a valid argument object from a description.

Workflow activities

Workflow engines expose tasks or activities that can be composed into a longer process. The engine can retry, compensate, schedule, and persist state across steps. The workflow definition remains the primary source of control flow.

Tool calling can sit inside a workflow, but if the model proposes the next activity, the workflow no longer fully determines the path. That is the point where the comparison becomes incomplete.

Tool calling keeps the contract and execution boundary, but makes the caller’s choice and argument construction probabilistic.

What’s Actually New?

Tool calling is not a new way to execute code. Its newness comes from combining familiar parts around a different caller.

1. Capability descriptions become executable context

The tool description is written for a model as well as for a developer. Its name, description, input schema, output behavior, examples, side effects, and limitations influence whether the model selects it.

That makes documentation operational. A vague description can produce a wrong selection even when the underlying implementation is perfect. A misleading description can make a dangerous tool appear safe.

2. The model participates in dispatch

In conventional dispatch, the caller or a deterministic router identifies the operation. With tool calling, the model can select the operation from a set of candidates. The dispatch problem becomes partly semantic.

The runtime should still reject unknown or unavailable tools. It should also treat selection quality as something to evaluate, not assume.

3. The interface has multiple audiences

An API contract serves code. A tool contract may be consumed by code, a model, and a human-facing host or operator. The schema constrains the request, while the natural-language description helps the model decide when the capability applies and may help a person understand an approval prompt.

Those parts have different failure modes. A schema can be valid while its description encourages the wrong use. A clear description can still produce arguments that fail a business rule.

4. Errors can become recovery context

In a conventional client, an error often returns to code that has a known recovery path. In a model-mediated loop, the error may be represented as a tool result that the model can use to correct an argument, ask a question, choose another tool, or stop.

That flexibility can improve recovery. It can also create loops, expose details that should remain internal, or teach the model to retry an operation that should not be retried.

5. Capability exposure becomes a context-budget decision

Every tool definition consumes attention and tokens. Exposing hundreds of low-quality tools can make selection harder and increase latency. Capability discovery, grouping, filtering, and deferred loading become part of interface design.

The question is no longer only, “Which APIs exist?” It is also, “Which capabilities should be visible in this context, for this task, to this caller?“

6. The execution model can vary

Current tool systems may defer discovery, filter tools per request, execute some tools on the provider side, issue parallel calls, or represent long-running work with a task handle. These are variations in discovery and execution, not a change to the authority boundary.

Someone still has to decide which operation is exposed, validate the request, enforce policy, handle side effects, and report what happened.

Where the Analogy Breaks

The RPC analogy is useful because it highlights a request, a contract, a remote operation, and a result. It becomes dangerous when it suggests that the caller has the same guarantees as ordinary code.

A valid request is not necessarily a correct request

Schema validation answers whether the arguments have an allowed shape. It does not answer whether the model selected the right order, interpreted the user’s intent correctly, or used a current value.

A description is not authorization

Tool metadata may explain an operation. It must not be the source of permission. The runtime needs an identity and policy decision that holds even when the model, prompt, or tool description is wrong.

A result is not automatically truth

The result may be stale, partial, malformed, or produced by an external system with its own uncertainty. The runtime should preserve source, timestamp, status, and error information where those details affect the next decision.

Retries are not harmless by default

Retrying a read may be reasonable. Retrying a refund, email, deployment, or record update may duplicate a side effect. Idempotency keys, operation status, and explicit compensation behavior matter.

Tool output is untrusted input

A tool can return text, markup, documents, or records that contain instructions. If that output is placed directly into model context, it can influence later decisions. A tool result needs an authority boundary just as retrieved web content does.

The caller can be confidently wrong

The model may select the most plausible tool rather than the correct one. It may fill an optional argument with an assumption. It may continue after an error because the response looks like a recoverable problem. Confidence in the generated text is not evidence that the operation is correct.

Under the Hood

A production tool call is more than a function name and a JSON object.

The model proposes an operation, but the runtime remains responsible for the boundary around its effects.

1. Define the capability contract

A useful contract includes at least:

• a stable name;
• a human-readable description of what the capability does and does not do;
• input schema and required fields;
• output schema or a documented result shape;
• side-effect classification;
• authorization requirements;
• idempotency and retry behavior;
• error categories that a caller can act on;
• latency, cost, and rate-limit expectations;
• confirmation and approval semantics where relevant;
• a completion model for asynchronous or long-running work;
• source and freshness information where the result represents external state.

JSON Schema is useful for expressing data shape. It is not a complete business contract. “amount is a number” does not mean “this caller may refund this amount for this order.”

2. Present the right capabilities

The host presents or exposes a set of available tools. That set may be static, filtered by task, selected by user role, discovered from another service, or loaded only when a category becomes relevant.

The smaller and clearer the set, the easier it is to evaluate selection. A broad registry can be powerful, but it makes descriptions, naming, collisions, and context budgets architectural concerns.

3. Receive a proposal

The model returns one or more structured proposals such as:

{
  "name": "get_order_status",
  "arguments": {
    "order_id": "A-1842"
  }
}

The format is structured, but the decision that produced it is still probabilistic. The execution host should treat it as an untrusted request from an intelligent but fallible caller.

4. Validate before execution

Validation should happen in layers:

1. Is the tool registered, enabled, and visible in this context?
2. Which user, workload, or delegated identity is the call made under, and does it have access?
3. Does the proposal satisfy the input schema?
4. Does it satisfy domain validation and current business policy?
5. Does the operation fit the current budget, rate limit, and side-effect rules?
6. Does it require a human approval step?

Only after those checks should the execution host invoke the capability.

5. Return an observable result

The result that re-enters model context should distinguish success, failure, and uncertainty. The execution host should preserve enough metadata for a later answer to avoid claiming more than the capability established.

For a read operation, that may mean source system, record identifier, retrieval time, and freshness. For a write operation, it may mean accepted, completed, rejected, or pending, plus an operation identifier that can be checked later.

6. Decide whether the loop continues

The runtime or model can decide that the interaction is complete, but the execution host should enforce hard bounds. Maximum tool turns, wall-clock time, token budget, cost budget, and repeated-call detection prevent a model from converting a recoverable ambiguity into an unbounded loop.

Tool calling sits between model context and governed capability, with evaluation, approval, budgets, tracing, and security around the path.

A Concrete Example

Consider a support assistant handling an order question.

The user says, “My order A-1842 is late. Can you refund the shipping charge?”

The assistant may need two capabilities:

• get_order_status, a read operation that returns shipment events and the current delivery state;
• request_shipping_refund, a consequential operation with eligibility rules and a possible approval requirement.

A safe sequence could be:

1. The model selects get_order_status and proposes A-1842.
2. The runtime validates that the support identity may inspect the order.
3. The order system returns the latest status, shipment history, source, and timestamp.
4. The model interprets the result and may propose request_shipping_refund with the order identifier and reason.
5. The capability or policy service evaluates current eligibility; the model does not establish it.
6. The execution host checks the support identity, order ownership, eligibility, refund limit, idempotency key, and approval rule.
7. The capability returns a confirmed refund, a pending review, or a rejection with a reason.
8. The model explains the result without claiming a refund was completed if the operation is only pending.

The model improves the interaction because it can connect an open-ended request to relevant capabilities. The execution boundary remains necessary because the model should not be trusted to enforce authorization, policy, eligibility, or side-effect safety.

The same pattern applies outside customer support. An IT assistant may inspect an incident before proposing a restart. A procurement assistant may gather invoice evidence before proposing approval. A deployment assistant may inspect a change and request execution. The names change, but the boundary does not: model proposal, independent validation, governed execution, observable outcome.

Failure Modes

Wrong selection or ambiguous arguments

Overlapping descriptions and incomplete user references lead to a plausible but wrong operation. Improve filtering and require clarification when the cost of error is high.

Schema success, business failure, or privilege confusion

A valid object can violate current state or policy, and a visible tool is not permission. Validate business rules and authorization inside the capability boundary.

Duplicate side effects

A timeout makes completion uncertain and the model retries. Use idempotency keys, status lookup, and explicit pending or unknown outcomes.

Prompt injection through tool output

Returned content can instruct the model to ignore its policy or take another action. Treat output as untrusted data and keep authority outside context.

Unbounded loops or stale contracts

The model keeps calling tools, or descriptions and schemas drift apart. Enforce budgets, stopping rules, versioned contracts, and release evaluations.

Sources

• Function calling, OpenAI API: a primary reference for defining and invoking model-proposed function calls.
• How tool use works, Anthropic Claude Platform Docs: a current reference for the tool-use request and result cycle.
• Tools, Model Context Protocol specification: a current reference for describing, discovering, and invoking tools through a protocol.
• Authorization, Model Context Protocol specification: a current reference for authorization semantics around protocol-hosted tools.
• Understanding JSON Schema: a primary reference for the schema constraints used to shape tool arguments.
• AIP-155: Request identification: a reference for idempotency keys and safe request retries.