Function calling in LLMs

Function calling in LLMs is a capability that allows a large language model to request the execution of a predefined external function and incorporate the function's return value into its response, rather than relying solely on information encoded in the model's weights.

Without function calling, a language model can only produce text based on what it learned during training and whatever was placed in its context window. That constraint is significant in customer service, where agents need to look up live order data, query account balances, check reservation status, or trigger actions in downstream systems. Function calling closes that gap by letting the model behave as an orchestrator that selects the right tool for each step, rather than a text generator that merely describes what could be done.

How function calling works

At inference time, the application passes a list of available functions to the model alongside the user's message. Each function is described with a name, a plain-language purpose, and a typed parameter schema. The model reads the conversation, determines whether any function is relevant, and if so, outputs a structured call object specifying which function to invoke and what arguments to pass. The application executes the function, appends the result to the conversation, and calls the model again so it can incorporate the result into its final reply.

Common patterns include:

• Single-call tools: The model issues one function call, receives the result, and responds. Typical for lookups like order status or account tier.
• Sequential calls: The model issues a series of function calls in order, each depending on the output of the previous one. Used in multi-step workflows such as cancellation flows that first verify identity, then check eligibility, then trigger the cancellation.
• Parallel calls: The model issues multiple function calls simultaneously to reduce latency. Supported in several frontier model APIs when the calls are independent of each other.

The OpenAI function calling documentation defines the canonical schema format that most other providers have since adopted, making function definitions largely portable across model families.

Why function calling matters for customer experience

Function calling is what transforms a large language model (LLM) from a conversational interface into an action-capable agent. A support agent that can only describe a refund process is categorically less useful than one that can actually issue the refund. For teams deploying agentic AI, function calling is the mechanism that connects the model's reasoning to the operational systems where customer outcomes are produced.

The limitation is reliability. Models occasionally call functions with incorrect arguments, call the wrong function for a given context, or fail to call a function when one is clearly warranted. These failure modes become critical in customer service because a malformed function call can corrupt backend state. Teams typically add input validation at the function layer and route sensitive operations through a human-in-the-loop (HITL) review step until they have built sufficient confidence in the model's calling accuracy. AI observability tooling is essential for catching argument errors in production before they affect customers at scale.

Function calling and agent orchestration

In systems with more than one tool or agent, function calling interacts directly with AI agent orchestration. The orchestration layer is responsible for defining which functions are available to each agent, enforcing authorization rules so that agents cannot call functions outside their scope, and aggregating results across multiple agents when a workflow spans several tools. Designing the function schema well is a prerequisite for reliable orchestration: ambiguous function names or overlapping parameter definitions produce the same call-selection errors at the orchestrator level that they produce in single-agent setups.

For a deeper dive, download Decagon's guide to agentic AI for customer experience.