Tool calling

Tool calling , also known as function calling , is a structured way to give LLMs the ability to make requests back to the application that called it. You define the tools you want to make available to the model, and the model will make tool requests to your app as necessary to fulfill the prompts you give it.

The use cases of tool calling generally fall into a few themes:

Giving an LLM access to information it wasn’t trained with

• Frequently changing information, such as a stock price or the current weather.
• Information specific to your app domain, such as product information or user profiles.

Note the overlap with retrieval augmented generation (RAG), which is also a way to let an LLM integrate factual information into its generations. RAG is a heavier solution that is most suited when you have a large amount of information or the information that’s most relevant to a prompt is ambiguous. On the other hand, if retrieving the information the LLM needs is a simple function call or database lookup, tool calling is more appropriate.

Introducing a degree of determinism into an LLM workflow

• Performing calculations that the LLM cannot reliably complete itself.
• Forcing an LLM to generate verbatim text under certain circumstances, such as when responding to a question about an app’s terms of service.

Performing an action when initiated by an LLM

• Turning on and off lights in an LLM-powered home assistant
• Reserving table reservations in an LLM-powered restaurant agent

If you want to run the code examples on this page, first complete the steps in the Getting started guide. All of the examples assume that you have already set up a project with Genkit dependencies installed.

This page discusses one of the advanced features of Genkit model abstraction, so before you dive too deeply, you should be familiar with the content on the Generating content with AI models page. You should also be familiar with Genkit’s system for defining input and output schemas, which is discussed on the Flows page.

At a high level, this is what a typical tool-calling interaction with an LLM looks like:

1. The calling application prompts the LLM with a request and also includes in the prompt a list of tools the LLM can use to generate a response.
2. The LLM either generates a complete response or generates a tool call request in a specific format.
3. If the caller receives a complete response, the request is fulfilled and the interaction ends; but if the caller receives a tool call, it performs whatever logic is appropriate and sends a new request to the LLM containing the original prompt or some variation of it as well as the result of the tool call.
4. The LLM handles the new prompt as in Step 2.

For this to work, several requirements must be met:

• The model must be trained to make tool requests when it’s needed to complete a prompt. Most of the larger models provided through web APIs, such as Gemini and Claude, can do this, but smaller and more specialized models often cannot. Genkit will throw an error if you try to provide tools to a model that doesn’t support it.
• The calling application must provide tool definitions to the model in the format it expects.
• The calling application must prompt the model to generate tool calling requests in the format the application expects.

Genkit provides a single interface for tool calling with models that support it. Each model plugin ensures that the last two of the above criteria are met, and the Genkit instance’s generate() function automatically carries out the tool calling loop described earlier.

Tool calling support depends on the model, the model API, and the Genkit plugin. Consult the relevant documentation to determine if tool calling is likely to be supported. In addition:

• Genkit will throw an error if you try to provide tools to a model that doesn’t support it.
• If the plugin exports model references, the info.supports.tools property will indicate if it supports tool calling.

Use the Genkit instance’s defineTool() function to write tool definitions:

  
 
 import 
 { genkit, z } 
 from 
 'genkit' 
 ; 
 
 
 
 
 import 
 { googleAI } 
 from 
 '@genkitai/google-ai' 
 ; 
 
 
 
 
 
 
 
 
 const 
 ai 
 = 
 genkit 
 ({ 
 
 
 
 
 plugins: [ 
 googleAI 
 ()], 
 
 
 
 
 model: googleAI. 
 model 
 ( 
 'gemini-2.5-flash' 
 ), 
 
 
 
 
 }); 
 
 
 
 
 
 
 
 
 const 
 getWeather 
 = 
 ai. 
 defineTool 
 ( 
 
 
 
 
 { 
 
 
 
 
 name: 
 'getWeather' 
 , 
 
 
 
 
 description: 
 'Gets the current weather in a given location' 
 , 
 
 
 
 
 inputSchema: z. 
 object 
 ({ 
 
 
 
 
 location: z. 
 string 
 (). 
 describe 
 ( 
 'The location to get the current weather for' 
 ), 
 
 
 
 
 }), 
 
 
 
 
 outputSchema: z. 
 string 
 (), 
 
 
 
 
 }, 
 
 
 
 
  
 async 
 ( 
 input 
 ) 
 => 
 { 
 
 
 
 
  
 // Here, we would typically make an API call or database query. For this 
 
 
 
 
  
 // example, we just return a fixed value. 
 
 
 
 
  
 return 
 `The current weather in ${ 
 input 
 . 
 location 
 } is 63°F and sunny.` 
 ; 
 
 
 
 
 }, 
 
 
 
 
 ); 
 
 
 

The syntax here looks just like the defineFlow() syntax; however, name , description , and inputSchema parameters are required. When writing a tool definition, take special care with the wording and descriptiveness of these parameters. They are vital for the LLM to make effective use of the available tools.

Include defined tools in your prompts to generate content.

Using generate() :

  
 
 const 
 response 
 = 
 await 
 ai. 
 generate 
 ({ 
 
 
 
 
 prompt: 
 'What is the weather in Baltimore?' 
 , 
 
 
 
 
 tools: [getWeather], 
 
 
 
 
 }); 
 
 
 

Using definePrompt() :

  
 
 const 
 weatherPrompt 
 = 
 ai. 
 definePrompt 
 ( 
 
 
 
 
 { 
 
 
 
 
 name: 
 'weatherPrompt' 
 , 
 
 
 
 
 tools: [getWeather], 
 
 
 
 
 }, 
 
 
 
 
  
 'What is the weather in {{location}}?' 
 , 
 
 
 
 
 ); 
 
 
 
 
 
 
 
 
 const 
 response 
 = 
 await 
 weatherPrompt 
 ({ location: 
 'Baltimore' 
 }); 
 
 
 

Using Prompt files:

  
 
 --- 
 
 
 
 
 tools 
 : [ 
 getWeather 
 ] 
 
 
 
 
 input 
 : 
 
 
 
 
  
 schema 
 : 
 
 
 
 
  
 location 
 : 
 string 
 
 
 
 
 --- 
 
 
 
 
 
 
 
 
 What is the weather in 
 {{ 
 location 
 }} 
 ? 
 
 
 

Then you can execute the prompt in your code as follows:

  
 
 // assuming prompt file is named weatherPrompt.prompt 
 
 
 
 
 const 
 weatherPrompt 
 = 
 ai. 
 prompt 
 ( 
 'weatherPrompt' 
 ); 
 
 
 
 
 
 
 
 
 const 
 response 
 = 
 await 
 weatherPrompt 
 ({ location: 
 'Baltimore' 
 }); 
 
 
 

Using Chat:

  
 
 const 
 chat 
 = 
 ai. 
 chat 
 ({ 
 
 
 
 
 system: 
 'Answer questions using the tools you have.' 
 , 
 
 
 
 
 tools: [getWeather], 
 
 
 
 
 }); 
 
 
 
 
 
 
 
 
 const 
 response 
 = 
 await 
 chat. 
 send 
 ( 
 'What is the weather in Baltimore?' 
 ); 
 
 
 
 
 
 
 
 
 // Or, specify tools that are message-specific 
 
 
 
 
 const 
 response 
 = 
 await 
 chat. 
 send 
 ({ 
 
 
 
 
 prompt: 
 'What is the weather in Baltimore?' 
 , 
 
 
 
 
 tools: [getWeather], 
 
 
 
 
 }); 
 
 
 

When combining tool calling with streaming responses, you will receive toolRequest and toolResponse content parts in the chunks of the stream. For example, the following code:

  
 
 const 
 { 
 stream 
 } 
 = 
 ai. 
 generateStream 
 ({ 
 
 
 
 
 prompt: 
 'What is the weather in Baltimore?' 
 , 
 
 
 
 
 tools: [getWeather], 
 
 
 
 
 }); 
 
 
 
 
 
 
 
 
 for 
 await 
 ( 
 const 
 chunk 
 of 
 stream) { 
 
 
 
 
 console. 
 log 
 (chunk); 
 
 
 
 
 } 
 
 
 

Might produce a sequence of chunks similar to:

  
 
 { 
 index 
 : 
 0 
 , 
 role 
 : 
 "model" 
 , 
 content 
 : [{text: 
 "Okay, I'll check the weather" 
 }]} 
 
 
 
 
 { 
 index 
 : 
 0 
 , 
 role 
 : 
 "model" 
 , 
 content 
 : [{text: 
 "for Baltimore." 
 }]} 
 
 
 
 
 // toolRequests will be emitted as a single chunk by most models 
 
 
 
 
 { 
 index 
 : 
 0 
 , 
 role 
 : 
 "model" 
 , 
 content 
 : [{toolRequest: {name: 
 "getWeather" 
 , input: {location: 
 "Baltimore" 
 }}}]} 
 
 
 
 
 // when streaming multiple messages, Genkit increments the index and indicates the new role 
 
 
 
 
 { 
 index 
 : 
 1 
 , 
 role 
 : 
 "tool" 
 , 
 content 
 : [{toolResponse: {name: 
 "getWeather" 
 , output: 
 "Temperature: 68 degrees 
 \n 
 Status: Cloudy." 
 }}]} 
 
 
 
 
 { 
 index 
 : 
 2 
 , 
 role 
 : 
 "model" 
 , 
 content 
 : [{text: 
 "The weather in Baltimore is 68 degrees and cloudy." 
 }]} 
 
 
 

You can use these chunks to dynamically construct the full generated message sequence.

When working with tools that might trigger multiple sequential calls, you can control resource usage and prevent runaway execution using the maxTurns parameter. This sets a hard limit on how many back-and-forth interactions the model can have with your tools in a single generation cycle.

Why use maxTurns?

• Cost Control: Prevents unexpected API usage charges from excessive tool calls
• Performance: Ensures responses complete within reasonable timeframes
• Safety: Guards against infinite loops in complex tool interactions
• Predictability: Makes your application behavior more deterministic

The default value is 5 turns, which works well for most scenarios. Each “turn” represents one complete cycle where the model can make tool calls and receive responses.

Example: Web Research Agent

Consider a research agent that might need to search multiple times to find comprehensive information:

  
 
 const 
 webSearch 
 = 
 ai. 
 defineTool 
 ( 
 
 
 
 
 { 
 
 
 
 
 name: 
 'webSearch' 
 , 
 
 
 
 
 description: 
 'Search the web for current information' 
 , 
 
 
 
 
 inputSchema: z. 
 object 
 ({ 
 
 
 
 
 query: z. 
 string 
 (). 
 describe 
 ( 
 'Search query' 
 ), 
 
 
 
 
 }), 
 
 
 
 
 outputSchema: z. 
 string 
 (), 
 
 
 
 
 }, 
 
 
 
 
  
 async 
 ( 
 input 
 ) 
 => 
 { 
 
 
 
 
  
 // Simulate web search API call 
 
 
 
 
  
 return 
 `Search results for "${ 
 input 
 . 
 query 
 }": [relevant information here]` 
 ; 
 
 
 
 
 }, 
 
 
 
 
 ); 
 
 
 
 
 
 
 
 
 const 
 response 
 = 
 await 
 ai. 
 generate 
 ({ 
 
 
 
 
 prompt: 
 
 
 
 
  
 'Research the latest developments in quantum computing, including recent breakthroughs, key companies, and future applications.' 
 , 
 
 
 
 
 tools: [webSearch], 
 
 
 
 
 maxTurns: 
 8 
 , 
 // Allow up to 8 research iterations 
 
 
 
 
 }); 
 
 
 

Example: Financial Calculator

  
 
 const 
 calculator 
 = 
 ai. 
 defineTool 
 ( 
 
 
 
 
 { 
 
 
 
 
 name: 
 'calculator' 
 , 
 
 
 
 
 description: 
 'Perform mathematical calculations' 
 , 
 
 
 
 
 inputSchema: z. 
 object 
 ({ 
 
 
 
 
 expression: z. 
 string 
 (). 
 describe 
 ( 
 'Mathematical expression to evaluate' 
 ), 
 
 
 
 
 }), 
 
 
 
 
 outputSchema: z. 
 number 
 (), 
 
 
 
 
 }, 
 
 
 
 
  
 async 
 ( 
 input 
 ) 
 => 
 { 
 
 
 
 
  
 // Safe evaluation of mathematical expressions 
 
 
 
 
  
 return 
 eval 
 (input.expression); 
 // In production, use a safe math parser 
 
 
 
 
 }, 
 
 
 
 
 ); 
 
 
 
 
 
 
 
 
 const 
 response 
 = 
 await 
 ai. 
 generate 
 ({ 
 
 
 
 
 prompt: 
 
 
 
 
  
 'Calculate the total value of my portfolio: 100 shares of AAPL, 50 shares of GOOGL, and 200 shares of MSFT. Also calculate what percentage each holding represents.' 
 , 
 
 
 
 
 tools: [calculator, stockAnalyzer], 
 
 
 
 
 maxTurns: 
 12 
 , 
 // Multiple stock lookups + calculations needed 
 
 
 
 
 }); 
 
 
 

What happens when maxTurns is reached?

When the limit is hit, Genkit stops the tool-calling loop and returns the model’s current response, even if it was in the middle of using tools. The model will typically provide a partial answer or explain that it couldn’t complete all the requested operations.

As most things in Genkit tools need to be predefined during your app’s initialization. This is necessary so that you would be able interact with your tools from the Genkit Dev UI. This is typically the recommended way. However there are scenarios when the tool must be defined dynamically per user request.

You can dynamically define tools using ai.dynamicTool function. It is very similar to ai.defineTool method, however dynamic tools are not tracked by Genkit runtime, so cannot be interacted with from Genkit Dev UI and must be passed to the ai.generate call by reference (for regular tools you can also use a string tool name).

  
 
 import 
 { genkit, z } 
 from 
 'genkit' 
 ; 
 
 
 
 
 import 
 { googleAI } 
 from 
 '@genkit-ai/google-genai' 
 ; 
 
 
 
 
 
 
 
 
 const 
 ai 
 = 
 genkit 
 ({ 
 
 
 
 
 plugins: [ 
 googleAI 
 ()], 
 
 
 
 
 model: googleAI. 
 model 
 ( 
 'gemini-2.5-flash' 
 ), 
 
 
 
 
 }); 
 
 
 
 
 
 
 
 
 ai. 
 defineFlow 
 ( 
 'weatherFlow' 
 , 
 async 
 () 
 => 
 { 
 
 
 
 
  
 const 
 getWeather 
 = 
 ai. 
 dynamicTool 
 ( 
 
 
 
 
 { 
 
 
 
 
 name: 
 'getWeather' 
 , 
 
 
 
 
 description: 
 'Gets the current weather in a given location' 
 , 
 
 
 
 
 inputSchema: z. 
 object 
 ({ 
 
 
 
 
 location: z. 
 string 
 (). 
 describe 
 ( 
 'The location to get the current weather for' 
 ), 
 
 
 
 
 }), 
 
 
 
 
 outputSchema: z. 
 string 
 (), 
 
 
 
 
 }, 
 
 
 
 
  
 async 
 ( 
 input 
 ) 
 => 
 { 
 
 
 
 
  
 return 
 `The current weather in ${ 
 input 
 . 
 location 
 } is 63°F and sunny.` 
 ; 
 
 
 
 
 }, 
 
 
 
 
 ); 
 
 
 
 
 
 
 
 
  
 const 
 { 
 text 
 } 
 = 
 await 
 ai. 
 generate 
 ({ 
 
 
 
 
 prompt: 
 'What is the weather in Baltimore?' 
 , 
 
 
 
 
 tools: [getWeather], 
 
 
 
 
 }); 
 
 
 
 
 
 
 
 
  
 return 
 text; 
 
 
 
 
 }); 
 
 
 

When defining dynamic tools, to specify input and output schemas you can either use Zod as shown in the previous example, or you can pass in manually constructed JSON Schema.

  
 
 const 
 getWeather 
 = 
 ai. 
 dynamicTool 
 ( 
 
 
 
 
 { 
 
 
 
 
 name: 
 'getWeather' 
 , 
 
 
 
 
 description: 
 'Gets the current weather in a given location' 
 , 
 
 
 
 
 inputJsonSchema: myInputJsonSchema, 
 
 
 
 
 outputJsonSchema: myOutputJsonSchema, 
 
 
 
 
 }, 
 
 
 
 
  
 async 
 ( 
 input 
 ) 
 => 
 { 
 
 
 
 
  
 /* ... */ 
 
 
 
 
 }, 
 
 
 
 
 ); 
 
 
 

Dynamic tools don’t require the implementation function. If you don’t pass in the function the tool will behave like an interrupt and you can do manual tool call handling:

  
 
 const 
 getWeather 
 = 
 ai. 
 dynamicTool 
 ({ 
 
 
 
 
 name: 
 'getWeather' 
 , 
 
 
 
 
 description: 
 'Gets the current weather in a given location' 
 , 
 
 
 
 
 inputJsonSchema: myInputJsonSchema, 
 
 
 
 
 outputJsonSchema: myOutputJsonSchema, 
 
 
 
 
 }); 
 
 
 

By default, Genkit repeatedly calls the LLM until every tool call has been resolved. You can conditionally pause execution in situations where you want to, for example:

• Ask the user a question or display UI.
• Confirm a potentially risky action with the user.
• Request out-of-band approval for an action.

Interruptsare special tools that can halt the loop and return control to your code so that you can handle more advanced scenarios. Visit the interrupts guide to learn how to use them.

If you want full control over this tool-calling loop, for example to apply more complicated logic, set the returnToolRequests parameter to true . Now it’s your responsibility to ensure all of the tool requests are fulfilled:

  
 
 const 
 getWeather 
 = 
 ai. 
 defineTool 
 ( 
 
 
 
 
 { 
 
 
 
 
  
 // ... tool definition ... 
 
 
 
 
 }, 
 
 
 
 
  
 async 
 ({ 
 location 
 }) 
 => 
 { 
 
 
 
 
  
 // ... tool implementation ... 
 
 
 
 
 }, 
 
 
 
 
 ); 
 
 
 
 
 
 
 
 
 const 
 generateOptions 
 : 
 GenerateOptions 
 = 
 { 
 
 
 
 
 prompt: 
 "What's the weather like in Baltimore?" 
 , 
 
 
 
 
 tools: [getWeather], 
 
 
 
 
 returnToolRequests: 
 true 
 , 
 
 
 
 
 }; 
 
 
 
 
 
 
 
 
 let 
 llmResponse; 
 
 
 
 
 while 
 ( 
 true 
 ) { 
 
 
 
 
 llmResponse 
 = 
 await 
 ai. 
 generate 
 (generateOptions); 
 
 
 
 
  
 const 
 toolRequests 
 = 
 llmResponse.toolRequests; 
 
 
 
 
  
 if 
 (toolRequests. 
 length 
 < 
 1 
 ) { 
 
 
 
 
  
 break 
 ; 
 
 
 
 
 } 
 
 
 
 
  
 const 
 toolResponses 
 : 
 ToolResponsePart 
 [] 
 = 
 await 
 Promise 
 . 
 all 
 ( 
 
 
 
 
 toolRequests. 
 map 
 ( 
 async 
 ( 
 part 
 ) 
 => 
 { 
 
 
 
 
  
 switch 
 (part.toolRequest.name) { 
 
 
 
 
  
 case 
 'getWeather' 
 : 
 
 
 
 
  
 return 
 { 
 
 
 
 
 toolResponse: { 
 
 
 
 
 name: part.toolRequest.name, 
 
 
 
 
 ref: part.toolRequest.ref, 
 
 
 
 
 output: 
 await 
 getWeather 
 (part.toolRequest.input), 
 
 
 
 
 }, 
 
 
 
 
 }; 
 
 
 
 
  
 default 
 : 
 
 
 
 
  
 throw 
 Error 
 ( 
 'Tool not found' 
 ); 
 
 
 
 
 } 
 
 
 
 
 }), 
 
 
 
 
 ); 
 
 
 
 
 generateOptions.messages 
 = 
 llmResponse.messages; 
 
 
 
 
 generateOptions.prompt 
 = 
 toolResponses; 
 
 
 
 
 } 
 
 
 

The Model Context Protocol (MCP) provides a powerful way to extend your tool-calling capabilities by connecting to external MCP servers. With MCP, you can:

• Access pre-built toolsfrom the MCP ecosystem without implementing them yourself
• Connect to external serviceslike databases, APIs, and file systems
• Share toolsbetween different AI applications
• Build extensible workflowsthat leverage community-maintained tools

MCP tools work seamlessly with Genkit’s tool-calling system, allowing you to mix custom tools with external MCP tools in the same generation request.

• Learn about Model Context Protocol (MCP) to extend your tool capabilities with external servers
• Explore interrupts to pause tool execution for user interaction
• See retrieval-augmented generation (RAG) for handling large amounts of contextual information
• Check out multi-agent systems for coordinating multiple AI agents with tools
• Browse the tool calling example for a complete implementation

Tool calling , also known as function calling , is a structured way to give LLMs the ability to make requests back to the application that called it. You define the tools you want to make available to the model, and the model will make tool requests to your app as necessary to fulfill the prompts you give it.

The use cases of tool calling generally fall into a few themes:

Giving an LLM access to information it wasn’t trained with

• Frequently changing information, such as a stock price or the current weather.
• Information specific to your app domain, such as product information or user profiles.

Note the overlap with retrieval augmented generation (RAG), which is also a way to let an LLM integrate factual information into its generations. RAG is a heavier solution that is most suited when you have a large amount of information or the information that’s most relevant to a prompt is ambiguous. On the other hand, if a function call or database lookup is all that’s necessary for retrieving the information the LLM needs, tool calling is more appropriate.

Introducing a degree of determinism into an LLM workflow

• Performing calculations that the LLM cannot reliably complete itself.
• Forcing an LLM to generate verbatim text under certain circumstances, such as when responding to a question about an app’s terms of service.

Performing an action when initiated by an LLM

• Turning on and off lights in an LLM-powered home assistant
• Reserving table reservations in an LLM-powered restaurant agent

If you want to run the code examples on this page, first complete the steps in the Get started guide. All of the examples assume that you have already set up a project with Genkit dependencies installed.

This page discusses one of the advanced features of Genkit model abstraction, so before you dive too deeply, you should be familiar with the content on the Generating content with AI models page. You should also be familiar with Genkit’s system for defining input and output schemas, which is discussed on the Flows page.

At a high level, this is what a typical tool-calling interaction with an LLM looks like:

1. The calling application prompts the LLM with a request and also includes in the prompt a list of tools the LLM can use to generate a response.
2. The LLM either generates a complete response or generates a tool call request in a specific format.
3. If the caller receives a complete response, the request is fulfilled and the interaction ends; but if the caller receives a tool call, it performs whatever logic is appropriate and sends a new request to the LLM containing the original prompt or some variation of it as well as the result of the tool call.
4. The LLM handles the new prompt as in Step 2.

For this to work, several requirements must be met:

• The model must be trained to make tool requests when it’s needed to complete a prompt. Most of the larger models provided through web APIs such as Gemini can do this, but smaller and more specialized models often cannot. Genkit will throw an error if you try to provide tools to a model that doesn’t support it.
• The calling application must provide tool definitions to the model in the format it expects.
• The calling application must prompt the model to generate tool calling requests in the format the application expects.

Genkit provides a single interface for tool calling with models that support it. Each model plugin ensures that the last two criteria mentioned in the previous section are met, and the genkit.Generate() function automatically carries out the tool-calling loop described earlier.

Tool calling support depends on the model, the model API, and the Genkit plugin. Consult the relevant documentation to determine if tool calling is likely to be supported. In addition:

• Genkit will throw an error if you try to provide tools to a model that doesn’t support it.
• If the plugin exports model references, the ModelInfo.Supports.Tools property will indicate if it supports tool calling.

Use the genkit.DefineTool() function to write tool definitions:

  
 
 package 
 main 
 
 
 
 
 
 
 
 
 import 
 ( 
 
 
 
 
  
 " 
 context 
 " 
 
 
 
 
  
 " 
 fmt 
 " 
 
 
 
 
  
 " 
 log 
 " 
 
 
 
 
 
 
 
 
  
 " 
 github.com/firebase/genkit/go/ai 
 " 
 
 
 
 
  
 " 
 github.com/firebase/genkit/go/genkit 
 " 
 
 
 
 
  
 " 
 github.com/firebase/genkit/go/plugins/googlegenai 
 " 
 
 
 
 
 ) 
 
 
 
 
 
 
 
 
 // Define the input structure for the tool 
 
 
 
 
 type 
 WeatherInput 
 struct 
 { 
 
 
 
 
 Location 
 string 
 `json:"location" jsonschema_description:"Location to get weather for"` 
 
 
 
 
 } 
 
 
 
 
 
 
 
 
 func 
 main 
 () { 
 
 
 
 
 ctx 
 := 
 context. 
 Background 
 () 
 
 
 
 
 
 
 
 
 g, err 
 := 
 genkit. 
 Init 
 (ctx, 
 
 
 
 
 genkit. 
 WithPlugins 
 ( 
 & 
 googlegenai 
 . 
 GoogleAI 
 {}), 
 
 
 
 
 genkit. 
 WithDefaultModel 
 ( 
 "googleai/gemini-1.5-flash" 
 ), 
 // Updated model name 
 
 
 
 
 ) 
 
 
 
 
  
 if 
 err 
 != 
 nil 
 { 
 
 
 
 
 log. 
 Fatalf 
 ( 
 "Genkit initialization failed: 
 %v 
 " 
 , err) 
 
 
 
 
 } 
 
 
 
 
 
 
 
 
 genkit. 
 DefineTool 
 ( 
 
 
 
 
 g, 
 "getWeather" 
 , 
 "Gets the current weather in a given location" 
 , 
 
 
 
 
  
 func 
 ( 
 ctx 
 context 
 . 
 Context 
 , 
 input 
 WeatherInput 
 ) ( 
 string 
 , 
 error 
 ) { 
 
 
 
 
  
 // Here, we would typically make an API call or database query. For this 
 
 
 
 
  
 // example, we just return a fixed value. 
 
 
 
 
 log. 
 Printf 
 ( 
 "Tool 'getWeather' called for location: 
 %s 
 " 
 , input.Location) 
 
 
 
 
  
 return 
 fmt. 
 Sprintf 
 ( 
 "The current weather in 
 %s 
 is 63°F and sunny." 
 , input.Location), 
 nil 
 
 
 
 
 }) 
 
 
 
 
 } 
 
 
 

The syntax here looks just like the genkit.DefineFlow() syntax; however, you must write a description. Take special care with the wording and descriptiveness of the description as it is vital for the LLM to decide to use it appropriately.

Include defined tools in your prompts to generate content.

Using genkit.Generate() :

  
 
 resp, err 
 := 
 genkit. 
 Generate 
 (ctx, g, 
 
 
 
 
 ai. 
 WithPrompt 
 ( 
 "What is the weather in San Francisco?" 
 ), 
 
 
 
 
 ai. 
 WithTools 
 (getWeatherTool), 
 
 
 
 
 ) 
 
 
 

Using genkit.DefinePrompt() :

  
 
 weatherPrompt 
 := 
 genkit. 
 DefinePrompt 
 (g, 
 "weatherPrompt" 
 , 
 
 
 
 
 ai. 
 WithPrompt 
 ( 
 "What is the weather in {{location}}?" 
 ), 
 
 
 
 
 ai. 
 WithTools 
 (getWeatherTool), 
 
 
 
 
 ) 
 
 
 
 
 
 
 
 
 resp, err 
 := 
 weatherPrompt. 
 Execute 
 (ctx, 
 
 
 
 
 ai. 
 WithInput 
 ( 
 map 
 [ 
 string 
 ] 
 any 
 { 
 "location" 
 : 
 "San Francisco" 
 }), 
 
 
 
 
 ) 
 
 
 

Using a .prompt file:

Create a file named prompts/weatherPrompt.prompt (assuming default prompt directory):

  
 
 --- 
 
 
 
 
 system 
 : 
 "Answer questions using the tools you have." 
 
 
 
 
 tools 
 : [ 
 getWeather 
 ] 
 
 
 
 
 input 
 : 
 
 
 
 
  
 schema 
 : 
 
 
 
 
  
 location 
 : 
 string 
 
 
 
 
 --- 
 
 
 
 
 
 
 
 
 What is the weather in 
 {{ 
 location 
 }} 
 ? 
 
 
 

Then execute it in your Go code:

  
 
 // Assuming prompt file named weatherPrompt.prompt exists in ./prompts dir. 
 
 
 
 
 weatherPrompt 
 := 
 genkit. 
 LookupPrompt 
 ( 
 "weatherPrompt" 
 ) 
 
 
 
 
 if 
 weatherPrompt 
 == 
 nil 
 { 
 
 
 
 
 log. 
 Fatal 
 ( 
 "no prompt named 'weatherPrompt' found" 
 ) 
 
 
 
 
 } 
 
 
 
 
 
 
 
 
 resp, err 
 := 
 weatherPrompt. 
 Execute 
 (ctx, 
 
 
 
 
 ai. 
 WithInput 
 ( 
 map 
 [ 
 string 
 ] 
 any 
 { 
 "location" 
 : 
 "San Francisco" 
 }), 
 
 
 
 
 ) 
 
 
 

Genkit will automatically handle the tool call if the LLM needs to use the getWeather tool to answer the prompt.

If you want full control over this tool-calling loop, for example to apply more complicated logic, set the WithReturnToolRequests() option to true . Now it’s your responsibility to ensure all of the tool requests are fulfilled:

  
 
 getWeatherTool 
 := 
 genkit. 
 DefineTool 
 ( 
 
 
 
 
 g, 
 "getWeather" 
 , 
 "Gets the current weather in a given location" 
 , 
 
 
 
 
  
 func 
 ( 
 ctx 
 context 
 . 
 Context 
 , 
 location 
 struct 
 { 
 
 
 
 
 Location 
 string 
 `jsonschema_description:"Location to get weather for"` 
 
 
 
 
 }) ( 
 string 
 , 
 error 
 ) { 
 
 
 
 
  
 // Tool implementation... 
 
 
 
 
  
 return 
 "sunny" 
 , 
 nil 
 
 
 
 
 }, 
 
 
 
 
 ) 
 
 
 
 
 
 
 
 
 resp, err 
 := 
 genkit. 
 Generate 
 (ctx, g, 
 
 
 
 
 ai. 
 WithPrompt 
 ( 
 "What is the weather in San Francisco?" 
 ), 
 
 
 
 
 ai. 
 WithTools 
 (getWeatherTool), 
 
 
 
 
 ai. 
 WithReturnToolRequests 
 ( 
 true 
 ), 
 
 
 
 
 ) 
 
 
 
 
 if 
 err 
 != 
 nil 
 { 
 
 
 
 
 log. 
 Fatal 
 (err) 
 
 
 
 
 } 
 
 
 
 
 
 
 
 
 parts 
 := 
 [] 
 * 
 ai 
 . 
 Part 
 {} 
 
 
 
 
 for 
 _, req 
 := 
 range 
 resp. 
 ToolRequests 
 () { 
 
 
 
 
 tool 
 := 
 genkit. 
 LookupTool 
 (g, req.Name) 
 
 
 
 
  
 if 
 tool 
 == 
 nil 
 { 
 
 
 
 
 log. 
 Fatalf 
 ( 
 "tool 
 %q 
 not found" 
 , req.Name) 
 
 
 
 
 } 
 
 
 
 
 
 
 
 
 output, err 
 := 
 tool. 
 RunRaw 
 (ctx, req.Input) 
 
 
 
 
  
 if 
 err 
 != 
 nil 
 { 
 
 
 
 
 log. 
 Fatalf 
 ( 
 "tool 
 %q 
 execution failed: 
 %v 
 " 
 , tool. 
 Name 
 (), err) 
 
 
 
 
 } 
 
 
 
 
 
 
 
 
 parts 
 = 
 append 
 (parts, 
 
 
 
 
 ai. 
 NewToolResponsePart 
 ( 
 & 
 ai 
 . 
 ToolResponse 
 { 
 
 
 
 
 Name:   req.Name, 
 
 
 
 
 Ref:    req.Ref, 
 
 
 
 
 Output: output, 
 
 
 
 
 })) 
 
 
 
 
 } 
 
 
 
 
 
 
 
 
 resp, err 
 = 
 genkit. 
 Generate 
 (ctx, g, 
 
 
 
 
 ai. 
 WithMessages 
 ( 
 append 
 (resp. 
 History 
 (), ai. 
 NewMessage 
 (ai.RoleTool, 
 nil 
 , parts 
 ... 
 )) 
 ... 
 ), 
 
 
 
 
 ) 
 
 
 
 
 if 
 err 
 != 
 nil 
 { 
 
 
 
 
 log. 
 Fatal 
 (err) 
 
 
 
 
 } 
 
 
 

Tool calling , also known as function calling , is a structured way to give LLMs the ability to make requests back to the application that called it. You define the tools you want to make available to the model, and the model will make tool requests to your app as necessary to fulfill the prompts you give it.

The use cases of tool calling generally fall into a few themes:

Giving an LLM access to information it wasn’t trained with

• Frequently changing information, such as a stock price or the current weather.
• Information specific to your app domain, such as product information or user profiles.

Note the overlap with retrieval augmented generation (RAG), which is also a way to let an LLM integrate factual information into its generations. RAG is a heavier solution that is most suited when you have a large amount of information or the information that’s most relevant to a prompt is ambiguous. On the other hand, if retrieving the information the LLM needs is a simple function call or database lookup, tool calling is more appropriate.

Introducing a degree of determinism into an LLM workflow

• Performing calculations that the LLM cannot reliably complete itself.
• Forcing an LLM to generate verbatim text under certain circumstances, such as when responding to a question about an app’s terms of service.

Performing an action when initiated by an LLM

• Turning on and off lights in an LLM-powered home assistant
• Reserving table reservations in an LLM-powered restaurant agent

If you want to run the code examples on this page, first complete the steps in the Get started guide. All of the examples assume that you have already set up a project with Genkit dependencies installed.

This page discusses one of the advanced features of Genkit model abstraction, so before you dive too deeply, you should be familiar with the content on the Generating content with AI models page. You should also be familiar with Genkit’s system for defining input and output schemas, which is discussed on the Flows page.

At a high level, this is what a typical tool-calling interaction with an LLM looks like:

1. The calling application prompts the LLM with a request and also includes in the prompt a list of tools the LLM can use to generate a response.
2. The LLM either generates a complete response or generates a tool call request in a specific format.
3. If the caller receives a complete response, the request is fulfilled and the interaction ends; but if the caller receives a tool call, it performs whatever logic is appropriate and sends a new request to the LLM containing the original prompt or some variation of it as well as the result of the tool call.
4. The LLM handles the new prompt as in Step 2.

For this to work, several requirements must be met:

• The model must be trained to make tool requests when it’s needed to complete a prompt. Most of the larger models provided through web APIs, such as Gemini and Claude, can do this, but smaller and more specialized models often cannot. Genkit will throw an error if you try to provide tools to a model that doesn’t support it.
• The calling application must provide tool definitions to the model in the format it expects.
• The calling application must prompt the model to generate tool calling requests in the format the application expects.

Genkit provides a single interface for tool calling with models that support it. Each model plugin ensures that the last two of the above criteria are met, and the Genkit instance’s generate() function automatically carries out the tool calling loop described earlier.

Tool calling support depends on the model, the model API, and the Genkit plugin. Consult the relevant documentation to determine if tool calling is likely to be supported. In addition:

• Genkit will throw an error if you try to provide tools to a model that doesn’t support it.
• If the plugin exports model references, the info.supports.tools property will indicate if it supports tool calling.

Use the Genkit instance’s tool() decorator to write tool definitions:

  
 
 from 
 pydantic 
 import 
 BaseModel, Field 
 
 
 
 
 from 
 genkit.ai 
 import 
 Genkit 
 
 
 
 
 from 
 genkit.plugins.google_genai 
 import 
 GoogleAI 
 
 
 
 
 
 
 
 
 ai 
 = 
 Genkit( 
 
 
 
 
  
 plugins 
 = 
 [GoogleAI()], 
 
 
 
 
  
 model 
 = 
 'googleai/gemini-2.5-flash' 
 , 
 
 
 
 
 ) 
 
 
 
 
 
 
 
 
 class 
 WeatherInput 
 ( 
 BaseModel 
 ): 
 
 
 
 
 location: 
 str 
 = 
 Field( 
 description 
 = 
 'The location to get the current weather for' 
 ) 
 
 
 
 
 
 
 
 
 
 
 
 
 @ai.tool 
 () 
 
 
 
 
 def 
 get_weather 
 (input: WeatherInput) -> 
 str 
 : 
 
 
 
 
  
 """Gets the current weather in a given location""" 
 
 
 
 
  
 # Replace with actual weather fetching logic 
 
 
 
 
  
 return 
 f 
 'The current weather in 
 {input 
 .location 
 } 
 is 63°F and sunny.' 
 
 
 

The syntax here looks just like the flow() syntax; however description parameter is required. When writing a tool definition, take special care with the wording and descriptiveness of these parameters. They are vital for the LLM to make effective use of the available tools.

Include defined tools in your prompts to generate content.

  
 
 result 
 = 
 await 
 ai.generate( 
 
 
 
 
  
 prompt 
 = 
 'What is the weather in Baltimore?' 
 , 
 
 
 
 
  
 tools 
 = 
 [ 
 'get_weather' 
 ], 
 
 
 
 
 ) 
 
 
 

Genkit will automatically handle the tool call if the LLM needs to use the get_weather tool to answer the prompt.

By default, Genkit repeatedly calls the LLM until every tool call has been resolved. You can conditionally pause execution in situations where you want to, for example:

• Ask the user a question or display UI.
• Confirm a potentially risky action with the user.
• Request out-of-band approval for an action.

Interruptsare special tools that can halt the loop and return control to your code so that you can handle more advanced scenarios. Visit the interrupts guide to learn how to use them.

If you want full control over this tool-calling loop, for example to apply more complicated logic, set the return_tool_requests parameter to True . Now it’s your responsibility to ensure all of the tool requests are fulfilled:

  
 
 llm_response 
 = 
 await 
 ai.generate( 
 
 
 
 
  
 prompt 
 = 
 'What is the weather in Baltimore?' 
 , 
 
 
 
 
  
 tools 
 = 
 [ 
 'get_weather' 
 ], 
 
 
 
 
  
 return_tool_requests 
 = 
 True 
 , 
 
 
 
 
 ) 
 
 
 
 
 
 
 
 
 tool_request_parts 
 = 
 llm_response.tool_requests 
 
 
 
 
 
 
 
 
 if 
 len 
 (tool_request_parts) 
 == 
 0 
 : 
 
 
 
 
  
 print 
 (llm_response.text) 
 
 
 
 
 else 
 : 
 
 
 
 
  
 for 
 part 
 in 
 tool_request_parts: 
 
 
 
 
  
 await 
 handle_tool(part.name, part.input)