Skip to content

Function Calling with tool support

Ollama API: chat request with tools https://github.com/ollama/ollama/blob/main/docs/api.md#chat-request-with-tools

Since Ollama 0.3.0, Ollama supports tools calling, blog post: https://ollama.com/blog/tool-support. A list of supported models can be found under the Tools category on the models page: https://ollama.com/search?c=tools

Define a list of tools

use a supported model

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
model := "mistral:7b"

toolsList := []llm.Tool{
    {
        Type: "function",
        Function: llm.Function{
            Name:        "hello",
            Description: "Say hello to a given person with his name",
            Parameters: llm.Parameters{
                Type: "object",
                Properties: map[string]llm.Property{
                    "name": {
                        Type:        "string",
                        Description: "The name of the person",
                    },
                },
                Required: []string{"name"},
            },
        },
    },
    {
        Type: "function",
        Function: llm.Function{
            Name:        "addNumbers",
            Description: "Make an addition of the two given numbers",
            Parameters: llm.Parameters{
                Type: "object",
                Properties: map[string]llm.Property{
                    "a": {
                        Type:        "number",
                        Description: "first operand",
                    },
                    "b": {
                        Type:        "number",
                        Description: "second operand",
                    },
                },
                Required: []string{"a", "b"},
            },
        },
    },
}

Set the Tools property of the query

  • set the Temperature to 0.0
  • you don't need to set the row mode to true
  • set query.Tools with toolsList
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
messages := []llm.Message{
    {Role: "user", Content: `say "hello" to Bob`},
}

options := llm.SetOptions(map[string]interface{}{
    option.Temperature: 0.0,
    option.RepeatLastN: 2,
    option.RepeatPenalty: 2.0,
})

query := llm.Query{
    Model:    model,
    Messages: messages,
    Tools:    toolsList,
    Options:  options,
    Format:   "json",
}

Run the completion

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
answer, err := completion.Chat(ollamaUrl, query)
if err != nil {
    log.Fatal("😡:", err)
}

// It's a []map[string]interface{}
toolCalls := answer.Message.ToolCalls

// Convert toolCalls into a JSON string
jsonBytes, err := json.Marshal(toolCalls)
if err != nil {
    log.Fatal("😡:", err)
}
// Convert JSON bytes to string
result := string(jsonBytes)

fmt.Println(result)

The result will look like this: 

1
[{"function":{"arguments":{"name":"Bob"},"name":"hello"}}]

Or you can use the ToolCallsToJSONString helper

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
answer, err = completion.Chat(ollamaUrl, query)
if err != nil {
    log.Fatal("😡:", err)
}

result, err = answer.Message.ToolCallsToJSONString()
if err != nil {
    log.Fatal("😡:", err)
}
fmt.Println(result)
The result will look like this: 
1
[{"function":{"arguments":{"name":"Bob"},"name":"hello"}}]

Note

Look here for a complete sample: examples/19-mistral-function-calling-tool-support

Or (better) you can use the ToolCallsToJSONString helper

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
answer, err := completion.Chat(ollamaUrl, query)
if err != nil {
    log.Fatal("😡:", err)
}

result, err := answer.Message.ToolCalls[0].Function.ToJSONString()
if err != nil {
    log.Fatal("😡:", err)
}
fmt.Println(result)

The result will look like this: 

1
{"name":"hello","arguments":{"name":"Bob"}}

Note

Look at these samples: