Working with OpenAI and F# – Using automated responses and functions

In this series so far, we've looked at working with OpenAI with datasets e.g. CSV data as input into the product, and how you can use F# to bring in external datasets to send up to OpenAI in order to ask questions of it. We've also observed how we can have it return back information in a machine-readable format rather than just plain text - recall that in our first post we showed how, with some selective prompting, OpenAI can return data as JSON. We've also focused on a more "interactive" flow of messages through scripts. In this example, I want to move towards automating the request / response rather than manually responding through F# Interactive by inverting the flow of messages using Function Tools, so that OpenAI asks you questions that better enable it to answer your original question (!)

Challenges with sending data to OpenAI

So far, we've looked at working with relatively small datasets that are sent to OpenAI to get answers. However, repeatedly sending up data in order for OpenAI to answer questions about a subset of it may not always be the optimal solution:

• You may have larger datasets which can increase the time taken for OpenAI to operate.
• You may have sensitive datasets which you do not / cannot share with OpenAI.
• Recall that OpenAI has no stateful history, which means that you need to send back the entire conversation thread on every request. Given that OpenAI charges you per token (think of this as an arbitrary portion of text - a character, word or chunks of words), sending up the same dataset time and again increases costs.
• You may have bespoke algorithms that you wish to control and not delegate to OpenAI to simply do whatever it feels in order to answer a question.
• You may wish to execute operations against local data (or third-party sources) which OpenAI cannot "see". As a simple example, renaming files - clearly OpenAI cannot touch files on the local file system. It could provide you with a set of commands which you then parsed, but it can't execute operations outside of its own internal scope.

Functions for the win

As a solution to this challenge, the OpenAI SDK includes support for Function Calling - the ability to provide OpenAI with details of local functions that it can orchestrate calls to. Here's how it works:

1. Your application supplies all the details to the available functions - inputs, outputs, name and description - as additional information whenever sending a message / question to OpenAI.
2. It may then respond with - instead of a textual response - a request for you to call a function (known as a tool request).
3. Your application calls the function requested and returns the data back to OpenAI (this is known as a tool response).
4. OpenAI will use that data to answer the original question that you asked.

A function tool request (pseudo code) for an F# function might look like this:

let getTasks category dateRange : TodoTask list =
    // fetch tasks from Todo database etc.

{
   "Function Name" : "GetTasks"
   "Arguments" : [
       "Category" : "Work"
       "DateRange" : "> 01 Nov 2024"
   ]
}

It's your application's job to parse that message and then actually call getTasks in your own application, before giving the data back to OpenAI in some plain-text format that it can understand. Note that getTasks can be any function that you want, it's your code - you can call a database, call another API - it's entirely up to you.

Modelling Function Tools with FSharp

OpenAI has its own SDK but we might prefer a more F# approach for modelling. Here's what a (slightly simplified) F# model might look like:

/// A function tool can be modelled as a function which takes in a map of (argument name -> value) and returns back a string response.
type FunctionTool = Map<string, obj> -> string
/// An argument can be one of three datatypes.
type PropertyType =
    | Enum of string list
    | String
    | Number
/// The configuration for a function tool includes its name and description which OpenAI uses to understand how to use it.
type FunctionToolConfig = {
    Name: string
    Description: string
    Parameters: Property list
    Func: FunctionTool
}

Let's take a simple example to start with - greeting someone with Hello World. We want to tell OpenAI to greet someone, but also to let us control the format of the greeting itself with our own code. The first thing to do is to supply to OpenAI the list of function tools that are at its disposal:

let greet name = $"Hello, %s{name}!"

let tools = [
    {
        Name = "Greet"
        Description = "Greets the user"
        Parameters = [ Property.Create "Name" "The name of the person to greet" String ]
        Func = fun args -> greet (string args["Name"])
    }
]

Armed with this, we can then start to communicate with OpenAI as normal. It will automatically request function calls as needed:

let chat = Conversation.StartNew(tools)
chat.SendMessage "Send a greeting to John"
chat.History()

(*

("UserChatMessage", "Send a greeting to John")                             // initial request from our local application
("AssistantChatMessage", "Calling Greet with arguments [|[Name, John]|]"); // Request from OpenAI to call Greet
("ToolChatMessage", "Hello, John!");                                       // Response from our local application
("AssistantChatMessage", "Hello, John!")|]                                 // Final response from OpenAI

*)

Behind the scenes, the Conversation type automatically handles AssistantChatMessages, calling the appropriate F# code that you define as needed. Obviously, OpenAI doesn't have to simply provide the response directly back - it will use the output however it needs to. In this example, it calls the Greet function but includes the output within a larger message:

chat.SendMessage "Send a greeting to John. Use the greeting within a larger sentence or two."
// Hello, John! I hope this message finds you well and brings a smile to your face. Enjoy your day!

It can also intelligently generate parameter values. Let's enhance the greeting function to take in a "friendliness factor" parameter, which the implementation would use to shape the greeting:

Property.Create "Friendliness" "How friendly the greeting should be, on the scale of 1-5" Number

Here are some sample requests we send through to OpenAI, along with the Function Tool requests it made. As you can see, OpenAI correctly parses the text and sends an appropriate value for friendliness.

chat.SendMessage "Send a terse greeting to John." // [Friendliness, 2]
chat.SendMessage "Send a greeting to John." // [Friendliness, 3]
chat.SendMessage "Send a warm greeting to John." // [Friendliness, 4]
chat.SendMessage "You're in a hurry and don't like John much. How should you greet him?" // [Friendliness, 1]

Orchestrating multiple function calls

This is already quite nice! But where things can really be more powerful is allowing OpenAI to orchestrate multiple function calls together itself. Let's add another tool, which can simply read all the contents from a text file:

FunctionToolConfig.Create
    "ReadFile"
    "Loads the contents of a file"
    [ Property.Create "Path" "The path to the file" String ]
    (fun args -> File.ReadAllText(args.["Path"] |> string))

As you can see, this is simply a wrapper around a standard System.IO.File method. But armed with this simple function we can ask OpenAI questions like the following:

chat.SendMessage "Send greetings to all the people listed in the GuestList.txt file."

This leads to the following key events in the conversation:

(*

Calling ReadFile with arguments [Path, GuestList.txt]
...
Calling Greet with arguments [Friendliness, 3]; [Name, John], Calling Greet with arguments [Friendliness, 3]; [Name, Samantha], Calling Greet with arguments [Friendliness, 3]; [Name, Fred]
...
Hello, John!
Hello, Samantha!
Hello, Fred!

*)

Observe how OpenAI has figured out which functions to call and in what order. It also sent all three greet function requests as a single "payload" back, rather than one at a time.

Ideas for Function Tools

There are countless applications for using Function Tools. Here are some simple examples:

• Bulk renaming or moving files based on specific - potentially fuzzy - contents within them.
• Recommending suitable locations for your next home move based on multiple datasets e.g. crime, price, distance to work.
• Rapidly creating a dynamic database access layer using a generic SQL-executing query function (OpenAI can easily generate valid TSQL given a schema).
• Calculating routes for a trip based on external datasets e.g. weather.

These are just some arbitrary examples. Clearly, using tools which rely on AI support is a somewhat nascent area and care needs to be applied in order to use these tools safely and effectively. The aim of this series was to give a gentle introduction into OpenAI and how we can use F# applications - and scripts - to execute specific activities. Sample source code for this series is available here. Enjoy!