Edit

Share via

Customize the configuration of AI functions

AI functions, currently in public preview, allow users to harness the power of Fabric's native large language models (LLMs) to transform and enrich their enterprise data. They're designed to work out-of-the-box, with the underlying model and settings configured by default. Users who want more flexible configurations, however, can customize their solutions with a few extra lines of code.

Important

This feature is in preview, for use in the Fabric 1.3 runtime and higher.

  • Review the prerequisites in this overview article, including the library installations that are temporarily required to use AI functions.
  • Although the underlying model can handle several languages, most of the AI functions are optimized for use on English-language texts.
  • During the initial rollout of AI functions, users are temporarily limited to 1,000 requests per minute with Fabric's built-in AI endpoint.

Customizing AI functions with pandas

By default, AI functions are powered by Fabric's built-in AI endpoint. The LLM's settings are globally configured in the aifunc.Conf class. If you work with AI functions in pandas, you can use the aifunc.Conf class to modify some or all of these settings:

Parameter Description Default
model_deployment_name
Optional		 A string that designates the name of the language model deployment that powers AI functions. You can choose from the models supported by Fabric. gpt-4o-mini
embedding_deployment_name
Optional		 A string that designates the name of the embedding model deployment that powers AI functions. text-embedding-ada-002
temperature
Optional		 A float between 0.0 and 1.0 that designates the temperature of the underlying model. Higher temperatures increase the randomness or creativity of the model's outputs. 0.0
seed
Optional		 An int that designates the seed to use for the response of the underlying model. The default behavior randomly picks a seed value for each row. The choice of a constant value improves the reproducibility of your experiments. openai.NOT_GIVEN
timeout
Optional		 An int that designates the number of seconds before an AI function raises a time-out error. By default, there's no time-out. None
max_concurrency
Optional		 An int that designates the maximum number of rows to be processed in parallel with asynchronous requests to the model. Higher values speed up processing time (if your capacity can accommodate it). 4

The next code sample shows how to override aifunc.Conf settings globally, so that they apply to all AI function calls in a given session:

# This code uses AI. Always review output for mistakes. 
# Read terms: https://azure.microsoft.com/support/legal/preview-supplemental-terms/

aifunc.default_conf.temperature = 0.5 # Default: 0.0
aifunc.default_conf.max_concurrency = 10 # Default: 4

df = pd.DataFrame([
        "Hello! How are you doing today?", 
        "Tell me what you'd like to know, and I'll do my best to help.", 
        "The only thing we have to fear is fear itself."
    ], columns=["text"])

df["translations"] = df["text"].ai.translate("spanish")
df["sentiment"] = df["text"].ai.analyze_sentiment()
display(df)

You can also customize these settings for each individual function call. Each AI function accepts an optional conf parameter. The next code sample modifies the default aifunc settings for only the ai.translate function call, using a custom temperature value. (The ai.analyze_sentiment call still uses the default values, because no custom values are set.)

# This code uses AI. Always review output for mistakes. 
# Read terms: https://azure.microsoft.com/support/legal/preview-supplemental-terms/

from synapse.ml.aifunc import Conf

df = pd.DataFrame([
        "Hello! How are you doing today?", 
        "Tell me what you'd like to know, and I'll do my best to help.", 
        "The only thing we have to fear is fear itself."
    ], columns=["text"])

df["translations"] = df["text"].ai.translate("spanish", conf=Conf(temperature=0.5))
df["sentiment"] = df["text"].ai.analyze_sentiment()
display(df)

To substitute a custom Azure OpenAI LLM resource in place of the native Fabric LLM, you can use the aifunc.setup function with your own client, as shown in the next code sample:

from openai import AzureOpenAI

# Example of creating a custom client
client = AzureOpenAI(
    api_key="your-api-key",
    azure_endpoint="https://your-openai-endpoint.openai.azure.com/",
    api_version=aifunc.session.api_version,  # Default "2024-10-21"
    max_retries=aifunc.session.max_retries,  # Default: sys.maxsize ~= 9e18
)

aifunc.setup(client)  # Set the client for all functions

Customizing AI functions with PySpark

If you're working with AI functions in PySpark, you can use the OpenAIDefaults class to modify the underlying language model that powers the functions. As an example, the following code sample uses placeholder values to show how you can override the built-in Fabric AI endpoint with a custom Azure OpenAI LLM deployment:

from synapse.ml.services.openai import OpenAIDefaults
defaults = OpenAIDefaults()

defaults.set_deployment_name("your-deployment-name")
defaults.set_subscription_key("your-subscription-key")
defaults.set_URL("https://your-openai-endpoint.openai.azure.com/")
defaults.set_temperature(0.05)

You can substitute your own values for the deployment name, subscription key, endpoint URL, and custom temperature value:

Parameter Description
deployment_name A string value that designates the custom name of your model deployment in Azure OpenAI or Azure AI Foundry. In the Azure portal, this value appears under Resource Management > Model Deployments. In the Azure AI Foundry portal, the value appears on the Deployments page. You can choose from the models supported by Fabric. By default, the native Fabric LLM endpoint deployment is set to gpt-4o-mini.
subscription_key An API key used for authentication with your LLM resource. In the Azure portal, this value appears in the Keys and Endpoint section.
URL A URL designating the endpoint of your LLM resource. In the Azure portal, this value appears in the Keys and Endpoint section. For example: "https://your-openai-endpoint.openai.azure.com/".
temperature A numeric value between 0.0 and 1.0. Higher temperatures increase the randomness or creativity of the underlying model's outputs. By default, the Fabric LLM endpoint's temperature is set to 0.0.

You can retrieve and print each of the OpenAIDefaults parameters with the next code sample:

print(defaults.get_deployment_name())
print(defaults.get_subscription_key())
print(defaults.get_URL())
print(defaults.get_temperature())

You can also reset the parameters as easily as you modified them. The following code sample resets the AI functions library so that it uses the default Fabric LLM endpoint:

defaults.reset_deployment_name()
defaults.reset_subscription_key()
defaults.reset_URL()
defaults.reset_temperature()

Related content