Edit

Share via

Customize speech models with fine-tuning

With custom speech, you can enhance speech recognition accuracy for your applications by using a custom model for real-time speech to text, speech translation, and batch transcription.

Tip

Bring your custom speech models from Speech Studio to the Azure AI Foundry portal. In Azure AI Foundry portal, you can pick up where you left off by connecting to your existing Speech resource. For more information about connecting to an existing Speech resource, see Connect to an existing Speech resource.

You create a custom speech model by fine-tuning an Azure AI Speech base model with your own data. You can upload your data, test and train a custom model, compare accuracy between models, and deploy a model to a custom endpoint.

This article shows you how to use fine-tuning to create a custom speech model. For more information about custom speech, see the custom speech overview documentation.

Tip

You can bring your custom speech models from Speech Studio to the Azure AI Foundry portal. In Azure AI Foundry, you can pick up where you left off by connecting to your existing Speech resource. For more information about connecting to an existing Speech resource, see Connect to an existing Speech resource.

Start fine-tuning

Custom speech fine-tuning includes models, training and testing datasets, and deployment endpoints. Each project is specific to a locale. For example, you might fine-tune for English in the United States.

In the Azure AI Foundry portal, you can fine-tune some Azure AI services models. For example, you can fine-tune a model for custom speech. Each custom model is specific to a locale. For example, you might fine-tune a model for English in the United States.

  1. Go to your project in the Azure AI Foundry portal. If you need to create a project, see Create an Azure AI Foundry project.

  2. Select Fine-tuning from the left pane.

  3. Select AI Service fine-tuning > + Fine-tune.

    Screenshot of the page to select fine-tuning of Azure AI Services models.

  4. In the wizard, select Custom Speech (speech to text fine-tuning) for custom speech. Then select Next.

  5. Enter the language, name, and description for the fine-tuning job. Then select Create.

Continue fine-tuning

Go to the Azure AI Speech documentation to learn how to continue fine-tuning your custom speech model:

View fine-tuned models

After fine-tuning, you can access your custom speech models and deployments from the Fine-tuning page.

  1. Sign in to the Azure AI Foundry portal.

  2. Select Fine-tuning from the left pane.

  3. Select AI Service fine-tuning.

    Screenshot of the page where you can view fine-tuned AI services models.

After you create a custom speech project, you can access your custom speech models and deployments from the Custom speech page.

  1. Sign in to the Speech Studio.

  2. Select the subscription and Speech resource to work with.

    Important

    If you train a custom model with audio data, select a service resource in a region with dedicated hardware for training audio data. See footnotes in the regions table for more information.

  3. Select Custom speech > Create a new project.

  4. Follow the instructions provided by the wizard to create your project.

Select the new project by name or select Go to project. Then you should see these menu items in the left panel: Speech datasets, Train custom models, Test models, and Deploy models.

Get the project ID for the REST API

When you use the speech to text REST API for custom speech, you need to set the project property to the ID of your custom speech project. You need to set the project property so that you can manage fine-tuning in the Azure AI Foundry portal.

Important

The project ID for custom speech isn't the same as the ID of the Azure AI Foundry project.

You can find the project ID in the URL after you select or start fine-tuning a custom speech model.

  1. Sign in to the Azure AI Foundry portal.

  2. Select Fine-tuning from the left pane.

  3. Select AI Service fine-tuning.

  4. Select the custom model that you want to check from the Model name column.

  5. Inspect the URL in your browser. The project ID is part of the URL. For example, the project ID is 00001111-aaaa-2222-bbbb-3333cccc4444 in the following URL:

    https://ai.azure.com/build/models/aiservices/speech/customspeech/00001111-aaaa-2222-bbbb-3333cccc4444/<REDACTED_FOR_BREVITY>

When you use the speech to text REST API for custom speech, you need to set the project property to the ID of your custom speech project. You need to set the project property so that you can manage fine-tuning in the Speech Studio.

To get the project ID for a custom speech project in Speech Studio:

  1. Sign in to the Speech Studio and select the Custom speech tile.

  2. Select your custom speech project.

  3. Inspect the URL in your browser. The project ID is part of the URL. For example, the project ID is 00001111-aaaa-2222-bbbb-3333cccc4444 in the following URL:

    https://speech.microsoft.com/portal/<Your-Resource-ID>/customspeech/a0a0a0a0-bbbb-cccc-dddd-e1e1e1e1e1e1

Related content