Share via

How to get custom model metrics?

Admin Rob 0 Reputation points
2025-08-04T04:59:47.63+00:00

Helllo, we are training custom classification and field exraction models in Document Intelligence Studio. We need to see the metrics (e.g. accuracy, precision, recall, F1, confusion matrix) of the newly trained model to decide whether we can apply it or not. We cannot find it either in UI, or in API.

Is there a way to get this information? Please note, it is not about Azure Language Studio (you can get model metrics there), but about Document Intelligence Studio which seems to have very limited options.

Thank you

Azure AI Document Intelligence
0 comments

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Alex Burlachenko 13,330 Reputation points Volunteer Moderator
    2025-08-04T08:41:05.7433333+00:00

    Rob hi,

    when u train a custom model, the metrics exist but they're tucked away in the training result. u gotta use the api to fetch them. after training, make a get request to the 'get operation details' endpoint. look for the 'analyzeResult' field it holds the juicy stuff like precision, recall, and f1.

    a quick curl example to grab the training result

    curl -X GET "https://{your-endpoint}/formrecognizer/documentModels/{modelId}/operations/{operationId}?api-version=2023-07-31" -H "Ocp-Apim-Subscription-Key: {your-key}"

    also, if u're using python, the sdk wraps this nicely. just call get_operation_details and dig into the response.

    try logging the entire api response, u might spot them hiding in a nested field.

    this might help in other tools too )) when apis don’t surface metrics upfront, the raw response often has gold buried inside. worth looking into the http traffic while training, tools like fiddler or postman can save u hours.

    and hey, if u're comparing models, keep an eye on the 'confidence' scores in the analyze result. they’re not full metrics, but they hint at performance.

    good luck with your models! hope this unblocks u ))

    Alex

    and "yes" if you would follow me at Q&A - personaly thx.
P.S. If my answer help to you, please Accept my answer

    https://ctrlaltdel.blog/

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.