Proper Error Handling in API Management

@Edward Hakin

A couple of follow-up questions to clarify your situation further:

1. Could you share specific configurations for the inbound, backend, and outbound sections in your policy? This will help pinpoint any interactions affecting error handling.
2. Are you observing any patterns with the 400x errors (like specific API calls) that consistently fail to trigger the on-error section?

<policies>
    <inbound>
        <base />
        <authentication-managed-identity resource="https://cognitiveservices.azure.com" 		output-token-variable-name="managed-id-access-token" ignore-error="false" />
        <set-header name="Authorization" exists-action="override">
            <value>@("Bearer " + (string)context.Variables["managed-id-access-token"])</value>
        </set-header>
        <set-backend-service backend-id="[pool-ai-load-balancer]" />
    </inbound>
    <backend>
        <!--Set count to one less than the number of backends in the pool to try all backends until the backend pool is temporarily unavailable.-->
        <retry count="2" interval="0" first-fast-retry="true" condition="@(context.Response.StatusCode == 429 || (context.Response.StatusCode == 503 && !context.Response.StatusReason.Contains("Backend pool") && !context.Response.StatusReason.Contains("is temporarily unavailable")))">
            <forward-request buffer-request-body="true" />
            <choose>
                <when condition="@(context.Response.StatusCode == 429)">
                    <set-variable name="response429Body" value="@(context.Response.Body.As<string>(preserveContent: true))" />
                    <set-variable name="response429Reason" value="@(context.Response.StatusReason)" />
                </when>
            </choose>
        </retry>
    </backend>
    <outbound>
        <base />
    </outbound>
    <on-error>
        <base />
        <choose>
            <when condition="@(context.Response.StatusCode >= 400)">
                <choose>
                    <when condition="@(context.Variables.ContainsKey("response429Body"))">
                        <return-response>
                            <set-status code="429" reason="@((string)context.Variables["response429Reason"])" />
                            <set-body>@((string)context.Variables["response429Body"])</set-body>
                        </return-response>
                    </when>
                    <otherwise>
                        <return-response>
                            <set-status code="@(context.Response.StatusCode)" reason="@((string)context.LastError.Reason)" />
                            <set-body>@(
                                    context.Response.Body != null && context.Response.Body.As<string>(preserveContent: true) != ""
                                        ? context.Response.Body.As<string>(preserveContent: true)
                                        : (string)context.LastError.Message
                                )</set-body>
                        </return-response>
                    </otherwise>
                </choose>
            </when>
        </choose>
    </on-error>
</policies>

This more or less follows the outline / example from this git repo, but with some changes: https://github.com/Azure-Samples/AI-Gateway/blob/main/labs/backend-pool-load-balancing/policy.xml

No 400x errors fail to on-error, except 429 errors.

@Edward Hakin

Thank you for the detailed explanation. You’re correct that the current setup effectively handles 429 errors using custom logic in the <backend> and <on-error> sections.

For other 4xx or 5xx responses like 401 or 404, the <on-error> section isn’t triggered, as APIM considers these normal backend responses instead of policy errors.

To address these cases, could you try adding the following logic to the <outbound> section? This should let you inspect and modify error responses as needed:

<outbound>
    <base />
    <choose>
        <when condition="@(context.Response.StatusCode >= 400)">
            <set-body>@{
                var body = context.Response.Body.As<string>(preserveContent: true);
                return string.IsNullOrEmpty(body) ? "An error occurred." : body;
            }</set-body>
            <!-- Optionally log to Event Hub here -->
        </when>
    </choose>
</outbound>

This approach won't trigger <on-error>, but it does allow you to handle the response body for all backend errors. Please let us know if this is helpful.

@Edward Hakin

I wanted to follow up to see if you’ve had an opportunity to look over my earlier response. Please let me know if you have any further questions.