Notifications Hub Completely Down

Question

Notifications Hub Completely Down

Jason Brooks 10

Starting early this morning, we can't seem to make any requests to an Azure Notifications Hub. Any attempt to hit an endpoint results in a timeout.

I went to check the access policies tab in the Notifications Hub namespace to make sure there wasn't an issue with the connection string, but that entire tab returns "No results" under policies. Thinking someone may have accidentally deleted our connection policy, I went to create a new policy, but after creating it and refreshing, the tab still says "No results".

None of our customers are able to register new push devices or send notifications. Is this entire service currently down? The status page isn't indicating any issues.

Screenshot 2025-06-03 at 8.34.24 AM

Oleg Krymskyi 5 Reputation points

2025-06-03T13:02:38.55+00:00

I have the same problem
John Clark 0 Reputation points

2025-06-03T13:03:33.6+00:00

We are also experiencing this issue.
Lucas Baran 10 Reputation points

2025-06-03T14:34:00.19+00:00

We are having the same problem! so far no helpful response from Microsoft on timeline to the service being restored
Lucas Baran 10 Reputation points

2025-06-03T14:34:48.51+00:00

We are having the same problem!
Prabhavathi Manchala 2,470 Reputation points Microsoft External Staff Moderator

2025-06-03T14:55:00.1433333+00:00
Hi Jason Brooks,

You're seeing timeouts when trying to access Notification Hub endpoints. The Access Policies tab shows No results, even when you try to add a new one. Because of this, push notifications aren't working, and device registrations are failing.

This likely points to a backend issue with Notification Hub in your region, specifically with Access Policies (SAS keys), which are needed for authentication and authorization.

1). Verify if the issue is specific to your region using Azure CLI or PowerShell

Sometimes the Azure portal might not show data, even though it’s available through the CLI or REST API.

Try running this command in Azure CLI:

az notification-hub authorization-rule list \ --namespace-name <your-namespace> \ --resource-group <your-resource-group>

If it also returns nothing or times out, it likely means there’s a backend issue with your Notification Hub.

2). Try to Create and use a New SAS Policy Programmatically

To create a new SAS key, run this command in Azure CLI

az notification-hub authorization-rule create \ --namespace-name <your-namespace> \ --resource-group <your-resource-group> \ --notification-hub-name <your-hub-name> \ --name "MyNewPolicy" \ --rights Send Listen

Then, get the connection string with

az notification-hub authorization-rule keys list \ --namespace-name <your-namespace> \ --resource-group <your-resource-group> \ --notification-hub-name <your-hub-name> \ --name "MyNewPolicy"

Try using this connection string in your application to see if it restores functionality temporarily.

3). Check Azure Resource Health & Support Tickets

Go to Azure Service Health in the portal and check Resource Health for your Notification Hub and namespace. If it shows degraded or unavailable, raise a support request right away.

Help + support - Microsoft Azure

Please refer to the following documents for a better understanding.

Push notification service issues

Monitor notification hubs
Oleg Krymskyi 5 Reputation points

2025-06-03T17:36:48.7133333+00:00

@Prabhavathi Manchala It didn't help. Here is the screen with error I have:

Besides, we deployed our notification hub a month ago, and haven't touch it after that.

And suddenly have to be updated something though the CLI.

It seems like a problem on MS side
Prabhavathi Manchala 2,470 Reputation points Microsoft External Staff Moderator

2025-06-03T17:42:02.63+00:00

Hi,

If you're deploying through east us region,

It appears there is an ongoing outage affecting Azure Notification Hubs in the East US region. You can track the status here: Azure Status Page.

We've reached out to the product team for more details and will keep you updated here. Apologies for the inconvenience caused.
Lucas Baran 10 Reputation points

2025-06-03T19:01:51.4466667+00:00

Just got an update from Microsoft

Current Status: We are bringing up a new cluster. Once complete, it will undergo validation and transition to the active cluster to restore service functionality. We are updating our estimated time to mitigation to approximately 4 hours, though recovery may complete in closer to 3 hours if no additional issues arise. We will provide the next update within 60 minutes—or sooner if there are significant developments.

1 answer

Your answer

Oleg Krymskyi 5 Reputation points

2025-06-03T13:02:38.55+00:00

I have the same problem
John Clark 0 Reputation points

2025-06-03T13:03:33.6+00:00

We are also experiencing this issue.
Lucas Baran 10 Reputation points

2025-06-03T14:34:00.19+00:00

We are having the same problem! so far no helpful response from Microsoft on timeline to the service being restored
Lucas Baran 10 Reputation points

2025-06-03T14:34:48.51+00:00

We are having the same problem!
Oleg Krymskyi 5 Reputation points

2025-06-03T17:36:48.7133333+00:00

@Prabhavathi Manchala It didn't help. Here is the screen with error I have:

Besides, we deployed our notification hub a month ago, and haven't touch it after that.

And suddenly have to be updated something though the CLI.

It seems like a problem on MS side
Prabhavathi Manchala 2,470 Reputation points Microsoft External Staff Moderator

2025-06-03T17:42:02.63+00:00

Hi,

If you're deploying through east us region,

It appears there is an ongoing outage affecting Azure Notification Hubs in the East US region. You can track the status here: Azure Status Page.

We've reached out to the product team for more details and will keep you updated here. Apologies for the inconvenience caused.
Lucas Baran 10 Reputation points

2025-06-03T19:01:51.4466667+00:00

Just got an update from Microsoft

Current Status: We are bringing up a new cluster. Once complete, it will undergo validation and transition to the active cluster to restore service functionality. We are updating our estimated time to mitigation to approximately 4 hours, though recovery may complete in closer to 3 hours if no additional issues arise. We will provide the next update within 60 minutes—or sooner if there are significant developments.

Answer 1

Hi @Jason Brooks ,

Please confirm that your notification hubs are working okay now. Yesterday I tested mine in East US and it seems to be working fine. Below is latest update as of this writing:

What happened?

Between 06:52 UTC and 21:45 UTC on 03 June 2025, a platform issue impacted underlying service instances impacting Notification Hubs in the East US region. Customers experienced errors when sending notifications to recipients hosted in this region. Notifications or registrations attempted during the impact window will not be recovered, as the compute layer responsible for storing them was unavailable.

What do we know so far?

We identified that the service instances responsible for processing requests had become unhealthy. This unavailability of the compute layer prevented customer notifications from being processed correctly, resulting in the service disruption described above.

How did we respond?

06:00 UTC on 03 June 2025 – We received an alert via internal service telemetry indicating Notification Hub availability degradation in the East US region.

06:52 UTC on 03 June 2025 – Customer impact began.

07:00 UTC on 03 June 2025 – We identified an Active cluster as not being able to serve the traffic in the East US region.

08:30 UTC on 03 June 2025 – Our team's investigation found that the issue was with service instances on unhealthy cluster nodes. We engaged other internal teams to work on mitigating efforts to get the cluster running again.

13:00 UTC on 03 June 2025 – After further investigation, a parallel mitigation workstream was begun to build a new cluster. Work also continued to recover the previous cluster.

20:41 UTC on 03 June 2025 – The parallel cluster was deployed and transitioned to an active status to handle Notifications and traffic in East US. Customers should have experienced increasing success from this point. Teams monitored closely to validate complete mitigation.

21:45 UTC on 03 June 2025 – After a period of observation and confirmation from customers, we confirmed that Notification Hubs service requests had returned to pre-incident levels and customer impact was mitigated.

What happens next?

Our team will be completing an internal retrospective to understand the incident in more detail. We will publish a Preliminary Post Incident Review (PIR) within approximately 72 hours, to share more details on what happened and how we responded. After our internal retrospective is completed, generally within 14 days, we will publish a Final Post Incident Review with any additional details and learnings.

To get notified when that happens, and/or to stay informed about future Azure service issues, make sure that you configure and maintain Azure Service Health alerts – these can trigger emails, SMS, push notifications, webhooks, and more: https://aka.ms/ash-alerts

For more information on Post Incident Reviews, refer to https://aka.ms/AzurePIRs

The impact times above represent the full incident duration, so are not specific to any individual customer. Actual impact to service availability may vary between customers and resources – for guidance on implementing monitoring to understand granular impact: https://aka.ms/AzPIR/Monitoring

Finally, for broader guidance on preparing for cloud incidents, refer to https://aka.ms/incidentreadiness

Please click Accept Answer if the above was helpful.

Thanks.

-TP

Prabhavathi Manchala 2,470 Reputation points Microsoft External Staff Moderator

2025-06-04T08:55:37.2366667+00:00

Hello @Jason Brooks,

Just checking in to see if the provided answer helped. If it did, please click "Accept the answer” and Yes for the answer. So, it can benefit other community members reading this thread. If you have any further queries, do let us know.

Share via

Notifications Hub Completely Down

1 answer

Your answer