Resolved

We have monitored the service for stability and this incident shall now be closed.

Monitoring

Email queues on the Broadcom infrastructure are back to normal levels. Customers sending and receiving emails will no longer experience email delays.

We are closely monitoring the email service and will provide update as soon as more information becomes available.

Update

Email queues on Broadcom infrastructure are dropping at a steady rate. Customers will start seeing previously queued emails delivering. We have engaged our platform provider to help us further with our investigation.

We will provide an update as soon as more information becomes available.

Update

We are starting to see queued emails delivering but at a slow rate. We are continuing to investigate the issue and also working on processing the queued emails at a higher speed. We will provide an update as soon as more information becomes available.

Update

We are continuing to investigate. This issue is only impacting a subset of our NAM infrastructure and it is already out of service. Emails currently queued on our infrastructure will attempt delivery once we identify and address the underlying cause. Please note that there is no impact to protection services, and all other aspects of our services remain functional.

We will provide an update as soon as more information becomes available.

Investigating

We are investigating a recurrence of this issue. A small subset of customers provisioned on NAM based infrastructure may experience delays sending or receiving emails. We will deliver queued email once service is restored. As a workaround, we have taken the impacted infrastructure out of service and as a result, new emails are no longer impacted and are delivering without any delays.

We will provide an update as soon as more information becomes available.

Monitoring

Email queues on the Broadcom infrastructure are back to normal levels. Customers sending and receiving emails will no longer experience email delays.

We are closely monitoring the email service and will provide update as soon as more information becomes available.

Identified

Email queues on Broadcom infrastructure are dropping at a steady rate. Customers will start seeing previously queued emails delivering without any issue.

We will provide an update as soon as more information becomes available.

Update

We are starting to see queued emails delivering but at a slow rate. We will provide an update as soon as more information becomes available.

Update

The incident has been narrowed to an operational issue with our service platform provider Amazon Web Services who have experienced power issue in the US-EAST-2 Region. See the following link for details:

https://health.aws.amazon.com/health/status

As a workaround, we have taken the impacted infrastructure out of service and as a result, new emails are no longer impacted and are delivering without any delays. Emails previously queued on the impacted infrastructure will be delivered once the issue is resolved.

We will provide an update as soon as more information becomes available.

Investigating

We are aware of a small subset of customers provisioned on NAM based infrastructure experiencing delays sending or receiving emails. We will deliver queued email following our retry schedule once service is restored.

We are handling this issue as a high priority and have engaged all appropriate teams at Broadcom. Please note that there is no impact to protection services, and all other aspects of our services remain functional.

We will provide an update as soon as more information becomes available.

Began at:

Affected components