SITCLOUD Email Outage 4.4.14

There is a current outage happening from 3AM on SITCLOUD, and was resolved at 9AM.

Emails will be slow coming in until queues completed. Any questions please call our office.

UPDATE: Email from upstream provider with cause, resolution and fix.

From: Peter Black
Sent: Friday, 4 April 2014 12:51 PM
To: cloud
Subject: Unplanned Outage – Exchange
Importance: High

Good morning

This morning 3 am Data Outsource experiences a major outage with the Exchange Services.

At 6 am the Exchange Team had fully identified and started work in rectifying this issue and had to make emergency changes and by 9 am this was available again. By 10am  the queue was clear and emails functioning as normal.

On behalf of my team and myself I strongly apologise for this service outage.

 

Why did this service fail?

As you may be aware be brought this service online in the last Month due to the closure of the Singapore platform with very little notice. Since then we have been working long hours to get this Service as Highly Available as possible. The Load balancer and Replication Service was pending Hardware which is 3 weeks late and due to arrive Monday. This should stop the 2 failures from happening again in future as further redundancy will be there.

 

What is planned to stop this issue in future?

A second load balancer due Monday and new hardware will double the redundancy of the system and strongly minimise the possibility of an outage.
If you have any further questions please feel free to contact me.

 

Thank You
Wayne Taylor – CIO

 

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.