Recently Salesforce.com database in North America NA14, suffered a nasty outage which lasted over two days and a 4 hours’ worth of data loss due to system failure. Salesforce CEO Marc Benioff tried to play the data loss down saying the data loss happened during the US night time and hence data loss is not significant. However, Salesforce also did not reveal the number of customers affected by this outage.
As per Salesforce.com the root cause of the issue is power failure to the Washington, D.C. (WAS) datacenter which is caused due to a main critical board that tripped open a circuit breaker pair. The breakers are used to segment power from the datacenter ‘s universal power supply ring to direct into different rooms. This fault created an uncertain power condition, which led to a redundant breaker not closing to activate the backup feed because that electronic circuit breaker could not confirm the state of the problem board.
The data loss was caused by the file discrepancies in the NA14 database caused by database cluster failure. Additionally, it was found that the file discrepancies had been copied to the database in the WAS data center and that the local backup had not yet completed in the CHICAGO (CHI)data center. Because the file discrepancies had been copied to the standby copy of the database in the WAS data center, a site switch back to this copy of the NA14 instance in WAS was not a viable option to restore service to NA14 customers. Additionally, because the local backup of the instance in the CHI data center had not completed, this copy could not have been used to correct the file discrepancies and restore service. The root cause of the database failure was determined to be a latent firmware bug that was exposed due to increased traffic volume resulting from a combination of factors.
This was a nasty blow to Salesforce. Gartner VP Yefim Natis told InformationWeek in an interview, noting that Microsoft Azure and Amazon AWS have not been immune to outages either. “Outages happen to everyone. They vary in impact. This one was bad. People lost data. They didn’t have access to systems for a long period of time.”
This outage has affected customers on the US west coast and while service was restored on Wednesday after nearly a full day of down time, the instance continued to experience a degradation of service for quite some time. Salesforce Trust page showed the below message during this service degradation –
“The NA14 instance continues to operate in a degraded state. Customers can access the Salesforce service, but we have temporarily suspended some functionality such as weekly exports and sandbox copy functionality. In addition, search indexing may be delayed.”
While Salesforce stated that they have taken corrective measures to ensure such incidents do not occur in future; however, this incident has raised questions on Salesforce’s cloud capability with growing demand and aging storage.
Gartner VP Yefim Natis told “This happening once will be forgiven”. However, if this Salesforce outage is an indication of things to come for Salesforce’s systems, customers may start looking for other solutions.
Microsoft Dynamics CRM: Zero Data loss so far
Microsoft Dynamics CRM uses highly reliable Microsoft Azure datacenters and provides 99.9% financially backed SLA. Dynamics CRM Online has so far ZERO DATA LOSS and is compliant to EU Model clauses and many other regulatory certifications.
Microsoft Dynamics CRM Online stores FIVE (5) copies of data. TWO Synchronous for high availability in the same datacenter and TWO Asynchronous for backup in different datacenters within same continent and ONE nightly backup. Microsoft offers geographical redundancy to its customers by replicating data between two geographically distant sites in same continent.
The following World map shows Microsoft Dynamics CRM Online data centers across the globe.
This ensures higher degree of data integrity and data loss prevention in case of any outage or database failure. Microsoft also provides 5 layers of security protection, including EU and ISO27001 standards compliance. Microsoft Dynamics CRM Online is also IRAP certified for Australian Customers.