Research in Motion (RIM) has issued an official statement that seeks to explain why there was a significant problem with its wireless email system earlier this week.
According to this company, many of its North American customers experienced delays sending and receiving messages over a period of about half a day.
The company’s statement on this issue says:
RIM has determined that the incident was triggered by the introduction of a new, noncritical system routine that was designed to provide better optimization of the system’s cache. The system routine was expected to be nonimpacting with respect to the real-time operation of the BlackBerry infrastructure, but the pretesting of the system routine proved to be insufficient.
Problems on Problems
But even after its employees had diagnosed the reason for the outage, RIM’s problems didn’t end there. They were unable to immediately fix the primary email system, so it was decided to switch to a a back-up system.
As RIM’s statement says:
After isolating the resulting database problem and unsuccessfully attempting to correct it, RIM began it’s fail-over process to a backup system
Although the backup system and fail-over process had been repeatedly and successfully tested previously, the fail-over process did not fully perform to RIM’s expectations in this situation and therefore caused further delay in restoring service and processing the resulting message queue.
Hope for the Future
The company apologized, and while it didn’t promise something like this wouldn’t happen again, it says it is doing what it can to prevent it.
In RIM’s own words:
RIM apologizes to customers for inconvenience resulting from the service interruption. RIM’s root cause analysis and system enhancement process with respect to this incident is ongoing, and RIM has already identified certain aspects of its testing, monitoring and recovery processes that will be enhanced as a result of the incident and in order to prevent recurrence.