Columbia University Information Technology

Service Alerts

RESOLVED SERVICE ALERTS

Resolved Service Alerts

Resolved Service Alerts

  • Email - Cyrus
    12/09/2008 - 08:31pm to 12/12/2008 - 04:00pm

    December 11, 2008

    On December 9, one of our email servers suffered significant hardware failure.  Approximately 5% of Columbia email users may have problems accessing their email.  No mail has been lost, but visible symptoms include an empty inbox and no access to email folders.

    Affected users can access temporary folders to send and receive new email effective 11:00 a.m. today.  Mail that has been queued since yesterday will be delivered over the next few hours.  From Cubmail, select “Public Folders” from the drop-down menu to the right of “Open Folder.”  Under “Private,” select the mailbox with your username.  For instructions on accessing this folder from an email client other than Cubmail, please visit http://www.columbia.edu/acis/email/cyrus/privatepublicfolders.html or call the CUIT Help Desk at x41919.

    We are projecting full resolution and access to all email by tomorrow morning.  We apologize for this inconvenience and regret the impact of this outage for our affected customers.  For continued updates, please visit http://cuitalerts.columbia.edu/home.

  • Email - Cyrus
    12/11/2008 - 11:40am to 12/11/2008 - 01:29pm
    Mail for affected customers is now being delivered to an alternate folder.  This will allow these customers to read and respond to any new mail coming in as today at 11AM. Mail that has been queued since yesterday will not be available right away.  It will be delivered, out of order, over the next few hours. 

    These customers can find their mail under "Public Folders", in most mail clients.  In CubMail, on the upper right corner next to"Open Folder", they can click on the folder list dropdown.  At the bottom of the list is a "Public Folders" hierarchy, and within that a "private" subfolder (visible only to most customers affected by the outage).  Within private, there will be a mailbox with their username on it, e.g. "abc5432".  If they select that folder, it will bring up their new mail, and they can read it normally.

     

    Cubmail is currently the most effective means of getting new mail. For assistance with mail clients such as Outlook or Thunderbird please call our CUIT helpline at 212 854 1919

     

    We are projecting full resolution and access to all email by tomorrow morning.  We apologize for this inconvenience and regret the impact of this outage for our affected customers.

  • Email - Cyrus
    12/11/2008 - 03:09pm to 12/12/2008 - 04:00pm

    Access to most customers' mailboxes has been restored. Mailboxes can be accessed via Cubmail and Mail clients such as Outlook and Thunderbird as well.

     

    Please note the following:

     

    Some mailboxes are still not available.  This means ONLY that the mail server software can't see those mailboxes yet.  We are still restoring mailboxes, and more will appear as we continue to work.

     

    For some users the inbox is not available yet. New mail is still being delivered to Public Folders.private.<username> where <username> is each user's name.

     

    Customers should avoid creating new mail folders or move messages until  the recovery is completed.

  • Email - Cyrus
    12/12/2008 - 09:11am to 12/12/2008 - 04:10pm

    As of this morning we continue the work to repair the mailbox database and project to complete all repairs by this afternoon.

     

    Of the mailboxes that were not available last night, 90% have been fully recovered and are now accessible by customers via CubMail or mail clients such as Outlook and Thunderbird. Once repairs are complete we will resume normal delivery of messages to customers' to inboxes.

     

    Customers should avoid creating any new mail folders or moving email messages until all repairs are complete.

     

     

  • Email - Cyrus
    12/12/2008 - 05:07pm to 12/16/2008 - 12:20pm

    The restoration and repair of affected customers' email accounts was completed at 3:57pm EST 12/12/08.   Mail service has returned to its normal state.  All mail in temporary shared folders is now accessible in standard inboxes, where they should be.

    Affected customers will see messages delivered during the course of the outage to be out of order (clustered in a bunch, appearing to have been delivered sometime between 5pm Friday and 2am Saturday).  We apologize for, and regret, the impact that this extraordinary outage has had for affected customers.

    CORRECTION:  No mail for any affected customer is anticipated to be lost as a result of this outage. A previous version of this alert erroneously indicated that a very small number of customers may have experienced such a problem.  Apologies for the confusion.