MXroute - Notice history

All systems operational

Billing Panel - Operational

100% - uptime
May 2023 · 100.0%Jun · 100.0%Jul · 100.0%

Previous page

Next page

May 2023
Jun 2023
Jul 2023

DirectAdmin Panel - Operational

100% - uptime
May 2023 · 100.0%Jun · 100.0%Jul · 100.0%

Previous page

Next page

May 2023
Jun 2023
Jul 2023

IMAP - Operational

100% - uptime
May 2023 · 100.0%Jun · 100.0%Jul · 100.0%

Previous page

Next page

May 2023
Jun 2023
Jul 2023

SMTP - Operational

100% - uptime
May 2023 · 100.0%Jun · 100.0%Jul · 100.0%

Previous page

Next page

May 2023
Jun 2023
Jul 2023

Webmail - Operational

100% - uptime
May 2023 · 100.0%Jun · 100.0%Jul · 100.0%

Previous page

Next page

May 2023
Jun 2023
Jul 2023

Support - Operational

100% - uptime
May 2023 · 100.0%Jun · 100.0%Jul · 100.0%

Previous page

Next page

May 2023
Jun 2023
Jul 2023

Notice history

Jul 2023

Spam filters on Arrow server
  • Resolved
    Resolved

    This incident has been resolved.

  • Identified
    Identified

    Spam filters on the arrow.mxrouting.net server are not performing as desired after upgrading to the new system. More spam is coming in than is desirable. We are working on this.

Arrow Spam Filtering
  • Completed
    July 17, 2023 at 7:05 PM
    Completed
    July 17, 2023 at 7:05 PM

    Spam filtering on the Arrow server should be better and improved. An email will be sent to users on the Arrow server explaining further.

  • In progress
    July 17, 2023 at 6:48 PM
    In progress
    July 17, 2023 at 6:48 PM

    Spam filtering might be a bit "wonky" for a while as we perfect these improvements, but the worst result should be receiving an email that you prefer not to accept. Please do not open support tickets about spam received that shouldn't have been until this maintenance is closed.

  • Planned
    July 17, 2023 at 6:24 PM
    Planned
    July 17, 2023 at 6:24 PM

    We are expecting a temporary increase in inbound spam for users on the arrow.mxrouting.net server while we work on improvements to it's spam filtering.

Witcher Server
  • Resolved
    Resolved

    This incident has been resolved.

  • Investigating
    Investigating

    We are currently investigating an issue with the witcher.mxrouting.net server.

Jun 2023

Outbound Relays
  • Update
    Update

    This morning one of our outbound relay servers ran out of memory. This caused some outbound email to be stuck in queue for longer than intended. While we keep a hot spare ready for this particular server at all times, it was not engaged as the event did not trigger the right type of alert. We've fixed this and all of the delayed outbound mail is processing, likely done by the time this update is posted.

    Measures will be taken to ensure first that this does not happen again, and second that the length of the event is significantly reduced if we fail to prevent it from happening again.

  • Resolved
    Resolved

    This incident has been resolved.

  • Investigating
    Investigating

    We are currently investigating a possible issue with outbound relays.

Tuesday Server Maintenance
  • Completed
    June 09, 2023 at 1:27 AM
    Completed
    June 09, 2023 at 1:27 AM

    Fixing Crossbox and then done.

  • Planned
    June 09, 2023 at 1:09 AM
    Planned
    June 09, 2023 at 1:09 AM

    We are performing emergency maintenance on the tuesday.mxrouting.net server in order to prevent extended downtime tomorrow. Our datacenter provider for that server (the only server we have with that provider) has informed us that they are moving physical locations tomorrow, and that they expect a 3-4 hour outage. We're not doing that. We're migrating the server into our "home" datacenter in Virginia immediately.

    During this maintenance you can expect brief hiccups that appear to resolve themselves very quickly. We've gotten very good at migrating these servers in production, and being one of our smallest (in disk usage) servers, this should go over very smoothly.

    We will:

    • Disable all services on the server
    • Run one final rsync of data
    • Switch DNS
    • Bring services back online

    This will result in no loss of data, and only the most brief of outages. Typically an outage of this length goes unnoticed by our customers, this status page is merely a formality.

Banshee Server Issue
  • Resolved
    Resolved

    This incident has been resolved. Postmortem and email to customers will be a bit delayed while the details are worked out.

  • Update
    Update

    We're still performing an rsync of the data, and it's intended that the next step takes much less time than this one. When this is over we'll detail in a postmortem report what went wrong, how we'll prevent it from happening to such a degree in the future, and we'll talk about compensation (the latter will be in an email to the relevant customers).

  • Update
    Update

    The rsync of data to the new server is still ongoing. It's roughly 3/4 of the way done.

  • Update
    Update

    The fastest path to resolution right now is for us to migrate the Banshee server to another system. This is unfortunately going to take a significant number of hours, and any attempt to tell you how many hours that would be would have to be a lie, it simply can't be known right now. It'll take no less than the time it takes, and we'll make every effort to make that number as small as possible.

    The Banshee server is a legacy cPanel server, and this wasn't how we wanted to retire it's hardware. There is in fact no hardware issue, but for some reason the OS is hosed beyond reasonable repair. All data is intact, and all inbound emails should be held and received after this is complete.

  • Identified
    Identified

    Now it just keeps booting to BIOS config and it won't do anything else. BIOS config is strangely lacking in the proper options that it should contain, options which might help to overcome this. So we're sending datacenter techs back out to see just what in the hell. Everything should be fine soon, no reason to suspect that this will be a long term outage.

  • Update
    Update

    The server goes offline almost immediately after coming back online. This server lacks an IPMI, so we are having a KVM attached. With that said, while booted into rescue mode all disks appear fine, this does not appear to be a hardware issue. We just can't get any strong insight into it without a KVM.

  • Investigating
    Investigating

    Problem reappeared. We are currently investigating this incident.

  • Resolved
    Resolved

    This incident has been resolved.

May 2023

Acadia Issue
  • Resolved
    Resolved

    This incident has been resolved.

  • Investigating
    Investigating

    We are currently investigating an outage of the Acadia server. Please note that this ONLY applies to customers on the acadia.mxroute.com server.

May 2023 to Jul 2023

Next