This morning the Heracles server (heracles.mxrouting.net, only for users on that server) experienced IMAP connection failures due to reaching configured limits in Dovecot. A total of 3,899 connections failed from 10:08:51 to 14:55:39 UTC. This is verified to have impacted customers across 1,465 domains on the server (total 7552 domains on Heracles). This was caused by two colliding factors:
A DDOS attack against the IMAP server.
Customers who violated our policy against warmup services flooding thousands of IMAP connections per minute using either instantly[.]io or a comparable service.
Resolutions:
Increased service limits.
Created monitoring system that catches this specific issue.
Created secondary monitoring system that ensures the previous monitoring system remains healthy.
Suspended customers using warmup services.
