BitMEX’s downtime on 13 March prompted by DDoS attack, not ‘hardware issues’

BitMEX’s Chief Technology Officer [CTO], Samuel Reed, had provided a gist of the attack that hit the exchange platform on 13 March. Now, BitMEX’s Chief Executive Officer [CEO], Arthur Hayes, has come out to update the community with more info, stating that the BitMEX team will be addressing the issue soon. He elaborated on the same in a blog post, one that expanded on the two DDoS attacks that took place on 13 March.

The CTO had previously noted that the exchange was under a botnet attack, the same it had warded off on 15 February. According to the aforementioned blog, the attackers had identified their target in February and were waiting for an opportune time to attack. The attackers deployed a “specially-crafted query to the Trollbox feature,” one that prompted the database’s query optimizer to run an inefficient query plan.

As Reed explained, even though the CPU reached 100% usage, 99.6% of the CPU was in IO wait [in an ideal state]. The team had previously misjudged this to be a hardware failure.

The same attack was launched at 12:56 UTC, but BitMEX was able to block the traffic in both cases and cleared the queue manually to resume operations. The blog went on to say,

“As part of our internal post mortem, the BitMEX team identified 156 accounts for which Last Price stops were clearly erroneously triggered on ETHUSD, caused by the unintended late processing of market orders during the first downtime at 02:16 UTC.”

It added,

“For each stop that triggered erroneously during this period, BitMEX calculated the delta to the printed Index Price and refunded the user. A total of 40.297 XBT was refunded.”

The attack that contributed to the exchange going offline for 45 minutes invoked many speculations from other crypto-exchanges and traders. However, BitMEX assured the community that it has been testing its oldest and vulnerable parts of the system to make sure such an attack does not repeat itself.

In fact, BitMEX was not the only exchange to undergo a DDoS attack recently. Earlier in February, OKEx and Bitfinex were both attacked within hours. At the time, OKEx CEO Jay Hao had claimed that the attacker was one of its competitions who “could not compete with OKEx’s competitive products and technologies.”