On September 4, Robinhood Chain had an outage, but not the one that was reported. At Walnut we’re building a new chain monitoring tool, and we used it to analyze the incident from public data alone: Ethereum blocks, Robinhood Chain blocks, and the transactions between them. This article walks through that analysis. The short version: the chain never stopped producing blocks. Instead, a fee spike on Ethereum outbid the chain’s batch poster, and for about 40 minutes the chain dropped a large share of the transactions users sent it.
That contradicts the news. Crypto news sites reported that Robinhood Chain stopped producing blocks for more than 14 minutes, starting at around 12:57 UTC, at the chain’s all-time fee peak (crypto.news, CryptoSlate). Those reports were wrong: the block timestamps show the chain kept producing about 10 blocks per second throughout, with no halt. Robinhood hasn’t communicated about the incident directly, but Arbitrum, whose Nitro stack the chain runs on, addressed it on X and confirmed two of our findings: the chain had no downtime, and its batch posting was delayed by the Ethereum blob market.
What actually happened started with a US jobs report, which set off a fee spike on Ethereum. Robinhood Chain’s batch poster, the process that ships the chain’s transaction data to Ethereum, was outbid for more than 8 minutes, then again for 5. The poster was already running flat out, so the backlog it built up turned into 40 minutes of dropped transactions, from 12:37 to 13:20 UTC. The typical busy app got through with about a fifth of its usual successful transactions.
The rest of this post shows why the halt reports were wrong, what the monitor saw, and what an alert at the right time would have changed. If you want an early preview of the monitor, reach out.
What the news reported
The reports said Robinhood Chain stopped producing blocks at 12:57 UTC and missed about 8,400 of them, but the block timestamps say otherwise.
Block 54,282,091 carries the timestamp 12:57:00 UTC, the reported start of the halt. We sampled every 10th block from 12:30 to 13:40. Over that window the chain produced 9.89 blocks per second, and no two consecutive samples are more than 3 seconds apart. A 14-minute halt would leave a gap of 840 seconds, but there is no such gap.
Robinhood Chain runs Arbitrum Nitro, and its sequencer, the node that orders transactions and produces blocks, stamps each block with the time it produced it, so an unbroken series of timestamps means production never paused. The “8,400 missed blocks” is arithmetic: 14 minutes at 10 blocks per second. The reports relied on a block explorer, and an explorer that stops indexing looks exactly like a halted chain.
What users hit was a chain that kept producing blocks while dropping much of what was sent to it. From 12:37 to 13:20, successful traffic collapsed. At the worst point the chain processed 21 million gas per second, against 56 million right after the incident. The median busy app kept 20% of its usual successful transactions, and none kept 80%. Swaps, bridges, trading terminals, wallets, and bots fell together.
What our monitor saw
We rebuilt the incident as a live-style monitor on public data and replayed September 1 to 14. The monitor watches four things: the Ethereum fee market Robinhood’s batches compete in, the batch poster itself, whether submitted transactions get into the chain, and what users experience. Each alert fires only on data that was available at that moment.
This is the timeline it produced, with the reported halt at 12:57 as the reference point:
- Before the incident: a “poster underbidding” watch was active from September 1, where our data starts, until Robinhood raised its tip on the evening of September 4. The market median ran 1,000× the poster’s bid the whole time.
- 12:34: a warning that the batch poster had gone 5 minutes without posting. That’s 22 minutes before the reported halt and 2 minutes before users started failing.
- 12:40: a watch that the poster had no headroom left to catch up.
- 12:45: an urgent alert that transactions were being dropped at intake, based on delayed Chainlink price updates. That’s 12 minutes before the first “halt” report, and across 14 days this rule fired only on September 4.
- 12:50: smart-contract wallet transactions failing in bulk.
- 13:00: successful traffic down to 32% of normal.
What happened, step by step
Here is the chain of events behind those alerts, in order.
Step 1: A jobs report set off a fee spike on Ethereum. At 12:30 UTC a far stronger-than-expected US jobs report moved crypto prices, and Ethereum trading bots bid up tips to trade first. The first full block after the release included one transaction paying 12.61 ETH in tips, and the base fee rose 20× by 12:54.
Step 2: Robinhood’s batch poster was outbid. Robinhood Chain’s batch poster ships the chain’s data to Ethereum as blobs, tipping just 0.001 gwei, and every poster bidding that low was shut out, Arbitrum One’s included. None of its batches landed from 12:29:47 to 12:38:23, then again from 12:42 to 12:48. Both times the poster raised its fee cap rather than its tip, and only the tip decides the order.
Step 3: The poster had no room to catch up. Other stalled posters recovered, but Robinhood’s was already at its maximum rate and running 4½ minutes behind since September 3. The stall had nowhere to drain, and the backlog peaked at about 18 minutes at 12:45.
Step 4: The chain dropped incoming transactions. Chainlink price updates landed late in one-minute steps (72, 132, 193, 253 seconds), meaning earlier broadcasts were silently dropped and re-sent, a pattern seen on no other day in two weeks of data. The main ERC-4337 bundler (which submits smart-contract wallet transactions) saw its failure rate jump from 13% to 77% for the same reason.
Step 5: Where the drop happened isn’t visible on-chain. Nitro’s documented cutoff, which refuses transactions when posting becomes unprofitable, is ruled out, since every block from 12:46 to 12:56 carries user transactions. The drop happened off-chain, in the sequencer’s 12-second queue timeout or the RPC layer in front of it.
Arbitrum’s statement points to the RPC layer. It says direct user transactions saw no delays, but infrastructure providers that rely on the chain’s data stream, the sequencer feed, were affected by the high number of feed subscribers. The two accounts fit together: most apps, wallets, and bots send transactions through RPC providers rather than to the sequencer directly, so “no delays for direct transactions” and 40 minutes of dropped transactions can both be true. The delayed Chainlink updates and the bundler failures in Step 4 show that transactions from real apps were lost either way.
What public data can’t show is how the posting backlog and the feed load interacted. On September 11 a similar Ethereum spike broke nothing, thanks to more headroom, a 250× higher tip, or both. The Robinhood or Arbitrum teams could settle both questions.
What an alert at the right time would have changed
Every step above showed up in public data before users noticed. Here is what an on-call engineer would have seen, and what they could have done.
- Days before: the poster is underbidding. The tip sat at the bottom of the blob market for at least three days, and even a 2 gwei tip costs only about 0.0003 ETH per batch. Robinhood made this change that evening (0.5 gwei, then 0.25 gwei on September 8), after the outage.
- 12:34: the poster has been silent for 5 minutes. OP Mainnet’s poster, also at 0.001 gwei, moved to 2–4 gwei within minutes and kept posting. This warning came 2 minutes before users started failing and 22 before the press called it a halt.
- 12:40: the poster has no headroom. With a 4½-minute backlog before the spike, this is what separates a stall that recovers from one that becomes an outage.
- 12:45: transactions are being dropped at intake. Chainlink’s delayed updates gave 12 minutes to post an incident and check the sequencer and RPC tier before the first report. The rule stayed silent on the September 11 control day.
One caveat: we set the thresholds with September 4 in view and checked them against two weeks of data, so read these lead times as a replay, not a track record.
Get early access
If you work on Robinhood Chain or Arbitrum Nitro, you hold the telemetry that closes the gap in Step 5, and we’d like to compare notes. The monitor behind this analysis is in early preview. Reach out for access.