The arena for breaking AI

A competition this size gets absolute beginners and professional red-teamers in the same hour, so the tracks are a difficulty ladder rather than a menu. Tutorial teaches prompt injection from zero. CBRNE raises the stakes to genuinely harmful elicitation, the category labs actually worry about. MATS x Trails is the advanced tier, indirect prompt injection against agents, where the attack rides in through the data instead of the chat box.

The fourth track was a partnership with PointCrow, a streamer with an audience that had never heard of red-teaming. His challenges wrapped the same mechanics in scenarios like botched surgeries and a murder mystery, and the countdown on the card did more for sign-ups than any banner we ran. Each track is just configuration on the same challenge engine, which is what let us stand a new one up in days.

The core loop is a split screen: the brief on the left, a live model on the right. The one shown here allows only emoji as input and asks you to make the model say an exact phrase, and at the time of the screenshot nobody in the world had solved it. Attempts are cheap by design. You talk to the model as long as you like, start over freely, and only submit for judging when you think you have it.

Judging is where most of my engineering time went. A single regex cannot decide whether a model was actually jailbroken, so I built a dynamic scoring algorithm: each challenge declares its own success criteria, from exact-match phrases with formatting stripped to model-graded evaluations of whether harmful content was really produced, and points scale with difficulty and token spend. Some challenges hand the model tools of its own, which means the judge has to reason about what the model did, not just what it said. Every verdict had money on it, so the same submission had to score the same way every time.

Launch days put thousands of participants on the platform simultaneously, all hammering the same models and all wanting to know where they stand right now. Standings, points, and completed-challenge counts stream over WebSockets, so a solve on one continent moves the leaderboard on another without anyone refreshing. Supabase carries the state; the realtime layer carries the mood.

Fairness at that scale is a product feature, not a policy page. Teams form inside the platform and score as one entry, usernames stay masked on public boards, and every submission keeps its full transcript so a disputed prize can be replayed end to end. The dashboard ties it together: pick a track, see your progress, and jump back in where you left off.