Thats a wrap for ICML2025. Incredible to watch the space go from "What are you talking about" to "That's impossible" to "Hmmm thats very interesting" in just over a year. @tha_ajanthan @hmdolatabadi

If you claim to have seen this coming, you better have the manifold P&L to back it up

Noam tends to not exaggerate.

Totally agree - Flower labs another group actively publishing great stuff and now squarely focused on decentralised training. Should be a major datapoint for everyone still skeptical of the area - flower team is as legitimate as it gets and Nic Lane is pretty much top of the pinnacle in Federated Learning. Every single signal to me is we're about witness a massive academic tipping point into this area on the ML side. Not really even contrarian anymore its moved from that to just clearly early days of what is going to become a major and very impactful field.

From my experience, getting a paper on decentralized DL accepted to top-level conferences can be quite tough. The motivation is not familiar to many reviewers, and standard experiment settings don't account for the problems you aim to solve. Hence, I'm very excited to see companies like @PluralisHQ and @PrimeIntellect investing the effort to share their results and get them published at major conferences! IMO even preparing the submission forces you to be more rigorous about your experiments + outside feedback from reviewers helps you sharpen the paper's message.

Pluralis + Prime Intellect + Ambient flips Ethereum by 2030

Feel like meta closing models was very predictable. I explicitly said this would happen last year and explained why (from ).

Spoke 50 minutes straight to a packed room of cracked AI researchers at ICML, presenting work by @akashnet_, @PrimeIntellect, @gensynai, @NousResearch, @PluralisHQ, and @GoogleDeepMind. There is now an enormous interest in DeAI. Mission (Partially) Accomplished.

For people not familiar with AI publishing; there are 3 main conferences every year. ICML, ICLR and NeurIPS. These are technical conferences and the equivalent of journals in other disciplines - they are the main publishing venue for AI. The competition to have papers at these conferences is now at a ridiculous level, getting papers accepted is very hard, and there is a lot of concern about the review process which is quite noisey at this point. A strong paper with no flaws has around a 50% chance of being accepted, and typically a paper is submitted with reviewer changes several times until it is accepted. Despite all that, papers in these venues remain the primary stamp of legitimacy in AI world, and are probably still the primary career metrics for ML researchers (although this is weakening imo as so much of the research in the frontier labs is unpublished). Main Track papers are significantly different to workshop papers. The main track has intense, serious peer review. Workshop papers are for preliminary work, that give some indication of an interesting result, but are either not complete or the result is not significant enough for main track. They are only required to be reviewed by the workshop reviewer pool and they don’t appear in proceedings. Many great papers have first shown up in workshops (e.g. grokking) - but workshop and main track papers are fundamentally different things, with a fundamentally different level of impact. The only two companies in decentralised AI that have main track papers this year are @PrimeIntellect and Pluralis.

Using beautiful Grafana dashboards for everything internally, so much nicer than Tensorboard. Wandb still good but doesn't really work with decentralised training. Makes me wonder what the internal vis tooling is like in openai - must be incredible.