Back to France after some time in sunny California and happy Copenhagen. Mistral, Photoroom, Meta (xformers, FairScale, R&D), EyeTribe (acq) Mostly writing around AIhttps://bsky.app/profile/bentheegg.bsky.socialhttps://bsky.app/profile/bentheegg.bsky.social/post/3luvknhtolk2p"The Serial Scaling Hypothesis" (https://arxiv.org/abs/2507.12549v1, Liu et al) is interesting I think, not as new as it completely looks (autoregressive models are used serially, models have depth,..) but feels like a good formalization and intuition as of where current GPT based LLMs will typically fail26 Jul 2025 21:57 +0000at://did:plc:vgqqpolbhvqsasdfplrrbcsg/app.bsky.feed.post/3luvknhtolk2phttps://bsky.app/profile/bentheegg.bsky.social/post/3lua6f4orr22fIn the coming age of agents, I think vibe coding will die out, same lasting power as prompt engineering. For things LLMs excell at, you might as well stick to higher level directives and let it own the work, Claude Code is a good example. 1/218 Jul 2025 09:52 +0000at://did:plc:vgqqpolbhvqsasdfplrrbcsg/app.bsky.feed.post/3lua6f4orr22fhttps://bsky.app/profile/bentheegg.bsky.social/post/3ltt7ulsrjs2jStill not a lot of ML talk on bsky (at least in my feed), hence paper Sunday: my two most interesting recent reads - H Nets arxiv.org/abs/2507.07955 - Energy Based Transformers arxiv.org/abs/2507.02092 https://arxiv.org/abs/2507.0795513 Jul 2025 06:14 +0000at://did:plc:vgqqpolbhvqsasdfplrrbcsg/app.bsky.feed.post/3ltt7ulsrjs2jhttps://bsky.app/profile/bentheegg.bsky.social/post/3ltqvesbhok2mLittle bit of personal news, shared in other circles already: I'm moving to Mistral in August, after three years at Photoroom. I'm really proud of what we built in the ML team with relatively limited means, lasting SOTA on the existing foundations (saliency segmentation) while growing a lot on genAI12 Jul 2025 08:01 +0000at://did:plc:vgqqpolbhvqsasdfplrrbcsg/app.bsky.feed.post/3ltqvesbhok2mhttps://bsky.app/profile/bentheegg.bsky.social/post/3lsri5a7buk2nAlex Nichol is one the rare many-hits researchers of the field, with on top of that a track record of practical models which affect the public/ship. That Meta wouldn't target him is pretty rich [contains quote post or other embedded content]29 Jun 2025 20:12 +0000at://did:plc:vgqqpolbhvqsasdfplrrbcsg/app.bsky.feed.post/3lsri5a7buk2nhttps://bsky.app/profile/bentheegg.bsky.social/post/3lseukquinc27Automatically generate a fused megakernel in triton.. diving in, but if it works half as well as it reads it would already be quite something. Aligns with torch.compile of course https://github.com/mirage-project/mirage/tree/mpk24 Jun 2025 19:49 +0000at://did:plc:vgqqpolbhvqsasdfplrrbcsg/app.bsky.feed.post/3lseukquinc27https://bsky.app/profile/bentheegg.bsky.social/post/3lsbd675ob22dSharing that Photoroom open sourced _Dataroom_, as promised some time ago. Accompanying blog post and mini thread https://github.com/photoroom/dataroom https://www.photoroom.com/inside-photoroom/building-a-modern-data-stack 1/N23 Jun 2025 10:00 +0000at://did:plc:vgqqpolbhvqsasdfplrrbcsg/app.bsky.feed.post/3lsbd675ob22dhttps://bsky.app/profile/bentheegg.bsky.social/post/3lrx3eugd722iStill haven't tried Cursor, but I recently moved from Github Copilot to Continue with Codestral (free API), and it's absurd how much better Continue with Codestral is (vs. Copilot with expensive and slow models). Made me realize that there is zero moat in this field, at least for Copilot.19 Jun 2025 08:14 +0000at://did:plc:vgqqpolbhvqsasdfplrrbcsg/app.bsky.feed.post/3lrx3eugd722ihttps://bsky.app/profile/bentheegg.bsky.social/post/3lrkjvxgdek2aSelf adapting language models, still early but fascinating prospects. There's a dimensionality curse of course: since the dimensions the LLM can touch per generated token are very small as such, needs a massive lever / dimension reduction to be able to self improve. arxiv.org/pdf/2506.1094314 Jun 2025 08:29 +0000at://did:plc:vgqqpolbhvqsasdfplrrbcsg/app.bsky.feed.post/3lrkjvxgdek2ahttps://bsky.app/profile/bentheegg.bsky.social/post/3lrgp5zpf5s2wGreat write up of AMD new offerings, catching the nvidia train on the software side it seems. 3x speedup on MI300X since release, was required but still great to grab https://morethanmoore.substack.com/p/amds-ai-future-is-rack-scale-helios?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F26243d18-3049-4e78-8be0-ace5128f5e62_4000x2250.jpeg&open=false12 Jun 2025 19:53 +0000at://did:plc:vgqqpolbhvqsasdfplrrbcsg/app.bsky.feed.post/3lrgp5zpf5s2whttps://bsky.app/profile/bentheegg.bsky.social/post/3lqsnvauxvs2idatago now available with webdataset compatibility (streaming tarballs, so you get the data as it arrives). Just pip install datago and give it a whirl if you'd like ? Speed without the dataloader processes, and typical ViT/DiT pre-processing baked in. example code here https://github.com/Photoroom/datago/blob/main/python/benchmark_webdataset.py04 Jun 2025 20:37 +0000at://did:plc:vgqqpolbhvqsasdfplrrbcsg/app.bsky.feed.post/3lqsnvauxvs2ihttps://bsky.app/profile/bentheegg.bsky.social/post/3lqgzgnkpcs2nGreat link with https://bsky.app/profile/dbreunig.bsky.social/post/3lqfeiqn3ks2f [contains quote post or other embedded content]31 May 2025 05:31 +0000at://did:plc:vgqqpolbhvqsasdfplrrbcsg/app.bsky.feed.post/3lqgzgnkpcs2nhttps://bsky.app/profile/bentheegg.bsky.social/post/3lqdpnni5ls2wSageAttention3 paper reads great, and looks like B200s just got a good value boost. QAT or PTQ-free use of FP4, I expected this to be much more complicated or come later to be honest. Only at the attention level and LLMs are most often MLP bottlenecked but stil arxiv.org/abs/2505.11594 https://arxiv.org/abs/2505.1159429 May 2025 21:58 +0000at://did:plc:vgqqpolbhvqsasdfplrrbcsg/app.bsky.feed.post/3lqdpnni5ls2whttps://bsky.app/profile/bentheegg.bsky.social/post/3lovrfrfkz225Striking in retrospect how some leaders position at the time of the first atomic bomb (“other countries won’t get it”), Truman for instance, then space age, then now AI age, rhyme. Same as before, I think a bunch of places are bound to be SOTA AI, ideas and progress cannot be pinned to a wall11 May 2025 15:27 +0000at://did:plc:vgqqpolbhvqsasdfplrrbcsg/app.bsky.feed.post/3lovrfrfkz225https://bsky.app/profile/bentheegg.bsky.social/post/3lotwueuyrs2uApple readying in house server grade AI processors feels quite bizarre to me: Apple Intelligence has been underwhelming so far, so current status is probably far from end game. But lowering the code now into hardware limits future flexibility, bad timing ? https://www.tomshardware.com/pc-components/cpus/apple-reportedly-readies-baltra-processors-for-ai-servers10 May 2025 22:00 +0000at://did:plc:vgqqpolbhvqsasdfplrrbcsg/app.bsky.feed.post/3lotwueuyrs2u