Assistant Professor @cislmu.bsky.social @lmu.dehttps://bsky.app/profile/valentinhofmann.bsky.socialhttps://bsky.app/profile/valentinhofmann.bsky.social/post/3mg5zsmq6qc2k📢 Life update 📢 After a wonderful time at @ai2.bsky.social, I've joined @cislmu.bsky.social at @lmu.de as a tenure-track assistant professor in NLP. Thrilled to be back in Europe and to start a lab in Munich's flourishing AI ecosystem! 🎉03 Mar 2026 14:58 +0000at://did:plc:rhp5dsdnpmp2yiuplenu67th/app.bsky.feed.post/3mg5zsmq6qc2khttps://bsky.app/profile/valentinhofmann.bsky.social/post/3m4iyodzg3s2lExcited to see our #COLM2025 paper on fluid benchmarking highlighted by @eval-eval.bsky.social! They are worth a follow if you are into LLM eval research. 🔬 [contains quote post or other embedded content]31 Oct 2025 17:25 +0000at://did:plc:rhp5dsdnpmp2yiuplenu67th/app.bsky.feed.post/3m4iyodzg3s2lhttps://bsky.app/profile/valentinhofmann.bsky.social/post/3m363zowdps2yCheck out this #EMNLP2025 paper led by @minhducbui.bsky.social and @carolin-holtermann.bsky.social showing dialect prejudice remains a major issue in current LLMs. Example: GPT-5 associates German dialect speakers with being uneducated and steers them toward stereotyped jobs (e.g., farmworkers). 👇 [contains quote post or other embedded content]14 Oct 2025 16:01 +0000at://did:plc:rhp5dsdnpmp2yiuplenu67th/app.bsky.feed.post/3m363zowdps2yhttps://bsky.app/profile/valentinhofmann.bsky.social/post/3lyxt5fx6pc24📢 New #COLM2025 paper 📢 Standard benchmarks give every LLM the same questions. This is like testing 5th graders and college seniors with *one* exam! 🥴 Meet Fluid Benchmarking, a capability-adaptive eval method delivering lower variance, higher validity, and reduced cost. 🧵 [contains quote post or other embedded content]16 Sep 2025 17:16 +0000at://did:plc:rhp5dsdnpmp2yiuplenu67th/app.bsky.feed.post/3lyxt5fx6pc24https://bsky.app/profile/valentinhofmann.bsky.social/post/3lu3qag5pbs2iAttending #ICML2025? Don't miss this TokShop panel, which will explore: 🔮 The Future of Tokenization 🔮 Featuring a stellar lineup of panelists - mark your calendar! ✨ [contains quote post or other embedded content]16 Jul 2025 15:28 +0000at://did:plc:rhp5dsdnpmp2yiuplenu67th/app.bsky.feed.post/3lu3qag5pbs2ihttps://bsky.app/profile/valentinhofmann.bsky.social/post/3lrbinxdnz22dLLMs can appear unbiased on the surface but still perpetuate racist views in subtle ways. What causes this discrepancy? 🔍 In our upcoming #ACL2025 paper, we find a pattern akin to racial colorblindness: LLMs suppress race in ambiguous contexts, leading to biased outcomes. [contains quote post or other embedded content]10 Jun 2025 18:13 +0000at://did:plc:rhp5dsdnpmp2yiuplenu67th/app.bsky.feed.post/3lrbinxdnz22dhttps://bsky.app/profile/valentinhofmann.bsky.social/post/3lor2mbuzqs2cThrilled to share that this is out in @pnas.org today! 🎉 We show that linguistic generalization in language models can be due to underlying analogical mechanisms. Shoutout to my amazing co-authors @weissweiler.bsky.social, @davidrmortensen.bsky.social, Hinrich Schütze, and Janet Pierrehumbert! [contains quote post or other embedded content]09 May 2025 18:29 +0000at://did:plc:rhp5dsdnpmp2yiuplenu67th/app.bsky.feed.post/3lor2mbuzqs2chttps://bsky.app/profile/valentinhofmann.bsky.social/post/3lmuog6ljds23Delighted there will finally be a workshop devoted to tokenization - a critical topic for LLMs and beyond! 🎉 Join us for the inaugural edition of TokShop at #ICML2025 @icmlconf.bsky.social in Vancouver this summer! 🤗 [contains quote post or other embedded content]15 Apr 2025 18:11 +0000at://did:plc:rhp5dsdnpmp2yiuplenu67th/app.bsky.feed.post/3lmuog6ljds23https://bsky.app/profile/valentinhofmann.bsky.social/post/3lkvr2kh6ns2zHumans store thousands of multi-word expressions like "of course" in their mental lexicon, but current tokenizers don't support multi-word tokens. Enter SuperBPE, a tokenizer that lifts this restriction and brings substantial gains in efficiency and performance! 🚀 Details 👇 [contains quote post or other embedded content]21 Mar 2025 17:40 +0000at://did:plc:rhp5dsdnpmp2yiuplenu67th/app.bsky.feed.post/3lkvr2kh6ns2zhttps://bsky.app/profile/valentinhofmann.bsky.social/post/3li3esnga4s2tExcited to share IssueBench, the most extensive benchmark for LLM political bias! 📊 We find surprising consistency across models, with notable differences in Qwen on China-related issues. All examined LLMs also show strong alignment with Democrat voters. More details below! 👇 [contains quote post or other embedded content]13 Feb 2025 19:01 +0000at://did:plc:rhp5dsdnpmp2yiuplenu67th/app.bsky.feed.post/3li3esnga4s2thttps://bsky.app/profile/valentinhofmann.bsky.social/post/3lgzc4uqjts2cGreat to see the International AI Safety Report highlight research on dialect prejudice, including our work on covert racism in LLMs! https://www.nature.com/articles/s41586-024-07856-5 [contains quote post or other embedded content]31 Jan 2025 05:43 +0000at://did:plc:rhp5dsdnpmp2yiuplenu67th/app.bsky.feed.post/3lgzc4uqjts2chttps://bsky.app/profile/valentinhofmann.bsky.social/post/3lcl4v2eans2b📢 New paper 📢 What generalization mechanisms shape the language skills of LLMs? Prior work has claimed that LLMs learn language via rules. We revisit the question and find that superficially rule-like behavior of LLMs can be traced to underlying analogical processes. 🧵05 Dec 2024 16:51 +0000at://did:plc:rhp5dsdnpmp2yiuplenu67th/app.bsky.feed.post/3lcl4v2eans2b