NLP PhD @ USC Previously at AI2, Harvard mattf1n.github.iohttps://bsky.app/profile/mattf.nlhttps://bsky.app/profile/mattf.nl/post/3m3ftzr6xmk2mWe discovered that language models leave a natural "signature" on their API outputs that's extremely hard to fake. Here's how it works 🔍 📄 arxiv.org/abs/2510.14086 1/ https://arxiv.org/abs/2510.14086v117 Oct 2025 17:59 +0000at://did:plc:cdkorg27cbvytdyjomxpdsgs/app.bsky.feed.post/3m3ftzr6xmk2mhttps://bsky.app/profile/mattf.nl/post/3lschh2aecf22I didn't believe when I first saw, but: We trained a prompt stealing model that gets >3x SoTA accuracy. The secret is representing LLM outputs *correctly* 🚲 Demo/blog: mattf1n.github.io/pils 📄: arxiv.org/abs/2506.17090 🤖: https://huggingface.co/dill-lab/pils-32-llama2-chat-7b 🧑‍💻: github.com/dill-lab/PILS23 Jun 2025 20:49 +0000at://did:plc:cdkorg27cbvytdyjomxpdsgs/app.bsky.feed.post/3lschh2aecf22https://bsky.app/profile/mattf.nl/post/3lqs6p6ikkc2rIt appears that the only fonts with optical sizes that work with pdflatex are the computer/latin modern fonts. I would kill for a free pdflatex-compatible Times clone with optical sizes so my small text can look good in ArXiv/conference submissions.04 Jun 2025 16:05 +0000at://did:plc:cdkorg27cbvytdyjomxpdsgs/app.bsky.feed.post/3lqs6p6ikkc2rhttps://bsky.app/profile/mattf.nl/post/3lkhlxypqlk2jIf you are writing a paper for #colm2025 and LaTeX keeps increasing your line height to accommodate things like superscripts, consider using $\smash{2^d}$, but beware of character overlaps.16 Mar 2025 02:32 +0000at://did:plc:cdkorg27cbvytdyjomxpdsgs/app.bsky.feed.post/3lkhlxypqlk2jhttps://bsky.app/profile/mattf.nl/post/3lizubmokgk2mThis project was made feasible by the excellent open-source LLM training library @fairseq2.bsky.social; I highly recommend giving it a look! It made both SFT and DPO a piece of cake 🍰 [contains quote post or other embedded content]25 Feb 2025 21:58 +0000at://did:plc:cdkorg27cbvytdyjomxpdsgs/app.bsky.feed.post/3lizubmokgk2mhttps://bsky.app/profile/mattf.nl/post/3lizu3xpzvt2u🧵 Adapting your LLM for new tasks is dangerous! A bad training set degrades models by encouraging hallucinations and other misbehavior. Our paper remedies this for RAG training by replacing gold responses with self-generated demonstrations. Check it out here: https://arxiv.org/abs/2502.1025 Feb 2025 21:55 +0000at://did:plc:cdkorg27cbvytdyjomxpdsgs/app.bsky.feed.post/3lizu3xpzvt2uhttps://bsky.app/profile/mattf.nl/post/3ld4rjbeznk25Putting together an unofficial usc Beamer template, I noticed that the USC style guide lists 4 formats for “cardinal red” but each of them is different: PMS 201 C is #9D2235 CMYK: 7, 100, 65, 32 is #A1003D RGB: 135, 27, 30 is #991B1E HEX: #990000 Is this normal? The CMYK is especially egregious.12 Dec 2024 17:16 +0000at://did:plc:cdkorg27cbvytdyjomxpdsgs/app.bsky.feed.post/3ld4rjbeznk25https://bsky.app/profile/mattf.nl/post/3lctlybxeds2mIn Vancouver for NeurIPS but don't have Taylor Swift tickets? You can still spend the day going through our tutorial reading list: https://cmu-l3.github.io/neurips2024-inference-tutorial/ Tuesday December 10, 1:30-4:00pm @ West Exhibition Hall C, NeurIPS09 Dec 2024 01:43 +0000at://did:plc:cdkorg27cbvytdyjomxpdsgs/app.bsky.feed.post/3lctlybxeds2mhttps://bsky.app/profile/mattf.nl/post/3lcnou4wzws2sCurious about all this inference-time scaling hype? Attend our NeurIPS tutorial: Beyond Decoding: Meta-Generation Algorithms for LLMs (Tue. 1:30)! We have a top-notch panelist lineup. Our website: https://cmu-l3.github.io/neurips2024-inference-tutorial/06 Dec 2024 17:18 +0000at://did:plc:cdkorg27cbvytdyjomxpdsgs/app.bsky.feed.post/3lcnou4wzws2shttps://bsky.app/profile/mattf.nl/post/3lbuop2kyp52fThese folks have had a huge impact on my research [contains quote post or other embedded content]26 Nov 2024 18:39 +0000at://did:plc:cdkorg27cbvytdyjomxpdsgs/app.bsky.feed.post/3lbuop2kyp52fhttps://bsky.app/profile/mattf.nl/post/3lbkpcuh3a325This is niche but the LLM360 logo always reminds me of the 2014 iOS game Oquonie22 Nov 2024 19:23 +0000at://did:plc:cdkorg27cbvytdyjomxpdsgs/app.bsky.feed.post/3lbkpcuh3a325https://bsky.app/profile/mattf.nl/post/3lbjhgvnwls2yHottest new research challenge: find the lost LLM head! [contains quote post or other embedded content]22 Nov 2024 07:30 +0000at://did:plc:cdkorg27cbvytdyjomxpdsgs/app.bsky.feed.post/3lbjhgvnwls2yhttps://bsky.app/profile/mattf.nl/post/3lbipzrs67k26Everyone follow Sean! He's been working nonstop to perfect our upcoming NeurIPS tutorial [contains quote post or other embedded content]22 Nov 2024 00:31 +0000at://did:plc:cdkorg27cbvytdyjomxpdsgs/app.bsky.feed.post/3lbipzrs67k26https://bsky.app/profile/mattf.nl/post/3lb5psxsgms2kI made a map! Thank you to my 2019 self for providing the code https://github.com/mattf1n/Relief-Map17 Nov 2024 15:27 +0000at://did:plc:cdkorg27cbvytdyjomxpdsgs/app.bsky.feed.post/3lb5psxsgms2khttps://bsky.app/profile/mattf.nl/post/3laz2colzgc27Today I learned you can add a citation link to your GitHub repo. citation-file-format.github.io https://citation-file-format.github.io15 Nov 2024 18:52 +0000at://did:plc:cdkorg27cbvytdyjomxpdsgs/app.bsky.feed.post/3laz2colzgc27https://bsky.app/profile/mattf.nl/post/3lav3hj3fkk2hI’m proud of this tikz drawing I made today for our upcoming NeurIPS tutorial on decoding (our paper: arxiv.org/abs/2406.16838)14 Nov 2024 05:02 +0000at://did:plc:cdkorg27cbvytdyjomxpdsgs/app.bsky.feed.post/3lav3hj3fkk2hhttps://bsky.app/profile/mattf.nl/post/3larf45ube22aUSC NLP folks are on Bluesky! Follow my amazing colleagues here go.bsky.app/KUwSZ6W [contains quote post or other embedded content]12 Nov 2024 17:44 +0000at://did:plc:cdkorg27cbvytdyjomxpdsgs/app.bsky.feed.post/3larf45ube22ahttps://bsky.app/profile/mattf.nl/post/3l5vfrfzhyf2tJust landed in Philly for COLM where I’ll be presenting my work on extracting secrets from LLM APIs at the Wednesday afternoon poster sesh. Please reach out if you wanna hang and talk about sneaky LLM API hacks, accountability, and the geometry of LLM representations! arxiv.org/abs/2403.09539 https://arxiv.org/abs/2403.0953907 Oct 2024 03:51 +0000at://did:plc:cdkorg27cbvytdyjomxpdsgs/app.bsky.feed.post/3l5vfrfzhyf2thttps://bsky.app/profile/mattf.nl/post/3knlpsq4ju32y“Media companies aren't on their workers' side in just the same way that Amazon wasn't on publishers' side. The promise of a new, broader copyright [for AI] is like the promise of DRM: it's a prison pretending to be a fortress” https://pluralistic.net/2024/03/13/hey-look-over-there/13 Mar 2024 16:41 +0000at://did:plc:cdkorg27cbvytdyjomxpdsgs/app.bsky.feed.post/3knlpsq4ju32y