PhD @ UniMelb NLP, with a healthy dose of MT Based in 🇮🇩, worked in 🇹🇱 🇵🇬 , from 🇫🇷https://bsky.app/profile/rapha.devhttps://bsky.app/profile/rapha.dev/post/3mgzc37pr522ecool analogy for AI & jobs: - when ATMs came, the number of bank tellers rose, bc ATMs lowered the cost of running bank branches, so more branches opened - but in the 2010s, the number of bank tellers plummetted, bc mobile banking made branches unnecessary14 Mar 2026 11:08 +0000at://did:plc:mrcvpcuihersd3cpdcf77kvf/app.bsky.feed.post/3mgzc37pr522ehttps://bsky.app/profile/rapha.dev/post/3mgzbumxwfk2ea *great* chart to teach confounding variables14 Mar 2026 11:05 +0000at://did:plc:mrcvpcuihersd3cpdcf77kvf/app.bsky.feed.post/3mgzbumxwfk2ehttps://bsky.app/profile/rapha.dev/post/3m4fxehxdn22yThis is some legit really impressive work!! [contains quote post or other embedded content]30 Oct 2025 12:24 +0000at://did:plc:mrcvpcuihersd3cpdcf77kvf/app.bsky.feed.post/3m4fxehxdn22yhttps://bsky.app/profile/rapha.dev/post/3m3gzwc7gu22qWhoa the #WMT25 results on MT Evaluation are wild! ChrF outperforms pretty much all neural metrics 🙀18 Oct 2025 05:17 +0000at://did:plc:mrcvpcuihersd3cpdcf77kvf/app.bsky.feed.post/3m3gzwc7gu22qhttps://bsky.app/profile/rapha.dev/post/3luxh45k4rc2vin Vienna for ACL, presenting Tulun, a system for low-resource in-domain translation, using LLMs Tuesday @ 4pm Working w 2 real use cases: medical translation into Tetun 🇹🇱 & disaster relief speech translation in Bislama 🇻🇺27 Jul 2025 15:59 +0000at://did:plc:mrcvpcuihersd3cpdcf77kvf/app.bsky.feed.post/3luxh45k4rc2vhttps://bsky.app/profile/rapha.dev/post/3lr32nv4zyk2bCool paper, at the intersection of grammar and LLM interpretability. I like that they use linguistic datasets for their experiments, then get results that can contribute to linguistics as a field too! (on structural priming vs L1/L2) [contains quote post or other embedded content]08 Jun 2025 04:47 +0000at://did:plc:mrcvpcuihersd3cpdcf77kvf/app.bsky.feed.post/3lr32nv4zyk2bhttps://bsky.app/profile/rapha.dev/post/3lpxi3suvus25Our paper on who uses tetun.org, and what for, got published at the LoResMT 2025 workshop! An emotional paper for me, going back to the project that got me into a machine learning PhD in the first place.25 May 2025 01:11 +0000at://did:plc:mrcvpcuihersd3cpdcf77kvf/app.bsky.feed.post/3lpxi3suvus25https://bsky.app/profile/rapha.dev/post/3lonou2hhqc25My favourite ICLR paper so far. Methodology, findings and their implications are all very cool. In particular Fig. 2 + this discussion point: [contains quote post or other embedded content]08 May 2025 10:20 +0000at://did:plc:mrcvpcuihersd3cpdcf77kvf/app.bsky.feed.post/3lonou2hhqc25https://bsky.app/profile/rapha.dev/post/3lo6wkrhugk27Incredible paper, finding that large companies can game the LMArena through statistical noise (via many model submissions), over-sampling of their models, and overfitting to Arena-style prompts (without real gains on model reasoning) The experiments they run to show this are pretty cool too! [contains quote post or other embedded content]02 May 2025 13:28 +0000at://did:plc:mrcvpcuihersd3cpdcf77kvf/app.bsky.feed.post/3lo6wkrhugk27https://bsky.app/profile/rapha.dev/post/3lnhw63f6sk23Cool summary of issues with multilingual LLM eval, and potential solutions! If you're doubtful of all these non-reproducible evals on translated multiple choice questions, this paper is for you [contains quote post or other embedded content]23 Apr 2025 09:50 +0000at://did:plc:mrcvpcuihersd3cpdcf77kvf/app.bsky.feed.post/3lnhw63f6sk23https://bsky.app/profile/rapha.dev/post/3lmjjecph2k25GlotEval - a unified framework for multilingual eval of LLMs, on 7 different tasks, by @tiedeman.bsky.social @helsinki-nlp.bsky.social Just wish it supported eval of closed models (e.g. through LiteLLM?) https://github.com/MaLA-LM/GlotEval11 Apr 2025 07:41 +0000at://did:plc:mrcvpcuihersd3cpdcf77kvf/app.bsky.feed.post/3lmjjecph2k25https://bsky.app/profile/rapha.dev/post/3lcjp4hmydk2jOur paper on generating bilingual example sentences with LLMs got best paper award @ ALTA in Canberra! arxiv.org/abs/2410.03182 We work with French / Indonesian / Tetun, find that annotators don't agree about what's a "good example", but that LLMs can align with a specific annotator.05 Dec 2024 03:12 +0000at://did:plc:mrcvpcuihersd3cpdcf77kvf/app.bsky.feed.post/3lcjp4hmydk2jhttps://bsky.app/profile/rapha.dev/post/3lbk3gb4aa22rAnother example of why we need evals that take clinical risk into account when training NLP models for health https://slator.com/openais-whisper-faces-bad-press-for-hallucinations-in-healthcare-transcription/ https://slator.com/openais-whisper-faces-bad-press-for-hallucinations-in-healthcare-transcription/?utm_source=Slator+Newsletter&utm_campaign=feb8ae2b68-Slator+Weekly+-+Dec+11%2C+2020_COPY_01&utm_medium=email&utm_term=0_28e0ad4ca5-feb8ae2b68-165558614&mc_cid=feb8ae2b6822 Nov 2024 13:27 +0000at://did:plc:mrcvpcuihersd3cpdcf77kvf/app.bsky.feed.post/3lbk3gb4aa22rhttps://bsky.app/profile/rapha.dev/post/3lbeevrd7xc2hthis guy lives rent free in my hippocampus20 Nov 2024 07:01 +0000at://did:plc:mrcvpcuihersd3cpdcf77kvf/app.bsky.feed.post/3lbeevrd7xc2hhttps://bsky.app/profile/rapha.dev/post/3lbeeluvuac2hIs productionisation (and move to gimmicks like CoT in o1-preview) at OpenAI and Anthropic a sign that scaling laws are slowing? And if so, where are we headed in LLMs? Slightly pretentious but enjoyable read: https://www.generalist.com/briefing/the-bitter-religion#the-reckoning20 Nov 2024 06:55 +0000at://did:plc:mrcvpcuihersd3cpdcf77kvf/app.bsky.feed.post/3lbeeluvuac2h