AI Safety Fellow @Anthropic | PhD at University of Edinburgh | LLM Hallucinations | Clinical NLP | Opinions are my own. Personal page: https://aryopg.github.iohttps://bsky.app/profile/aryopg.bsky.socialhttps://bsky.app/profile/aryopg.bsky.social/post/3lo6uxauenk2wMMLU-Redux just touched down at #NAACL2025! 🎉 Wish I could be there for our "Are We Done with MMLU?" poster today (9:00-10:30am in Hall 3, Poster Session 7), but visa drama said nope 😅 If anyone's swinging by, give our research some love! Hit me up if you check it out! 👋02 May 2025 13:00 +0000at://did:plc:vf5vmhvup4pewx6ae5onnz4n/app.bsky.feed.post/3lo6uxauenk2whttps://bsky.app/profile/aryopg.bsky.social/post/3ll4ev5rhps24Today, I'm starting as an AI Safety Fellow @anthropic.com ! 🚀 Super excited to collaborate and learn from some of the brightest minds in AI! 🌟24 Mar 2025 08:51 +0000at://did:plc:vf5vmhvup4pewx6ae5onnz4n/app.bsky.feed.post/3ll4ev5rhps24https://bsky.app/profile/aryopg.bsky.social/post/3lcmv65g7fc2ySuper Cool work from Cohere for AI! 🎉 However, this highlights a concern raised by our MMLU-Redux team (arxiv.org/abs/2406.04127): **error propagation to many languages**. Issues in MMLU (e.g., "rapid intervention to solve ebola") seem to persist in many languages. Let's solve the root cause first? [contains quote post or other embedded content]06 Dec 2024 09:38 +0000at://did:plc:vf5vmhvup4pewx6ae5onnz4n/app.bsky.feed.post/3lcmv65g7fc2yhttps://bsky.app/profile/aryopg.bsky.social/post/3lba2ql4u6c2bI’ll be travelling to London from Wednesday to Friday for an upcoming event and would be very happy to meet up! 🚀 I'd love to chat about my recent works (DeCoRe, MMLU-Redux, etc.). DM me if you’re around! 👋 DeCoRe: arxiv.org/abs/2410.18860 MMLU-Redux: arxiv.org/abs/2406.0412718 Nov 2024 13:48 +0000at://did:plc:vf5vmhvup4pewx6ae5onnz4n/app.bsky.feed.post/3lba2ql4u6c2b