Hey folks!

Over the last few weeks we’ve been cooking up something special, and we’re so excited that it’s out in the world for you to try! 

🙌 Introducing: HallOumi 

HallOumi - a hallucination detection model that provides per-sentence verification of any content with sentence-level scores, citations and explanations detailing why a particular response claim is grounded based on the provided context (document) or not.

👉 Try HallOumi today! 👈

Hallucinations are frequently cited as one of the most critical challenges in deploying generative models across both commercial and personal applications. At Oumi, we’re excited to be working to fix this.

To train HallOumi yourself, see our recipe on GitHub! We also open-sourced our interactive demo so you can test HallOumi at home.

👋 Say Hi IRL (aka Events)

✅ 🗓️ March 28th:  Seattle Startup Summit @ Seattle

Oussama, Manos, and Matthew (left to right in the picture above) had a great time chatting with folks at the Seattle Startup Summit a few weeks ago. 

⏭️ 🗓️ April 24th - April 28th: ICLR 2025 @ Singapore

Manos and Oussama will be attending ICLR. Let’s chat if you’re there!

⏭️ 🗓️ April 29th: Llamacon @ San Francisco 

Catch Matthew and William at LlamaCon! Let us know if you’ll be there, and be sure to say hi!

👀 Other releases you might have missed:

• 🦙 Get Started with Llama 4!

  • Meta’s new family of Llama 4 models is already available in Oumi!  Take a look at our quick walkthrough on how to fine-tune Llama 4 with Oumi! Jeremy will get you started in just a few lines of code.

    • We shared a breakdown on how you can easily fine-tune, evaluate, and run inference on Llama4 for chat/data synthesis using Oumi here, too!

• 🚀 Custom Evaluations Made Easy with Oumi!

  • In a recent blog post, we dive into how Oumi makes custom evaluations a breeze — no complex setups, no hassle. All you need is your data, a simple function to assess the model responses, and a quick config change to select the model you want to evaluate. Start creating custom evaluations today!

• 🤯 Need to label a dataset? Looking to run inference at scale?

  • In this walkthrough, Matthew covers Batch Prediction on the Oumi platform, demonstrating local inference with SmolLM, remote inference with Claude 3.7, and batch inference using Open AI's batch API!

• 📊 Evaluate any model against any benchmark with Oumi

  • Here is an overview of evaluating models using Oumi, as well as creating your own custom benchmarks.

• 🧑‍⚖️ How you can use LLMs as a judge!

  • Discover how to leverage LLM judges in Oumi for dataset exploration, custom evaluation creation, and quality assurance tasks.

📁 Resource Roundup:

A quick refresher on where to find everything you need based on your interests, as well as how to get involved in research and more:

🔥 Keep Up with our Latest Changes: The Oumi platform’s github repository is your one-stop-shop for the latest technical updates from Oumi.

✋ Sign Up for Research Projects: If you have a research idea, we’d love to chat. Sign up here for a 15-minute research discussion. 

We have two dedicated forums for research projects in our Discord:

• 💬 project-ideas: a board where anyone can share and discuss research ideas.

• 💬 active-research-projects: when a project is/becomes active, post it in here to discuss, get help etc.

‼️ We can sponsor resources for ideas and projects that have strong community backing and need more compute! Post your idea in project-ideas, and let us know what you need. 

📹 Weekly Walkthrough Tutorials: We launched a video tutorial series in which Matthew and team will do a detailed walkthrough of a key feature of the Oumi platform.

🗓️ Community Office Hours: We’re hosting weekly office hours to discuss research projects and the Oumi platform. Grab some time and let’s chat!

We know your inbox (and attention) is precious. Thank you for being part of the Oumi community and for reading our newsletter.

Here’s to shaping the future of open-source AI together,

The Oumi Team

Have questions or feedback? Reply to this email or drop into our Discord community. We’d love to hear from you!