Label Studio 1.13: Refreshed Interface and New GenAI Evaluation Templates

We’re excited to announce the 1.13 release of Label Studio! This update includes a refreshed UI and some new Generative AI templates for you to use.

Introducing A Refreshed Label Studio Interface

You’ll probably notice that in this release, we’ve updated the colors and fonts used in our Label Studio interface, giving it a sleek new look while maintaining the same intuitive navigation you’re familiar with. All Label Studio tools, features, and settings are still in the same place, ensuring that this will be a smooth transition. But the new fonts and colors should make the interface more pleasant to look at and read. We hope you enjoy the new look and feel!

Templates for Evaluating LLM Performance

As more and more people are looking to use LLMs both in commercial and research applications, it’s becoming more and more important to evaluate LLM performance. After all, how do you know that the LLM is giving answers you can trust? And while we’ve announced some exciting new evaluation features in our enterprise-focused HumanSignal Platform, we’re also excited to bring you new templates for evaluating LLM responses in Label Studio Community Edition:

• Response Moderation - does the response from the LLM answer the prompt that has been given?
• Response Grading - how good is the response from the LLM?
• Side-by-Side Comparison - which model is better at answering the prompt provided?
• Evaluate RAG with Human Feedback - how well does the whole RAG pipeline perform based on human feedback?
• Evaluate RAG with Ragas - how well does the whole RAG pipeline perform according to Ragas metrics?

All of these templates allow you to leverage your models to automatically generate answers to the prompts in your dataset, while leveraging the power of a Human in the Loop to build confidence in the LLMs and their answers. You can also retrain the model in an active learning cycle to improve as you go.

For more about these new templates, you can read the blog post or check out the new template documentation.

Upcoming Webinar: Fine-Tuning Enterprise RAG Knowledge Bases with Label Studio, ChatGPT, and Ragas

If you’re interested in learning to evaluate RAG with Ragas, join HumanSignal’s Jo Booth for a hands-on workshop/webinar where he’ll show you how to use Label Studio, ChatGPT, and Ragas to evaluate and fine-tune an enterprise RAG knowledge base. The webinar is on August 15, 2024 at 12 EDT / 9AM PDT. You can register here.

Happy labeling!