Meta’s Llama 3.1 405B Model Outperforms GPT-4 in Leaked Benchmarks

KEYTAKEAWAYS

TABLE OF CONTENTS

• KEY TAKEAWAYS
• DISCLAIMER
• WRITER’S INTRO

CONTENT

Early benchmark data for Meta AI’s upcoming Llama 3.1 models, including the massive 405B parameter version, shows surprising performance gains. The open-source model potentially surpasses OpenAI’s GPT-4 in several key AI benchmarks.

In a significant development for the artificial intelligence community, leaked benchmarks for Meta AI’s upcoming Llama 3.1 405B model suggest it may outperform OpenAI’s GPT-4, the current industry leader. This revelation, posted on the LocalLLaMA subreddit, marks a potential milestone as the first instance of an open-source model surpassing a leading closed-source large language model (LLM).

Meta introduced Llama 3 in April 2024 as a new generation of open-source LLMs, with initial releases of 8B and 70B parameter models. The company announced plans for a more ambitious 400 billion parameter model, which is still in training. The leaked data includes benchmarks for the 8B, 70B, and the massive 405B parameter models.

The benchmarks show Llama 3.1 outperforming GPT-4 in several crucial areas, including GSM8K, Hellaswag, BoolQ, and various MMLU categories (humanities, STEM, and other). However, it lags in HumanEval and MMLU-social sciences tests, indicating room for improvement.

It’s important to note that these results reflect the base models’ performance. The upcoming instruction-tuned versions are expected to yield even better results across various benchmarks.

This development underscores the growing influence and capability of open-source AI initiatives. Meta’s commitment to the open-source ethos, as stated in their blog post launching Llama 3, aims to democratize access to advanced AI models and tap into the global developer community’s collective intelligence.

While GPT-5 may challenge Llama 3.1’s emerging dominance in the future, the impressive performance of this open-source model against GPT-4 highlights the potential for widespread innovation and enhancement in AI technology.

>> Also read: What is GPT-4o? OpenAI’s Most Advanced AI Model Yet

▶ Buy Crypto at BingX

Sign up to claim 5,000+ USDT in rewards & 20% off trading fees!