Meta's Intense AI Race to Outperform OpenAI's GPT-4 Revealed in Court Documents
The recently unsealed court documents from the case Kadrey v. Meta unveil an intense internal drive among Meta's top AI executives to surpass OpenAI’s GPT-4 model with their forthcoming Llama 3 AI model. "Honestly... Our goal needs to be GPT-4," stated Meta's Vice President of Generative AI, Ahmad Al-Dahle, in a message from October 2023 addressed to researcher Hugo Touvron. "We have 64k GPUs coming! We need to learn how to build frontier and win this race."
Competitive Drive Within Meta
While Meta has publicly released their AI models, internally, the focus was on surpassing proprietary competitors like Anthropic and OpenAI, both of which keep their models exclusive behind APIs. The benchmarks were models like Claude from Anthropic and GPT-4 from OpenAI.
Discussions within Meta also touched upon the French AI startup Mistral, with an apparent dismissive attitude. "Mistral is peanuts for us," Al-Dahle mentioned, suggesting confidence in Meta's superiority.
Pressure and Allegations
Aggression in Meta's approach is underlined in the messages, with executives expressing a strong drive to obtain the necessary data to effectively train the Llama models. One message read, "Llama 3 is literally all I care about." This fervor has led to allegations of cutting corners, particularly regarding the use of copyrighted data.
"Is there anything you wanted to use but couldn't for some stupid reason?" - Ahmad Al-Dahle
According to the messages, Touvron and Al-Dahle discussed improving Llama 3’s dataset and considered using LibGen, a collection of copyrighted works, to enhance performance.
Efforts to Close Performance Gaps
Meta CEO Mark Zuckerberg has articulated intentions to close the gap between Llama models and closed models from companies like OpenAI and Google. The pressure to achieve this milestone is evident from the internal communications.
In a letter from July 2024, Zuckerberg expressed optimism, stating, "This year, Llama 3 is competitive with the most advanced models and leading in some areas. Starting next year, we expect future Llama models to become the most advanced in the industry."
Llama 3's Competitive Release
Llama 3, released in April 2024, successfully competed against leading models from Google, OpenAI, and Anthropic, also outperforming other open competitors such as Mistral. However, the legality of the data used for training the model is under intense examination due to its potentially copyrighted nature, sanctioned for use by Zuckerberg himself.