News

Why Does Meta Insist on Open-Source AI? Look at Unix/Linux

• By David Ramel
• 07/24/2024

Nearly alone among the tech giants, Meta is still championing open-source foundational AI models as the way forward for AI development.

While cloud giants like Google, Microsoft and Amazon have been investing heavily in proprietary AI models and spewing out for-pay AI-powered wares to make up the money, Meta has been pushing for open-source AI models, which it believes will lead to more innovation and better AI systems.

Though it's bucking the proprietary trend, Meta yesterday reaffirmed its commitment to open source AI models with the release of Llama 3.1, saying it was in a class of its own in terms of flexibility, control and state-of-the-art capabilities rivaling the best proprietary models.

A few months ago Llama 3 cracked the top five the AI Leaderboard, the only non-proprietary model to do so.

None other than CEO Mark Zuckerberg used Meta's release of Llama 3.1 to highlight the company's commitment to open-source AI in the blog post, "Open Source AI Is the Path Forward."

In supporting the company's stance, he reminded everyone of the history of Unix/Linux.

In the early days of high-performance computing, the major tech companies of the day each invested heavily in developing their own closed source versions of Unix. It was hard to imagine at the time that any other approach could develop such advanced software. Eventually though, open source Linux gained popularity -- initially because it allowed developers to modify its code however they wanted and was more affordable, and over time because it became more advanced, more secure, and had a broader ecosystem supporting more capabilities than any closed Unix. Today, Linux is the industry standard foundation for both cloud computing and the operating systems that run most mobile devices -- and we all benefit from superior products because of it.

Zuckerberg believes AI will progress in the same way, and his lengthy post goes into great detail supporting open-source AI, explaining why it's good for developers and the world.

"The bottom line is that open source AI represents the world's best shot at harnessing this technology to create the greatest economic opportunity and security for everyone," he concluded.

Meta is not alone on its open source path, of course, though the "Big 3" cloud giants, AWS, Microsoft and Google, do favor proprietary models and associated software (though not exclusively). Currently the industry shakes out something like this in terms of open-source/proprietary AI models and associated software:

Tech Companies Championing Open-Source AI Models

• Meta: A leading advocate for open-source AI, releasing models like Llama.
• Hugging Face: A platform dedicated to hosting and sharing open-source AI models and tools.
• Stability AI: Known for image generation models like Stable Diffusion, also releasing open-source LLMs.
• EleutherAI: A community-driven research lab focused on research and developing open-source AI models.
• NVIDIA: While not in the LLM business, it offers open-source AI tools and frameworks alongside its hardware.
• Apache Software Foundation: Hosts several open-source AI projects.
• Together AI: Known for RedPajama-INCITE-3B.
• Cerebras: Known for their Cerebras-GPT model.
• Alibaba: Offering its Qwen model as open source.
• Mistral: Actively releasing open-source models.
• BLOOM: Contributing to the open-source AI community.
• xAI: Engaging in open-source AI initiatives.
• EleutherAI: A grassroots collective that releases open-source models like GPT-Neo and GPT-J.
• Red Hat: Supports and contributes to open-source AI projects like TensorFlow and PyTorch.

Tech Companies Championing Closed Proprietary AI Models

• OpenAI: While releasing some open-source tools, primarily focused on proprietary models like GPT-3 and GPT-4.
• Google: Develops and deploys large proprietary AI models like Bard and LaMDA, while also shipping open-source projects like Gemma.
• Microsoft: Mostly collaborates with OpenAI on proprietary models, but did create some of its own -- like Turin-NLG -- while also working on related open-source projects like LMOps and JARVIS.
• Amazon/AWS: Created Titan proprietary model. Reports last year said it was working on an Olympus LLM. Offers AI services and provides models through services like SageMaker and Bedrock.
• Baidu: A Chinese tech giant with a focus on proprietary AI models like ERNIE and PaddlePaddle.
• Anthropic: Known for its proprietary model, Claude.

Open-Source Advantages
Typically, the open source backers cite these advantages of open-source AI models:

• Community-driven innovation: Rapid development and improvement through collective effort.
• Transparency and accountability: Open codebase allows for scrutiny and ethical evaluation.
• Accessibility and democratization: Wider availability to researchers, startups, and individuals.
• Cost-effective: Potential for lower development and deployment costs.
• Customization and flexibility: Tailoring models to specific needs without vendor lock-in.

Proprietary Advantages
The proprietary camp, meanwhile, typically lists benefits including:

• Intellectual property protection: Exclusive ownership of technology and algorithms.
• Revenue generation: Potential for significant profit through licensing or direct sales.
• Control and customization: Full authority over model development and deployment.
• Competitive advantage: Differentiation from competitors through unique capabilities.
• Data privacy and security: Enhanced protection of sensitive data.

As far as performance, one study determined that "Closed Models Outperform Open Models, at Staggering Cost."

In terms of generating foundation models, data from the 2024 AI Index report shows that Meta is right near the top, sandwiched between cloud giants Google and Microsoft and ahead of OpenAI.

Llama 3.1
Getting back to Llama 3.1, Meta listed these takeaways from the release:

• Meta is committed to openly accessible AI. Read Mark Zuckerberg's letter detailing why open source is good for developers, good for Meta, and good for the world.
• Bringing open intelligence to all, our latest models expand context length to 128K, add support across eight languages, and include Llama 3.1 405B—the first frontier-level open source AI model.
• Llama 3.1 405B is in a class of its own, with unmatched flexibility, control, and state-of-the-art capabilities that rival the best closed source models. Our new model will enable the community to unlock new workflows, such as synthetic data generation and model distillation.
• We're continuing to build out Llama to be a system by providing more components that work with the model, including a reference system. We want to empower developers with the tools to create their own custom agents and new types of agentic behaviors. We're bolstering this with new security and safety tools, including Llama Guard 3 and Prompt Guard, to help build responsibly. We're also releasing a request for comment on the Llama Stack API, a standard interface we hope will make it easier for third-party projects to leverage Llama models.
• The ecosystem is primed and ready to go with over 25 partners, including AWS, NVIDIA, Databricks, Groq, Dell, Azure, Google Cloud, and Snowflake offering services on day one.
• Try Llama 3.1 405B in the US on WhatsApp and at meta.ai by asking a challenging math or coding question.

"Until today, open source large language models have mostly trailed behind their closed counterparts when it comes to capabilities and performance," Meta said. "Now, we're ushering in a new era with open source leading the way. We're publicly releasing Meta Llama 3.1 405B, which we believe is the world's largest and most capable openly available foundation model. With more than 300 million total downloads of all Llama versions to date, we're just getting started."