Daily AI Newsletter: High-performing language models on the energy of a lightbulb

In today’s email:

🏎️ Meet Sohu, the fastest AI chip of all time.
🔥 Google brings its Gemini AI to Gmail to help you write and summarize emails
🤯 ‘No Bot is Themselves Anymore:’ Character AI Users Report Sudden Personality Changes to Chatbots
🧰 10 new AI-powered tools and resources. Make sure to check the online version for the full list of tools.

Researchers power high-performing language models on the energy of a lightbulb by eliminating matrix multiplication.

Researchers from the University of California Santa Cruz, UC Davis, LuxiTech, and Soochow University have developed a new method to run AI language models without matrix multiplication, potentially reducing power consumption and reliance on GPUs. Detailed in a recent preprint paper, their approach involves creating a custom language model using ternary values and a new computational mechanism called a MatMul-free Linear Gated Recurrent Unit (MLGRU). This redesign allows the models to operate efficiently on simpler hardware like FPGA chips, drastically cutting energy use compared to traditional models that rely heavily on GPUs.

The researchers compared their MatMul-free model to a conventional Llama-2-style model across several benchmarks, demonstrating competitive performance with significantly lower power consumption and memory usage. Their optimized implementation showed up to a 61 percent reduction in memory consumption during training. Although the current models, with up to 2.7 billion parameters, are not as complex as state-of-the-art models like GPT-4, the study's findings suggest that scaling up the MatMul-free approach could yield similar or even superior performance levels with fewer resources.

This innovation could have profound implications for the accessibility and sustainability of AI technology, particularly for deployment on resource-constrained hardware such as smartphones. The researchers believe that with further development and investment, their method could support the creation of large-scale, high-performance language models that are both energy-efficient and cost-effective.

Harness your data for holistic customer journeys

The key to delivering better digital experiences? It’s a unified, end-to-end view of your customer journey—and your ability to turn data into actionable insights.

Dig into Amplitude's guide to delivering better digital experiences to learn how to:

Assess and build your data strategy.
Establish a single source of truth and democratize data.
Build a solid foundation for AI adoption.

Get the Guide

Meet Sohu, the fastest AI chip of all time.

Etched, a promising startup founded by Harvard dropouts Gavin Uberti and Chris Zhu, is developing a unique AI chip called Sohu, designed specifically for running transformer models. Unlike general-purpose GPUs, Sohu, an application-specific integrated circuit (ASIC) built using TSMC’s 4nm process, delivers superior inferencing performance while consuming less energy. Uberti claims that one Sohu server can replace 160 Nvidia H100 GPUs, making it a faster, cheaper, and more environmentally friendly option for businesses needing specialized AI chips.

Etched's focus on transformers, a dominant model architecture in generative AI, sets it apart from competitors. Transformers are the backbone of many advanced AI models, including OpenAI’s video-generating Sora and Google’s text-generating Gemini. By eliminating unnecessary hardware and software components, Sohu achieves streamlined performance and efficiency. Etched's approach has attracted significant investment, with the company recently closing a $120 million Series A funding round, bringing their total funding to $125.36 million.

Despite the competitive AI chip market and the potential for transformers to be surpassed by new models, Etched remains optimistic. The company plans to launch the Sohu Developer Cloud to allow customers to preview the chip’s capabilities, aiming to drive further sales. With unnamed customers already reserving millions in hardware, Etched hopes to carve out a significant niche in the AI chip industry. However, the challenges faced by previous AI chip startups highlight the uncertainties and high stakes in this rapidly evolving field.

OpenAI delays ChatGPT’s new Voice Mode

$Story illustration$

In May, OpenAI showcased a highly realistic "advanced voice mode" for its ChatGPT platform, promising a rollout to paying users within weeks. However, the launch has been delayed due to ongoing issues. OpenAI announced on its Discord server that the release, initially planned for late June, is now postponed to July. They are focusing on enhancing the model’s content detection capabilities and preparing infrastructure to handle real-time responses at scale.

The new Voice Mode might not be available to all ChatGPT Plus users until the fall, pending internal safety and reliability checks. This delay does not impact the rollout of other new features, such as video and screen sharing, demonstrated at OpenAI’s spring event. These features, including solving math problems from images and explaining device settings, are now accessible on both smartphone and desktop clients.

OpenAI's Voice Mode, which can understand and convey emotions, sparked controversy due to the default "Sky" voice resembling actress Scarlett Johansson's. Johansson's legal team is investigating the voice's development after she declined OpenAI's licensing offers. OpenAI has since removed the voice, denying unauthorized use or employing a soundalike.