Author: Bao Yilong
Source: Hard AI
This week, global headlines were dominated by tariff issues, but the tech industry's focus was on the intensive actions in the AI field.
Over the weekend, Meta unexpectedly released the Llama 4 series late at night, claiming &#34;native multimodal + tens of millions of context windows,&#34; and for the first time disclosed a lightweight version that can run on a single H100. Previously, OpenAI announced that the O3 and O4-mini models would be launched in a few weeks, while confirming that GPT-5 has been delayed for several months due to technical integration and computing power deployment issues.
DeepSeek, in collaboration with a research team from Tsinghua University, released a new paper this week on Scaling during inference, proposing a learning method called Self-Principle Commentary Tuning (SPCT) and constructing the DeepSeek-GRM series models. By combining meta-reward models to achieve scaling during inference, performance approaches that of the 671B large model, suggesting that DeepSeek R2 is nearing completion.
<h2>Meta Strongly Launches Llama 4, with Multimodal and Ultra-Long Context as Highlights</h2>
On Saturday, Meta officially released the Llama 4 series models. The entire Llama 4 series adopts a mixture of experts (MoE) architecture and achieves native multimodal training, completely bidding farewell to the era of Llama 3's pure text models. The released models include:
<blockquote>
<ul>
<li>
Llama 4 Scout (17B active parameters, 109B total parameters, supports 10 million+ Token context window, can run on a single H100 GPU);
</li>
<li>
Llama 4 Maverick (17B active parameters, 400B total parameters, context window 1 million+, performance surpasses GPT-4o and Gemini 2.0 Flash);
</li>
<li>
and the powerful Llama 4 Behemoth preview (288B active parameters, 2 trillion total parameters, trained using 32,000 GPUs and 30 trillion multimodal Tokens).
</li>
</ul>
<img src="https://imageproxy.pbkrs.com/https://wpimg-wscn.awtmt.com/d122b5df-a9c2-457d-ba6f-72c09425de54.png?x-oss-process=image/auto-orient,1/interlace,1/resize,w_1440,h_1440/quality,q_95/format,jpg" alt="" original-src="https://imageproxy.pbkrs.com/https://wpimg-wscn.awtmt.com/d122b5df-a9c2-457d-ba6f-72c09425de54.png"/>
</blockquote>
The newly announced Llama 4 Maverick and Llama 4 Scout will be open-source software. However, the new license for Llama 4 imposes certain restrictions on usage, such as companies with over 700 million monthly active users needing to apply for special permission, and compliance with multiple branding and attribution requirements during use.
Jeremy Howard, former president of Kaggle and founder of fast AI, stated that while he appreciates open-source, both Llama 4 Scout and Maverick are large MoE models that cannot run on consumer-grade GPUs even after quantization, which is a significant loss for the accessibility of the open-source community
<img src="https://imageproxy.pbkrs.com/https://wpimg-wscn.awtmt.com/943f5c97-0be0-4c6f-8217-c0eb9d162f45.png?x-oss-process=image/auto-orient,1/interlace,1/resize,w_1440,h_1440/quality,q_95/format,jpg" alt="" original-src="https://imageproxy.pbkrs.com/https://wpimg-wscn.awtmt.com/943f5c97-0be0-4c6f-8217-c0eb9d162f45.png"/>
Meta emphasizes that Llama 4 Scout and Llama 4 Maverick are its &#34;most advanced models to date&#34; and the &#34;best versions in terms of multimodality in their class.&#34;
<blockquote>
<ul>
<li>Scout Highlights: Extremely fast, natively supports multimodality, has an industry-leading 10 million+ token multimodal context window (equivalent to processing over 20 hours of video!), and can run on a single H100 GPU (after Int4 quantization).</li>
<li>Maverick Performance: Outperformed GPT-4o and Gemini 2.0 Flash in multiple mainstream benchmarks, with reasoning and coding capabilities comparable to the newly released DeepSeek v3, but with less than half the activation parameters of the latter.</li>
</ul>
</blockquote>
Users on X are also amazed by the performance of the Scout model, especially its ability to run on a single GPU and support ultra-long context windows.
<img src="https://imageproxy.pbkrs.com/https://wpimg-wscn.awtmt.com/5f0ae847-07fe-4948-b611-8e46a736d453.png?x-oss-process=image/auto-orient,1/interlace,1/resize,w_1440,h_1440/quality,q_95/format,jpg" alt="" original-src="https://imageproxy.pbkrs.com/https://wpimg-wscn.awtmt.com/5f0ae847-07fe-4948-b611-8e46a736d453.png"/>
The most eye-catching is Llama 4 Behemoth. Currently, Behemoth is still in training, but Meta positions it as &#34;one of the smartest LLMs in the world.&#34; This &#34;behemoth,&#34; with 288 billion activation parameters and a total of 2 trillion parameters, has been trained on 30 trillion multimodal tokens across 32,000 GPUs, showcasing Meta's strong capabilities in the AI field.
Some users on X pointed out the performance potential of Behemoth's training, emphasizing that it has already shown the ability to surpass several top models, such as Claude 3.7 and Gemini 2.0 Pro, at this stage.
<img src="https://imageproxy.pbkrs.com/https://wpimg-wscn.awtmt.com/91e10b19-dce8-45ac-9393-daac10e07b1d.png?x-oss-process=image/auto-orient,1/interlace,1/resize,w_1440,h_1440/quality,q_95/format,jpg" alt="" original-src="https://imageproxy.pbkrs.com/https://wpimg-wscn.awtmt.com/91e10b19-dce8-45ac-9393-daac10e07b1d.png"/>
Other users on X mocked Meta's &#34;burning money&#34; strategy while expressing surprise at the parameter scale of Llama 4.
<img src="https://imageproxy.pbkrs.com/https://wpimg-wscn.awtmt.com/53d3da5e-825b-4773-8142-dbc7eaef3a33.png?x-oss-process=image/auto-orient,1/interlace,1/resize,w_1440,h_1440/quality,q_95/format,jpg" alt="" original-src="https://imageproxy.pbkrs.com/https://wpimg-wscn.awtmt.com/53d3da5e-825b-4773-8142-dbc7eaef3a33.png"/>
Previously, The Information reported on Friday that, under pressure from investors to demonstrate returns on investment, Meta plans to invest up to $65 billion this year to expand its AI infrastructure
<h2>OpenAI Confirms O3 and O4-mini Launching Soon, GPT-5 Free Strategy Creates a Stir</h2>
Alongside the release of Llama 4, OpenAI CEO Sam Altman confirmed on social media that O3 and O4-mini will be launched in the coming weeks, while GPT-5 will be available to the public in the next few months.
<img src="https://imageproxy.pbkrs.com/https://wpimg-wscn.awtmt.com/58c9f38a-c566-41a8-87ac-77e5c278f881.png?x-oss-process=image/auto-orient,1/interlace,1/resize,w_1440,h_1440/quality,q_95/format,jpg" alt="" original-src="https://imageproxy.pbkrs.com/https://wpimg-wscn.awtmt.com/58c9f38a-c566-41a8-87ac-77e5c278f881.png"/>
Although there are no further details about O3 and O4-mini, Altman stated that OpenAI has made significant improvements to the O3 model in many aspects, which will surely satisfy users.
<img src="https://imageproxy.pbkrs.com/https://wpimg-wscn.awtmt.com/7fc40185-23f4-4953-a458-190c81c51df8.png?x-oss-process=image/auto-orient,1/interlace,1/resize,w_1440,h_1440/quality,q_95/format,jpg" alt="" original-src="https://imageproxy.pbkrs.com/https://wpimg-wscn.awtmt.com/7fc40185-23f4-4953-a458-190c81c51df8.png"/>
In fact, the capabilities and release timing of GPT-5 are the main focus of the market. According to Altman, GPT-5 will integrate multiple features such as voice, Canvas, search, and Deep Research, becoming the core of OpenAI's unified model strategy.
This means that GPT-5 will no longer be a single model, but a comprehensive system that integrates various tools and functions. Through this integration, GPT-5 will be able to autonomously use tools, determine when to think deeply and when to respond quickly, thus handling various complex tasks. OpenAI's initiative aims to simplify the internal model and product system, allowing AI to truly achieve convenience on demand.
Even more exciting is that GPT-5 will offer unlimited usage rights to free users, while paid users will experience a higher intelligence version. Previously, Altman mentioned in a deep conversation with Silicon Valley analyst Ben Thompson that due to the influence of DeepSeek, GPT-5 will consider allowing users to use it for free.
However, due to the repeated delays in the release of GPT-5, some netizens have created the following timeline to poke fun.
<img src="https://imageproxy.pbkrs.com/https://wpimg-wscn.awtmt.com/61a2ca05-92a9-4105-9324-e48026a2ba9d?x-oss-process=image/auto-orient,1/interlace,1/resize,w_1440,h_1440/quality,q_95/format,jpg" alt="" original-src="https://imageproxy.pbkrs.com/https://wpimg-wscn.awtmt.com/61a2ca05-92a9-4105-9324-e48026a2ba9d"/>
<h2>DeepSeek Partners with Tsinghua to Release New Paper</h2>
DeepSeek and the research team from Tsinghua University jointly released a new paper on Scaling during reasoning this week, proposing a learning method called Self-Principled Critique Tuning (SPCT) and constructing the DeepSeek-GRM series models. **This method dynamically generates evaluation principles and critique content through online reinforcement learning (RL), significantly enhancing the scalability of general reward modeling (RM) during the reasoning phase, and introduces a meta reward model (meta RM)
Further optimize expansion performance.
<img src="https://imageproxy.pbkrs.com/https://wpimg-wscn.awtmt.com/286a6a95-50ad-40a9-a06f-d3568907d6ad.png?x-oss-process=image/auto-orient,1/interlace,1/resize,w_1440,h_1440/quality,q_95/format,jpg" alt="" original-src="https://imageproxy.pbkrs.com/https://wpimg-wscn.awtmt.com/286a6a95-50ad-40a9-a06f-d3568907d6ad.png"/>
The core of the SPCT method lies in transforming &#34;principles&#34; from a traditional understanding process into a part of reward generation, enabling the model to dynamically generate high-quality principles and comments based on the input questions and their answers. This method includes two stages:
<blockquote>
<ul>
<li>Rejective fine-tuning as a cold start phase, helping the model adapt to different input types;</li>
<li>Rule-based online reinforcement learning further optimizes the generated content, enhancing reward quality and reasoning scalability.</li>
</ul>
</blockquote>
To optimize the voting process, the research team introduced a meta reward model. This model filters out low-quality samples by judging the correctness of the generated principles and comments, thereby improving the accuracy and reliability of the final output.
<img src="https://imageproxy.pbkrs.com/https://wpimg-wscn.awtmt.com/0b2aefb9-8ad3-4085-8c37-970515a36c72.png?x-oss-process=image/auto-orient,1/interlace,1/resize,w_1440,h_1440/quality,q_95/format,jpg" alt="" original-src="https://imageproxy.pbkrs.com/https://wpimg-wscn.awtmt.com/0b2aefb9-8ad3-4085-8c37-970515a36c72.png"/>
Experimental results show that DeepSeek-GRM-27B significantly outperforms existing methods and models in multiple RM benchmark tests, especially demonstrating excellent performance in reasoning scalability. By increasing reasoning computational resources, DeepSeek-GRM-27B has shown strong potential for performance improvement, proving the advantages of reasoning phase expansion strategies.
This achievement not only advances the development of general reward modeling but also provides a new technical path for the application of AI models in complex tasks, and it may even be showcased in DeepSeek R2.
Some overseas forum users jokingly remarked that DeepSeek has consistently followed the rhythm of being a &#34;late paper model,&#34; which may put pressure on competitor Llama-4.
<img src="https://imageproxy.pbkrs.com/https://wpimg-wscn.awtmt.com/6e6ccced-ea1e-4365-8327-17cd12125c32.png?x-oss-process=image/auto-orient,1/interlace,1/resize,w_1440,h_1440/quality,q_95/format,jpg" alt="" original-src="https://imageproxy.pbkrs.com/https://wpimg-wscn.awtmt.com/6e6ccced-ea1e-4365-8327-17cd12125c32.png"/>

DPSK

OpenAI

Meta released the Llama 4 series, emphasizing multimodal capabilities and ultra-long context windows, with some models to be open-sourced. OpenAI confirmed that O3 and O4-mini are about to launch, while the release of GPT-5 has been postponed but will be free. DeepSeek and Tsinghua University published a new paper proposing the SPCT method and meta-reward model, significantly enhancing reasoning expansion performance

Wallstreetcn

DeepSeek

Alphabet

Meta Platforms

A week of tariff news flooding the screens, the AI circle is also "stirring": Llama 4 is here, O3 and O4-mini are also on the way, and DeepSeek R2 and GPT-5 are not far off?