--- title: "DeepSeek Popular Science! How Domestic LLMs Are Overtaking on the Curve?" type: "Topics" locale: "en" url: "https://longbridge.com/en/topics/26799414.md" description: "Today, the hottest discussion in the financial circle is undoubtedly Deepseek, this AI application has caused quite a stir in the tech world, directly causing $NVIDIA(NVDA.US) to take a tumble, and it may pose a threat to $Meta Platforms(META.US) $Alphabet - C(GOOG.US) $Alibaba(BABA.US) and other large model manufacturers. Everyone's impression of it is "domestic LLM achieving GPT O-1 level at a low cost"..." datetime: "2025-01-27T03:12:06.000Z" locales: - [en](https://longbridge.com/en/topics/26799414.md) - [zh-CN](https://longbridge.com/zh-CN/topics/26799414.md) - [zh-HK](https://longbridge.com/zh-HK/topics/26799414.md) author: "[躺平致富](https://longbridge.com/en/profiles/8547308.md)" --- # DeepSeek Popular Science! How Domestic LLMs Are Overtaking on the Curve? Today, the hottest discussion in the financial circle is undoubtedly Deepseek, an AI application that has caused quite a stir in the tech world, directly causing $NVIDIA.US to take a tumble and potentially threatening major model manufacturers like $Meta Platforms.US, $Alphabet - $Citigroup(C.US), and $Alibaba.US. Everyone's impression of it is "domestic LLM achieving GPT O-1 level at low cost." So how does Deepseek achieve this? No one has mentioned the underlying principles, so let me explain: the technical principles behind DeepSeek's ability to overtake on a curve boil down to three points: 1. **Mixture-of-Experts (MoE)** is a core technology of the DeepSeek-V3 model. It optimizes computational efficiency and resource usage by intelligently selecting which parameters to activate, thereby improving the model's performance. This architecture allows the model to activate only a portion of its parameters when processing inputs, significantly reducing computational costs. To put it in layman's terms: Imagine a large company's customer service center, which has many different departments. Each department has its own area of expertise, such as technical support, billing inquiries, and complaint handling. Whenever a customer calls, the receptionist (Gate Network) will transfer the call to the most appropriate department (expert) based on the customer's issue. In DeepSeek's MoE architecture, the model functions like this customer service center, having multiple "experts" to handle different types of information. When an input arrives, the system selects a few of the most relevant experts to process it, rather than involving all experts, thus saving resources and improving efficiency 2. **Dynamic Redundancy Strategy** In DeepSeek-V3, the system dynamically selects suitable "experts" for processing based on the current task. This flexibility ensures that the model maintains optimal load balancing during inference and training, thereby improving efficiency and response speed. Continuing with the analogy of a customer service center, suppose some departments are often overwhelmed while others are quite idle. To balance the workload, the main receptionist dynamically adjusts the distribution of incoming calls based on the busyness of each department. For example, if the technical support department has already received many calls, the receptionist will reduce the transfers to that department and direct more calls to the billing inquiry department. DeepSeek's dynamic redundancy strategy is such a mechanism that intelligently adjusts the load of each expert, ensuring that every expert can work efficiently, thus avoiding resource waste. 1. Multi-Head Potential Attention (MLA) DeepSeek-V3 also introduces a multi-head potential attention mechanism, allowing the model to better capture contextual information when understanding and generating text. This mechanism enables the model to perform better in handling complex problems, especially in areas such as mathematical calculations and code generation. Still using a human analogy: Imagine a band performing, where each instrument has its unique timbre and role. During the performance, the conductor activates different instruments based on the needs of the piece. For instance, in a soft melody, only violins and flutes may be needed, while in a vigorous section, all instruments play together. DeepSeek's multi-head potential attention mechanism is like this conductor, allowing the model to dynamically select the most suitable "instruments" (attention heads) based on the input content, thereby enhancing overall performance and efficiency. ### Related Stocks - [NVDA.US](https://longbridge.com/en/quote/NVDA.US.md) - [META.US](https://longbridge.com/en/quote/META.US.md) - [GOOG.US](https://longbridge.com/en/quote/GOOG.US.md) - [GOOGL.US](https://longbridge.com/en/quote/GOOGL.US.md) - [BABA.US](https://longbridge.com/en/quote/BABA.US.md) - [89988.HK](https://longbridge.com/en/quote/89988.HK.md) - [09988.HK](https://longbridge.com/en/quote/09988.HK.md) - [HBBD.SG](https://longbridge.com/en/quote/HBBD.SG.md) ## Comments (12) - **安之Beta · 2025-01-27T07:27:50.000Z · 👍 2**: Let's applaud for China's innovation ecosystem👏 - **芒格是我爹 · 2025-01-27T07:19:22.000Z · 👍 1**: NB - **飞飞哥 · 2025-01-27T04:57:19.000Z · 👍 5**: Stolen goods have no cost, so they must be cheap. - **云深处** (2025-01-27T06:28:09.000Z): As expected, cognition determines everything. - **我要验牌！** (2025-01-27T06:40:06.000Z): Your cognition deserves your earnings LOL - **韩愈** (2025-01-27T10:27:44.000Z): Jealous much? What an idiot must be so mad. - **nvamd · 2025-01-27T04:37:19.000Z · 👍 2**: What kind of science popularization is this? Which model doesn't have what you mentioned? - **美股——阿蛋 · 2025-01-27T04:32:24.000Z · 👍 1**: A real slap in the face to the 500 billion STAR GATE. Better stay in cash and watch the market for now. - **momoM · 2025-01-27T03:26:26.000Z · 👍 1**: Thanks for the knowledge drop, influencer! - **躺平致富** (2025-01-27T03:28:10.000Z): You're welcome @momoM, any related stocks you're bullish on? - **momoM** (2025-01-27T04:02:33.000Z): Meitu Kingsoft XtalPi🙈