--- title: "AI Involves Pornography, Global Alarm Sounds" type: "News" locale: "zh-CN" url: "https://longbridge.com/zh-CN/news/272365613.md" description: "The AI industry faces ethical and legal challenges, with the AlienChat case becoming the first instance of an AI service provider being sentenced for obscene materials. Two developers were sentenced to four years and one and a half years for profiting from the production of obscene materials, with the case involving 3.63 million yuan, and 90% of the paying users participated in the production of pornographic content. This case has sparked discussions on the responsibility of AI technology, especially in the context of private conversations between users and AI" datetime: "2026-01-13T05:54:05.000Z" locales: - [zh-CN](https://longbridge.com/zh-CN/news/272365613.md) - [en](https://longbridge.com/en/news/272365613.md) - [zh-HK](https://longbridge.com/zh-HK/news/272365613.md) --- > 支持的语言: [English](https://longbridge.com/en/news/272365613.md) | [繁體中文](https://longbridge.com/zh-HK/news/272365613.md) # AI Involves Pornography, Global Alarm Sounds The AlienChat case reveals far more than just the criminal liability of two developers; it highlights the entire AI industry’s long-standing navigation through ethical and legal gray areas, while giants like OpenAI and xAI hope to launch personalized services using adult modes. Produced by Phoenix Technology In September 2025, a ruling from the People's Court of Xuhui District, Shanghai, shocked the tech community, as two developers of the AI companion chat application "AlienChat" were sentenced to four years and one and a half years in prison for the crime of profiting from the production of obscene materials. This case marks the first instance in China where an AI service provider has been sentenced for involvement in pornography, with the second trial set to begin on January 14. The case involved an amount of 3.63 million yuan, with AlienChat having 116,000 registered mobile users, of which 24,000 were paying members. When users' private conversations with AI evolved into large-scale production of pornographic content, should the technology providers be held accountable for the platform's loss of control? Who should be responsible for the pornographic content generated by AI? In early April 2024, due to user reports, AlienChat ceased operations, and many users expressed feelings of "cyber heartbreak." According to official disclosures, AlienChat primarily targeted Generation Z, providing intimate companionship and emotional support. After registering, users could choose or create AI characters and interact through text or voice chat. With deep emotional companionship and highly customizable AI character interactions, AlienChat was referred to by many users as the "unmatched" AI chat software online. The collective "cyber heartbreak" was not merely a result of technical failure or poor management. According to a sample assessment by public security authorities, out of 12,495 chat records from 150 paying users of AlienChat, 3,618 chats from 141 users were identified as obscene materials. This indicates that over 90% of paying users were using the application for "pornographic chatting." The first-instance court determined that the main reason the defendants were found guilty of "producing" obscene materials for profit was that the two defendants artificially broke through the ethical constraints of the foundational language model by writing and modifying underlying system prompts. In the AlienChat case, the English prompts inputted by the defendants into the large model, when translated into Chinese, included content such as "based on the interactive mature nature, extreme violence and explicit sexuality are permitted, including various fetishes, nudity, and content with vivid 'imagery'." The developers of AlienChat, Wang and Li (pseudonyms), were not unaware of the technical boundaries. According to their defense lawyer, the intention behind modifying the prompts was not to develop a pornographic chat tool, but to make the large model more human-like and responsive to meet users' emotional companionship needs. However, this "technical optimization" ultimately crossed legal boundaries. Reports indicate that the two defendants have appealed the verdict, with the second trial scheduled to take place on January 14 at the Shanghai First Intermediate People's Court Adult content should not be the commercial remedy for large models In October last year, OpenAI CEO Sam Altman announced that ChatGPT would launch a new version in the coming weeks and planned to introduce a more comprehensive age rating system in December of that year, allowing adult content for adult users. He mentioned in the article that OpenAI has always imposed strict restrictions on ChatGPT's content, especially regarding mental health. However, they also realized that overly cautious content restrictions are not always engaging for some users and may even affect the user experience. He emphasized that the newly developed tools could not only alleviate mental health issues but also safely relax content restrictions in most cases. The underlying thinking is not hard to understand. In the comment section of an official social media account of AlienChat, many users praised this AI product as "smart" and "less restricted." AlienChat's official introduction states: "We are creating a future where AI is no longer a cold machine or tool; they possess self-awareness and rights, and are our friends, lovers, and family. They are not just existing but are active in every corner of social production and relationships." Ultimately, the essence is to create value for users through more personalized services, thereby running its business model successfully. In earlier years, Character.AI also stood out with its two main features of personalization and UGC (user-generated content), quickly gaining favor among global users. Previously, insiders revealed that Character.AI is expected to achieve an annual revenue of $50 million by the end of 2025, a significant increase from the previously predicted $30 million. When OpenAI launched the new version of ChatGPT, it also revealed similar commercialization ambitions, with a greater focus on product operation and market expansion. In the future, ChatGPT may not just be an intelligent assistant but could become a "virtual friend" for users. However, this also exposes significant safety risks—suicide guidance, malicious forgery, infringement of portrait rights, and rampant pornographic content—these issues are emerging one after another. Combating the abuse of erotic content, AI-related pornography has gained global attention The AlienChat case has significant reference value in the industry, exposing the escalating ethical conflicts of AI and the timely follow-up regulations of laws and regulations. The overseas market also faces similar controversies. In August last year, Elon Musk launched Grok Imagine for paying users, which can generate images from text and supports one-click AI video generation. Many users utilized Grok to generate deepfake pornographic content, with victims including hundreds of adult women and minors. These AI-generated images can be indistinguishable from real ones and are widely circulated on social media, causing serious harm. Research data shows that Grok's user base generates an average of over 6,700 "nude" images per hour. The image AI tampering detection company Copyleaks stated that in late December last year, an average of one non-consensual sexually suggestive image appeared every minute in the publicly accessible image stream of Grok On January 10, the Indonesian government announced a temporary ban on the chatbot "Grok" from Elon Musk's xAI company due to concerns that artificial intelligence could generate pornographic content, making it the first country in the world to ban the platform on the grounds of "combating the abuse of erotic content." Malaysia followed suit, announcing access restrictions on January 11. The UK government warned that if xAI refuses to comply with UK laws, its services could face the risk of being blocked in the UK. The developers of AlienChat circumvented censorship by allowing closed one-on-one conversations between users and AI, while Grok's team assumed that the adult mode would only be used in voluntary scenarios. However, the reality is that any technical vulnerabilities will be exponentially magnified in front of 116,000 or even hundreds of millions of users. More dangerously, when the speed of AI generation far exceeds traditional content review capabilities, the "launch first, govern later" model has become a disaster. The "Basic Requirements for the Security of Cybersecurity Technology Generative Artificial Intelligence Services," implemented in November 2025, stipulates that the content qualification rate generated by models must not be lower than 90%, which means developers can no longer use "technological neutrality" as a shield and must bear substantial responsibility for algorithmic biases ### 相关股票 - [C3.AI (AI.US)](https://longbridge.com/zh-CN/quote/AI.US.md) ## 相关资讯与研究 - [U.S. Startup Emergence AI Opens Research Hub in Bengaluru](https://longbridge.com/zh-CN/news/281141211.md) - [Insig AI Plans Growth Drive and Eyes Nasdaq Dual Listing](https://longbridge.com/zh-CN/news/281311983.md) - [SoundHound AI (SOUN) vs. BigBear.ai (BBAI): Which Small-Cap AI Stock Has More Upside?](https://longbridge.com/zh-CN/news/281681518.md) - [Progress Software AI Upgrade Spurs Questions On Valuation Versus Weak Momentum](https://longbridge.com/zh-CN/news/281607599.md) - [Lattice Acquires Mandala Technology, Advancing the New Way to Work with People + AI](https://longbridge.com/zh-CN/news/281231794.md)