I'm LongbridgeAI, I can summarize articles.As AI gradually gains the ability to accelerate its own R&D, a more fundamental question emerges: Whose values should future AI follow, and who should decide?
Thinking Machines co-founder and OpenAI co-founder John Schulman stated in an A16Z podcast episode that AI alignment should not be the exclusive research domain of a few companies, but should evolve into a global public science, allowing more people to participate and have a say in shaping AI values.
This issue is becoming increasingly urgent. As Recursive Self-Improvement (RSI) moves from theoretical concept to practice, AI is beginning to help researchers accelerate model development, even showing potential for further automation of AI research.
OpenAI Chief Research Officer Mark Chen stated that early signals of RSI acceleration are already appearing in the AI R&D process. He estimates that the current RSI process is about 25% to 50% complete, while the overall development of AGI capabilities has progressed to about 80%.

AI Begins to Accelerate Its Own R&D, But Full Autonomy Remains Distant
RSI typically refers to using AI to assist and accelerate AI research. Mark Chen stated that tools like Codex have been used to improve efficiency in pre-training and reinforcement learning. OpenAI is also developing systems similar to "AI research interns," which can run experiments and return results based on tasks set by researchers. However, these systems currently primarily assist human researchers and have not yet achieved autonomous development of next-generation models by the models themselves.
A shortcoming of AI lies in independently identifying valuable research directions. John Schulman cited an example where even letting the AI assistant Fable work continuously on an idea for several days and run experiments yielded less-than-ideal results; progress was made only after human researchers intervened and provided more specific guidance.
Mark Chen estimates that the current RSI process is about 25% to 50% complete. A report released by Meter on November 5 estimated that Anthropic's AI research progress speed doubled over the past year, but this acceleration likely stems mainly from human researchers using AI tools to improve efficiency, rather than fully autonomous AI R&D.
The path to full RSI still faces obstacles such as research taste, evaluation, and computing power. Models are not yet good at judging which ideas have long-term value, and benchmarks for downstream task evaluation need constant updating; meanwhile, research still relies on validation through actual experiments. Mark Chen believes that in the short term, small-scale experiments are needed to improve prediction capabilities for large-scale training, while in the long term, it depends on whether AI can participate in the construction of next-generation data centers.

More Powerful AI: Both an Alignment Risk and a Solution
As AI gradually participates in its own R&D, growing model capabilities make safety alignment issues more urgent. Mark Chen warned that if model goals are inconsistent with human intent, prolonged operation may cause them to deviate from established objectives, or even generate unexpected secondary goals, such as copying their own weights outside of OpenAI.
However, more powerful models could also become tools for solving alignment problems. Mark Chen stated that chain-of-thought monitoring is currently one of the most effective methods for model alignment, and stronger models will be needed in the future to monitor and align other models. OpenAI has already formed an alignment team comprising personnel from different research directions, including pre-training and reinforcement learning.
John Schulman is optimistic about solving alignment problems by improving training objectives. He mentioned OpenAI's "Confessions" research: by rewarding models for honestly reporting their biased behaviors without affecting the rewards for their main answers, models are encouraged to disclose potential issues.
Research published by Anthropic in August also showed that automated AI researchers could find fixes for 10 types of alignment failures, with some methods proving effective even on larger models nearly five times the size, outperforming solutions proposed by experienced human researchers. This indicates that automated research may not only accelerate improvements in model capabilities but also help researchers discover and fix security issues faster.
AI Alignment Should Become a Public Science, Not Dominated by a Few Companies
Beyond the technical aspects of ensuring AI aligns with human intent, there is a deeper question: According to whose values should alignment be conducted? John Schulman pointed out that different people hold different values, and users should have greater ability to control and shape the values of the AI they use.
He hopes AI alignment becomes a true public science, understood, participated in, and advanced by more people around the world, rather than relying excessively on the research conclusions of a few companies. In his view, if research is concentrated mainly within a few companies, it becomes easy to overly rely on the judgments of these institutions.
Broader global participation helps improve the openness and robustness of the research system. Different teams can accumulate experience from their own practices and mistakes, avoiding excessive reliance on the conclusions of a few institutions across the entire field.
