What is recursive self-improvement?

Recursive self-improvement happens when a system enters a feedback loop of self-improvement that makes it better at further self-improvement. For example, an AI capable of human-level AI research could design an AI capable of genius-level AI research, which could design an AI capable of supergenius-level AI research, and so on. This process could quickly result in a vastly more capable system.

While we humans also improve our capabilities by studying new topics, practicing new skills, etc., we can’t easily alter our biology.^[1] AI agents, on the other hand, are computer programs that can be edited, or run on more and faster hardware. Such advantages might enable them to recursively self-improve much faster than humans can

Few AI systems to date have been designed to pursue recursive self-improvement as an explicit goal, and not everybody agrees that recursive self-improvement by AIs is even possible. However, if it is possible, recursive self-improvement is probably an instrumentally convergent strategy that a wide range of AIs might want to pursue.