A project of Rethink Priorities
The Synthetic Cognitive Systems Initiative is a research group focused on the philosophical and technical questions raised by advanced AI systems. As AI technologies rapidly advance, we need systematic research that bridges the gap between technical understanding and philosophical inquiry.
We are motivated by grand philosophical questions that bear on how we will incorporate them into our world.
We believe in the importance of grounding speculation in experimentation and direct observation.
Our work combines philosophy, machine learning, consciousness research, model building, and economics to tackle these complex questions.
The rapid advancement of artificial intelligence systems presents society with novel philosophical and practical challenges that extend beyond near-term safety concerns. As AI systems become increasingly sophisticated in their behavior, exhibiting what appear to be preference, reasoning, and even rudiments of self-reflection, we face difficult questions about the appropriate framework for understanding them. Are these systems merely executing statistical patterns, or do they possess something approximating genuine mental states? What would it mean for an artificial system to have interests, experiences, or welfare considerations? These questions are not merely theoretical—the decisions we make now about how to conceive of, interact with, and regulate AI systems may shape technological trajectories for decades to come.
The Worldview Investigations Team at Rethink Priorities has developed several ongoing research programs that form the foundation of this initiative.
Report on risk attitudes in agents: Under a grant from OpenAI, we produced three papers exploring how AI preferences should relate to those of their users.
Ongoing study: Current grant-supported project on understanding and influencing risk attitudes in contemporary LLMs.
Digital Consciousness Model: Building a Bayesian model for calibrating beliefs about consciousness in current AI systems.
FIG fellowship program: Supervising new professionals doing technical work related to AI consciousness and welfare.
Research focus: Refining technical work on introspection, preferences, and personalities to elucidate model capabilities relevant to moral status.
Senior researchers with expertise across philosophy, economics, and computer programming.
Senior Researcher
Senior Researcher
Senior Researcher
Senior Researcher
Senior Researcher