The Stakes of Emerging Technology: Why Your Career Choice Matters More Than You Think
The next several decades will be defined by unprecedented, transformative technological breakthroughs. These advances hold the potential to deliver massive, widespread gains for humanity—but they also could push us to the very edge of catastrophic collapse. What our collective future looks like will ultimately depend on whether we can learn to successfully manage the risks that come with these new powers. That means if you want to leave a lasting positive mark on the world, one of the most meaningful actions you can take is to contribute to work that ensures our most powerful emerging technologies are built to serve all of humanity, rather than putting us at risk. Here’s why that work matters so deeply right now.
Breakthroughs in modern biotechnology—from CRISPR gene editing to mRNA technology—have given researchers incredible new tools to cure chronic genetic conditions, beat back deadly infectious diseases, and even wipe out the mosquito populations that spread malaria across the globe. But this rapid, explosive growth in our ability to alter living organisms also makes it far easier for a catastrophic pandemic to be unleashed, even completely unintentionally. Today, researchers already have the ability to fine-tune and synthesize viruses that are known to be extremely deadly. For now, this capability is limited to just a few dozen well-vetted, trusted scientific experts. But as the cost of biotech tools continues to plummet, this power will become accessible to thousands of people around the world. If we continue to develop tools that can create new pandemic threats long before we build robust systems to protect against them, the outcome could be devastating.
Advances in artificial intelligence are no less consequential. Today’s cutting-edge AI systems already can explain the humor in jokes, replicate distinct artistic styles, and apply common-sense reasoning across a wide range of unrelated fields. In the near future, they will cut years off the timeline for new life-saving drug discovery and speed up critical research into affordable green energy. All signs point to this rapid progress continuing unabated. If that trend holds, the most advanced AI systems will outperform humans not just on specific, narrow tasks, but across nearly all cognitive and practical domains. Advanced AI could end up being used to lock in systemic discrimination on an unprecedented scale, and give dictators more power to control their populations than ever before. Worse, we could ultimately lose full control of these systems entirely, once they become too powerful for humans to rein in.
These two technologies stand out because their catastrophic risks carry an unmatched combination of massive global scale, actionable solutions, and widespread neglect from current research and policy efforts. Forecasters on Metaculus, a popular community-driven prediction platform, estimate that synthetic biology carries a 1% risk of a catastrophe that kills at least 95% of the global human population by the end of this century. For advanced unaligned AI, that estimate jumps to 6%. And yet, barely any researchers, policymakers, or funders are focused on addressing these existential risks—even though there is no shortage of meaningful work that can be done right now. If more committed, talented people join the effort to steer these powerful technologies toward safe, equitable outcomes, we can ensure they deliver broad benefits to all people, both today and for generations to come.
But what does that work look like for you, personally? What are the highest-impact steps you can take to contribute to this goal?
Your single most important decision
If your goal is to create meaningful, positive change for the world, one decision matters more than any other: how you choose to build your career.
