The Rethink Priorities Existential Security Team's Strategy for 2023

Written By Ben Snodin

Update on 24th of July 2023: See a new page on our work helping launch new projects tackling existential risk here.

Update on 9th of June 2023:

Since I wrote this post we have made some changes to our strategy following some reflection during our team retreat last week. The major changes are

  • I, Jam, Marie, and Renan are dropping our plans to work on strategic clarity research later in the year.
  • We'll drop the "flexible time for high-impact opportunities" for now - we'll still be open to significantly pivoting based on changing circumstances and new information, but the bar is now higher and this is less of an explicit feature of our plans.
  • A major (but not the only) motivation for the above two changes is that focusing solely on the longtermist entrepreneurship program seems good - the plan for that program is very ambitious and we are a relatively small team.
  • Our work helping launch longtermist projects will focus only on AI-related projects in 2023. We're making this change because the majority of the projects we've found to be most promising so far are AI-related, and this narrower scope allows us to build deeper expertise and stronger networks relevant to the projects we'll help launch.
  • Note that these changes don't affect Linch's work - Linch continues to work on his own strategic clarity-related research agenda.

At the time of writing this post is only a month old, so why are we making relatively significant changes already? As mentioned in the original post, the strategy described in the post reflects plans that we started to draft in February and that we started to execute in March, so in reality we've been executing and gathering data for 2-3 months since forming our initial plans. We're also entering fairly new territory for the team and for longtermist-EA-motivated efforts in general, so we might expect our strategy to evolve fairly quickly.

Summary

This post contains a moderately rough, high-level description of the Rethink Priorities Existential Security team’s (XST’s) strategy for the period April-October 2023.[1]

  • XST is a team of researchers focused on improving the world according to a longtermist outlook through research and other projects, and is part of Rethink Priorities (RP).
  • Note that until very recently we were called the General Longtermism team (GLT). We have now renamed ourselves the Existential Security team (XST), which is slightly more descriptive and more closely reflects our focus on reducing existential risk.
  • XST’s three focus areas for 2023 will be:[2]
    • Longtermist entrepreneurship (65%): Making highly impactful longtermist projects happen by finding and developing ideas for highly promising longtermist projects, identifying potential founders, and supporting them as they get these projects started. Our main activities will be:
    • Strategic clarity research (25%): Research that helps shed light on high-level strategic questions relevant for the EA community and for people working on reducing existential risk. This year, we plan to focus on high-level EA movement-building strategy questions (such as “What kind of EA movement do we want?” or “What’s the optimal portfolio among priority cause areas we should aim at building?”), and possibly on high-level questions that seem important for assessing whether and how to help launch entrepreneurial projects. Most of our work on this will happen in the second half of the year.
    • Flexible time for high-impact opportunities (10%): Time allocated for i) team members working on projects that they are very keen on and ii) highly impactful and time-sensitive projects that arise due to changes in external circumstances.
  • Concrete outputs we’ll aim for:
    • 5 project idea memos by the end of June that are of a standard equal to or better than the 2023 Q1 megaproject speedruns that we posted on the EA Forum in February.
    • 1 strategic clarity research output by the end of October.
    • 1 new promising project launched by the end of October.
    • 11 publicly shared research or project idea outputs by the end of the year.[3]
  • From mid-May onwards, we’re planning to have 4 FTE executing this strategy: me (Ben), Marie, Jam, and Renan.[4] Linch is pursuing a separate research agenda related to longtermist strategic clarity.
  • The high-level timeline is:
    • March: The team winds down current projects and begins work on executing the team strategy from the start of April.
    • April-July: The team focuses on the entrepreneurship program, and works on founder-first activities, founder support, and project research. The project research is focused on generating a new prioritization model and shallow project ranking by the end of April, and 5 project ideas memos by the end of June for a potential meeting of promising entrepreneurs in July/August.
    • August-October: Jam and Ben continue working on the entrepreneurship program, while Marie and Renan switch to strategic clarity research. 
    • Start of November: We have a checkpoint to review how things have gone and decide how to continue.

Summary of Rethink Priorities jargon

  • RP = “Rethink Priorities”
  • XST = “The [Rethink Priorities] Existential Security team”
  • SP = “The [Rethink Priorities] Special Projects team”
  • AIGS = “The [Rethink Priorities] AI Governance & Strategy team”
  • The LT department = “The [Rethink Priorities] Longtermism department”

Preamble

From roughly November last year up to March this year, the main focus for the team was revisiting our strategy and making plans for 2023 (though XST team members also continued with various ongoing individual projects). This post is based on an internal strategic planning document that gradually evolved as a result of this thinking, in the period February - April 2023.

I expect this post will be of most interest to people who are interested in following what the team is up to, including people who are potentially interested in joining the team, collaborating with us, or funding us. 

If you are interested in any of the above things, some information that might be helpful:

  • We’re tentatively planning to open a hiring round, most likely in Q3 this year, where we’re hoping to significantly expand the team. You can register your interest in joining the team in our team’s expression of interest form, and we’ll notify you when our hiring round opens (the Rethink Priorities newsletter will also notify subscribers when our hiring round opens).
  • We’re keen to hear from potential founders, advisors, or other collaborators. Please contact jam at rethinkpriorities dot org.
  • Feel free to also pitch us projects by emailing jam at rethinkpriorities dot org - but note that by default we're focusing on only a few highly promising projects and are mostly not doing light-touch support / advice for other projects.
  • We’re also keen to hear from potential funders – our main bottleneck for hiring currently is funding. We hope that we will be able to raise funds when we begin fundraising in earnest in the coming months but this is far from guaranteed and will likely take time. If you’re interested in funding our work, please email janique at rethinkpriorities dot org.
  • We’ll generally continue to share updates in the Rethink Priorities newsletter – you can sign up here.

Note that this post is fairly rough – I could have spent significantly more time polishing it, but other things felt higher priority, and it seems better to put it online in its current state than to keep it private. One particular issue is that the document the post is based on has been gradually developing since February, and in the meantime, we’ve started executing the strategy and naturally our plans have evolved a little – so some of the information might be a little out of date.

I (Ben Snodin) am the main author of this post, although the text is based on earlier drafts written by other members of XST, the strategy was decided in a fairly collaborative way within XST, and we’ve had lots of input from others inside and outside Rethink Priorities.

Introduction

XST was created in early 2022 under the leadership of Linch Zhang, sitting in the Longtermism department (LT department) alongside the AI Governance & Strategy team (AIGS). XST doesn’t yet have a strong collective view on which priorities within longtermism are most important on the margin, so we lean towards operating a longtermist portfolio that looks good from several plausible longtermist outlooks – though in practice we are always focused on reducing existential risk from one source or another.

From within this broad longtermist outlook, we also have a much broader remit compared to the rest of Rethink Priorities: XST is much more open than other teams and departments to producing endpoints that are not research outputs.

In 2022, a major focus was identifying and helping create longtermist megaprojects – highly scalable projects that have large inputs (financial and human capital) and outputs (existential risk reduction). See here for a summary of all our activities in 2022.

Due to the changes in the post-FTX funding landscape, we have decided to move away from the focus on megaprojects. In 2023,[5] we’ll have three main focus areas, in order of priority:

  1. A longtermist entrepreneurship program
  2. Strategic clarity research
  3. A small fraction of time reserved for other high-impact opportunities yet to be determined

The overarching goal with those focus areas will be to experiment and gain information about which potential activities are most impactful for the longtermist community considering the new landscape and what our comparative advantage is as a team. 

In 2022, XST was maximally ambitious and very keen to scale a lot. In 2023 we expect to scale more slowly.

Longtermist entrepreneurship program

We’ll plan to spend 65% of our time on our longtermist entrepreneurship program this year. Concretely, we’ll develop detailed proposals for projects we think are most promising in terms of longtermist impact, and identify and support potential founders who might set up those or other longtermist projects.

Supporting longtermist entrepreneurship was the area we were most excited about after some strategic deep dives in early 2023 because it:

  • Has significant impact potential
  • Has been very neglected in the EA community
  • Is a potential comparative advantage of RP due to our historically strong operations team, and
  • Makes the Rethink Priorities “longtermist portfolio” significantly more diverse given the existing work from the AIGS team

Supporting longtermist entrepreneurship can be seen as a successor to the work the team did in 2022 on supporting potential megaprojects. Even though we originally focused on megaprojects, most of the ideas we found to be most promising were not really “mega”: some of them were highly scalable, but most seemed potentially high impact even at a small scale. 

Even post-FTX, supporting longtermist entrepreneurship has consistently been each XST team member’s top choice of strategic direction, despite quite a thorough three-month exploration of other options for XST strategy.

Three components of the longtermist entrepreneurship program

We think of support for longtermist entrepreneurship as being made up of three components:

  1. Project research: i) identifying the most promising project ideas, ii) turning those ideas into detailed, actionable plans for potential founders
  2. Founder search: finding excellent founders and matching them with project ideas – either matching an exceptional founder with a good idea (“founder-first”) or a good founder with an exceptional idea (“project-first”). This may also involve cofounder matching
  3. Founder support: supporting founders while they spin up projects and, potentially, for some time after that

Our comparative advantage is likely to be project research (although RP has a comparative advantage with at least part of founder support thanks to the Rethink Priorities Special Projects team (SP)). We’ll be particularly keen to seek advice and partnerships with people who can help with founder search and founder support.

The next three sections give a bit more detail on each of these components.

Project research

A core component of our longtermist entrepreneurship program will be project research: generating detailed reports and plans for highly promising project ideas. Project research can be broken down into i) prioritization research and ii) detailed research into project ideas.

For our prioritization research for megaprojects in 2022, we put together an internal database of project ideas, developed a simple weighted factor model based on five criteria (existential risk reduction, cost, tractability, downside risk, and RP’s comparative advantage), and established buckets of projects based on their average rank and the variance between the ranks given by different scorers. Most of that work was not public, but we published a rough list of project ideas with just the projects’ titles and some links to more information.

For our prioritization research in 2023, we’ll:

  1. Collect project ideas from the longtermist community and come up with our own ideas
  2. Create an improved version of the weighted factor model we used in 2022
  3. Rank the project ideas using the weighted factor model
  4. Use that ranking to decide which ideas to investigate in more detail

We’ll then investigate the most promising project ideas in more detail with two aims:

  1. Getting a clearer sense of their potential value, including checking for potential show-stoppers and severe downside risks. We expect that in some cases, detailed investigation will lead us to abandon the idea because of, e.g., downside risks.
  2. Creating a plan for making the idea happen that is detailed enough to show to potential founders and to give founders a good starting point for initiating the project

The speedrun series we published earlier this year shows some relatively shallow investigations of promising projects.

Founder search

In order to help launch projects, we need to somehow come into contact with founders who we can provide with project ideas and support in other ways.

Some avenues for founder search will be:

  • Using our networks
    • A special case of this is a group of highly competent potential founders that Mike McCormick is coordinating and providing support for via his “40k” project
  • Using existing and future public expression of interest forms, posts on the EA Forum, etc.
  • Possibly, creating programs or job openings at RP for potential founders to apply to

Founder support

Founder support refers to essentially everything we will do to support founders other than providing them with project ideas. 

Support for new founders/projects will happen later in the process than project research and founder search, and it’s less clear what we’ll want to do in this area than it is for the other two areas. However, possible support could include:

  • Cofounder matching
  • Advice on initial fundraising
  • Early input on project strategy
  • Other advice/mentorship
  • Research support (e.g., providing founders with RA time)
  • Employment with RP (for the “founders in residence” experiment, see next section)
  • Operational support provided by SP

Three experiments we’ll run in 2023

There are three main ways we will experiment with longtermist entrepreneurship support in 2023:

  1. “Project first,” including via Mike McCormick’s group: i) generate five detailed project ideas by the end of June 2023, take them to a meeting of Mike McCormick’s group of potential founders, find potential founders through that meeting, and then support those founders. ii) also take our top project ideas to other potential founders who we identify as potentially being a good fit
  2. “Founder first”: spend a bit of time looking for exceptional founders who we then find projects for
  3. Founders in residence: use our networks to look for candidates for hiring 1-2 founders in residence (in some form) on contracts 3-12 months long

See the next three sections for a bit more detail on each of these experiments.

Project first

Mike McCormick has assembled a group that includes highly competent entrepreneurs and executives who are interested in longtermist entrepreneurship, with a goal of helping these people start impactful longtermist projects. We understand that one key bottleneck for him and his group is having excellent, detailed project ideas for founders to run with. Since generating these detailed project ideas seems to be a relative strength of ours, working closely with Mike could be extremely productive.

Mike is tentatively planning to hold a meeting of a group of entrepreneurs in Q3 2023, and this seems like an excellent opportunity for XST team members to present a set of detailed project ideas. In addition to possibly directly leading to founders working on these ideas, this could be a great way to establish links between XST team members and potential founders, for example, helping with future cofounder matching, understanding the bottlenecks founders face, finding advisors for the longtermist entrepreneurship program, and finding founders for future project ideas.

With this in mind, we have an internal team deadline to generate five detailed plans for highly promising projects by June 30, 2023. Setting ourselves this deadline now means that

  • We will have a healthy amount of pressure to focus on a key goal for 2023, namely creating detailed plans for highly promising project ideas
  • If/when Mike’s 2023 entrepreneur meeting happens, we’ll be well positioned to make the most of the event and gain the benefits mentioned above
  • We’ll have detailed plans to share with founders we find through other founder search activities

Aside from interactions with Mike McCormick’s group of entrepreneurs, we’ll also plan to connect with potential founders who might be a good fit for the projects we’ve identified as most promising. This seems more likely to happen from July onwards rather than before July, since i) by July we plan to have some detailed plans for highly promising projects, and ii) in July we plan to de-emphasize project research which gives us more capacity for founder search.

Founder first

The “founder-first approach” refers to identifying unusually talented founders and trying to pair them with high-impact projects, rather than identifying high-impact projects and trying to directly recruit a founder for that specific project (the “project-first approach” described in the previous section). The key idea here with the founder-first approach is that we launch projects that are potentially less promising in the abstract because the founder fit is unusually good.

An important motivator for this is the thought that projects will be far more successful if we find an unusually good match between the founder and a project. Plausibly, this is as large of a driver of eventual impact as the choice of project itself (since projects that are much higher impact in the abstract may accomplish nothing or be net negative due to poor execution).

Founders in residence

“Founders in residence” refers to an approach where we give people contracts with RP, allow them to explore potential project ideas, and eventually spin up a highly promising project.

Some advantages of this approach are

  • It offers job security and stability to potential founders, making the transition to longtermist entrepreneurship lower risk and removing a key barrier for some potential founders.
  • It provides founders the opportunity to gain a deep understanding of the relevant space and to feel that they have ownership over the project they eventually choose to pursue.
  • The founders in residence can potentially advise us while we carry out the other two experiments, if they have relevant expertise that we are missing.

However, the approach has some disadvantages:

  • It might attract people who are strong on research but weak on entrepreneurship, and these people might be less effective at eventually creating highly impactful organizations than other groups we might try to engage with.
  • It requires us to put more trust into potential founders since they are given the RP brand and allowed the freedom to pursue ideas that they consider to be worthwhile.
  • It has a larger financial cost: unlike with other approaches, we need to pay the founder a salary and pay other employment costs.
  • It poses a higher risk to RP: perfect vetting isn’t possible and employing someone always comes with risks, e.g., they could turn out to exhibit harmful behaviors that damage organizational culture.
  • It poses other costs for RP such as operational costs (even before any new project has been created).

Reducing the level of pay and the length of the employment contract would make the disadvantages less severe but would also potentially reduce the strength of the advantages.

While we are wary of spreading ourselves too thin in 2023 by trying many different approaches, it seems worth doing relatively low-cost experiments in this area. With this in mind, we tentatively plan to look for people who might be a good fit for founder-in-residence roles and facilitate agreements for 1-2 people to join RP on 3-12 month contracts within the next six months.

Getting external advice

Given our lack of experience with launching entrepreneurial projects, a key determinant of our success at supporting longtermist entrepreneurship this year will be the quality of external advice we receive from relevant experts. Because of this, we will expend a lot of early effort on establishing relationships with relevant experts.

Strategic clarity research

In addition to our longtermist entrepreneurship program, we will also engage in strategic clarity research in 2023.

We use the term “strategic clarity research” to refer to work that helps shed light on high-level strategic questions relevant for the EA community and for people working on reducing existential risk. The “customers” for this work could be

  • The EA community and people working on reducing existential risk in general.
  • Particular people at particular orgs who want clarity on particular questions (what Rethink Priorities has referred to as the “consulting model”).
  • Ourselves, particularly helping resolve uncertainties that arise when considering whether and how to best launch potential longtermist entrepreneurial projects.

The motivation for work in this area is

  • This kind of thinking seems undersupplied in the EA community currently.
  • There are particular instances of EA decision-makers wanting clarity on certain relevant questions.
  • This work could eventually (on the timescale of six months or more) help with generating and prioritizing entrepreneurial projects we might want to help launch.

One concern with entering this area is that it might be easy to do a poor job and not make much progress, especially if the work is being done by a relatively junior team like XST.

Our initial focus for this work will be EA movement-building research. This seems like a good starting point because i) EA movement-building research seems like an important area that is especially relevant currently, ii) people seemed enthusiastic about us doing work in this area earlier this year at the Summit on Existential Security and EAG SF,[6] and iii) XST has some movement-building experience and continuing contact with movement-building projects, e.g., through Condor Camp.

However, as time goes on we will be open to investigating topics in other areas, especially topics that come up as we do research into potential longtermist entrepreneurial projects.

Topics for EA movement-building research could include

  • What we’ve concretely achieved with past EA movement building, whether and to what extent this differs by approach, and what the relevant downside risks have been.
  • How should we approach movement building/field building?
  • What kind of EA movement do we want?
  • What’s the optimal portfolio among priority cause areas we should aim at building?
  • How cautious should we be with growing or slowing down the movement?

Note that we will hold off on strategic clarity research for Q2 2023 while we focus on the longtermist entrepreneurship program, and start with strategic clarity research in Q3 2023. We’ll plan to spend 25% of our time on strategic clarity research this year.

Flexible time for high-impact opportunities

We will reserve 10% of our time for currently-unspecified projects that seem impactful at particular points this year. This time is available i) for team members to work on projects that they are very keen on and ii) for highly impactful and time-sensitive projects that arise due to changes in external circumstances.

Motivations:

  • Allow team members to work on projects that they personally find motivating or personally consider especially impactful. (Note that this is alongside RP’s 10% professional development time.)
  • Allow the team to take on potential projects that seem highly impactful and time-sensitive due to external events ("rapid response"). As a generalist team with a good funding buffer, we might be especially well-placed to make these kinds of pivots.

A lot of XST’s work in 2022 could be considered to be in this category. For example:

Another relevant example from outside of XST is that RP temporarily pivoted ~50% of its research staff to address questions related to the COVID-19 pandemic in March 2020.

Long-term visions

I think having long-term visions for what we want to achieve with our strategic priorities is extremely helpful. First, these visions can help provide direction and keep us on a productive path when we make updates to our strategy as time goes on, and second, working towards ambitious visions can be very motivating.

Visions for longtermist entrepreneurship:

  • There is a thriving longtermist entrepreneurship ecosystem, with several conferences per year, active online networks, and a welcoming and appealing environment for highly competent people who are curious about longtermist entrepreneurship.
  • Longtermist entrepreneurship is a lower-risk career path, with excellent support and guidance.
  • XST is a base of excellence for longtermist entrepreneurship support, having produced many highly-regarded longtermist projects and organizations, and with an established and well-regarded entrepreneurship program that attracts top longtermist entrepreneurial talent and that others seek to copy.
  • The main bottleneck for longtermist entrepreneurship is funding, thanks to the abundance of excellent, detailed project proposals, highly talented longtermist entrepreneurs, and effective structures to combine those two things to make highly impactful longtermist projects.

Visions for strategic clarity research:

  • The LT department (or RP more broadly) is a base of excellence for independent (i.e., from other EA orgs) thinking on key worldview/strategic clarity questions.
  • XST has nurtured and developed leading thinkers in high-level longtermist strategy.
  • XST research outputs have significantly influenced longtermist EA thinking on key strategic questions.

Visions for flexible time/rapid response:

  • The LT department is exceptionally well-placed to perform effective rapid response, thanks to a semi-formal group of researchers/staff who are prepared for rapid response work (including research and ops), have relevant skills and training, and have significant financial resources that can be quickly and flexibly deployed.
  • The LT department contains a team of highly talented, creative thinkers who are given the freedom to pursue the high-risk, high-reward research projects that they find most promising (maybe similar to places like Bell Labs).

Acknowledgments

This post is a project of Rethink Priorities. It was written by Ben Snodin. Thanks to Peter Wildeford, Marie Davidsen Buhl, Renan Araujo, Jam Kraprayoon, and members of the Rethink Priorities AIGS team, as well as others for helpful feedback. If you like our work, please consider subscribing to our newsletter. You can explore our completed public work here.

  1. ^

    See also this post on what XST did in 2022.

  2. ^

    Percentages indicate roughly what fraction of our time we plan to spend on these focus areas from April 2023 to October 2023 (inclusive).

  3. ^

    This might include one or more of the 5 project idea memos or strategic clarity research output if those don’t contain sensitive information.

  4. ^

    Before mid May we have 3 FTE executing on this strategy, due to Renan being occupied with other work.

  5. ^

    Between March and October. We'll review our impact and strategy again in Nov-Dec.

  6. ^

    See also this very rough document: Proposal: We should do more EA movement building research

Ben Snodin

Ben Snodin is a Senior Researcher at Rethink Priorities working on risks and opportunities from advanced nanotechnology. Before joining RP, Ben was a Senior Research Scholar at the Future of Humanity Institute. Ben holds an MPhys (combined undergrad and master’s) in Physics and a PhD in Theoretical Chemistry from Oxford University, and has worked in finance as a quantitative analyst.

Previous

Examining pathways through which narrow AI systems might increase the likelihood of nuclear war
Next

Intermediate goals for reducing risks from nuclear weapons: A shallow review (part 1/4)