Measure is unceasing

Forecasting Newsletter: January 2022

Highlights

Index

You can sign up for this newsletter on substack, or browse past newsletters here. If you have a content suggestion or want to reach out, you can leave a comment or find me on Twitter.

Prediction Markets & Forecasting Platforms

Polymarket

Since Polymarket settled with the CFTC last month (a), they have moved out of US jurisdiction. They are now operating from Panama (a). Their terms of use now prohibit users from the US from participating on the platform, and likewise prohibits use of a VPN to circumvent this prohibition (a). They have now started using UMA (a) as a market resolver. Personally, I am glad Polymarket has survived.

Manifold Markets

Manifold Markets (a), a new play-money prediction market, has kept its frantic development pace. They will be “launching” on February the 8th, from their current open beta. They outline some of their hopes and plans in their substack newsletter (a).

I created some markets on whether I will consider the Principles of Intelligent Behavior in Biological and Social Systems fellowship to be “a success” (a), on the number of subscribers to this newsletter (a), and on whether my current employer will still be alive by the end of 2022 (a). The process was extremely painless, and I recommend that readers give it a try.

CSET-Foretell is now INFER

CSET-Foretell moved from being hosted by CSET (a) at Georgetown to being hosted by ARLIS (a) at Maryland. ARLIS also received an $8M grant (a) from Open Philanthropy for forecasting specifically. 

This is a fairly large amount of EA funding for forecasting projects. For reference, Metaculus recently received $250k (a), and Phil Tetlock and associates previously received two (a) ~$500k (a) grants. Open Philanthropy also previously gave ~$55M CSET (a), as well as later top-up (a) grants (a). But this was for a broader think tank, rather than for forecasting specifically.

In the December issue of this newsletter, I assessed that the move from CSET to ARLIS was probably a negative development, partially because I thought that funding from Open Philanthropy was much better than government funding. As it happens, ARLIS has just now received funding from Open Philanthropy as well. Multiple people also reached out to comment that the move was probably neutral or positive, on account of ARLIS' deeper involvement with the US government. My independent impression is that I still dislike the move, but my all-things-considered view is that it’s probably ok and I was wrong. As to how wrong, we’ll see.

Separately, because their Pro Forecaster program still only pays $20/hour, my team—Samotsvety Forecasting, which overwhelmingly won the last two seasons—might not be participating going forward, though we are trying to negotiate with them. I talked to a few super-forecasters about this, and $20/hour isn’t going to get ARLIS the best forecasters. Their open call for pro forecasters can be found here (a).

But this isn’t a problem unique to INFER. Generally, forecasting platforms such as Metaculus (a), Hypermind (a), or the Social Science Prediction Platform (a), just don’t pay that much to forecasters. This leads to predictable problems, such as forecasters moving to crypto prediction markets, or experiencing burnout after forecasting in an unsustainable way for little reward, as happened to some top covid forecasters.

Metaculus

SimonM (a) kindly curated the top comments from Metaculus this past January (a). They are:

On the potential invasion of Ukraine by Russia:

Metaculus' updated public figure predictions can be found here (a).

Forecasting meetup

Nathan Young and I are organizing a forecasting and prediction markets online meetup at 7:00 PM UTC on the 9th of February in the LessWrong online Walled Garden. The LessWrong Walled Garden is great because you can move around and leave and join conversations as you wish, allowing better flowing conversations, mentions Nathan. The event will officially finish 2 hours after it starts, but anyone is welcome to stay for longer.

Odds and ends

The sixth Makridakis Competition (a)—a well-known open competition to evaluate and compare the accuracy of different time series forecasting methods—has been announced. Total prizes—amongst many rounds and many forecasters—sum up to $300,000 (a). The deadline for the first submission point is the 6th of March, though there is a trial round in February. More details can be found on their webpage (a)

Aver (a), a prediction market built on top of the Solana chain, announced a $7.5M seed round (a). My impression is that they will start with sports and crypto, which are easier to resolve. But I imagine that they will also have the flexibility to also experiment with some markets whose probabilities are valuable to the general public. Based on the typical amount sold during the first financing round, their valuation is probably between $30M and $80M.

Insight Prediction has launched a real-money beta with limited access and is looking for members of the prediction market community to test the site and give feedback. They are running a Russian Invasion of Ukraine Market (a), as well as a weekly US covid total death markets (a). Their rationale is that these might make some people uncomfortable, but they think that these markets are the most important for policymakers and users in terms of the information they provide. I agree with that reasoning. 

On the negative side, Insight Prediction had previously been stuck in development for a long time. They were originally planning to launch in June or July 2021, though at the time one of the funders refused to take a $20 bet I offered on their timelines. Reliable anonymous sources have also expressed some skepticism about the project—not necessarily in the sense of being a scam, but rather in terms of their plans being unfocused. 

Prediction market players who want to participate in the early access beta can reach out per email.

Announcing the Forecasting Newsletter $10,000 Micro-Grant Program

After the apparent success of ACX grants (a), Misha Yagudin and I received $10k from an anonymous donor to give out as micro-grants through this newsletter.

Some examples of projects we’d be excited to fund might be:

The application form is HERE (a). Feel free to apply for more than $10k: we don’t anticipate having much difficulty getting more funding for promising applications, and we may refer these to other funders (e.g., the EA Infrastructure Fund (a)) if we can’t. 

We will be accepting applications until March 15, though we may extend this period if we don’t receive enough high-quality submissions. We preliminarily plan to make decisions by April the 1st.

Otherwise, Luke Muehlhauser comments (a) that forecasting related projects might be a good fit for the EA Infrastructure Fund (a). Jonas Vollmer, who runs EA Funds, confirms this (a)

For larger projects, the Survival and Flourishing Fund, backed by philanthropists Jaan Tallinn and Jed McCaleb, is organizing the distribution of around $6M-$10M in grants (a) this June, with applications due on Feb 21. They generally only accept applications from registered charities, but speculation grants (a) might be a good fit for smaller projects (40%).

Blog Posts

Scott Alexander posts Predictions for 2022 (a) and grades his own probabilities for 2021 (a). Zvi gives his own probabilities for them. (a).

Ege Erdil writes about retrospective forecasting (a): In order to use Bayes' theorem, we need to somehow get the probabilities before the thing we are interested in happened. He links to this interesting blogpost/paper (a) extracting a 42% that the Confederacy would win the US civil war from the value of their gold bonds in Amsterdam. On this topic, see also Lustick and Tetlock’s The simulation manifesto.

Note to the future: All links are added automatically to the Internet Archive, using this tool (a). “(a)” for archived links was inspired by Milan Griffes (a), Andrew Zuckerman (a), and Alexey Guzey (a).

h/t to Nathan Young, Ozzie Gooen and Clay Graubard for comments and suggestions on this edition.

Jefferson, in his forecast, had anticipated this, as the “rock upon which the old Union would split.” He was right. What was conjecture with him, is now a realized fact.

Alexander H. Stephens, Cornerstone Speech.