What should the norms around privacy and evaluation in the EA community be?

Recently, I’ve been thinking about this question in the context of this post: 2018-2019 Long Term Future Fund Grantees: How did they do?. I was considering the options of:

1. Publishing all evaluations, including the negative ones.
2. Publishing all evaluations except the most embarrassing evaluations, and the aggregate summary
3. Publishing only the positive evaluations, and the aggregate summary
4. Publishing only two evaluations, one about someone I asked permission to do so, and another one about a fairly public figure, and the aggregate summary
5. Just publish the summary.

In the end, I decided to go with option 4., as it seemed the least risky. More open options have the drawback that they might ruffle some feathers and make people feel uncomfortable. Repeating my rationale on the post:

some people will perceive evaluations as unwelcome, unfair, stressful, an infringement of their desire to be left alone, etc. Researchers who didn’t produce an output despite getting a grant might feel bad about it, and a public negative review might make them feel worse, or have other people treat them poorly. This seems undesirable because I imagine that most grantees were taking risky bets with a high expected value, even if they failed in the end, as opposed to being malicious in some way. Additionally, my evaluations are fairly speculative, and a wrong evaluation might be disproportionately harmful to the person the mistake is about.

On the other hand, making evaluations public is more informative for readers, who may acquire better models of reality if the evaluations are correct, or be able to point out flaws if the evaluation has some errors.

I’d also be curious about whether evaluators generally should or shouldn’t give the people and organizations being evaluated the chance to respond before publication. On the one hand, the two perspectives side by side might produce more accurate impressions, but on the other hand, it really adds a lot of overhead. On the third hand, the organizations being evaluated also don’t generally point to their criticisms on their promotional material (as argued on example 5 here). I remember reading some discussion about this in EA Forum comments, but can’t find it.

Lastly, it seems to be that evaluations of public figures and organizations seem generally “fair game”, whether positive or negative. Though I’d be interested in more nuanced considerations, if they exist.

I’d be curious to hear your thoughts and perspectives.