Sample Projects & Writing

Projects

GiveWell’s Cost-Effectiveness Models

I was the point person on GiveWell’s cost-effectiveness analyses from 2016 to 2018. As the point person on GiveWell’s cost-effectiveness models, I:

  • created models for estimating the cost-effectiveness of new interventions
  • coordinated with colleagues and external organizations to update existing models to account for new analyses and data
  • spearheaded major changes to make the model more transparent, easier to maintain, and less likely to contain errors

Archived copies of the primary models are available as they were when I joined the organization and when I left.[1]

CoverageCritic

I created CoverageCritic.com, where I help consumers understand the wireless industry and find cell phone plans that are well-matched to their needs and budgets. On CoverageCritic, you can see the Plan Finder tool I developed as well as a bunch of content I’ve written. Here are links to a few example articles:

Writing

Beware of Scoring Systems

Organizations that evaluate products and services often use math-heavy evaluation rubrics. While these rubrics often make evaluations appear rigorous or scientific, the rubrics rarely work well in practice. In my article Beware of Scoring Systems, I explain why this is the case while using lists of college rankings as illustrative examples.

The Optimizer’s Curse & Wrong-Way Reductions

I wrote the article The Optimizer’s Curse & Wrong-Way Reductions to (a) point out limitations of some approaches to cost-effectiveness modeling and (b) draw attention to issues that I thought the effective altruism community wasn’t paying enough attention to. Here is the summary I included in the original post:

I spent about two and a half years as a research analyst at GiveWell. For most of my time there, I was the point person on GiveWell’s main cost-effectiveness analyses. I’ve come to believe there are serious, underappreciated issues with the methods the effective altruism (EA) community at large uses to prioritize causes and programs. While effective altruists approach prioritization in a number of different ways, most approaches involve (a) roughly estimating the possible impacts funding opportunities could have and (b) assessing the probability that possible impacts will be realized if an opportunity is funded.

I discuss the phenomenon of the optimizer’s curse: when assessments of activities’ impacts are uncertain, engaging in the activities that look most promising will tend to have a smaller impact than anticipated. I argue that the optimizer’s curse should be extremely concerning when prioritizing among funding opportunities that involve substantial, poorly understood uncertainty. I further argue that proposed Bayesian approaches to avoiding the optimizer’s curse are often unrealistic. I maintain that it is a mistake to try and understand all uncertainty in terms of precise probability estimates.

Footnotes

  1. GiveWell’s current cost-effectiveness model can be found here.