Measuring our outcomes

We believe that tracking our outcomes provides evidence of our success or failure in working towards our overall goal, which is to contribute to effective improvements in health around the world, now and in the future. 

The overlapping purposes served by outcomes tracking include: 

• Guiding our strategy: our ongoing evaluation of our outcomes shapes the programs that we provide to seek such impacts in the future.

• Providing accountability to donors and funders: Our donors and funders, like us, are typically very interested in the outcomes associated with our activities. We consider it important to hold ourselves accountable to them by sharing our outcomes with them too.

No. We care about supporting medical students and doctors to have a higher positive impact with their careers, and these outcomes are a clear way to track that - but they’re not the only way.

We are very interested in some of the small- and medium-sized changes that happen early, that may indicate a future impactful career. This includes enrolling in educational courses (online or in person), volunteering, internships, and applying for jobs (even if initially unsuccessful). We also track how satisfied participants are with our courses, and the wellbeing of our participants. Contact us for a full list of the information that we track - the list evolves over time. 

We use a combination of the following: 

• Emails: Approximately every six to 12 months, we send a short email to recent program participants (anyone not emailed in the previous round) and to our whole mailing list. We also invite respondents by email to complete an optional survey.

• Word of mouth updates: we like to stay in touch with our community members and the past participants of our programs. This means we sometimes find out about career changes incidentally through our conversations with them.

We treat all responses confidentially by default. Here are more details: 

• Aggregated, de-identified results can help our team and funders to understand Hi-Med's impacts and to continue to improve our programs.  

• The overall numbers of career changes or donation pledges we’ve recorded are included on our website. 

We provide the opportunity to opt in to add your name, alongside your reported impacts, to a confidential case studies document. The provision of case studies (by those who opt-in) helps us to better understand how we contributed to the changes that people have reported to us. This can also inform how we can better support others in the future.

If you wish to change or withdraw the consent that you have given us previously, for example in order to have your data deleted from our records, then you should email us here (the Hi-Med Evaluation team hi.med.evals@gmail.com and broader team info@highimpactmedicine.org).

Every 6-12 months, we contact people who signed up to our mailing list to ask if they’ve made any career changes, plans for changes, pledges, or if they have other updates to which they think Hi-Med contributed.

• Sometimes, people have already benefited from knowing about the community, staying in the loop about news, or reviewing our jobs newsletter each month. This is why we consider it worthwhile to ask our whole mailing list for impacts they attribute (partly) to Hi-Med, not just our course participants.

As of May 2026, the question we send is as follows:

__________________

Has your involvement with Hi-Med to date contributed to any of the following? (Please reply and put an x next to all options that apply.)

   [  ] Starting a new paid role 

   [  ] Applying to a new paid role

   [  ] Volunteering in a new role 

   [  ] Starting a postgraduate degree

   [  ] Changing your topic of study

   [  ] Undergoing other education or training

   [  ] Planning any of the above 

   [  ] Pledging to make ongoing donations to effective charities (percentage of income: __%)

   [  ] Other

   [  ] Nothing (no impact)

Please provide further details here (optional).

We may reach out to you for clarifications. 

   [  ] I would prefer you not to contact me for clarifications.

If you’d prefer to set up a call with a Hi-Med team member to discuss this question, please let me know.

Confidentiality

We treat all responses confidentially by default. Aggregated, anonymized results can help our team and funders to understand Hi-Med's impacts and to continue to improve. 

Bonus question: would you like to opt-in to your response being recorded alongside your name in an internal impact document (which will only be shared with team members/contractors and current or prospective funders)? [Y / N]

________________________

In 2025, our question was similar, but we grouped together some categories that we now keep separate. In 2024, our question was open-ended, as we were engaging in a case-study-first approach to data collection then.

Yes. Here’s what we do at the moment: 

• We include an “Other” option whenever we ask our question about impacts that our community members attribute to Hi-Med. We hope this allows participants an opportunity to express negative feedback if they haven’t already.

• Hi-Med has an internal process designed specifically for reporting opportunities for improvement. We review these reports at least once a month, and discuss the progress we are making to address each opportunity in our monthly team meetings.

• We also monitor responses to our anonymous feedback form, which you can find here.

This point matters a lot to us. We prioritize being extremely clear about what we know, and what we don’t know, about our influence on causing outcomes. Here’s what we do: 

1. We do know how many people have stated that Hi-Med contributed to them making career changes, plan changes, or donation pledges. So that’s what we track.

2. We have limited confidence in our estimates of the extent to which Hi-Med (versus other factors) contributed to the things we’re tracking, but we can make reasonable estimates.

   • In all our impact multiplier estimates, we include estimates of the degree to which Hi-Med contributed, which is never 100%. If we look at the total value of all the career changes to which we’ve contributed, the 90% confidence interval for the Hi-Med contribution is 0.5%-12%. Usually we contribute a small amount, but occasionally we contribute a lot more. We attribute the rest of the value of those career changes to the individual themselves and to the other factors (outside of Hi-Med) that also contributed. Often, we think that involvement with Hi-Med speeds up steps that likely would have happened without us, but over a longer time interval. In a few important cases, people have shared that we opened up options to them that they would never have considered otherwise.

   • We defer to participants regarding whether they think Hi-Med contributed to the changes that they report. For those who opt-in, we get a detailed case study and try to get more information on the factors that contributed to their changes, including but not limited to Hi-Med. This process is inherently subjective, but those who opt-in to provide case studies are helping us immensely, because they allow current and future funders/donors to understand how we had an impact in specific individuals’ lives. This is especially important for people who’ve gone on to do high-impact things.

   • Please get in touch with us if you want access to our impact multiplier estimates document.

Since 2025, we’ve been using a two-tiered approach to tracking outcomes: 

1. Our main way of collecting outcomes data is with a very short email with a single, simple checkbox question.

   • This method minimizes friction for our respondents. It doesn’t require respondents to click on any links, or even interact with a within-email form. To respond, participants just check what option applies to them and then hit “reply”.

2. We also offer the chance for people to complete a survey, so that we can learn more about the impacts we’re having (among those who are willing to share). There are two options: a shorter survey and a longer one. 

   • For every response we get to the shorter survey, someone on our Monitoring & Evaluation team pays for an antimalarial bednet via the Against Malaria Foundation. For the longer survey, the same person pays for five antimalarial bednets for each response. 

   • These donations are made out-of-pocket, not out of the Hi-Med budget.

Since 2025, we’ve administered a team satisfaction survey every ~6 months. The results are only reported in our internal dashboard due to the small size of our team (and the potential for re-identification of survey responders).

We also ask our team members if they think Hi-Med has helped with any positive career-related or donation-related changes for them. In our impact multiplier estimates, we categorize these changes differently because we recognize that there’s much more potential for social desirability bias to negatively influence our ability to ascertain our level of impact in those cases.

Please get in touch with us!

We welcome engagement of all kinds, including critical engagement about what we’re doing. You can email us or use our anonymous feedback form.