Andrew Wong, M.D., is part of a team developing the DATA-MD curriculum, which aims to teach learners how to evaluate clinical machine learning (ML) and artificial intelligence (AI) tools, an area that is currently underdeveloped across medical education and training. The goal of this curriculum is to transform how ML and AI are taught both at Michigan Medicine and throughout the nation, leading to better utilization of AI tools in clinical practice. The project is funded by a Graduate Medical Education Innovation Grant and Wong is receiving project support through RISE.
Machine learning (ML) and artificial intelligence (AI) are increasingly becoming a regular part of medical professionals’ toolkits. Machine learning, a type of AI that involves training computers with data to perform dedicated tasks, such as disease diagnosis, has been applied across the entire health care spectrum. Recent ML algorithms have demonstrated equal or better performance than clinicians in fields like dermatology, radiology and ophthalmology.
Yet when it comes to teaching health care trainees how to successfully utilize and evaluate ML and AI tools, there is a gap in current medical education with most residents and medical students having never received formal training in ML and AI topics. Andrew Wong, M.D., wants to change that.
“Data shows that trainees are very undertrained and unprepared for the changes coming their way in this field,” shared Wong. “We want to empower them to use ML and AI tools in the clinical setting, which will ultimately help them do their jobs better.”
Wong, who is currently a Chief Medical Resident in Internal Medicine, is part of a team that developed a new ML and AI curriculum for medical trainees, the Data Augmented, Technology Assisted Medical Decision-Making (DATA-MD) curriculum. The initiative is co-led by Cornelius James, M.D., a Clinical Assistant Professor of Internal Medicine, Pediatrics and Learning Health Sciences. This standardized curriculum on ML and AI will be utilized by learners at Michigan Medicine with the goal of expanding throughout the nation.
Cornelius James, M.D.
“The potential for AI and ML tools to augment diagnostic decision making tools is beginning to be realized within health care,” shared James. “We must start preparing clinicians to engage with these technologies. The DATA-MD curriculum is a step in this direction.”
Wong and James received funding support for DATA-MD through the Graduate Medical Education Innovation Fund, which supports work that leads to pioneering education of future physicians. Wong was also selected as a Cohort 3 Innovator in the RISE Innovation Development Program, which develops innovators in medical education at Michigan Medicine who embrace curiosity, creativity, critical thinking, initiative and intelligent risk-taking to support education experimentation and discovery.
The team spent a year developing the one-of-a-kind curriculum before piloting it with internal medicine residents in May 2023. The curriculum contains four key learning modules — an introduction to ML and AI, fundamentals of epidemiology and biostatistics, use of ML/AL to augment diagnostic decision-making, and ethics and legal implications of ML/AI. The initial pilot centered on two in-person small group sessions covering content from each of the four modules.
A learner who participated in the pilot had this to share: “The [DATA-MD] curriculum was certainly eye-opening! It taught me things that I did not know about AI and machine learning and gave me the skills to learn more in the future.”
Wong and the team are now looking at their next steps for reaching a wider audience. “Our data shows that the pilot was effective,” shared Wong. “We are currently working to broaden our curriculum to other specialties and branch out nationally through our partnership with the AMA and NAM.”
While the curriculum’s primary audience is residents and medical students, DATA-MD was designed to be broadly applicable to learners of all levels and backgrounds, and could expand to other health sciences disciplines, such as nursing or pharmacy. At this time, the course is not publicly available, but the team is working on an online module course that they hope to make available within the next year.
RISE’s Impact
For Wong, his biggest benefit from being part of the RISE Innovation Development Program has been the mentorship and networking opportunities.
“RISE helps you meet all the key players. It attracts those who are passionate about medical education and have key skills and experience within the field,” he said.
Innovators who participate in the RISE Innovation Development Program receive support in the form of monthly sessions with members of their cohort, innovation coaching and individual meetings with members of the RISE Core Team. During the program, innovators will engage in a community of practice and learn the fundamentals of change management, leadership and implementation science necessary to lead innovation in medical education.
Wong appreciates the mentorship he has received specifically from RISE Core Team members Paula Thompson, Ph.D., program director, and Nikki Bibler Zaidi, Ph.D., evaluation and assessment director.
“Leveraging their expertise as people who have developed curricula in the past has been a big help. They offered feedback and guidance on the structure of the curriculum and helped with evaluation tools such as surveys and quantitative analysis,” Wong shared.
Wong is particularly grateful for RISE’s assistance in the evaluation area. “Without this support, tracking down experts to assist with evaluation would have been much more difficult.”
The work Wong and his team are doing is a big undertaking. Right now there is a national curriculum deficit in ML and AI throughout medical education — everything they are developing is novel content for which no prior curricula are available to offer insight. Being able to accurately evaluate the DATA-MD curriculum is key to making improvements that will allow future learners to successfully understand these topics.
“I want to empower learners and medical professionals to better understand ML and AI so they can confidently utilize these tools to aid their clinical decision,” shared Wong. “Over time, these tools will only continue to become more pervasive and I hope that this curriculum will help them navigate that space.”
To stay informed about RISE opportunities and education innovation happening at Michigan Medicine, join our community.
Photo credits: Camren Clouthier