Artificial intelligence, which is bringing us everything from self-driving cars to personalized ads on the web, is also invading the world of medicine.
In radiology, this technology is increasingly helping doctors in their jobs. A computer program that assists doctors in diagnosing strokes garnered approval from the U.S. Food and Drug Administration earlier this year. Another that helps doctors diagnose broken wrists in X-ray images won FDA approval on May 24.
One particularly intriguing line of research seeks to train computers to diagnose one of the deadliest of all malignancies, pancreatic cancer, when the disease is still readily treatable.
That's the vision of Dr. Elliot Fishman, a professor of radiology at Johns Hopkins Medicine in Baltimore. Artificial intelligence and radiology seem like a natural match, since so much of the task of reading images involves pattern recognition. It's a dream that's been decades in the making, Fishman says.
"When I started in radiology, they said, 'OK, don't worry about reading the chest X-rays because the computers will read them,' " Fishman says. "That was 35 years ago!"
Computers still can't perform the seemingly simple task of reading a chest X-ray, despite sky-high expectations and more than a little hype around the role of artificial intelligence. Fishman is undaunted as he turns this technology on pancreatic cancer.
And that disease is a huge challenge. Only 7 percent of patients given a pancreatic cancer diagnosis are alive five years later. One reason the disease is so deadly is that doctors usually diagnose it when it's too late to remove the tumors with surgery. Fishman and his team want to change that, by training computers to recognize pancreatic cancer early. Working with Johns Hopkins computer science students and faculty, they are helping develop a tumor-detecting algorithm that could be built into CT scanner software.
Americans get 40 million CT scans of the abdomen every year, for everything from car accidents to back pain. Imagine if a computer program with expert abilities could look for pancreas tumors in all those scans.
"That's the ultimate opportunity — to be able to diagnose it before you have any symptoms and at a stage where it's even maybe too subtle for a radiologist to be able to detect it," says Dr. Karen Horton, chair of the Johns Hopkins radiology department and Fishman's collaborator on the project.
The challenge lies in teaching a computer to detect what a well-trained doctor knows to look for.
"Elliot and I are very subspecialized so we're really, really good," Horton says matter-of-factly. "We see more pancreatic cancer than probably anyone in the world."
She says if the computer algorithm could capture their collective knowledge about how to diagnose pancreatic cancer and give that expertise to the typical doctor, "you could be, I would argue, better than us, but certainly as good as us — which would mean better than most of the practicing radiologists."
Even a program perfectly attuned to finding patterns can't reliably recognize cancer if it hasn't been trained on reliable starting material.
When it comes to developing AI, "sometimes people say, 'oh just take a bunch of cases and put them in a computer and the computer will figure out what to do'," Fishman says. "That's nonsensical."
The Felix Project at Johns Hopkins, as the pancreas effort is called, pours a huge amount of human time, labor and intellect into training computers to recognize the difference between a normal pancreas and one with a tumor.
Of all the internal organs to deal with, "the pancreas is the hardest," Fishman says. "The kidney looks like a kidney, the liver's a big thing." On the other hand, he says, "The pancreas is a very soft organ, it sits way in the middle and the shape varies from patient to patient. Just finding the pancreas, even for radiologists, is at times a challenge."
Eva Zinreich, a retired oncologist, is up for that challenge. She is one of a team of medical experts who spend their days poring over CT scans and teaching the computer how to recognize the pancreas, other organs, and then, tumors within the pancreas.
She sits at a computer workstation, wielding a digital paintbrush.
"I'll show you in 3D because that's the fun stuff, ok?" she says as she sets about coloring in the aorta and other blood vessels on a scan.
Next, she colors the pancreas yellow.
"You see that shaded area?" she asks. "That's the tumor," and she proceeds to color it red.
It will take her almost four hours just to mark up this single scan. Four medical experts have been working full-time for well over a year on this project. They've done this painstaking work on scans from about 1,000 healthy people, and their tally of pancreatic cancer images is now approaching 1,000 as well, Fishman says.
They are feeding their annotated scans into the project's computer program and gradually teaching it to recognize the same signs of a tumor that radiologists now pick out of the scans.
At another workstation in the lab, radiologist Linda Chu is trying to make the computer system even more adept than Elliot Fishman and Karen Horton are at recognizing pancreas cancers. She's developing ways for the computer to look for patterns in the scan that the human eye can't pick out. It's interpreting textures in the images, rather than shapes and shading.
Chu says she's making tentative progress. For example, she's been training the software to identify subtle clues that distinguish between a benign cyst and cancer.
"We don't truly understand what the computer is seeing, but clearly the computer is able to see something in the images that us humans cannot comprehend at this point," Chu says.
But this is also part of the challenge of AI — if the computer highlights something that a human expert can't see, and it's not clear how it arrived at that conclusion, can you trust it?
"That's what makes the research interesting!" Chu says.
Computer science students from the Johns Hopkins University main campus are key to developing the software that's learning how to read and interpret the images that flow from Fishman's lab.
The Lustgarten Foundation, which is focused on pancreatic cancer, has provided nearly $4 million over two years to fund the Felix Project. Horton says if it's successful, all the information they collected on healthy people can be used as a starting point to study tumors elsewhere in the body.
"You could have Felix kidney, Felix liver, Felix lung, Felix, heart," she says. And they could all go together into the scanner software.
The project is named after the "Felix Felicis" good-luck potion, from the Harry Potter books. And, absent an effective magic spell, the laborious process is a reminder that success in bringing artificial intelligence to medicine will not be as simple as dumping piles of data into a computer and trusting that an algorithm will sort it all out.
You can contact Richard Harris at firstname.lastname@example.org.
MARY LOUISE KELLY, HOST:
Artificial intelligence is getting a foothold in medicine. Last week, the Food and Drug Administration approved a computer algorithm that can help doctors identify wrist fractures. Researchers at Johns Hopkins have much more ambitious plans, too. They are using AI to train computers to do something very difficult, diagnose pancreatic cancer. NPR's Richard Harris visited what is called The Felix project.
RICHARD HARRIS, BYLINE: Elliot Fishman has spent his career as a radiologist pushing one vision, to bring powerful computing into his field of medicine.
ELLIOT FISHMAN: When I started radiology, they said, OK, don't worry about reading the chest X-rays because in two years, the computers will read them. That's 35 years ago.
HARRIS: Computers still can't perform that seemingly simple task, despite sky-high expectations and more than a little hype around the role of artificial intelligence. Fishman is undaunted. In fact, he's taken on one of the deadliest malignancies, cancer of the pancreas.
FISHMAN: Five year survival is 7 percent.
HARRIS: One reason it's so deadly is doctors usually diagnose the disease when it's too late to remove the tumors with surgery. Fishman and his team want to change that by training computers to recognize pancreatic cancer early. Dr. Karen Horton, chair of the radiology department, is part of this effort.
KAREN HORTON: Elliot and I are very subspecialized, so we are really, really good, right? So we see more pancreatic cancer than probably anybody in the world. So if you could put - box in a computer every case that Elliot and I saw and let the computer learn from that, you could be, I would argue, better than us but certainly as good as us, which would be better than most of the practicing radiologists.
HARRIS: The idea is to build pancreatic cancer detection into CT scanner software. Americans get 40 million CT scans of the abdomen every year for everything from car accidents to back pain. Horton says imagine if a computer program with expert abilities could look for pancreas tumors in all of those scans.
HORTON: And that's the ultimate opportunity, to be able to diagnose it before you have any symptoms and at a stage where maybe it's even too subtle for a radiologist to be able to detect it.
HARRIS: The challenge lies in bringing this idea to fruition, Fishman says.
FISHMAN: You know, this idea sometimes people say is, oh, you just take a bunch of cases, you put them into the computer, the computer will figure out what to do. That's nonsensical.
HARRIS: Even a program perfectly attuned to finding patterns can't figure out what's cancer and what's not if it doesn't have reliable starting material. So the core of The Felix Project at Hopkins is to pour a huge amount of human time, labor and intellect into training computers to recognize the difference between a normal pancreas and one with a tumor. Of all the internal organs, Fishman says...
FISHMAN: The pancreas is the hardest. You know, the kidney looks like a kidney. The liver is a big thing, the spleen - the pancreas is a very soft organ, sits way in the middle - you saw the pictures - and the shape varies from patient to patient. Just finding the pancreas even for radiologists at times is a challenge.
This is my lab. So we do a lot of the 3-D here. And Eva will show you how we do some of the segmentation.
EVA ZINREICH: Eva Zinreich. How are you?
HARRIS: I'm Richard Harris. It's nice to meet you.
Eva is Dr. Eva Zinreich, a retired oncologist who now comes in to train computers to recognize the difference between a healthy pancreas and one with disease.
ZINREICH: So I show you in 3-D because that's the fun stuff, OK?
HARRIS: She mouses across the screen and starts filling in the various parts of a CT scan with a digital paintbrush. She starts with the biggest blood vessel, the aorta. The computer program helps her draw within the lines.
ZINREICH: It's pretty good, right?
ZINREICH: Now we can do whatever organ you would like to see.
HARRIS: Well, let's do the pancreas.
ZINREICH: OK, let's do the pancreas.
HARRIS: She colors the pancreas yellow and then pulls out a red pallet to mark a darker area inside the organ.
ZINREICH: You see this shaded area? That's the tumor.
HARRIS: It looks like a pretty big tumor.
ZINREICH: It's a very large tumor.
HARRIS: It will take her almost four hours just to mark up this single scan. Four medical experts have been working full time for more than a year on this project, and they've done this painstaking work for about 1,000 healthy volunteers and they're now approaching 1,000 images that have pancreatic tumors. In another workstation in the lab, Linda Chu is trying to make the computer system even better than Elliot Fishman and Karen Horton by having the computer look for signs of cancer in the scans beyond what the human eye can see.
LINDA CHU: I believe that there's more to the images.
HARRIS: She says she's making progress - for example, training software to identify subtle clues that distinguish between a benign cyst and cancer.
CHU: We don't truly understand what the computer is seeing, but clearly the computer is able to see something in the images that us humans cannot comprehend at this point so...
HARRIS: That's kind of exciting.
CHU: Yeah, very exciting work.
HARRIS: But it's also part of the challenge of AI. If the computer highlights something that a human expert can't see and you don't know how it arrived at that conclusion, can you trust it?
CHU: But that's what makes the research interesting.
HARRIS: Horton says if The Felix Project for pancreatic cancer succeeds, all the information they collected on healthy people can be used to study other organs.
HORTON: You could have Felix kidney, Felix liver, Felix lung, Felix heart. And but you have to attack it that way, each organ - what's the pathology, what's the cancer? - if you want to focus on cancer.
HARRIS: It's not as simple as dumping piles of data into a computer and just letting the algorithm sort it all out. Richard Harris, NPR News. Transcript provided by NPR, Copyright NPR.