What Does Google’s Breast Cancer Screening A.I. Actually Do?
The program was slightly better than human radiologists at spotting abnormalities in mammograms
/https://tf-cmsv2-smithsonianmag-media.s3.amazonaws.com/filer/2e/05/2e0547bc-39a3-4857-8d4c-17b791e70723/mamogram-yellow-box.jpg)
Google’s latest artificial intelligence tool designed to analyze mammograms is, by some measures, at least as effective as human radiologists, according to a new report. But some critics question whether researchers are applying A.I. to the right problem when it comes to finding and treating breast cancer.
About one-in-eight women in the United States and the United Kingdom develop breast cancer, and it’s the second deadliest cancer to women after skin cancer. That’s why most doctors recommend women over 45 get mammograms every year or two.
Mammograms are used to screen for breast cancer and catch it early, when it’s small and easier to treat. Spotting abnormalities on a mammogram isn't simple and some can go unnoticed. False negatives give tumors a year or more to grow and spread, so improved accuracy is a common goal. Not every abnormal result means cancer, so if radiologists notice something suspicious, the patient is referred to get a biopsy, during which a small piece of tissue from the suspicious area is removed either surgically or with a hollow needle and analyzed by a pathologist. Only about one-fifth of biopsies done in the U.S. turn out to be cancerous.
“There are many radiologists who are reading mammograms who make mistakes, some well outside the acceptable margins of normal human error,” Massachusetts General Hospital’s director of breast imaging Constance Lehman, who was not involved in the study, tells the New York Times’ Denise Grady. “This paper will help move things along quite a bit. There are challenges to their methods. But having Google at this level is a very good thing.”
A team of international researchers, which included experts from Google’s Deepmind, the Cancer Research U.K. Imperial Centre, Northwestern University, and Royal Surrey County Hospital, evaluated the tech giant’s latest foray into healthcare in a report funded by Google and published last week in the journal Nature.
First, the research team trained the A.I. system with a total of 91,000 mammograms and the corresponding biopsy results. Then, using an additional 27,000 mammograms, the team had the program predict which showed signs of cancer and compared its accuracy to the original evaluations.
Lastly, they had the A.I. read 500 more mammograms and then showed the same images to six radiologists who hadn’t seen them before.
“We took mammograms that already happened, showed them to radiologists and asked, ‘Cancer or no?’ and then showed them to A.I., and asked, ‘Cancer, or no?’” Mozziyar Etemadi, a specialist in cancer and A.I. at Northwestern University and co-author on the study, tells Grady.
At times, the A.I. identified suspicious tissue that the radiologists missed, and at other times, the radiologists caught something the program missed. But overall, the A.I. was more accurate. False positives were reduced by 5.7% in the U.S. and 1.2% in the U.K., and false negatives were reduced by 9.4% and 2.7% respectively.
The new study also isn’t without criticism, however. They don’t provide a demographic breakdown of the populations they studied, and unfortunately, algorithms in healthcare have a history of racial bias. It wouldn’t have been impossible to address: A previous mammogram-reading A.I. developed at MIT showed equal accuracy across races.
Other critics point out that women already face an overdiagnosis problem when it comes to breast cancer. Creating a device that’s really good at finding abnormalities in mammograms could make that issue worse.
When mammogram results are considered abnormal, it might still turn out to be non-cancerous, or one of three kinds of cancer: harmless; slow-moving and dangerous; or fast-moving and dangerous. A biopsy-proven cancer could be any of those three kinds, but only slow-moving, dangerous cancers can be and need to be cured.
Oh look, it's an example of AI being used to solve the wrong problem! https://t.co/A6P92XPwjk Study shows that AI can find more cancers on mammograms, but that's the wrong goal. We should aim to save lives, not turn more healthy women into cancer patients.
— Christie Aschwanden (@cragcrest) January 2, 2020
As science journalist Christie Aschwaden, explains on Twitter, overdiagnosis is not only a problem of false positives—where non-cancerous lumps are treated as suspicious after a mammogram—but leads to women with harmless cancers undergoing unnecessary surgery and chemotherapy.
In the end, the goal of creating an AI tool for breast cancer screening is to reduce the workload for radiologists—who are in short supply—and get faster results to patients facing breast cancer scares.
As cancer survivor and clinical nurse Theresa Brown writes in an opinion article about AI and cancer research for CNN:
[Facing a diagnosis for cancer,] “ordinary life looks like a dark maze of dead ends and shadowy twists and turns that move you forward, but ultimately take you nowhere. What the patient most wants is a light to penetrate those shadows.”
If Google’s AI can someday speed up the weeks-long process of waiting for a diagnosis and provide clearer results, Brown writes, it would provide that light for patients. In the past, computer-aided detection had the opposite effect, but improvements to AI and quality of data may improve outcomes.
“We can learn from the mistakes with CAD and do it better,” says Lehman, per the New York Times. “These systems are picking up things a human might not see, and we’re right at the beginning of it.”