AI-enabled imaging and diagnostics previously thought impossible

In partnership with health organizations globally, we’re researching robust new AI-enabled tools focused on diagnostics to assist clinicians. Drawing from diverse datasets, high-quality labels, and state-of-the-art deep learning techniques, we are making models that we hope will eventually support medical specialists in diagnosing disease. We’re excited to further develop this research towards new frontiers—and to demonstrate that AI has the ability to enable novel, transformative diagnostics.

Using sound to help researchers detect tuberculosis in India

India reports over 2 million Tuberculosis (TB) cases yearly—Google Research’s foundation model helps researchers build AI models that can flag early signs of TB through sound, potentially improving diagnosis for over 35 million people.
Learn more

Information Access

Dermatology Images

Information Access

Improving access to skin disease information

We conducted research to see if we could identify skin, hair, and nail conditions through computer vision AI and image search capabilities. Our research concluded that we can identify hundreds of conditions, including more than 80% of the conditions seen in clinics and more than 90% of the most commonly searched conditions. The work was highlighted in both Nature Medicine and JAMA Network Open.

Dermatology Images

A resource for dermatology images

Less than 20% of dermatology textbook images are estimated to feature dark skin tones. The Skin Condition Image Network (SCIN), created in collaboration with Stanford Medicine, addresses this by offering over 10,000 crowdsourced images, freely available as an open-access resource to help bridge important gaps for AI development, medical research, and equitable healthcare tools.
Learn more

Using AI to improve lung cancer detection

A promising step forward for predicting lung cancer

Lung cancer leads to over 1.8 million deaths per year world wide, accounting for almost one in five cancer deaths, and is the largest cause of cancer mortality. Our research, published in Nature Medicine, shows that deep learning may eventually help physicians more accurately screen for lung cancer and identify the disease even in incidental lung cancer detection workflows. Read the post

Novel biomarkers for non-eye related conditions

Anemia Detection

Computer Vision

Anemia Detection

Detecting hidden signs of anemia from the eye

The human eye can reveal signs of underlying diseases like anemia, a condition that affects 1.6 billion people worldwide causing tiredness, weakness, dizziness and drowsiness. In research published in Nature Biomedical Engineering, we were able to use deep learning to quantify hemoglobin levels and detect anemia with de-identified photographs of the back of the eye. This result means it’s possible that someday providers may be able to detect the disease with a simple non-invasive screening tool. Read the post

Computer Vision

Using computer vision to assess cardiovascular risk

Assessing the risk of cardiovascular diseases is the first and most critical step toward reducing the likelihood that a patient suffers a cardiovascular event in the future. By applying deep learning techniques to retinal images, we’ve been able to reveal factors associated with the risk of a major cardiovascular event like a heart attack or stroke, as published in Nature Biomedical Engineering. This research could help scientists generate more targeted hypotheses and drive a wide range of future research. Read the post

Using AI to improve breast cancer detection

Clinical Practice

Deep Learning

Clinical Practice

Studying how AI can help breast cancer screening in clinical practice

Breast cancer screening helps detect cancer earlier, but diagnosing breast cancer accurately and consistently remains a challenge, with half of all women experiencing a false-positive over a 10-year period. In Nature, we demonstrated the potential of our AI model to analyze de-identified retrospectively collected screening mammograms with similar or better accuracy than clinicians. Now, we’re collaborating on an investigative device research study to understand how the model can help in clinical practice to reduce the time from screening mammography to diagnosis, narrowing the assessment gap and improving the patient experience. Read the post

Deep Learning

Applying deep learning to metastatic breast cancer detection

In our pathology research published in the Archives of Pathology & Laboratory Medicine as well as The American Journal of Surgical Pathology, we showed how a proof-of-concept assistance tool (LYNA) could use deep learning to increase the accuracy of metastatic breast cancer detection. Read the post

Detecting signs of disease from external images of the eye

Exploring how external eye photos can reduce the need for specialized equipment

We’re doing research and building AI models that can not only decipher important information from retinal images, but rather from external eye images. In our research published in The Lancet Digital Health, we show that a deep learning model can predict the presence of diabetic retinal disease and other biomarkers such as HbA1c or eGFR from external eye images alone. This could reduce the need for specialized equipment and expand access to care for the growing population of patients with diabetes or other chronic diseases. Read the post

Further research in imaging and diagnostics

We continue to advance AI-enabled imaging research in other domains, expanding this technology to facilitate transformative diagnostics.

AI Advances

AI Learning

AI Diagnosis

AI Advances

Exploring AI advancements in radiotherapy planning to improve efficiency

Building off of work done with the University College London Hospitals and published in JMIR Publications, we are collaborating with Mayo Clinic to study the use of AI to help clinicians plan radiotherapy treatment for cancer. We’ve joined forces to research, train and validate an algorithm to assist physicians with segmenting healthy tissue and organs from tumors to reduce treatment planning time and improve the efficiency of radiotherapy, hopefully allowing clinicians to spend less time planning and more time with their patients. Read the post

AI Learning

Using machine learning to detect deficient coverage in colonoscopy screenings

Colorectal cancer (CRC) is a global health problem and the second deadliest cancer in the United States, resulting in an estimated 900K deaths per year. By alerting physicians to missed regions of the colon wall, our algorithm has the potential to lead to the discovery of more adenomas, thereby increasing the adenoma detection rate and decreasing the rate of interval colorectal cancer, as published in IEEE Transactions on Medical Imaging. Read the post

AI Diagnosis

Using AI to identify the aggressiveness of prostate cancer

To diagnose the severity of prostate cancer, biopsies are analyzed and given a Gleason grade, which is scored on comparisons to healthy cells. In work published in JAMA Oncology and JAMA Network Open, we explored whether an AI system could accurately Gleason grade prostate biopsies, and our results indicated that the deep learning system has the potential to support expert-level diagnoses. Read the post