According to the Compendium of Medicine (Gujin Yitong Daquan), "Looking, listening, probing and feeling the patient's pulse are the guiding principles for diagnosis". The first principle, looking, refers to observation of the patient's spirit, complexion, expression, posture, etc., so as to speculate whether his or her internal organs have been affiliated with any disease via facial observation. Recently, Professor Ji Xiangyang from the Department of Automation, Tsinghua University worked closely with Professor Zheng Zhe, vice president of Beijing Fuwai Hospital, to develop a coronary heart disease risk prediction algorithm based on face recognition. It is the first of its kind to predict the risk of disease through facial analysis.
As common diseases that pose serious threats to human health, cardiovascular and cerebrovascular diseases are chronic diseases caused by various factors over a long time. While being afflicted, the patient's face will gradually form features that are different from those of ordinary people. It had been found long ago that facial symptoms such as thinning or greying hair, wrinkles, creases on the earlobes, macular degeneration (usually around the eyelids) and keratopathy (deposits of fat and cholesterol) were linked to an increased risk of heart disease. How can we predict and quantify the risk of heart disease in patients by analyzing changes in their facial features, so that general practitioners can rapidly gain substantial diagnostic experience? Artificial intelligence offers a promising opportunity for a significant breakthrough in this area.
5,796 patients from eight hospitals in China were enrolled for the study. They received diagnostic information through procedures such as coronary angiography or coronary computed tomography angiography (CCTA). Patients were randomly divided into two groups—the training group (5,216 patients, accounting for 90%) and the validation group (580 patients, accounting for 10%). Nurses used cameras to capture a set of four facial photos for each patient: one frontal photo, two side profile photos and one top-view photo. Meanwhile, the patients were interviewed and registered to collect data on their socioeconomic statuses, lifestyles and medical histories. In addition, the radiologist examined the patients' angiograms and assessed the severity of their heart attacks based on the number and location of blood vessels that shrank by 50% or more. It's worth noting that these information will be used later to create, train, and validate the AI's deep learning algorithms.
Detection flow diagram
The researchers then tested the algorithm on 1,013 patients from nine Chinese hospitals, who were mostly Han Chinese and were registered between April 2019 and July 2019. The study showed that the deep learning algorithm was superior to existing methods for predicting heart disease risk (the Diamond-Forrester model and the CAD Consortium Clinical Score). In the validation group, the algorithm correctly detected heart disease in 80% of cases (true positive rate, or "sensitivity") and incorrectly detected heart disease in 61% of cases (true negative rate, or "specificity"). In contrast, the sensitivity and specificity of the test group were 80% and 54% respectively. "The additional clinical information did not improve its performance, which means it can easily predict underlying heart disease based on facial photos alone," Professor Ji Xiangyang said. He also said the cheeks, foreheads and noses of the patients provided more information than any other spot on their faces. However, additional research is required to further improve the specificity of the algorithm, as the high false positive rate may lead to unwanted anxiety and inconvenience to patients, potentially resulting in an increased number of unnecessary tests being performed.
The cheeks, forehead and nose provided more information for the algorithm than any other spot on the face did
A paper about the study was published in the European Society of Cardiology (ESC), a leading journal in the study of heart disease. Lin Shen, PhD student of Fuwai Hospital of Chinese Academy of Medical Sciences, and Li Zhigang and Fu Bowen, PhD students from the Department of Automation, Tsinghua University, are co-first authors of the paper. Professors Ji Xiangyang and Zheng Zhe are the corresponding authors of the paper.
"In general, this study highlights the new potential of AI algorithms in medical diagnostics," Charalambos Antoniades, professor of cardiovascular medicine at the University of Oxford in the United Kingdom, and his student, Dr. Christos Kotanidis, wrote in an editorial article entitled "Selfies in cardiovascular medicine: welcome to a new era of medical diagnostics", "The significance of this research is that its deep learning algorithm requires only a simple facial image as data input to make it scalable." They also said that using selfies as a screening method is simple and effective for screening the general population, and that testers can then make a more comprehensive clinical assessment based on the results. The selection process will allow stratified populations to enter the healthcare system for first-line diagnostic tests of CCTA. This approach will benefit screening for cardiovascular disease in underdeveloped and low-income areas. It's worth noting that Antoniades and Kotanidis also highlighted the limitations of the study, including the low specificity of the test which means that the test needs to be refined and validated in a larger population, and raised ethical concerns regarding the practice of "judging people by their looks."
Professor Ji Xiangyang is the director and a tenured professor of the Institute of Brain and Cognitive Science, Department of Automation, Tsinghua University, who has long been committed to exploring the potential of intelligent perception and artificial intelligence algorithms in medical diagnosis. The development of facial recognition based coronary heart disease risk prediction algorithms indicates that more similar tools will emerge in the field of medicine in the future. When that happens, perhaps we can achieve a rough assessment of our physical condition without leaving home. Then by taking necessary precautions in advance, we can effectively prevent "minor illnesses" from escalating into "serious illnesses", as mentioned in the Yellow Emperor's Internal Canon (Huangdi Neijing).
