*Contributed equally
The present study created an artificial intelligence (AI)-automated diagnostics system for uterine cervical lesions and assessed the performance of these images for AI diagnostic imaging of pathological cervical lesions. A total of 463 colposcopic images were analyzed. The traditional colposcopy diagnoses were compared to those obtained by AI image diagnosis. Next, 100 images were presented to a panel of 32 gynecologists who independently examined each image in a blinded fashion and diagnosed them for four categories of tumors. Then, the 32 gynecologists revisited their diagnosis for each image after being informed of the AI diagnosis. The present study assessed any changes in physician diagnosis and the accuracy of AI-image-assisted diagnosis (AISD). The accuracy of AI was 57.8% for normal, 35.4% for cervical intraepithelial neoplasia (CIN)1, 40.5% for CIN2-3 and 44.2% for invasive cancer. The accuracy of gynecologist diagnoses from cervical pathological images, before knowing the AI image diagnosis, was 54.4% for CIN2-3 and 38.9% for invasive cancer. After learning of the AISD, their accuracy improved to 58.0% for CIN2-3 and 48.5% for invasive cancer. AI-assisted image diagnosis was able to improve gynecologist diagnosis accuracy significantly (P<0.01) for invasive cancer and tended to improve their accuracy for CIN2-3 (P=0.14).
Every year, ~500,000 women are affected with cervical cancer worldwide and ~270,000 women succumb to this disease (
The traditional biopsy routine for cervical cancer diagnosis is that gynecologists manually observe the uterine cervix with a colposcope and decide where to obtain a tissue sample for more detailed microscopic examination. There are problems to this method. First, colposcopes are large and expensive. Second, gynecologists require a great deal of practical experience in deciding correctly from which part of the cervix is best to obtain the tissue.
To address this shortcoming, the present study created a system of AI-assisted image diagnosis (AISD) for cervical lesions. This AI system can guide the inexperienced in their selection of the best biopsy sites. If AISD for cervical lesions could be normalized for use in professional practice, the biopsy itself might become obsolete, or used only when absolutely needed for a definitive opinion. This economical and simple improvement in diagnostic capabilities would reduce the burden for gynecologists and could be expanded to medical facilities in localities, regions and advancing countries that have fewer medical resources. This would be conducive for provision of proper medical treatments and decreasing the overall cervical cancer burden (
Tanaka
The aim of the present study was to achieve AISD for cervical lesion using images taken by Smartscopy and report the performance assessment of AISD for cervical lesions taken by colposcope. This system could be subsequently applied toward Smartscopy images.
The present study was a cooperative research project with Kyocera Corporation, a maker of advanced smartphones and AI software. University Clinical Research Review Committee approved this research [17257(T7)-8]. All methods were performed in accordance with the relevant guidelines and regulations.
Colposcopy and biopsy were performed on 463 patients by gynecologic oncologists at the Osaka University Hospital between January 2010 and August 2019. The median age of the patients was 46 years (range 23-82). This is a retrospective study in which the patient data was fully de-identified. The present study was approved by the Institutional Review Board and the Ethics Committee of the Osaka University Hospital [approval no. 17257(T7)-8]. The researchers obtained informed consent from participants of the survey on the questionnaire, which was anonymous. The present study included only those who consented to participate.
A total of 463 images from 463 patients taken by colposcope were analyzed. The images were of pathological cervical lesions processed with acetic acid prior to biopsy. These images were cropped to 224x224 pixels and saved as JPEG files. Gynecologic oncologists annotated the images according to pathological lesions (
A randomly selected subset of 115 of the 463 images was employed as a ‘test dataset’ and the remaining 348 images were used as the training dataset (
The number of images was also increased. The use of triple images for the training dataset was investigated by adding rotated or blurred images and quadruple images were tested by changing the hue, chroma (purity or intensity of color) and brightness (HSV), as is the standard practice in computer image analysis.
GoogLeNet (Inception v1) (
During the period between October of 2020 and January of 2021, 100 images (25 images for each pathology category) were presented to a panel of 32 gynecologists in the Osaka University Graduate School of Medicine, Niigata University Graduate School of Medicine, Kanazawa Medical University, University of Occupational and Environmental Health, Kawasaki Medical University, Hiramatsu Obstetrics and Gynecology Clinic, Saito Women Clinic, Ladies Clinic Yagi and Maki Ladies Clinic. They diagnosed each image as belonging to one of the four categories. Next, they re-diagnosed every image after the AI diagnosis was revealed to them. Changes in human diagnosis and the accuracy of the AI-image diagnosis was assessed.
Using Medcalc (
The average accuracy of diagnosis for pathological lesions solely by AI was 43.5%. For the four categories, the accuracy was 57.8% for normal, 35.4% for CIN1, 40.5% for CIN2-3 and 44.2% for invasive cancer (
To improve accuracy, the number of images per slide for the training dataset were changed.
Next, whether increasing the number of images per slide could improve the accuracy of image diagnosis, as is standard practice in computer science was investigated. Tripling the number of images for the training dataset by adding rotated and blurred images and quadrupling the images by changing HSV was also investigated. However, none of these efforts improved upon the accuracy of using a single image.
The accuracy of the human diagnosis of cervical pathological images by gynecologists before knowing the diagnosis from AI was 64.8% for normal, 54.4% for CIN1, 54.4% for CIN2-3 and 38.9% for invasive cancer. Once they became aware of the AI diagnosis, the human diagnosis accuracy was 63.3% for normal, 51.1% for CIN1, 58.0% for CIN2-3 and 48.5% for invasive cancer (
AI is being applied across various disciplines, including phonetic recognition, image recognition, face recognition and automated driving technology. Similarly, AI applications are expected to evolve rapidly in many medical fields (
Incorporating AI into medical practices is expected to improve the medical environment across Japan. Patients could receive safer and more adequate medical services, the overload of medical professionals could be reduced and new methods of diagnosis and treatment could be developed.
The Japanese Ministry of Health, Labor and Welfare has selected six important areas for AI development (
The number of AI image recognition software has seen dramatic recent increase and there have already been reports of AI automated diagnosis being conducted (
As pioneers in this field, Hu
Yuan
Furthermore, Xue
Tan
In all of these reports, cervical pathology was divided into two or three categories, atypical squamous cells of undetermined significance, low grade (L)SIL (normal and CIN1) and high grade (H)SIL (CIN2 and over). The present study is the first (to the best of the authors' knowledge) to report the evaluation of AI image diagnosis using four categories. The average accuracy was 43.5% particularly for CIN2-3 and 44% for invasive cancer. This is lower than the accuracy of the other two categories. To further improve AI accuracy, the number of training dataset images for various methods was increased, but this was unsuccessful.
In the future, diagnosis using images captured by the Smartscope will be evaluated. It is hypothesized that AI accuracy might be improved with improved context and timing of image-acquisition.
The present study reported, for first time to the best of the authors' knowledge, on the integration of AI and human image diagnosis for uterine cervical pathological lesions. It was evident that AI-assisted image diagnosis could significantly improve the gynecologist's accuracy for diagnosing of the category of invasive cervical cancer and it tended to improve diagnosis accuracy for CIN2-3, but not CIN1 and normal.
When comparing the initial accuracy of AI and humans diagnoses, the accuracy of humans was higher for normal and CIN1 (64.8 and 54.4%, respectively. AI-assisted accuracy was higher for CIN2-3 and invasive cancer (58 and 48.5%, respectively).
For mammography screening, AI advances could be used to increase screening accuracy by reducing missed cancers and false positives. Salim
Schaffter
Humans are still responsible for any AI-assisted diagnosis in Japan. At present, it need not be argued ‘Which is better, human or AI?’ or ‘Will humans be dumped into the dustbin of medical history?’ Instead, we are looking toward a way to realize the powerful potential of human and AI cooperation in medicine.
The present study has a limitation. The accurate diagnosis rate of AI-based diagnosis is in the 40% range, which cannot be used in clinical practice. This could be attributed to the evaluation of AI image diagnosis in four categories in the current study. In previous reports (
For four categories of cervical cancer pathology diagnosis, the accuracy of AI image diagnosis was 57.8% for normal, 35.4% for CIN1, 40.5% for CIN2-3 and 44.2% for invasive cancer. AI-assisted image diagnosis significantly improved the diagnostic accuracy of the gynecologist for invasive cancer and tended to improve slightly the gynecologist's accuracy for CIN2-3, but it did not improve the gynecologist's accuracy regarding the categories of CIN1 and normal cervix.
The authors would like to thank Dr GS Buzard (Department of Obstetrics and Gynecology, Osaka University Graduate School of Medicine) for his constructive criticism and editing of our manuscript.
The datasets during and/or analyzed during the current study available from the corresponding author on reasonable request.
YI designed the study and interpreted the results, AM wrote the manuscript, designed the study and interpreted the results, YU designed the study and interpreted the results, YT, RN, AM, MSh, TE, MSe, TE, TS, KY, HH, TN, TM, KH, JS, JY, YT and TK performed sample preparation. AM and YI confirm the authenticity of all the raw data. All authors reviewed and approved the final manuscript.
The present study was approved by the Institutional Review Board and the Ethics Committee of the Osaka University Hospital [approval no. 17257(T7)-8]. The researchers obtained informed consent from participants of the survey on the questionnaire, which was anonymous. The present study included only those who consented to participate.
Not applicable.
The present study was a cooperative research project with Kyocera Corporation.
Diagram of AI-image-assisted diagnosis for cervical lesions. The traditional biopsy routine for cervical cancer diagnosis is that gynecologists manually observe the uterine cervix with a colposcope and decide where to obtain a tissue sample for more detailed microscopic examination. However, colposcopes are large and expensive and gynecologists require a great deal of practical experience in deciding correctly from which part of the cervix is best to obtain the tissue. Smartscopy is cheap and simple improvement. This AI system can guide the selection of the best biopsy sites by doctors not yet well-practiced with such decisions. It could be expected to be of help to reduce the burden of gynecologists and expand to medical facilities in advancing countries. AI, artificial intelligence.
Example of an annotated image. The left image is of a cervical pathological lesion processed with acetic acid prior to biopsy. The right image is annotated by a gynecologic oncologist, who specified the pathological lesion.
The distribution of images.
Normal | CIN1 | CIN2-3 | Invasive cancer | Total | ||||||
---|---|---|---|---|---|---|---|---|---|---|
Images | 120 | 120 | 113 | 110 | 463 | |||||
Training | Test | Training | Test | Training | Test | Training | Test | Training | Test | |
90 | 30 | 90 | 30 | 85 | 28 | 83 | 27 | 348 | 115 |
CIN, cervical intraepithelial neoplasia.
The accuracy of AI image diagnosis.
Accuracy (%) | |
---|---|
Normal | 57.8 |
CIN1 | 35.4 |
CIN2-3 | 40.5 |
Invasive cancer | 44.2 |
Total | 43.5 |
AI, artificial intelligence; CIN, cervical intraepithelial neoplasia.
Accuracy of AI image diagnosis of each group.
Group 1 25% of training case (%) | Group 2 50% of training case (%) | Group 3 75% of training case(%) | Group 4 100% of training case (%) | |
---|---|---|---|---|
Normal | 48.2 | 47.7 | 53.6 | 57.8 |
CIN1 | 19.9 | 29.7 | 30.1 | 35.4 |
CIN2-3 | 30.4 | 29.5 | 42.9 | 40.5 |
Invasive cancer | 54.1 | 52.5 | 46.3 | 44.2 |
Total | 36.4 | 37.9 | 42.1 | 43.5 |
AI, artificial intelligence; CIN, cervical intraepithelial neoplasia.
Significance of AI-assisted image diagnosis.
Lesions | Initial | AI-assisted | P-value |
---|---|---|---|
Normal | 518/800 (64.8%) | 506/800 (63.3%) | 0.57 |
CIN 1 | 435/800 (54.4%) | 409/800 (51.1%) | 0.21 |
CIN 2-3 | 435/800 (54.4%) | 464/800 (58.0%) | 0.14 |
Invasive cancer | 311/800 (38.9%) | 388/800 (48.5%) | <0.01 |
AI, artificial intelligence; CIN, cervical intraepithelial neoplasia.
Summary of AI reports.
Author (year) | Subject | (Refs.) |
---|---|---|
Hu |
Pioneer of automated visual evaluation of cervigrams | ( |
Xue |
AI assistance in colposcopy imaging judgment | ( |
Yuan |
High performance of AI diagnostic system | ( |
Xue |
Automated visual evaluation on smartphones | ( |
Miyagi |
AI colposcopy combined with HPV types | ( |
Tan |
AI assistance in thin-prep cytological test images | ( |
AI, artificial intelligence; HPV, human papillomavirus.