Breast cancer is the most common malignancy in women worldwide and despite advances in research and therapy, it remains a highly lethal malignant disease that constitutes a heavy burden to public health in >154 countries, with an ~600,000 per year (1,2). In Chile, it has been the leading cause of cancer-associated death since 2009 with 11.8% of deaths from cancer in women per year and a high level of mortality in elder patients. The average number of patients diagnosed with this pathology has improved considerably in recent years; several symptoms are prevalent, especially in the population with inadequate access to national health systems (3,4). The recommended treatment according to the molecular subtypes agreed upon in the St. Gallen consensus has shown favorable effects on patient survival (5,6). According to the St. Gallen protocol, the histopathological aspects of the mammary tumor are defined in a standardized manner, and recommendations for adjuvant therapies are decided based on observation of tumor biopsies under the microscope (5). In this context, the diagnosis of tumors using a minimal panel of molecular techniques, where immunodetection of hormonal receptors [estrogen receptor (ER) and progesterone receptor (PR)], Ki67 antigen, and epidermal growth factor receptor 2 (HER2), is central to pathological analyses due to their fundamental role in appropriately establishing the therapeutic strategy and prognosis for each patient, and for ultimately reducing breast cancer-associated deaths (5,7,8).

Specifically, HER2 protein is overexpressed in 10–25% of invasive breast cancer cases (2,7). Detection of HER2 in tumor cells through histological analysis is an important measure of prognostic and predictive outcomes in invasive tumors (2,9,10). The detection of HER2 by microscopy is more challenging than for the other antigens due to the complex pattern of HER2 expression within cells, which often requires the use of complementary techniques such as fluorescent in situ hybridization (FISH) to detect the gene encoding ERBB2 (8,11,12). The American Society for Clinical Oncology (ASCO) has proposed a specific qualitative algorithm for HER2 using immunohistochemistry (IHC), based on the intensity of the signal, the completeness of the pattern of expression in the cell membrane, and the approximate minimum percentage of expression (8). Following this approach, invasive tumor samples can be categorized as HER2 0, 1+, 2+, or 3+, where the last category corresponds to upregulated expression protein and represents cases that should be treated with anti-HER2 therapy (12,13). Being able to adequately differentiate the HER2-IHC 2+ category (equivocal) from a false positive, or 3+, by traditional microscopy is crucial from several points of view, including appropriate referral to the oncology service and subsequent suitable adjuvant treatment with trastuzumab in this pathology and in gastric cancer (14–16). However, due to the analytical complexity of this biomarker, the HER2 category represents an important obstacle when it comes to determining the histopathological diagnosis under the St. Gallen protocol, with unfavorable consequences on the diagnostic procedure, treatment strategy, and public health (17).

Analyzing the intensity and distribution of the IHC signal in situ provides important information for the diagnosis and prognosis of cancer. Visual scoring of markers is currently employed, using light microscopy with medium magnification capacity (×20 or ×40 objective lenses) (15,18). The tumor can be scored as 0/1+ (negative), 2+ (equivocal), or 3+ (positive) according to official guidelines such as those proposed by ASCO (8,12); however, the microscopic observation of the IHC performed by the pathologist is highly time-consuming and suffers from high inter- and intra-observer variability, with the results being conditioned by subjective factors (19,20). In contrast, analysis performed by computational tools shortens the time required and decreases inter-operator variation in the evaluation of staining levels (20,21). Used for the first time for immunostaining in 1980, computerized image analysis has since been applied in several studies, in combination with innovations in laboratory techniques for protein and gene detection; some examples include AQUA Technology (22), HERmark (23), Mammaprint (24), Oncotype Dx (25), and PAM50 Breast Cancer Intrinsic Classifier (26). Unfortunately, most assays that provide quantitative data work with reference centers and laboratories that do not fit the needs and capabilities of public health systems in Latin America (27). This strengthens the importance of developing low-cost technologies for high-precision analysis and generating digital references of existing material. In addition to this problem, the existing computerized analysis techniques of immunohistochemical signals only integrate 1–2 image parameters to qualify the diagnosis through defined thresholds based on controls (15,21,23). Due to these limitations, innovative techniques are needed to integrate a large number of quantitative parameters to build a mathematical classification model that could be optimized by data training itself (28,29).

Machine learning (ML) and observation with computational tools have been used in a wide variety of clinical tasks ranging from the segmentation of medical images to generation, classification, and clinical prediction (30–33). Different research centers and biotechnology corporations have been exploring the use of artificial intelligence (AI) and ML in key clinical areas (30,34–36). Specifically, ML classified as Supervised Learning allows for the resolution of classification problems where the machine must learn to be able to predict discrete values. This means that the machine can predict the most likely category, class, or label for new examples, or solve regression problems to predict the value of a continuous variable (28,37). In certain problems, the response variable is not normally distributed, for example, a coin toss can only have two outcomes: Positive or negative. In these cases, the use of logistic regression as a classification model is relatively popular and has been used regularly in both industry and medicine (38). The performance of this response is shaped thanks to a mathematical optimization procedure; the use of the cost function in conjunction with the downward gradient that derives from the maximum likelihood makes it possible to find weight coefficients for each input parameter that maximize the performance of the classifier (39).

Despite the promising properties described for the ML classification, the constant increase in the complexity of learning algorithms has led to the frequent appearance of ‘black box’ models, where the interpretation of their internal functioning is very difficult with traditional analytical methods (40). These models are increasingly used in decision-making in important contexts, such as in clinical development, implying that the demand for transparency is essential when decisions based on AI are unjustifiable in real life (41,42). In this context, the concept of explainability makes it possible to detect and correct the training bias, provides robustness by detecting disturbances in the prediction, and establishes arguments that allow us to understand changes in the performance of the classification tasks. Different systems that are coupled to ML models allow opening the black box, such as decision trees, visualization of inputs and outputs through graphs, and Shapley Additive exPlanation (SHAP) values, where the latter stands out primarily for being applicable to any classification model, with easily understandable graphic interpretation and allowance of both explanation and advice (41,43).

The primary question of this study is whether it is possible to obtain a classification as good as pathologists' using ML and photomicrographs features. Additionally, we attempt to understand how the classifier is modified when including information provided by FISH.

The complex expression pattern of HER2 within cells presents a challenge for pathologists and the high inter-observer variability highlights the necessity of tools that allow for improved predictive results to better tailor treatment strategies. Deep learning automatic-assisted diagnosis is the most suitable and potentially accurate solution. Automatic quantification of images allows for the factorization of two key points highlighted in the HER2 classification algorithm recommended by ASCO. First, the visual field signal intensity in the low, medium, and high categories, a continuous variable of 256 dimensions (8-bit depth), is internationally used in other immune detection techniques in brightfield and fluorescence formats (45,46). The second parameter corresponds to the completeness of the staining around the cell membrane, whose quantitative estimation still represents a significant challenge for the pathologist. In the present study, the signal was transformed into an inverted binary, where the surrounding blanks by the nuclear signal were converted into particles with a size ranging between 250 and 2,500 pixels2 and medium circularity when working with images with a standardized resolution of 1,000 pixels in width (47). This strategy allows for the generation of statistically significant differences when comparing amplified vs. normal HER2 expression samples (set of positive vs. negative controls), based on a reduced analytical complexity in comparison with current analytical methods of membrane signals from other research groups that use a skeletonization technique or cell segmentation through the specialized filters (48,49). Moreover, our automated process also used a low complexity method to extract the IHC signal by decomposing the colors into RGB data matrices pixel by pixel, where the combinatorial intensity in red, green, and blue allowed the pixels to be separated. A promising approach to refine the signal could be to use the technique proposed by Fu et al (18), with a filter detection complex for brown color generated by the DAB chromogen typically used in immunohistochemistry.

With the parameters obtained, it was possible to train a logistic regression ML model for classification and explainability, and we were able to show significant changes in the classification performance when the training was performed based on the diagnosis supplemented with FISH results: When evaluating the basic performance parameters, the classifier showed high precision and accuracy in the two trained models (IHC: 0.84 and 0.72 vs. IHC + FISH: 0.97 and 0.89, respectively), highlighting that both values increased when the FISH criterion was added. This obtained data increases the performance and seems to be primarily explained by reducing the output impact of the parameter associated with the global intensity of IHC MGV and increasing the weight of the subcellular location of the protein, in this case towards the plasma membrane through the positive cells, the criterion of completeness (COUNT). In this regard, although IHC takes advantage of the high affinity of the antigen-antibody reaction that allows identifying the expression of biomarkers or proteins, its results, known as immunoexpression, immunostaining, and/or chromogenic signal intensity, are influenced by multiple factors from the pre-analytical to the post-analytical phase, and depending on the performance of these stages, several results can be obtained even using the same antibody, affecting clinical interpretation (50,51). If each stage of the preanalytical phase of histopathology is analyzed in detail, the first step of taking a sample for biopsy and fixing the tissue in formaldehyde solution is one of the most important steps, as the result of the IHC is highly dependent on the time required to fix a sample after extraction: The cold ischemic time must be as low as possible, since a prolonged cold ischemic time leads to tissue acidosis, enzymatic degradation, and altered immunoreactivity, resulting in artifactual lower antigenic expression (50,51). Conversely, over-fixing may occur if fixing times exceed 72 h in small samples such as core breast biopsies. This increases variability and can generate false negatives/positives at the time of interpretation when qualitatively assessing the staining intensity (50). Finally, another preanalytical factor is the histological paraffin section technique. In IHC, the optimal section thickness is 3 µm, when these sections are thicker (>5 µm), they can generate an overgrowth of the staining intensity during the IHC-reveal stage, due to overlapping of the cell layers, which saturates the reaction and may produce an more intense signal, inappropriately augmenting intensity instead of the completeness of the cell membrane, which can generate a false diagnosis of +3 HER2, translating into a completely different treatment and prognosis (52–54).

Based on the performance improvements obtained by including the FISH results as an extra dimension in the reference diagnosis, it is hypothesized that the addition of more features to the ML algorithm training, that include the construction of a training supervision matrix based on unsupervised learning will improve predictive ability. This complementary information may include the basic histological subtype of the tumor, the expression pattern in lymph node metastases, the response to treatment, and patient survival, to better study the predictive variables of each tumor (2,55–57). With this increase in the dimension of the known output variables, the performance of the test may potentially improve, specifically in the recall values, which were low in this study (IHC: 0.43 and IHC + FISH: 0.55), this may assist in determining the proportion of actual positives that were correctly identified (58). Low false positive detection is very attractive in practice regarding the implications of a diagnosis of breast cancer and referring the patient to the oncology service, with tremendous burden in terms of complexity of health services, costs, and impact to the patient in vain (2,59). In addition, the general performance of the test through the ROC analysis, showed that this HER2 classifier had AUC values of 0.81 and 0.94 for training with the IHC and IHC + FISH sets respectively, a very positive scenario since values between 0.8–0.9 are considered very good for complex and high-throughput detection, such as in blood biochemical profiling (60,61).

Finally, the classification model presented here can potentially be trained with a larger number of samples to generate a robust digital pathological reference, through a free access interoperability platform for public health services, just as pathology laboratories do in developed countries, applying data management systems currently available on the market, but at a high cost (62–64). Furthermore, the methods and technology proposed here have been developed motivated by previous research in this field, to test the performance of different classification algorithms for breast cancer, through the unitary evaluation of multiple molecular markers in existing protocols such as Ki67, ER, and PR (6,65,66), or in proposals under development for molecular classification including new markers such as PDL1, Claudin, MCL-1, and TRPV1 (67–70). The ability of ML classifiers to decrease the inter-observer variability improves the accuracy of breast cancer diagnoses, which may have important implications since treatment is based on the subtype of tumor diagnosed (5,54,71). This approach has the potential to reduce breast cancer mortality with a significant impact on health systems and affected populations. Our approach however has some limitations. The model presented here is only limited to diagnosis and does not provide analytical evidence to improve tumor prognosis, as it does not integrate other relevant immunohistochemical biomarkers such as Ki67, or hormone receptors to subclassify the tumor according to the risk and the treatment corresponding to the standard categories (6). Additionally, the model requires an image preprocessing phase, which increases the level of complexity and time required for analysis. Therefore, the implementation of deep learning models based on convolutional neural networks that allow working directly with the images and integrating other biomarkers in a 4-dimensional hyperimage (32,35), could be explored in the future to propose an easy-to-use procedure to be assessed in clinical trials.

In conclusion, our current findings suggest that ML classifiers may be an important contribution to pathology services and pathologists as, despite their limitations, they can assist in obtaining a more accurate diagnosis of HER2 immunohistochemical expression as a tool to predict future potential patient outcomes.