eISSN: 1896-9151
ISSN: 1734-1922
Archives of Medical Science
Current issue Archive Manuscripts accepted About the journal Special issues Editorial board Abstracting and indexing Subscription Contact Instructions for authors Ethical standards and procedures
SCImago Journal & Country Rank
vol. 13
Clinical research

Application of artificial neural networks for predicting presence of non-sentinel lymph node metastases in breast cancer patients with positive sentinel lymph node biopsies

Tomasz Nowikiewicz
Paweł Wnuk
Bogdan Małkowski
Andrzej Kurylcio
Janusz Kowalewski
Wojciech Zegarski

Arch Med Sci 2017; 13, 6: 1399–1407
Online publish date: 2016/05/05
Article file
- application.pdf  [0.09 MB]
Get citation
JabRef, Mendeley
Papers, Reference Manager, RefWorks, Zotero


Breast cancer is the most common malignant disease in women in Poland. In 2010, a total of 15,981 new cases were diagnosed in our country [1, 2]. 22.4% of cancer incidence in women was accounted for breast cancer, being the cause of 12.8% of oncological deaths [2]. With regards to women, neoplasm of breast (along with trachea, bronchus and lung) contribute to the highest number of years of life lost [3].
Axillary lymph node conserving treatment considerably reduces the risk of complications of axillary lymphadenectomy [4–6]. This was possible due to wider use of sentinel lymph node biopsy (SLNB) in breast cancer patients without clinically suspicious axillary lymph nodes [4–6].
The involvement of other axillary lymph nodes is expected in about 40% of breast cancer patients with metastatic SLNs [7]. More than half of these patients do not benefit from complementary axillary lymph node dissection (ALND) leading to lymphadenectomy-related complications.
The presence of metastases in non-sentinel lymph nodes (nSLN) mostly depends on the size of metastases (AJCC classification [8]) found in the sentinel lymph node (SLN). In a study by Kohrt et al., if isolated tumor cells (ITC) were prezent in SLN, metastatic nSLNs were found in 4.7% of patients. In case of SLN micrometastases nSLNs metastases concerned 42% of patients.. The size of SLN metastases exceeded 2 mm in 71% of examined patients [9].
The addition of other risk factors led to the development of systems for numerical evaluation of the probability of finding metastases in nSLN. The available scoring scales include clinical data obtained from pre- and post-SLNB records. High clinical value of these systems has been proven in multiple studies [10–20]. It was also demonstrated that use of these scales may help in the therapeutic decision-making process [21–23].
The methodology of developing the nomograms was based on selection of factors influencing the rate of occurrence of metastases in nSLNs in a statistically significant manner. Different systems use multivariate analysis including different (for each of them) sets of selected data. The Memorial Sloan-Kettering Cancer Center (MSKCC) nomogram includes eight clinical parameters: size of pathology, multifocality, histologic form with histologic tumor grade, involvement of vascular structures, estrogen receptor status, number of resected lymph nodes during SLNB: with and without metastases, and method of detection of metastases in SLN (also available on-line: http://nomograms.mskcc.org/breast/) [10]. Other scales evaluate fewer variables – from seven (MOU) [17] to only three (Tenon [18], Stanford [9] and Cambridge [19]).
The aim of this study was to present a new model of predicting the incidence of metastases in nSLNs in early breast cancer patients.

Material and methods

Analyzed group

One thousand five hundred eighty-three patients with early-stage breast cancer were scheduled for SLNB between January 2004 and December 2012 in the Bydgoszcz Oncology Center. Metastatic SLNs were found in 348 patients (22%) – SLN(+) patients.
The study included all consecutive patients with pathologically proven early invasive breast cancer without clinically and radiologically suspected lymph nodes (cN0). Patients who underwent neoadjuvant treatment were excluded from the study. Complementary ALND was performed in the majority of SLN(+) patients – 94% (327). These patients comprised a study group subjected to a retrospective analysis. Data obtained as a result of the analysis were used for statistical calculations and for creation of a new prognostic model.
Involvement of nSLNs was identified in 32.1% (105/327) of patients following ALND (Figure 1) – nSLN(+) patients.
Prospective evaluation of the new test was based on a group of 365 consecutive patients hospitalized in the Bydgoszcz Oncology Center and subjected to SLNB. SLN metastases were diagnosed in 73 patients and selective ALND was performed in 67. The proportion of nSLN(+) patients was 35.8% (24/67).
The treatment protocol was approved by the Local Bioethical Committee and all patients signed informed consent. The analysis had no influence on therapeutic process (surgical procedures, adjuvant treatment). Collected clinical data were stored with no possibility of patients being identified by third parties.

Selection of evaluated data

Factors increasing the risk of tumor metastases in nSLN were defined. Obtained results were presented in the form of a nomogram, which was developed by using an artificial neural network for statistical calculations.
The correlation between nSLN involvement and numerous variables including selected epidemiological data (menopausal status, age, carcinoma of contralateral breast, family history and gynecological history), primary tumor features (histological type, size, palpability and multifocality, comedo component, suspicious microcalcifications in mammography, grading, estrogen (ER) and progesterone receptor (PR) expression and HER2 status, molecular type of the tumor and Ki-67 index), details of diagnostic and therapeutic management (methods of preoperative tumor diagnostics, need for mastectomy, SLN mode, type of SLN metastasis and method of detection, presence of intravascular tumor emboli, infiltration of lymph node capsule, number of dissected SLNs, number of involved SLNs) were calculated.

Clinicopathological data acquisition

ER, PR, Ki-67 and HER2 levels were assessed immunohistochemically (IHC). Tumors were deemed positive for these receptors if at least 10% of invasive tumor cells in a section exhibited nuclear staining. Histologic grading was defined according to the Scarff-Bloom-Richardson system. HER2 positivity was defined as a 3+ staining intensity score at immunohistochemical analysis for HER2 protein or for HER2 gene amplification by fluorescence in situ hybridization (FISH).
Based on IHC or FISH findings of ER, PR, HER2, and Ki-67 expression, the study population was divided into five subtypes: luminal A (ER positive and/or PR positive, HER2 negative and Ki-67 ≤ 14%); luminal B1 (ER positive and/or PR positive, HER2 negative and Ki-67 > 14%); luminal B2 – luminal HER2 (ER positive and/or PR positive and HER2 positive, irrespective of Ki-67 expression); HER2-overexpressed (ER negative, PR negative and HER2 positive); and basal-like – triple-negative breast cancer (TNBC: ER negative, PR negative and HER2 negative).

Statistical analysis

The initial stage of calculations included a univariate analysis of the influence of evaluated clinical parameters on the presence of metastases in nSLN. The differences between groups were evaluated by the 2 test. It allowed to define factors associated with significantly increased rates of tumor metastases in nSLNs.
Multivariate analysis for creating a new nomogram was done. Unlike former prognostic systems, all the factors (including those not showing a statistically significant influence on the presence of metastases in nSLN) were included. Statistical calculations were performed using an artificial neural network with MLP 40-5-2 structure and BFGS learning algorithm (SOS error function, activation function of the neurons in the hidden layer – logistic function and activation function of the neurons in the output layer – logistic function). Predictive accuracy was assessed by calculating the area under the receiver operating characteristic (ROC) curve (AUC).
SPSS for Windows version 17.0 (SPSS Inc., Chicago, IL) was used for all statistical analyses. A p-value below 0.05 was considered statistically significant.
The predictive value of other prognostic systems (MSKCC [10], Tenon [18] and Stanford [9] nomogram) was also assessed in analyzed subjects.
Patients with complete sets of necessary clinical data resulting from the methodology of each test were qualified for calculations.


Three hundred and twenty-five women and 2 men were included in the retrospective group. Median age was 53 years (range: 23–80 years). Median histopathological size of excised primary invasive carcinoma was 21 mm (range: 4–70 mm). Detailed characteristics of the analyzed patients are presented in Table I.
Median number of harvested SLNs was 2 (range: 1–13, mean: 2.8). Median number of metastatic SLNs was 1 (range: 1–6, mean: 1.4).
Median number of removed lymph nodes at the completion of ALND was 14 (range: 1–37, mean: 15.9). Median number of metastatic non-sentinel lymph nodes was 3 (range: 1–25, mean: 4.3).
For some of the analyzed variables, such as SLN capsule infiltration (p < 0.0000001), tumor emboli (p < 0.0000001), number of involved SLNs (p < 0.000001), size of SLN metastasis (p < 0.000014), method of the diagnosis of metastasis (p < 0.008), and clinical staging (p < 0.008), univariate analysis showed a significant correlation with the presence of metastatic nSLN lesions.
Other analyzed factors did not show a significant correlation (p > 0.05).
Based on a subsequent multivariate analysis (using an artificial neural network) including all baseline factors, the variables used in the new prognostic system were defined. Among the data that increased the rate of nSLN(+) patients in a statistically significant manner, the developed nomogram included three out of six of the above-mentioned variables (SLN capsule infiltration, size of SLN metastasis, tumor emboli). In the case of the remaining variables, statistical calculations included another 14 analyzed factors – Table II.
An AUC coefficient of 0.87 was obtained (95% confidence interval (CI): 0.81–0.92). Sensitivity was 69%, specificity 86%, accuracy 80%, positive predictive value (PPV) 70% and negative predictive value (NPV) 15% (retrospective group).
For the MSKCC nomogram, the calculated AUC value was 0.71 (95% CI: 0.65–0.78), for Stanford 0.68 (95% CI: 0.61–0.74), for Tenon 0.67 (95% CI: 0.61–0.73). Figures 2–5 show ROC curves for particular computational models.
Clinical value of the new system was validated by means of a prospective analysis of a group of patients subjected to SLNB after the system had been developed. Sensitivity was 77%, specificity 46%, accuracy 66%, positive predictive value (PPV) 72% and negative predictive value (NPV) 48% (prospective group).


Methods of statistical calculations used to create the previously published prognostic tests were based on a multivariate analysis of some selected variables. Data included in that analysis concerned clinical parameters which significantly increase the rate of finding metastatic lesions in nSLN [9, 10, 17, 24]. However, it seems that using that kind of methodology of calculation could be a direct cause of problems with obtaining higher values of the AUC coefficient than the ones already obtained (including limitations concerning sensitivities and specificities of particular nomograms). This was the cause of lack of progress in developing new tests.
Using more sophisticated mathematical methods (artificial neural networks) allowed better results to be achieved, especially concerning the value of the AUC coefficient (and thus, sensitivity and specificity of the developed system).
The observations presented above corroborate the results of probability calculations concerning occurrence of metastatic nSLNs obtained with three other prognostic tools (MSKCC, Tenon, Stanford). In the analyzed group of patients (retrospective group) only the MSKCC nomogram obtained AUC values exceeding the expected level of 0.70. Two other systems scored less satisfactory results.
SLN(+) patients diagnosed with metastases not exceeding 2 mm in size (ITC, micrometastases) do not require ALND as complementary surgery [25, 26]. The results of the randomized ACOSOG Z0011 trial [27] by Guliano et al. and the panel expert recognition for the therapeutic option suggested in that study gained at a conference in St Gallen [25, 28] increased the possibility of axillary conservative treatment of selected patients with SLN micrometastases. Use of nomograms (especially those with the highest prognostic value) can make the choice of further treatment notably easier.
Due to the method used for scoring of evaluated variables and the type of predicted risk of nSLN metastases, this system is considered a nomogram. It was based on multivariate analysis of available clinical data conducted using an artificial neural network. Calculations estimate the probability of occurrence of the predicted event (presence of nSLN metastases) for each individual patient.
Clinical parameters most frequently used in the previously developed statistical tools included the size of the primary tumor and the type of metastatic lesion in SLN [9, 10, 17–24]. In these calculations, both variables showed a statistically significant influence on the probability of finding metastatic nSLNs (univariate analysis). However, among the data evaluated in the new nomogram, only the latter parameter turned out to be decisive, while most of the variables used by the artificial neural network had a p-value of less than 0.05.
The expression of estrogen and progesterone receptors and overexpression of HER2 in breast cancer patients are important predictive and prognostic factors.
Determination of the ER, PgR, HER2, and Ki-67 status is essential in order to divide invasive breast cancers into biological types [30]. Defining which group the cancer belongs to influences planning of further treatment. Assumptions concerning mainly the details of systemic treatment were included in generally accepted standards of treatment of patients with breast cancer. Clinical relevance of individual prognostic factors (ER, HER2, histological grading) was also taken into account [25, 28, 30–34].
Molecular type of tumor has been tested by other groups for association with nSLN metastasis [9, 10, 18], but there seems to be no significant correlation between molecular type and metastasis. The Ki-67 index was also evaluated for prediction of nSLN metastasis, but it did not significantly correlate with nSLN metastases [18]. Inclusion of the value of the Ki-67 mitotic index and biological type of breast cancer is therefore an additional measure of innovation of the examined calculation method.
In the group of prospectively analyzed patients (stage of initial clinical evaluation of the new nomogram) the increase in sensitivity and positive predictive value (PPV) of 77% and 72% vs. 69% and 70%, respectively (for the retrospective group) was observed. Significantly lower specificity (46% vs. 86%) and accuracy (66% vs. 80%) were found. This means that a patient identified with the new computational model as suspicious – nSLN(–) – does not require verification of that diagnosis in an invasive way (surgery). On the other hand, suspected nSLN(+) may be a vicious circle in half of the patients and thus requires further diagnostics (ALND).
The presented assumptions require further verification through use of the new model in a larger group of prospectively studied patients.
Apparent shortcomings of this new prognostic tool – a large number of variables and need for employing an artificial neural network for calculations – are in fact its advantages. They allow for acquiring a high AUC value, which was unprecedented in previous numerical systems. Therefore, the calculated probability of occurrence of nSLN metastasis strongly correlated with the results of postoperative histopathological examination.
However, the complex statistical analyses resulting from the chosen method of data processing will soon be replaced by the on-line calculator available on the site of Bydgoszcz Oncology Center (www.cobydgoszcz.pl). The availability of this tool will facilitate, in our center as well, the use of a scoring system in everyday clinical practice and in prospective studies.
The nomogram, following the example of previous solutions of this type, received its own name – BRDA, which is an acronym for Before Rapid Dissection Answer. The name of the test also refers to the specifics of the geographical location of Bydgoszcz Oncology Center.
In conclusion, in the analyzed group of patients, only the MSKCC nomogram and the new model for prediction of nSLN involvement exhibited AUC values exceeding the expected value of 0.70. To develop the new test, variables such as biological tumor subtype and Ki-67 index value were used. Both factors are known to be of predictive value for assessment of treatment results among breast cancer patients. However, until now they have not been used to predict the presence of nSLN metastatic lesions.
The BRDA nomogram performs well in prospective validation on patient series. The overall assessment of clinical usefulness of this test will be possible after applying it in prospective studies. It should also be used in other patient populations.

Conflict of interest

The authors declare no conflict of interest.


1. Wojciechowska U, Didkowska J. Nowotwory w Polsce w 2012 roku. Nowotwory J Oncol 2013; 63: 197-216.
2. Didkowska J, Wojciechowska U. Nowotwory piersi w Polsce i Europie – populacyjny punkt widzenia. Nowotwory J Oncol 2013; 63: 111-8.
3. Pikala M, Maniecka-Bryla I. Years of life lost due to malignant neoplasms characterized by the highest mortality rate. Arch Med Sci 2014; 10: 999-1006.
4. Krag DN, Anderson SJ, Julian TB, et al. Sentinel-lymph-node resection compared with conventional axillary-lymph-node dissection in clinically node-negative patients with breast cancer: overall survival findings from the NSABP B-32 randomized phase 3 trial. Lancet Oncol 2010; 11: 927-33.
5. Mansel RE, Fallowfield L, Kissin M, et al. Randomized multicenter trial of sentinel node biopsy versus standard axillary treatment in operable breast cancer: the ALMANAC Trial. J Natl Cancer Inst 2006; 98: 599-609.
6. Kim T, Giuliano AE, Lyman GH. Lymphatic mapping and sentinel lymph node biopsy in early-stage breast carcinoma: a meta-analysis. Cancer 2006; 106: 4-16.
7. Pernas S, Gil M, Benítez A, et al. Avoiding axillary treatment in sentinel lymph node micrometastases of breast cancer: a prospective analysis of axillary or distant recurrence. Ann Surg Oncol 2010; 17: 772-7.
8. Singletary SE, Connolly JL. Breast cancer staging: working with the sixth edition of the AJCC Cancer Staging Manual. CA Cancer J Clin 2006; 56: 37-47.
9. Kohrt HE, Olshen RA, Bermas HR, et al. New models and online calculator for predicting non-sentinel lymph node status in sentinel lymph node positive breast cancer patients. BMC Cancer 2008; 8: 66.
10. Van Zee KJ, Manasseh DM, Bevilacqua JL, et al. A nomogram for predicting the likelihood of additional nodal metastases in breast cancer patients with a positive sentinel node biopsy. Ann Surg Oncol 2003; 10: 1140-51.
11. Cserni G, Boross G, Maráz R, et al. Multicentre validation of different predictive tools of non-sentinel lymph node involvement in breast cancer. Surg Oncol 2012; 21: 59-65.
12. Houvenaeghel G, Bannier M, Nos C, et al. Non sentinel node involvement prediction for sentinel node micrometastases in breast cancer: nomogram validation and comparison with other models. Breast 2012; 21: 204-9.
13. D’Eeredita G, Troilo VL, Giardina C, et al. Sentinel lymph node micrometastasis and risk of non-sentinel lymph node metastasis: validation of two breast cancer nomograms. Clinical Breast Cancer 2010; 10: 445-51.
14. Coutant C, Olivier C, Lambaudie E, et al. Comparison of models to predict nonsentinel lymph node status in breast cancer patients with metastatic sentinel lymph nodes: a prospective multicenter study. J Clin Oncol 2009; 27: 2800-8.
15. Park J, Fey JV, Naik AM, et al. A declining rate of completion axillary dissection in sentinel lymph node-positive breast cancer patients is associated with the use of a multivariate nomogram. Ann Surg 2007; 245: 462-8.
16. Dauphine CE, Haukoos JS, Vargas MP, et al. Evaluation of three scoring systems predicting non sentinel node metastasis in breast cancer patients with a positive sentinel node biopsy. Ann Surg Oncol 2007; 14: 1014-9.
17. Coufal O, Pavlík T, Fabian P. Predicting non-sentinel lymph node status after positive sentinel biopsy in breast cancer: what model performs the best in a Czech population? Pathol Oncol Res 2009; 15: 733-40.
18. Barranger E, Coutant C, Flahault A, et al. An axilla scoring system to predict non-sentinel lymph node status in breast cancer patients with sentinel lymph node involvement. Breast Cancer Res Treat 2005; 91: 113-9.
19. Pal A, Provenzano E, Duffy SW. A model for predicting non-sentinel lymph node metastatic disease when the sentinel lymph node is positive. Br J Surg 2008; 95: 302-9.
20. Meretoja TJ, Leidenius MHK, Heikkilä PS, et al. International multicenter tool to predict the risk of nonsentinel node metastases in breast cancer. J Natl Cancer Inst 2012; 104: 1888-96.
21. Saidi RF, Dudrick PS, Remine SG, et al. Nonsentinel lymph node status after positive sentinel lymph node biopsy in early breast cancer. Am Surg 2004; 70: 101-5.
22. Degnim AC, Reynolds C, Pantvaidya G, et al. Nonsentinel node metastasis in breast cancer patients: assessment of an existing and a new predictive nomogram. Am J Surg 2005; 190: 543-50.
23. Perhavec A, Perme MP, Hocevar M, et al. Ljubljana nomograms for predicting the likelihood of non-sentinel lymph node metastases in breast cancer patients with a positive sentinel lymph node. Breast Cancer Res Treat 2010; 119: 357-66.
24. Hwang RF, Krishnamurthy S, Hunt KK, et al. Clinicopathologic factors predicting involvement of nonsentinel axillary nodes in women with breast cancer. Ann Surg Oncol 2003; 10: 248-54.
25. Goldhirsch A, Wood WC, Coates AS and panel members. Strategies for subtypes-dealing with the diversity of breast cancer: highlights of the St Gallen International Expert Consensus on the Primary Therapy of Early Breast Cancer 2011. Ann Oncol 2011; 22: 1736-47.
26. Nowikiewicz T, Śrutek E, Jankowski M, et al. Management and results of treatment of breast cancer patients in case of sentinel lymph node micrometastases. Neoplasma 2014; 61: 299-304.
27. Giuliano AE, Hunt KK, Ballman KV, et al. Axillary dissection vs. no axillary dissection in women with invasive breast cancer and sentinel node metastasis: a randomized clinical trial. JAMA 2011; 305: 569-75.
28. Winczura P, Senkus-Konefka E, Jassem J. Polish and international recommendations on the treatment of breast cancer. Nowotwory J Oncol 2013; 63: 58-65.
29. Huszno J, Badora A, Nowara E. The influence of steroid receptor status on the cardiotoxicity risk in HER2-positive breast cancer patients receiving trastuzumab. Arch Med Sci 2015; 11: 371-7.
30. Cheang MC, Chia SK, Voduc D, et al. Ki67 index, HER2 status, and prognosis of patients with luminal B breast cancer. J Natl Cancer Inst 2009; 101: 736-50.
31. Badowska-Kozakiewicz AM, Sobol M, Patera J, Kozłowski W. Immunohistochemical evaluation of human epidermal growth factor receptor 2 and estrogen and progesterone receptors in invasive breast cancer in women. Arch Med Sci 2013; 9: 466-71.
32. Kołacińska A, Chałubińska J, Błasińska-Morawiec M, et al. Pathological complete response in younger and older breast cancer patients. Arch Med Sci 2012; 8: 310-5.
33. Tvedskov TF, Bartels A, Jensen MB, et al. Evaluating TIMP-1, Ki67, and HER2 as markers for non-sentinel node metastases in breast cancer patients with micrometastases to the sentinel node. APMIS 2011; 119: 844-52.
34. Piotrowski G, Gawor R, Stasiak A, Gawor Z, Potemski P, Banach M. Cardiac complications associated with trastuzumab in the setting of adjuvant chemotherapy for breast cancer overexpressing human epidermal growth factor receptor type 2 – a prospective study. Arch Med Sci 2012; 8: 227-35.
Copyright: © 2016 Termedia & Banach. This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0) License (http://creativecommons.org/licenses/by-nc-sa/4.0/), allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material, provided the original work is properly cited and states its license.
Quick links
© 2019 Termedia Sp. z o.o. All rights reserved.
Developed by Bentus.
PayU - płatności internetowe