Beclin 1 promotes apoptosis and decreases invasion by upregulating the expression of ECRG4 in A549 human lung adenocarcinoma cells

  • Authors:
    • Wenyu Wang
    • Hongkun Fan
    • Xiaoyan Li
    • Gang Wu
    • Weida Zhao
    • Guihong Zhang
    • Guoqiang Zhao
    • Linwei Li
  • View Affiliations

  • Published online on: May 6, 2016     https://doi.org/10.3892/mmr.2016.5219
  • Pages: 355-360
Metrics: Total Views: 0 (Spandidos Publications: | PMC Statistics: )
Total PDF Downloads: 0 (Spandidos Publications: | PMC Statistics: )


Abstract

Although Beclin 1 has been demonstrated to exert an important role in cell autophagy during carcinogenesis, its biological function in lung cancer has yet to be fully elucidated. A previous study by our laboratory identified that knockdown of Beclin 1 promoted cell growth and inhibited apoptosis in the A549 lung cancer cell line. In the present study, a Beclin 1 lentiviral expression vector was constructed, and an A549 cell line was established with a steady expression of Beclin 1. Furthermore, the effect of Beclin 1 overexpression on cell invasion and apoptosis, changes in the activities of the apoptosis‑associated caspases‑3 and ‑9, and the overexpression of esophageal cancer‑related gene 4 (ECRG4) were examined. The results demonstrated that the overexpression of Beclin 1 in A549 cells reduced cell invasion by Matrigel invasion assay and promoted apoptosis by flow cytometric analysis (P<0.01) compared with Lenex‑packaged lentiviral particles and non‑transfected control groups. Furthermore, the overexpression of Beclin 1 in A549 cells increased the activities of caspases‑3 and ‑9 and the expression of ECRG4 (P<0.01) compared with Lenex‑packaged lentiviral particles and non‑transfected control groups. In conclusion, the overexpression of Beclin 1 promoted apoptosis and decreased invasion by upregulating the expression of ECRG4 in A549 lung adenocarcinoma cells. Therefore, the selection of Beclin l as a target for gene therapy represents a more effective method for the treatment of lung cancer.

Introduction

Beclin l (the mammalian counterpart of the yeast Atg6 gene) is an essential player in autophagy. Allelic loss or deficiency of the Beclin 1 gene has been demonstrated in human breast cancer, ovarian cancer and prostate cancer; in lung cancer, hepatocellular carcinoma, cervical cancer and lymphoma, the expression of Beclin 1 is very low/almost undetectable (14). A previous study identified that Beclin 1−/− mice died early in embryonic development (5). Although Beclin 1+/− mice were able to survive, the incidence of cancer was much higher in these animals. In addition, the measured in vivo cell autophagy activity was markedly decreased, and cells reproduced faster in Beclin 1-deficient animals. These findings clearly suggested that there is a close correlation between the inhibition of autophagy activity and the occurrence of cancer.

Although Beclin 1 has been demonstrated to exert an important role in cell autophagy during carcinogenesis, its biological function in lung cancer has yet to be fully elucidated. A previous study by our laboratory identified that knockdown of Beclin 1 promoted cell growth and inhibited apoptosis in the A549 lung cancer cell line (6). In the present study, a Beclin 1 lentiviral expression vector was constructed, and an A549 cell line was established with a steady expression of Beclin 1. The effects of Beclin 1 overexpression on cell invasion and apoptosis, changes in the activities of the apoptosis-associated caspases-3 and -9, and the expression of esophageal cancer-related gene 4 (ECRG4) were examined.

Materials and methods

A549 cell culture

Human lung adenocarcinoma cell line A549 was obtained from the Chinese Center for Type Culture Collection (Wuhan, China). An A549 cell suspension was added into a centrifuge tube containing 5 ml RPMI-1640 culture medium (Invitrogen; Thermo Fisher Scientific, Inc., Waltham, MA, USA) and centrifuged at 1,200 × g for 5 min. The supernatant was subsequently discarded, and fresh RPMI-1640 medium was added. Following the second centrifugation, cells were resuspended in 10 ml RPMI-1640 culture medium containing 10% fetal bovine serum (FBS) and cultured overnight in a 5% CO2 incubator at 37°C. A549 cells at a density of 5×105 cells/ml were seeded into a 6-well culture plate (2 ml per well). When cells had grown to 70–80% confluence, 2 ml of RPMI-1640 medium containing 10% FBS and different concentrations of G418 solution (Invitrogen; Thermo Fisher Scientific, Inc.) were added. A G418 concentration (800 μg/ml) that could cause cell death in 6–8 days was selected as the concentration to be used for subsequent cell selection and screening experiments.

Infection of A549 cells

A549 cells were divided into three groups: Cells infected with lentiviral particles packaged with the recombinant vector, pLenex-Beclin 1; those infected with lentiviral particles packaged with the empty vector, pLenex; and non-transfected A549 cells. Recombinant lentiviruses were generated from our previously constructed pLenex-Beclin 1 and pLenex vectors (6). At 24 h prior to virus infection, A549 cells in each group were digested with 0.25% trypsin solution, seeded in 6-well plates at a density of 1.5×106 cells per well, and cultured in RPMI-1640 culture medium containing 10% FBS. After 12 h, when the A549 cells had reached 100% confluence, the virus suspension was added at a concentration of 1.5×107 international units (IU). Following a further incubation for 12 h, the plates were washed three times with phosphate-buffered saline (PBS) and cultured with complete culture medium. The culture medium was changed every other day. Cell morphology and growth were observed every day using inverted microscopy. On day 3 following infection, A549 cells were cultured with 800 μg/ml G418 (neomycin)-selective culture medium to select cells that were able to tolerate G418-selective medium and stably grow in order to establish a stable cell line.

Detection of Beclin 1 and ECRG4 expression by western blot analysis

Non-infected A549 cells and two groups of infected cells were added into tubes containing RLT buffer. After the cells had been harvested and lysed, the protein concentration was determined using a bicinchoninic acid protein assay kit. Samples were dissolved in loading buffer, boiled for 5 min, and subsequently loaded onto 10% sodium dodecyl sulfate-polyacrylamide gel electrophoresis (SDS-PAGE) gels. Protein bands were subsequently transferred onto a polyvinylidene difluoride membrane. The membrane was blocked overnight at 4°C in Tris-buffered saline-Tween 20 (TBST) containing 5% defatted milk, incubated with a rat anti-human Beclin 1 or ECRG4 primary antibody (dilution, 1:150; Santa Cruz Biotechnology, Inc., Santa Cruz, CA, USA) for 2 h, washed three times and incubated with horseradish peroxidase-conjugated rabbit anti-rat secondary antibody (dilution, 1:2,000) for 1 h at room temperature. The bands were visualized using 3,3′-diaminobenzidine (Pierce Biotechnology, Appleton, WI, USA).

Detection of cell apoptosis by flow cytometry

Infected and non-infected cells were cultured for 48 h. Following trypsin digestion, cells were adjusted to a density of 5×105-1×106 cells/ml and fixed with 75% cold ethanol overnight. Following washing, cells were fully centrifuged at 1,200 g, mixed with 100 μl RNA enzyme (1 mg/ml; Promega, Inc., Madison, WI, USA) and incubated in a water bath at 37°C for 30 min. An aliquot of 10 μg/ml propidium iodide dye solution was subsequently added. Following an incubation for 30 min in the dark, PBS was added to dilute the cells. The cell suspension was filtered through a 300-mesh nylon sieve, and the rate of cell apoptosis was determined using a flow cytometer.

Detection of active caspase-3 and caspase-9

A total of ~5×105 cells in each group were centrifuged at 1,200 rpm for 5 min. The harvested cells were fixed in an ice-cold cell lysate buffer for 10 min at a density of 2×105 cells/50 μl. The microcentrifuge tubes were subsequently centrifuged at 15,000 rpm for 5 min at 4°C. The supernatants were then transferred to another set of ice-cold microcentrifuges. Samples (50 μl) were pipetted into a 96-well microplate, mixed, and sealed with paraffin wax. Following a 2 h incubation in the dark, the fluorescence was determined using a plate reader with an excitation wavelength of 380 nm and an emission wavelength of 460 nm.

Matrigel invasion assay

The Boyden chamber assay was performed. Matrigel previously stored at −20°C was kept on ice overnight. With pre-cooled tips, 100 μl Matrigel was pipetted into 300 μl ice pre-cooled serum-free medium and mixed thoroughly. After the polycarbonate ester membrane with a pore size of 8 μm was placed onto the Transwell plate between the upper and lower chambers of the Boyden chamber, the diluted Matrigel was added in the chamber to cover the entire PCS ester membrane and placed at 37°C for 30 min to make the Matrigel coagulate. The digested cells from each group were rinsed three times with PBS to prepare single cell suspensions (at a density of 1×106 cells/ml) with RPMI-1640 serum-free medium. Each group of cells was divided into two parts, one for the Trypan blue exclusion experiment and the other for invasion assay. The Trypan blue exclusion experiment demonstrated that cell viability was >90%. Samples (200 μl) of chemokines from the medium were added into the lower chamber of the Transwell culture plate, and 400 μl of cell suspension from each group was added into the upper chamber of the Transwell culture plate. Following culture at 37°C in 5% CO2 for 24 h, the cells were gently removed from the surface of the Matrigel and the polycarbonate ester membrane with wet cotton swabs. The upper chamber was subsequently removed, marked, fixed with ice pre-cooled ethanol for 30 min, stained with hematoxylin for 1 min, dehydrated in graded methanol (80%, 95%, 100%), and cleared with xylene. The PCS ester base membrane was carefully cut away from the base of the upper chamber, placed on glass slides and mounted with neutral resin. The number of cells attached to the lower surface of the PCS ester membrane was counted in six randomly chosen high-power (magnification, ×400) fields to calculate the average number. The experiments were performed in triplicate, and repeated twice.

Statistical analysis

Statistical analyses were performed using SPSS v.16 software (SPSS, Inc., Chicago, IL, USA). Numerical data were expressed as the mean ± standard deviation. Analysis of variance was used to compare means of multiple groups, and the t-test was used for comparison of means between two groups. P<0.05 was considered to indicate a statistically significant difference.

Results

Construction of the Beclin 1 lentivirus expression vector and steady infection

The Beclin 1 lentivirus expression vector was initially constructed, and the A549 cell line with Beclin 1 lentivirus steady infection was successfully established. After 2 weeks of G418 selection, anti-G418 resistant cells survived and formed a stable clone. As shown in Fig 1, green fluorescence was observed under an inverted fluorescence microscope in the two lentivirus-infected groups 48 h following lentivirus infection, and the infection efficiencies were >90%. Following 2 weeks of selection with G418, all cells exhibited green fluorescence, indicating that the lentiviral vector-mediated expression of Beclin 1 in the A549 cells had been successful and efficient. In contrast, no green fluorescence was observed in non-transfected A549 cells. In order to further determine whether the recombinant lentiviral vector had been successfully transduced into A549 cells and the Beclin 1 gene was translated into the corresponding protein, western blot analysis was performed. The results of the western blotting revealed that an ~60 kDa band was observed in all three groups of cells, and the density of this band was higher in cells infected with pLenex-Beclin 1-packaged lentiviral particles compared with those infected with pLenex-packaged lentiviral particles or non-transfected A549 cells, indicating that the expression of Beclin 1 protein in cells infected with pLenex-Beclin 1-packaged lentiviral particles was highly efficient (Fig. 2). This finding suggested that infection of A549 cells with a lentiviral vector carrying the Beclin 1 gene could induce the overexpression of Beclin 1, and that these infected A549 cells were able to be used as model cells for subsequent functional experiments.

Beclin 1 overexpression reduces the invasion of A549 cells

No significant differences were identified in terms of the average number of cells passing through the Transwell membrane between non-transfected A549 cells and A549 cells infected with pLenex-packaged lentiviral particles (15.8±2.7 vs. 18.2±3.1; P>0.05). However, the average number of cells passing through the Transwell membrane was significantly lower in A549 cells infected with pLenex-Beclin 1-packaged lentiviral particles (6.4±2.1) compared with non-transfected A549 cells and A549 cells infected with pLenex-packaged lentiviral particles (P<0.05) (Fig. 3), suggesting that the overexpression of Beclin 1 reduced the invasive ability of the A549 cells.

Beclin 1 overexpression promotes apoptosis of A549 cells

As determined from the flow cytometric analysis, no statistical differences were identified in the apoptotic rate between non-transfected A549 cells and A549 cells infected with pLenex-packaged lentiviral particles (9.12±1.21% vs. 6.87±1.17%; P>0.05). However, the apoptotic rate was significantly higher in A549 cells infected with pLenex-Beclin 1-packaged lentiviral particles (17.12±2.12%) compared with non-transfected A549 cells and A549 cells infected with pLenex-packaged lentiviral particles (P<0.05), indicating that the overexpression of Beclin 1 promoted apoptosis of the A549 cells (Fig. 4).

Beclin 1 overexpression increases the activity of caspase-3 and caspase-9 in A549 cells

No significant differences were identified in the activities of caspases-3 and caspase-9 between non-transfected A549 cells and A549 cells infected with pLenex-packaged lentiviral particles (1.28±0.15 vs. 1.23±0.14, and 0.67±0.11 vs. 0.63±0.08, respectively; P>0.05 for both). However, the activities of caspase-3 and caspase-9 were significantly higher in A549 cells infected with pLenex-Beclin 1-packaged lentiviral particles (1.81±0.26 vs. 1.16±0.18) compared with the non-transfected A549 cells and A549 cells infected with pLenex-packaged lentiviral particles (P<0.05 for both) (Fig. 5), suggesting that the overexpression of Beclin 1 increases the activities of caspase-3 and caspase-9.

Beclin 1 overexpression increases ECRG4 expression in A549 cells

The results of the western blot analysis showed that no significant differences were identified in the expression of ECRG4 between non-transfected A549 cells and A549 cells infected with pLenex-packaged lentiviral particles. However, the expression of ECRG4 was significantly higher in A549 cells infected with pLenex-Beclin 1-packaged lentiviral particles compared with non-transfected A549 cells and A549 cells infected with pLenex-packaged lentiviral particles (P<0.05 for both) (Fig. 6), suggesting that the overexpression of Beclin 1 increases the expression of ECRG4 in A549 cells.

Discussion

Autophagy is a form of programmed cell death that has an important role in biological growth and development, self-renewal, as well as disease development and tumor formation (7). A previous study has indicated that abnormal regulation of autophagy is directly associated with tumor formation (8). It is well known that Beclin 1 is a key regulator of autophagy. In the present study, non-small cell lung cancer A549 cells were successfully infected with a previously constructed recombinant lentiviral vector carrying the Beclin 1 gene. The overexpression of the autophagy-associated gene, Beclin 1, was revealed to markedly decrease invasion, promote apoptosis, and increase the activities of caspases-3 and caspase-9 and the expression of ECRG4 in A549 cells.

Invasion and metastasis are the most significant characteristics of cancer cells. The Matrigel invasion assay is a commonly used in vitro method for studying tumor cell invasion. In the present study, the average number of cells passing through the Transwell membrane was significantly lower in A549 cells infected with pLenex-Beclin 1-packaged lentiviral particles compared with the two control groups (P<0.01 for both). These results indicated that the overexpression of Beclin 1 reduced the invasive ability of the A549 cells. Therefore, Beclin 1 may have a putative tumor-suppressing function in lung cancer. In the present study, the overexpression of Beclin 1 was also shown to promote the apoptosis of A549 cells. This result demonstrated that the apoptotic rate of Beclin 1-overexpressing cells was significantly higher compared with those of control cells (P<0.01 for both). Previous studies indicated that, although autophagy is morphologically different from apoptosis, it may occur prior to and/or co-exist with apoptosis, and may be stimulated or induced by apoptosis, depending on the responses of cells to intracellular or extracellular stimuli (9,10). Inhibition of cell apoptosis and autophagy may lead to the escape of tumor cells from normal clearance mechanisms of the body, resulting in the formation of malignant tumors.

Although there are significant differences between autophagy and apoptosis in biochemical pathways, their functions may be linked. Autophagy does not depend on the involvement of caspases, and its most notable feature is the presence of autophagosomes, which are eventually removed via the lysosomal system (11). Caspases are a group of apoptotic cysteine proteases, the majority of which exist as non-active caspase, zymogen forms. The caspase family members are intracellular proteins that are essential for the implementation of apoptosis. Their substrates include the caspases themselves, and several other factors. Through the hydrolysis of substrates, caspases transduce the apoptotic signal, or they themselves directly act as the effectors for apoptosis, promoting cytoskeletal degradation and DNA fragmentation (12). In the caspase family of proteases, caspases-3 and -9 are key molecules in the intrinsic apoptotic pathway. Caspase-9 is able to act through the nuclear lamina to cause cell structural damage and apoptosis (13,14). Furthermore, ECRG4 is a candidate tumor suppressor in a variety of tumor types (1518). It has been previously shown that ECRG4 is involved in apoptosis and cell autophagy (19,20). In the present study, we identified that the overexpression of Beclin 1 increased the activities of caspase-3 and caspase-9 and the expression of ECRG4 in A549 lung cancer cells. These findings will provide novel insights into the role of Beclin 1 in the regulation of autophagy and apoptosis, and the interaction between autophagy and apoptosis in lung adenocarcinoma. However, the detailed molecular mechanisms have yet to be fully elucidated.

In conclusion, the overexpression of Beclin 1 promoted apoptosis and decreased invasion by upregulating the expression of ECRG4 in A549 lung adenocarcinoma cells. Therefore, the selection of Beclin 1 as a target for gene therapy represents a more effective method for the treatment of lung cancer.

Acknowledgments

This study was supported by the Chinese National Natural Science Foundation (no. U1304817) and the Zhengzhou City Science Research Project (no. 141PPTGHG298)

References

1 

Shen Y, Liang LZ, Hong MH, Xiong Y, Wei M and Zhu XF: Expression and clinical significance of microtubule-associated protein 1 light chain 3 (LC3) and Beclin 1 in epithelial ovarian cancer. Ai Zheng. 27:595–599. 2008.In Chinese. PubMed/NCBI

2 

Liang XH, Kleeman LK, Jiang HH, Gordon G, Goldman JE, Berry G, Herman B and Levine B: Protection against fatal Sindbis virus encephalitis by Beclin, a novel Bcl-2-interacting protein. J Virol. 72:8586–8596. 1998.PubMed/NCBI

3 

Qu X, Yu J, Bhagat G, Furuya N, Hibshoosh H, Troxel A, Rosen J, Eskelinen EL, Mizushima N, Ohsumi Y, et al: Promotion of tumorigenesis by heterozygous disruption of the beclin 1 autophagy gene. J Clin Invest. 112:1809–1820. 2003. View Article : Google Scholar : PubMed/NCBI

4 

Kanzawa T, Germano IM, Komata T, Ito H, Kondo Y and Kondo S: Role of autophagy in temozolomide induced cytotoxicity for malignant glioma cells. Cell Death Differ. 11:448–457. 2004. View Article : Google Scholar : PubMed/NCBI

5 

Yue Z, Jin S, Yang C, Levine AJ and Heintz N: Beclin 1, an autophagy gene essential for early embronic development, is a haploinsufficient tumor suppressor. Proc Natl Acad Sci USA. 100:15077–15282. 2003. View Article : Google Scholar

6 

Wang W, Fan H, Zhou Y, Duan P, Zhao G and Wu G: Knockdown of autophagy-related gene BECLIN1 promotes cell growth and inhibits apoptosis in the A549 human lung cancer cell line. Mol Med Rep. 7:1501–1505. 2013.PubMed/NCBI

7 

Lockshin RA and Zakeri Z: Caspase-independent cell deaths. Curr Opin Cell Biol. 14:727–733. 2002. View Article : Google Scholar : PubMed/NCBI

8 

Jia L, Dourmashkin LR, Allen PD, Gray AB, Newland AC and Kelsey SM: Inhibition of autophagy abrogates tumour necrosis factor alpha induced apoptosis in human T-lymphoblastic leukaemic cells. Br J Haematol. 98:673–685. 1997. View Article : Google Scholar : PubMed/NCBI

9 

González-Polo RA, Boya P, Pauleau AL, Jalil A, Larochette N, Souquère S, Eskelinen EL, Pierron G, Saftig P and Kroemer G: The apoptosis/autophagy paradox: Autophagic vacuolization before apoptotic death. J Cell Sci. 118:3091–3102. 2005. View Article : Google Scholar : PubMed/NCBI

10 

Tóth S, Nagy K, Pálfia Z and Réz G: Cellular autophagic capacity changes during azaserine-induced tumour progression in the rat pancreas. Up-regulation in all premalignant stages and down-regulation with loss of cycloheximide sensitivity of segregation along with malignant transformation. Cell Tissue Res. 309:409–416. 2002. View Article : Google Scholar : PubMed/NCBI

11 

Gozuacik D and Kimchi A: Autophagy as a cell death and tumor suppressor mechanism. Oncogene. 23:2891–2906. 2004. View Article : Google Scholar : PubMed/NCBI

12 

Mizushima N: Methods for monitoring autophagy. Int J Biochem Cell Biol. 36:2491–2502. 2004. View Article : Google Scholar : PubMed/NCBI

13 

Johnson CR and Jarvis WD: Caspase-9 regulation: An update. Apoptosis. 9:423–427. 2004. View Article : Google Scholar : PubMed/NCBI

14 

Fan TJ, Han LH, Cong RS and Liang J: Caspase family proteases and apoptosis. Acta Biochim Biophys Sin (Shanghai). 37:719–727. 2005. View Article : Google Scholar

15 

Li LW, Yu XY, Yang Y, Zhang CP, Guo LP and Lu SH: Expression of esophageal cancer related gene 4 (ECRG4), a novel tumor suppressor gene, in esophageal cancer and its inhibitory effect on the tumor growth in vitro and in vivo. Int J Cancer. 125:1505–1513. 2009. View Article : Google Scholar : PubMed/NCBI

16 

Li L, Zhang C, Li X, Lu S and Zhou Y: The candidate tumor suppressor gene ECRG4 inhibits cancer cells migration and invasion in esophageal carcinoma. J Exp Clin Cancer Res. 29:1332010. View Article : Google Scholar : PubMed/NCBI

Retraction. J Exp Clin Cancer Res. 30:192011. View Article : Google Scholar

17 

Li LW, Li YY, Li XY, Zhang CP, Zhou Y and Lu SH: A novel tumor suppressor gene ECRG4 interacts directly with TMPRSS11A (ECRG1) to inhibit cancer cell growth in esophageal carcinoma. BMC Cancer. 11:522011. View Article : Google Scholar : PubMed/NCBI

18 

Baird A, Lee J, Podvin S, Kurabi A, Dang X, Coimbra R, Costantini T, Bansal V and Eliceiri BP: Esophageal cancer-related gene 4 at the interface of injury, inflammation, infection, and malignancy. Gastrointest Cancer. 2014:131–142. 2014. View Article : Google Scholar

19 

You Y, Yang W, Qin X, Wang F, Li H, Lin C, Li W, Gu C, Zhang Y and Ran Y: ECRG4 acts as a tumor suppressor and as a determinant of chemotherapy resistance in human nasopharyngeal carcinoma. Cell Oncol (Dordr). 38:205–214. 2015. View Article : Google Scholar

20 

Matsuzaki J, Torigoe T, Hirohashi Y, Kamiguchi K, Tamura Y, Tsukahara T, Kubo T, Takahashi A, Nakazawa E, Saka E, et al: ECRG4 is a negative regulator of caspase-8-mediated apoptosis in human T-leukemia cells. Carcinogenesis. 33:996–1003. 2012. View Article : Google Scholar : PubMed/NCBI

Related Articles

Journal Cover

July 2016
Volume 14 Issue 1

Print ISSN: 1791-2997
Online ISSN:1791-3004

Sign up for eToc alerts

Recommend to Library

Copy and paste a formatted citation
APA
Wang, W., Fan, H., Li, X., Wu, G., Zhao, W., Zhang, G. ... Li, L. (2016). Beclin 1 promotes apoptosis and decreases invasion by upregulating the expression of ECRG4 in A549 human lung adenocarcinoma cells . Molecular Medicine Reports, 14, 355-360. https://doi.org/10.3892/mmr.2016.5219
MLA
Wang, W., Fan, H., Li, X., Wu, G., Zhao, W., Zhang, G., Zhao, G., Li, L."Beclin 1 promotes apoptosis and decreases invasion by upregulating the expression of ECRG4 in A549 human lung adenocarcinoma cells ". Molecular Medicine Reports 14.1 (2016): 355-360.
Chicago
Wang, W., Fan, H., Li, X., Wu, G., Zhao, W., Zhang, G., Zhao, G., Li, L."Beclin 1 promotes apoptosis and decreases invasion by upregulating the expression of ECRG4 in A549 human lung adenocarcinoma cells ". Molecular Medicine Reports 14, no. 1 (2016): 355-360. https://doi.org/10.3892/mmr.2016.5219