Hospital initiatives in promoting smoking cessation: A 12-year follow-up

  • Authors:
    • John T. Denny
    • Angela M. Denny
    • James T. Tse
    • Vincent J. Deangelis
    • Darrick Chyu
    • Enrique J. Pantin
    • Sloane S. Yeh
    • Shaul Cohen
    • Christine H. Fratzola
    • Alann Solina
  • View Affiliations

  • Published online on: July 1, 2016     https://doi.org/10.3892/etm.2016.3496
  • Pages: 1599-1603
Metrics: Total Views: 0 (Spandidos Publications: | PMC Statistics: )
Total PDF Downloads: 0 (Spandidos Publications: | PMC Statistics: )


Abstract

In the present study, the availability of smoking cessation programs (SCP) was surveyed in the same randomly selected USA hospitals in 2000 and 2012. A total of 102 USA hospitals were randomly selected for this survey. Each hospital website was searched for the topic of smoking cessation. In the second phase of the survey, the main switchboard number of each hospital was anonymously telephoned and the ‘stop smoking clinic’ was requested. The phone survey results showed that the percentage of hospital switchboard calls that were connected to a SCP remained identical at 47% in 2000 and 2012. The results for the internet availability of SCP on hospital websites improved from 30% in 2000 to 47% in 2012. There were more hospitals that added additional SCP information (27%) compared with those that removed SCP information (15%) by 2012. Among the 57% of hospitals that showed no change in internet SCP information, 22% remained positive for such information while 35% remained negative. The phone survey of hospitals showed that 47% of USA hospitals were able to connect a caller to a SCP in the years 2000 and 2012. While there was no reduction over the 12 years, there was no increase in the percentage of hospital switchboards that connected to a SCP. Availability of SCP information on hospital web sites improved to a limited extent; increasing from 30% of sites in 2000 to 47% in 2012. Providing SCP on a hospital website is easy and free, for example adding a link to QuitNet or QuitLink. The present study adds to information gathered 12 years earlier, and is unusual in being able to provide follow‑up data on the same set of hospitals studied previously.

Introduction

Tobacco use remains the largest single preventable cause of disease and death in the USA (1). Every year, approximately 443,000 persons in the United States succumb to tobacco-related illnesses. In addition, smoking has been estimated in the United States, to add $96 billion in direct medical expenses and $97 billion in lost productivity every year (2). This reinforces data extrapolated from studies in the 1980s suggests that smoking causes 25% of mortality among women and men aged 35–69 years in the USA (3). The lethal effects of smoking are not confined to America; ~32 million individuals among 28 European Union countries succumbed to smoking-related conditions over the last 50 years, including 27 million men and 5 million women. Furthermore, >50% of these smoking-related cases of mortality were individuals aged 35–69 years, in which the loss of potential lifespan compared with non-smokers is at least 20 years (4).

Hospitals are crucial sources of information for patients regarding health issues, including smoking. Hospitals are local to patients, their location is usually known to local residents, they are open at all times and it is well-known that they are accessible to all, without regard of the ability to pay. Hospitals are particularly important as a source of health-care and health information and guidance to minority and low-income populations, who typically have reduced access to health insurance and disposable income (5).

OMalley et al surveyed a population of 2,462 individuals of Hispanic and African descent in New York City regarding where they obtained their health information; 18% reported receiving health information from a hospital or doctors office, while 40% reported a physician or health professional as a source (6).

Since insurance is the most powerful predictor of access to the health care system, it was not surprising that the largest number of persons claiming to obtain health information from a doctor/health professional was privately insured (55% private, 28% public and 17% uninsured) (6). Among the insured population, similar proportions identified the doctor/health professional as a usual source of health information (45% private and 43% public); whereas only 28% of uninsured persons cited the doctor/health professional as a usual information source. Use of printed materials as information sources was highest among the privately insured (6). The rate of smoking among American adults >18 years old was 19.3% in 2011 (1).

Until recently men were more likely to smoke than were women (7). Although the overall smoking rate has declined markedly in the last three decades, the majority of the decline has been caused by a substantial reduction in the smoking rate of men.

Between 1965 and 1995, the total smoking rate of men declined from 52 to 27% (8). While the rate of smoking for women has historically been below that of men, their rate has declined at a lower rate, from 34% in 1965 to 23% in 1995 (9). The gender gap in smoking rates is even narrower among younger age groups. Actually, data from the Monitoring the Future project indicate that the smoking rate for female 8th and 10th grade students has exceeded that of their male counterparts since 1995 (9). Since the majority of smokers start smoking in their teens, the relatively high rate of underage female smokers suggests that the female smoking rate may eventually surpass that of males (8).

As women frequently balance child-rearing, career and other responsibilities, their time to seek health information is at a premium (10). Local, convenient sources of information that are available for extended hours that complement a busy schedule are thus a good match to potentially furnish health information, such as smoking cessation. Furthermore, the increasing omnipresence of wireless networks and internet access are also well-suited for ease of access, and 24-h availability (11). Hospitals as a source of smoking cessation information seem like a good fit to reach this increasing number of female smokers (12).

In 2002, the present authors reported in that it was more difficult than expected for consumers to access smoking cessation programs (SCP) via their local hospital (13). Access for patients when dialed the hospital switchboard and asking for the “stop-smoking clinic” was investigated. In addition, we examined whether hospital websites had information available on smoking cessation. The randomly selected hospital websites were searched for smoking cessation. The overall results indicated that among the hospital websites surveyed, only 30% contained information relating to SCP (13). The phone survey of hospital switchboards showed that 47% had a SCP available via phone inquiry (13).

In an accompanying editorial, Dr Alfred Soffer (Master FCCP) commented that “this relative paucity of a hospital-based effort to assist the smoker in the challenging task of cessation is particularly regrettable, since outpatients and especially inpatients frequently represent an unparalleled group to target for SCPsˮ (14).

Since our original 2002 study, substantial technological developments have occurred. Twelve years later, we have repeated our original inquiries to investigate to what extent the availability of SCPs through hospitals had changed. In the intervening period, patterns of phone use have been disrupted by the ubiquity of cellular phone availability. Similarly, the sophistication of the internet and the pattern of its use have increased markedly. The aim of the present study was to investigate how these technological disruptions have affected SCP availability via hospitals.

Materials and methods

Ethical approval

This study was approved by the Robert Wood Johnson Medical School/Rutgers Institutional Review Board (approval no. 0220100135).

First phase

Originally in 2000, 102 websites were randomly selected across the United States (13). The websites were visited, and the site was noted either for the presence or absence of a ‘search button’ to search within the site only, i.e. an intrasite search option (ISSO). If such a search button existed, it was utilized to search for SCP within that site. If such a search button was not present, then the site was searched manually for SCP information.

Telephone survey

In the second phase of the survey, the main switchboard number of the same 102 hospitals was anonymously called, and the ‘stop smoking clinic’ was asked for. Responses were categorized into ‘yes,’ if the hospital had an in-house SCP available, ‘no’, if there was none, and ‘no but referred’, if there was no in-house program but the caller was referred to another specific program with a phone number. In 2012, a follow-up of the original study was conducted using the same random list of hospitals in the USA, and an identical methodology. Due to the changes in the health care marketplace, only 91 of the original 102 hospitals remained open at the time of the follow-up survey (Table I). These results were then split into subgroups to identify trends in hospitals handling phone calls relating to SCP enquiries (Table II).

Table I.

Availability of smoking cessation resources in hospitals in the years 2000 and 2012.

Table I.

Availability of smoking cessation resources in hospitals in the years 2000 and 2012.

Year

Data resource20002012
Total surveyed, n10291
Phone survey, n (%)48 (47)43 (47)
ISSO, n (%)41 (40)  83 (91)a
Web survey, n (%)31 (30)  43 (47)b

a P<0.001 vs. year 2000

b P<0.02 vs. year 2000, as determined using the χ2 test. ISSO, internet search option.

Table II.

Differences in the availability of smoking cessation resources between the years 2000 and 2012 as indicated by telephone and website surveys.

Table II.

Differences in the availability of smoking cessation resources between the years 2000 and 2012 as indicated by telephone and website surveys.

ChangedUnchanged


Data resourceYes to noNo to yesYesNoTotal unchanged
Phone survey, n (%)16 (18)15 (16)29 (32)31 (35)60 (66)
Web survey, n (%)14 (15)25 (27)a20 (22)32 (35)b52 (57)

a P<0.05 vs. yes to no

b P<0.05 vs. yes, as determined using the χ2 test.

Website survey

An investigator blinded to the switchboard results examined each hospital's website in the follow-up survey of the hospital websites. Firstly, the website was scored for either the presence or absence of an ISSO, or a visible text box for searching only within that hospital website (Table I). If an ISSO was present, the following terms were searched for: ‘Stop-smoking’, ‘smoking cessation’ and ‘tobacco cessation’. If there was no ISSO, the hospital website was manually searched for the aforementioned search terms (Table I). To identify trends in the internet availability of smoking cessation at hospital web sites, Table II shows hospitals that added or deleted SCP information from their web sites between 2000 and 2012. Table II shows the number of hospitals that did not change.

Statistical analysis

Data were analyzed using the χ2 test. The results were expressed as percentages. P<0.05 was considered to indicate a statistically significant difference. The statistical software used was ‘Chi Square Statistics’ (www.math.hws.edu/javamath/ryan/ChiSquare.html) created as part of the Mathbeans Project. The java applets were created by David Eck and modified by Jim Ryan. The Mathbeans Project is funded by a grant from the National Science Foundation (grant no. DUE-9950473).

Results

Telephone survey

The phone survey results showed that the percentage of hospital switchboard calls that were connected to a SCP remained identical at 47% in 2000 and 2012 (Table I). While 18% of the hospital switchboards changed from ‘yes’ to ‘no’ in phone survey of the availability to a SCP, 16% of the hospitals changed from ‘no’ to ‘yes’ (Table II). Furthermore, 32% of the hospitals surveyed stayed ‘yes’ and 35% remained ‘no’ to phone survey of the availability to a SCP between 2002 and 2012 (Table II). The majority of hospitals (66%) showed no change between 2000 and 2012 in phone referral to SCP (Table II).

Website survey

The presence of an ISSO on hospital websites in availability of SCP significantly increased from 40% in 2000 to 91% in 2012 (Table I). The results for internet availability of SCP at hospital web sites significantly improved from 30% in 2000 to 47% in 2012 (Table II). Significantly more hospitals provided additional SCP information to hospital websites (27%) compared with those that removed SCP information (15%) by 2012 (Table II). Among 57% of hospitals which showed no change in internet SCP information, 22% remained ‘yes’ while 35% stayed ‘no’ (Table II).

Discussion

The present phone survey results showed that the percentage of hospital switchboard calls that were connected to a SCP remained identical at 47% in between 2000 and 2012 (Table I). While there was no decrement, there was no improvement in hospital switchboards that connected to a SCP. Nearly equal numbers converted from ‘yes’ to ‘no’ (18%) as changed from ‘no’ to ‘yes’ (16%) (Table II). The majority of hospitals (66%) showed no change between 2000 and 2012 in phone referral to SCP (Table II). However, the present results for American hospitals are improved compared with those collected by Bolliger et al for Swiss hospitals, of which 21% had SCP information (15).

The internet is now an indispensable component of daily life for people in many parts of the world, and has become a more relied on source of health information for patients (16,17). It also offers an additional means of affecting changes to behavior such as smoking (18).

The internet is a viable route to deliver evidence-based SCPs that have the potential to make a large population impact on reducing smoking prevalence. There is high demand for SCP on the internet. For many Americans, the internet has become the primary method for locating information regarding healthcare and treatment options, including SCP (19). In the US, approximately 7% (10.2 million) of adult internet users have searched for information on quitting smoking (19). Publicly available data from large search engines suggest that 4 million Americans search for resources regarding SCP each year (20). Cobb and Graham studied 655 individuals who searched for smoking cessation information on the internet to determine appropriate triage and treatment strategies. They recruited individuals that clicked on a link to a leading smoking cessation website (QuitNet.com) from within the results of a search engine query. Individuals were “intercepted” before seeing the QuitNet home page and were invited to participate in the study. Current smokers were more likely to seek information on how to quit and on medications, while former smokers were more interested in how to cope with withdrawal. It is important that hospitals participate across different channels of communication to provide SCP to the community. Just as some learners are “visual” learners, some smokers will prefer to obtain information visually (21). Others may want the ability to access SCP via phone lines. Cobb and Graham illustrated that there may be little crossover between patient preference for channels: “All participants rated withdrawal information and individually tailored information as being more useful, while displaying little interest in telephone counseling, expert support, or peer support” (20).

Conversely, a less affluent population without access to a personal physician may be more dependent on their local hospitals for health information. However, such a population may have less internet access and smartphones, and it is important for hospitals to be able to help connect smokers who call to SCP (22).

The quality of the health information found on the internet is often poor. One study back-traced 650,000 consumer searches regarding SCP and found that 34% of searches reached professional sites (19). Overall, only 76% of searchers were able to locate a professional SCP website. This is another reason why hospitals should be represented in search results when consumers search for SCP.

A concerning disconnect exists between consumer demand (as demonstrated by search behavior) and the sites actually produced by researchers and health professionals. This “demand gap” may contribute to low overall participation rates and decrease the potential impact of such systems (19). Further research is required to link online consumer preferences to intervention design decisions (19).

Rabius et al (23) recruited 6,451 subjects from an American Cancer Society website using a QuitLink hyperlink (the QuitLink.com website is no longer an active link in 2016). The subjects were randomized into different web-based SCPs, which differed in degree of customization and interactivity. After removing those with an indicator of baseline depression, the more interactive, tailored sites were significantly associated with higher quitting rates than the less interactive ACS site: 13 vs 10%. These results indicate that internet assistance in SCP is potentially cost-effective and suggest that tailored, interactive websites may help cigarette smokers who do not report an indicator of depression at baseline to quit and maintain cessation (23).

The current survey revealed that the presence of ISSO on hospital websites more than doubled between 2000 and 2012, from 40 to 91% (Table I). These text boxes for entering a term to search for within that hospital website make it much easier for consumers to locate specific information on a website. Overall, hospitals improved the mechanics of their websites via the addition of ISSOs more markedly than they did in adding information for SCPs.

Table I also depicts the results for internet availability of SCPs o hospital websites, which improved from 30% in 2000 to 47% in 2012. More hospitals improved in adding SCP information (27%) by 2012, than removed SCP information (15%) (Table II). The majority of hospitals (57%) showed no change in internet SCP information while 22% remained “yes”, while 35% stayed “no” (Table II).

Certain data suggest that specific populations can be better reached by internet sites in promoting SCP (24).

Cobb and Graham studied 655 individuals who searched for smoking cessation information on the Internet and recruited individuals that clicked on a link to a leading smoking cessation website (QuitNet.com) from within the results of a search engine query. Among these, 59% were female and overall tended to be younger than the previously characterized general internet population (20). This suggests that a hospital SCP requires a web presence in order to be more accessible to younger patients and female patients.

The young, female demographic is of interest to hospital marketing officers, as young females will likely be selecting obstetric services. Hospital choice may subsequently serve as a base for her growing family future medical care, such as pediatrics. Therefore, this demographic is typically in control of family-wide choices of where to obtain medical services. A SCP that is accessible to young women may create a family pool of lifelong patients (25,26).

It is important for hospitals to design websites to be usable and comprehendible by those with less education. Graphic design has been shown to be important in influencing the impact a SCP website may have. A study comparing text to text with graphics found that the graphics group more clearly understood the dangers of cigar smoking (27).

The present study surveyed the availability of SCP at the same random USA hospitals in 2000 and 2012. The phone survey of hospitals showed that only 47% of USA hospitals could connect a caller to a SCP in 2000 and 2012. The majority of hospitals (66%) showed no change in connecting callers to a SCP between 2000 and 2012. No progress in expanding accessibility of SCP through phone callers was made between 2000 and 2012.

Since 2000, use of the internet has increased exponentially (28). Hospitals have adapted their website design so that the percentage of sites with an ISSO increased from 40% in 2000 to 91% in 2012, making searching within a hospital website easier. Unfortunately, the availability of SCP information on those same hospital websites did not improve to the same extent; changing from 30% of sites in 2000 to 47% in 2012. This remains <50% of all hospitals that provide access to smoking cessation via their websites. Providing SCP on a hospital website can be easily and cheaply achieved by adding a link to QuitNet or QuitLink. Hospitals are regulated by the Joint Commission on Accreditation of Healthcare Organizations (JCAHO). JCAHO is able to enforce stringent rules regarding smoking and health, for example restricting smoking on hospital property. The JCAHO could decree that hospitals be required to be able to offer callers a referral to a SCP; and that hospital websites contain information regarding SCPs or at a minimum, a link to another SCP site. Hospitals aiming to promote health and SCPs should carefully evaluate the internet as a possible modality for treatment and as a gateway to other traditional programs. The present study adds to information on SCPs provided through hospitals from twelve years earlier, and is unusual in being able to provide follow-up data on the same hospitals studied previously.

Acknowledgements

The authors would like to thank Ms. Charlotte B. Denny for inspiration and proof-reading.

Glossary

Abbreviations

Abbreviations:

SCP

smoking cessation program

ISSO

intrasite search option

References

1 

Centers for Disease Control and Prevention (CDC): Vital signs: Current cigarette smoking among adults aged >18 years-united states, 2005–2010. MMWR Morb Mortal Weekly Rep. 60:1207–1212. 2011.

2 

Centers for Disease C and Prevention: Smoking-attributable mortality, years of potential life lost, and productivity losses-United States, 2000–2004. MMWR Morb Mortal Weekly Rep. 57:1226–1228. 2008.

3 

Jha P, Ramasundarahettige C, Landsman V, Rostron B, Thun M, Anderson RN, McAfee T and Peto R: 21st-century hazards of smoking and benefits of cessation in the United States. N Engl J Med. 368:341–350. 2013. View Article : Google Scholar : PubMed/NCBI

4 

Jha P: The 21st century benefits of smoking cessation in Europe. Eur J Epidemiol. 28:617–619. 2013. View Article : Google Scholar : PubMed/NCBI

5 

Baicker K and Finkelstein A: The effects of Medicaid coverage-learning from the Oregon experiment. N Engl J Med. 365:683–685. 2011. View Article : Google Scholar : PubMed/NCBI

6 

O'Malley AS, Kerner JF and Johnson L: Are we getting the message out to all? Health information sources and ethnicity. Am J Prev Med. 17:198–202. 1999. View Article : Google Scholar : PubMed/NCBI

7 

Giovino GA, Schooley MW, Zhu BP and Eriksen MP: Surveillance for selected tobacco-use behaviors-United States, 1900–1994. MMWR. CDC surveillance summaries: Morbidity and mortality weekly report. CDC surveillance summaries/Centers for Disease Control. 43:1–43. 1994.

8 

Hersch J: Gender, income levels and the demand for cigarettes. J Risk Uncertain. 21:263–282. 2000. View Article : Google Scholar

9 

Wallace JM Jr, Bachman JG, O'Malley PM, Johnston LD, Schulenberg JE and Cooper SM: Tobacco, alcohol and illicit drug use: Racial and ethnic differences among U.S. high school seniors, 1976–2000. Public Health Rep. 117(Suppl 1): S67–S75. 2002.PubMed/NCBI

10 

Sillence E, Briggs P, Harris PR and Fishwick L: How do patients evaluate and make use of online health information? Social science & medicine. 64:1853–1862. 2007. View Article : Google Scholar

11 

Baumgart DC: Personal digital assistants in health care: experienced clinicians in the palm of your hand? Lancet. 366:1210–1222. 2005. View Article : Google Scholar : PubMed/NCBI

12 

Eysenbach G and Kohler C: How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews. Bmj. 324:573–577. 2002. View Article : Google Scholar : PubMed/NCBI

13 

Denny JT, Ginsberg S, Papp D, Browne G, Morgan S, Kushins L and Solina A: Hospital initiatives in promoting smoking cessation: A survey of Internet and hospital-based programs targeted at consumers. Chest. 122:692–698. 2002. View Article : Google Scholar : PubMed/NCBI

14 

Soffer A: Hospitals vs. merchants of nicotine. Chest. 122:391–392. 2002. View Article : Google Scholar : PubMed/NCBI

15 

Bolliger CT, van Biljon X, Humair JP, El Fehri V and Cornuz J: Promoting hospital-based smoking cessation services at major swiss hospitals: A before and after study. Swiss Med Wkly. 138:427–431. 2008.PubMed/NCBI

16 

Bianco A, Zucco R, Nobile CG, Pileggi C and Pavia M: Parents seeking health-related information on the internet: Cross-sectional study. J Med Internet Res. 15:e2042013. View Article : Google Scholar : PubMed/NCBI

17 

Percheski C and Hargittai E: Health Information-seeking in the digital age. J Am Coll Health. 59:379–386. 2011. View Article : Google Scholar : PubMed/NCBI

18 

Civljak M, Stead LF, Hartmann-Boyce J, Sheikh A and Car J: Internet-based interventions for smoking cessation. Cochrane Database Syst Rev. 7:CD0070782013.PubMed/NCBI

19 

Cobb NK: Online consumer search strategies for smoking-cessation information. Am J Prev Med. 38(Suppl 3): S429–S432. 2010. View Article : Google Scholar : PubMed/NCBI

20 

Cobb NK and Graham AL: Characterizing Internet searchers of smoking cessation information. J Med Internet Res. 8:e172006. View Article : Google Scholar : PubMed/NCBI

21 

Munafò MR, Roberts N, Bauld L and Leonards U: Plain packaging increases visual attention to health warnings on cigarette packs in non-smokers and weekly smokers but not daily smokers. Addiction. 106:1505–1510. 2011. View Article : Google Scholar : PubMed/NCBI

22 

Kruse RL, Koopman RJ, Wakefield BJ, Wakefield DS, Keplinger LE, Canfield SM and Mehr DR: Internet use by primary care patients: where is the digital divide? Fam Med. 44:342–347. 2012.PubMed/NCBI

23 

Rabius V, Pike KJ, Wiatrek D and McAlister AL: Comparing internet assistance for smoking cessation: 13-month follow-up of a six-arm randomized controlled trial. J Med Internet Res. 10:e452008. View Article : Google Scholar : PubMed/NCBI

24 

Cobb CO and Graham AL: Use of ‘Non-assigned’ Interventions in a Randomized Trial of Internet and Telephone Treatment for Smoking Cessation. Nic Tob Res: Official journal of the Society for Research on Nicotine and Tobacco. 2014. View Article : Google Scholar

25 

Kaldenberg DO: Patient satisfaction and the role of choice. Marketing Health Services. 19:391999.

26 

Taylor SL and Cosenza RM: A conceptual choice model for hospital services. J Mark Theory Pract. 7:20–32. 1999. View Article : Google Scholar

27 

Strasser AA, Orom H, Tang KZ, Dumont RL, Cappella JN and Kozlowski LT: Graphic-enhanced information improves perceived risks of cigar smoking. Addict Behav. 36:865–869. 2011. View Article : Google Scholar : PubMed/NCBI

28 

DeBell M, Chapman C, Kleiner A, Lewis L, Hudson L and Shafer L: Computer and Internet Use. Editorial Note. 5:72003.

Related Articles

Journal Cover

September-2016
Volume 12 Issue 3

Print ISSN: 1792-0981
Online ISSN:1792-1015

Sign up for eToc alerts

Recommend to Library

Copy and paste a formatted citation
x
Spandidos Publications style
Denny JT, Denny AM, Tse JT, Deangelis VJ, Chyu D, Pantin EJ, Yeh SS, Cohen S, Fratzola CH, Solina A, Solina A, et al: Hospital initiatives in promoting smoking cessation: A 12-year follow-up. Exp Ther Med 12: 1599-1603, 2016
APA
Denny, J.T., Denny, A.M., Tse, J.T., Deangelis, V.J., Chyu, D., Pantin, E.J. ... Solina, A. (2016). Hospital initiatives in promoting smoking cessation: A 12-year follow-up. Experimental and Therapeutic Medicine, 12, 1599-1603. https://doi.org/10.3892/etm.2016.3496
MLA
Denny, J. T., Denny, A. M., Tse, J. T., Deangelis, V. J., Chyu, D., Pantin, E. J., Yeh, S. S., Cohen, S., Fratzola, C. H., Solina, A."Hospital initiatives in promoting smoking cessation: A 12-year follow-up". Experimental and Therapeutic Medicine 12.3 (2016): 1599-1603.
Chicago
Denny, J. T., Denny, A. M., Tse, J. T., Deangelis, V. J., Chyu, D., Pantin, E. J., Yeh, S. S., Cohen, S., Fratzola, C. H., Solina, A."Hospital initiatives in promoting smoking cessation: A 12-year follow-up". Experimental and Therapeutic Medicine 12, no. 3 (2016): 1599-1603. https://doi.org/10.3892/etm.2016.3496