Skip to main content

Do current approaches to assessing therapy related adverse events align with the needs of long-term cancer patients and survivors?


The increasing efficacy of cancer therapeutics means that the timespan of cancer therapy administration is undergoing a transition to increasingly long-term settings. Unfortunately, chronic therapy-related adverse health events are an unintended, but not infrequent, outcome of these life-saving therapies. Historically, the cardio-oncology field has evolved as retrospective effort to understand the scope, mechanisms, and impact of treatment-related toxicities that were already impacting patients. This review explores whether current systemic approaches to detecting, reporting, tracking, and communicating AEs are better positioned to provide more proactive or concurrent information to mitigate the impact of AE’s on patient health and quality of life. Because the existing tools and frameworks for capturing these effects are not specific to cardiology, this study looks broadly at the landscape of approaches and assumptions. This review finds evidence of increasing focus on the provision of actionable information to support long-term health and quality of life for survivors and those on chronic therapy. However, the current means to assess and support the impact of this burden on patients and the healthcare system are often of limited relevance for an increasingly long-lived survivor and patient population.


The evolution of the field of cardio-oncology is emblematic of an increasing awareness of the scope and impact of cancer therapy-related cardiac toxicities and a desire to limit these effects in both current and future patients. The literature is extant with study of the unintended and delayed but potentially severe cardiac effects of anthracyclines (as used for decades to treat some childhood cancers, breast cancer, etc.) [1,2,3,4,5,6,7]. Unfortunately, even for novel therapeutic approaches such as checkpoint inhibiting immunotherapy, the community again finds itself facing unanticipated and uncertain cardiac adverse events (AE) associated with the primary therapy [8, 9]. This phenomenon begs the questions of whether the cardio-oncology community specifically and cancer care community broadly (research, drug development, regulatory review, clinical practice, patients and their advocates) have improved in their ability to anticipate and support treatment-related AEs that may manifest in long-term cancer survivors.

The nexus of increasing therapeutic efficacy and the reality that 40% of the population will be diagnosed with cancer in their lifetime creates an important new public health challenge [10, 11]. The timespan of cancer therapy administration, care, and outcome is shifting from primarily acute treatment settings to a broad range of chronic adjuvant therapy and survivorship settings [12,13,14]. Treatment-related AEs may span in severity from potentially lethal cardiotoxicities to less dire but still debilitating systemic events, including fatigue, gastrointestinal issues, skin inflammation, and neuropathy [15,16,17,18,19,20,21]. The cumulative effect of these outcomes can vary considerably from patient to patient. AEs can inhibit the curative value of a therapy if these effects impede a patient’s ability or willingness to continue therapy [22]. Modulatory factors such as variable treatment adherence rates, drug-drug interactions, access to care, and patient comorbidities result in a range of patient experiences and healthcare system demands [23]. Even for a specific treatment-related AE (e.g., treatment of aromatase inhibitor–induced chronic pain or approaches to monitoring ejection fraction changes associated with anthracycline cardiotoxicity), the nature of the supportive approaches that are adopted may vary considerably from site to site [24, 25].

Observational studies and health record analyses demonstrate that cancer treatment-related AE (both moderate and severe) can also degrade patients’ or survivors’ overall health status, cause financial strain, and limit their ability to meet family obligations, work, or pursue fitness or hobbies [12, 23]. Supportive care to ameliorate AEs may require patients to procure a broad range of pharmacologic treatments, undergo monitoring and testing, change diet and exercise practices, seek out rehabilitation services or pursue complementary alternative medicine approaches like acupuncture [26,27,28,29,30]. In sum, the impact of AEs on long-term cancer patient and survivor quality of life are broad and diverse in their manifestation (Fig. 1).

Fig. 1
figure 1

Schematic pathway linking cancer treatment, survivorship, adverse events, and quality of life (Original figure by Syril Pettit)

Ideally, of course, our ability to predict, limit, support, and/or prevent cardiac and other AE associated with cancer therapy would evolve in parallel with therapy development and approval. In practice, however, the growth of the cardio-oncology field has been a function of the many medical epidemiology and cohort studies, practitioner case reports, and innovative translational and mechanistic studies initiated only after many decades of observing the profound manifestation of these toxicities as morbidity and mortality in cancer survivors. The cancer care community has an opportunity and obligation to transition towards a more proactive stance on the collection and use of AE information to improve patient outcomes and quality of life.

In order to support the growing population of survivors and patients receiving chronic cancer therapy, a broad set of stakeholders (including drug developers, regulators, insurers, clinicians, patients and their families) will require actionable and integrated risk and benefit information to support long-term health and quality of life. However, a successful transition towards greater proactivity and broadened time horizons for such assessments will require more than an aspiration. It will depend heavily on the consistent use of relevant metrics and robust data integration frameworks.

This paper explores comprehensive literature reviews to evaluate some of the contemporary systemic measures and frameworks used to assess and communicate potential AEs and their impact on patients. Because the systematic approaches to detecting, reporting, integrating, and acting upon on cancer therapy AEs are neither limited to nor unique for cardiovascular AEs – these reviews consider a breadth of endpoints in their discussions and analyses. The specific relevance of these approaches for aiding cardiologists, oncologists, and others in providing support to patients and survivors will be addressed.

Data sources and drivers

Before any oncology drug moves into clinical practice in the United States, the balance between acceptable AEs (risk) and efficacy (benefit) is influenced and assessed by pharmacologic drug design, nonclinical testing, clinical trials, and regulatory review, all of which are major investments spread across the private and public sectors. During this process, the risk:benefit ratio for the therapy is calibrated (by the drug developer and regulatory reviewers) against the lethality of the target cancer [31]. Broadly speaking, these approaches provide an accepted and protective means of balancing anticipated AEs with efficacy in the patient population [32]. Interindividual variability in response to treatment, heterogenous tumor types, and limited study durations have, however, been cited as challenges in the generation of population-level and/or individual patient-level biological outcomes [33]. As a result, investment in enhancing preclinical predictivity is a significant area of growth. The biomedical research community is pursuing the adoption of novel preclinical experimental platforms, innovative preclinical and clinical trial designs, the use of comparative effectiveness methods, and enhanced collection of patient-reported AE data to enhance the predictive relevance of premarket safety and efficacy data [34,35,36,37,38]. It is clear, however, that the current effort and investment in nonclinical oncology safety studies and clinical trials generates data that are more highly focused on and predictive of some outcomes (e.g., acute organ toxicities) than others (e.g., chronic pain or delayed onset events such as cardiac damage) [38, 39].

On the other end of the drug development horizon, and at the genesis of the field of cardio-oncology, are data and experiences from patients outside of a clinical trial or nonclinical study setting. This arena is now often referred to as the source for ‘real world evidence’ (RWE). RWE can be defined as information on health care that is derived from multiple sources outside typical clinical research settings, including electronic health records (EHRs), claims and billing data, product and disease registries, and data gathered through personal devices and health applications” [40]. The cancer community at large is actively exploring opportunities to leverage this type of approach across broad cancer therapy classes and patient demographics. These efforts seek to use RWE in relation to a marketed drug or set of therapeutic approaches to promote a “learning healthcare system” (LHS) in the United States [40,41,42]. The LHS concept, initiated by the Institute of Medicine in the early 2000s, promotes the generation of “the best evidence and to apply that evidence to the healthcare choices that each patient and provider make in collaboration; to drive the process of discovery as a natural outgrowth of patient care; and to ensure innovation, quality, safety, and value in health care.” [43]

Although RWE has been instrumental as a primary evidence base for delayed cardiac effects of cancer therapy, its feasibility as an operational means to realize a more iterative and interconnected healthcare system at large is less clear. The means to integrate RWE as a complement to regulatory safety evaluation via randomized controlled trials (RCTs) and/or as a means of generating novel efficacy, safety, or use information for marketed drugs remains uncertain [40,41,42]. For example, Sherman et al. (2017) cite the potential for RWE (e.g., postmarket surveillance or postmarket trials) to help refine dose-setting, subpopulation identification, and long-term safety considerations for novel cancer therapeutics that receive expedited initial approval. The model they describe, however, does not truly expand the traditional approach to drug evaluation and retains the longstanding emphasis on standard safety/efficacy endpoint collection and regulator-mediated evaluation and decision making. Novel clinical trial designs and settings (e.g., the National Institutes of Health Collaboratory), large-scale health record analysis (e.g., Million Veterans Program), and new patient-engaged networks (e.g., PCORnet) have also been cited as potential opportunities to generate RWE [44,45,46]. As these are all relatively new programs (less than 3 years), their impacts are not yet defined. Ultimately, the success and novelty of any RWE approach to informing healthcare will rely upon the generation of credible, fit for purpose, and otherwise unavailable information as well as viable channels to disseminate and use this information. If indeed RWE is intended to enhance the value of health data (AE-driven or otherwise) to a range of stakeholders, a more nuanced and diverse evaluation of stakeholder needs will be required.

To this end, approaches to better engage patients as the primary stakeholder abound. The movement to provide “patient-centered care” with “shared decision making” is in part driven by a desire to enhance therapeutic adherence and efficacy by engaging patients with understandable information about the benefits, risks, costs, and logistics of their treatment [47,48,49,50,51,52]. This too is a challenging space as, not unexpectedly, preferences for balance of QoL versus length of life vary from patient to patient (and can vary during the course of therapy) [53, 54]. As such, the measurable impact of new patient-centered interventions is an area of active study with regard to the role of patient satisfaction on therapeutic adherence and health outcome [55, 56]. Although the concepts of patient-centered care are widely embraced conceptually, the way in which these elements are pragmatically incorporated into practical care decisions or data generation incentives remains loosely defined [52].

Aligning information and patient need

Despite both the conceptual objectives described in the prior section and the considerable practical experiences in the cardio-oncology arena, to date, there has been minimal systematic assessment of the alignment of available integrative approaches in the published literature with the demographic and temporal realities of an ever-growing cancer survivor population. This review focuses on the source data and assumptions relating to the use of AEs as part of integrative evaluations used to inform therapy-related decisions and patient care for long term treatment or survivorship settings. As the value of these approaches will depend upon their flexibility and external validity, this study includes but will not be limited to cardio-oncology specific applications.

Approach to reviewing the literature

The discussions below reflect the integration of two distinct but complementary comprehensive literature reviews on the use of AE data in characterizing the impact of cancer therapy on long-term cancer patients and survivors. The first review focuses on economic modeling approaches as these were the primary format for integrated “valuation” of the impact of treatment-related AEs before the year 2016. This publication is the first systematic evaluation of the variance in these methodologies with a specific emphasis on their underlying assumptions and data sources. The review evaluates the diversity of AEs and costs evaluated, the range of populations studied, and the relevance of these metrics towards informing and supporting patient quality of life. The second review moves forward in time to recognize the 2016 release of novel integrated ‘value frameworks’ by several professional societies including American Society of Clinical Oncology and the European Society of Medical Oncology. These frameworks were proposed as novel, integrative tools to combine toxicity, efficacy, cost, and other factors deemed critical for treatment and supportive care decision-making. The second literature review identifies the ways in which the recent flurry of integrative value frameworks has (or has not) improved on pre-2016 approaches and the likelihood that these frameworks will support enhanced access to actionable information for patients and clinicians.

Specifically, the reviews address two foundational approaches in the context of long-term cancer therapy and survivorship:

  • Review 1: How have economic models been designed and populated to measure the impact of treatment-related AEs on cancer patients and the healthcare system?

  • Review 2: What recommendations have been promoted to improve the quality and/or relevance of AE-related input data for cancer care value frameworks?

The results of each literature assessment follow below individually and then are integrated to define common approaches, key strengths and limitations, and consensus recommendations for future needs. The specific relevance to the cardio-oncology arena is addressed.

Review 1: Cost as a proxy for AE impact – Integrative assessments before 2016

Over the last 10 years, quantitative efforts to capture the impact of treatment-related adverse events have primarily taken the form of economic studies and quality-adjusted life-year (QALY) models and projections These efforts are largely aimed at economic, regulatory, and/or policy audiences. The few available estimates of the cost of purchasing and administering cancer therapy, monitoring health while on therapy or after, and managing AE detection and care, point to a major societal investment—as much as $120 billion and growing annually [23, 57]. In the cardio-oncology arena, the cumulative direct costs of supportive care for delayed cardiac effects are difficult to source. However, several studies have looked at the cost effectiveness of interventions such as pre-emptive left ventricular ejection fraction screening and determined that it is likely to be a cost-effective measure towards detecting dysfunction before it translates into heart failure [58].

Almost irrespective of the cost figures themselves, the exercise of estimating ‘total’ cost associated with the long-term effects of therapy allows for a thoughtful examination of a range of clinical, lifestyle, financial, social, and temporal elements that extend well beyond the scope of the typical U.S.-based drug safety assessment. In the U.S., the Food and Drug Administration (FDA) is not mandated to consider financial impacts when making regulatory approval decisions for oncology drugs and thus does not consider cost factors in its decision making although such approaches are routinely used in evaluating drugs in Europe [59, 60]. The use of cost as a means of capturing the totality of treatment-related AEs on patients is an approach subject to some debate in economic, clinical, patient, and medical ethics communities [61, 62]. For example, in the context of anthracycline induced cardiac toxicity, the ‘value’ of dexrazoxane as a cardio-protective agent has been an ongoing focus for studies seeking to align data around its protective effects, safety, cost, and societal willingness to pay [63,64,65,66]. Despite the sometimes conflicting outcomes, these studies have a vital role in informing an evolving landscape of methods that integrate diverse data of relevance for characterizing long-term cancer patient care and support.

A 2013 review covered some of these topics as they related to studies between 1999 and 2009 with a primary focus on whether QoL, multiple dose administration, and multiple AEs were considered in the cost assessment [67]. We have extended this review by incorporating material from the years 2007–2017, enhancing the focus on the source of AE data and AE terminology (ontology), characterizing the target patient population to whom the cost/risk predictions apply, and exploring assumptions around the cost of AEs and related supportive services. Although these issues span across the globe, for purposes of limiting the healthcare delivery context referenced in this analysis, the discussions here are focused primarily on the United States. Specifically, the review below asks: How have economic models been designed and populated to measure the impact of treatment-related AEs on cancer patients and the healthcare system?

Search strategy

A semi-systematic search of articles between 2007 and 2017 was conducted using the following databases: PubMed, Web of Science, CINAHL Plus with Full Text, and EconLit. Additional studies were identified through a manual search of references in relevant articles (snowballing) and evaluation of resources from leading organizations in the cancer care arena in the United States (e.g., ASCO). This research focused on those studies which specifically seek to characterize the costs (economic, social, logistical) of managing and treating adverse effects of oncologic therapy. The search terms, exclusion criteria, and Preferred Reporting Items for Systematic Reviews and Meta-Analyses [PRISMA] diagram detailing the literature procurement, filtering, and review strategy are available as online supplemental materials (Additional file 1: Table S1-Table S2, Additional file 2: Figure S1). After comprehensive search, 631 unique citations were identified, 49 were deemed eligible for full text review, and 27 were deemed fit for this analysis. A summary of key findings follows below.


The results of this structured review (2007–2017) provide insight into both the procedural means and situational assumptions around defining the costs of AEs associated with oncologic therapy [68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94]. The studies covered a broad range of therapeutic drug classes, cancer types, and patient populations. The major methodologies used in the 27 studies reviewed were: a) mathematical modeling (Markov models) using historical data and assumptions around the probability that a hypothetical patient would move across different states of disease, health, and death at various points in their care, b) estimations of the hypothetical total cost of care in support of AE based on compilations of diverse sets and sources of data, c) prospective collection of AE incidence, treatment, and cost information for actual patients, or d) meta-analyses of other published cost studies. The studies employed one of two general approaches: estimation of the total cost of a therapeutic regime (drug costs, clinical visit costs, adverse effect costs, etc.) or assessment of the cost of one or more specific AEs associated with a designated cancer therapy. The methods for representation of the cost assessment varied across the studies and included calculation of additional quality adjusted life years (QALY)s relative to total treatment cost, incremental cost to avoid a particular AE, total accumulated costs during a given treatment period (primary treatment costs and AE-related costs), total accumulated costs to treat AE only, and costs per progression-free life-year (PFLY).

The authors looked across the 27 studies to assess their respective approaches assessing and sourcing cost, adverse event, quality of life, and patient demographic information. The specific treatment of these issues in each of the 27 articles reviewed for this study is available as Additional file 1: Table S3-S7. The analysis demonstrated that, despite the differences in the approaches and focus across these articles, several consistent themes and trends were evident (Table 1). As will be discussed further in Results, these themes point to systemic limitations in the availability of patient relevant and externally valid data sources and approaches.

Table 1 Common themes and conclusions identified in review of literature on AE cost determination

Review 2: 2016 value frameworks – Better tools for Cancer care decision making?

Beginning in 2016, the practice of integrative therapy impact assessment took a significant conceptual step forward with the release of five major “value frameworks.” Value frameworks were designed to inform policy decisions as well as pragmatic therapy choices by clinicians and patients for a broad range of cancers, patient demographics, and therapy classes. They aim to integrate data on efficacy, safety (AEs), patient QoL, and, in some cases, cost for specific therapeutic modalities [95,96,97,98,99]. Specifically, the 2016 frameworks and their self-proclaimed objectives are as follows:

  • American Society of Clinical Oncology (ASCO) Value Framework: “A framework that would enable a physician and patient to assess the value of a particular cancer treatment regimen given the patient’s individual preferences and circumstances” [95].

  • European Society for Medical Oncology’s (ESMO) Magnitude of Clinical Benefit Scale (MCBS): “The ESMO-MCBS is an important first step to the critical public policy issue of value in cancer care, helping to frame the appropriate use of limited public and personal resources to deliver cost effective and affordable cancer care” [100].

  • Institute for Clinical and Economic Review (ICER) Value Assessment Framework: “Ultimately, the purpose of the value framework is to form the backbone of rigorous, transparent evidence reports that, within a broader mechanism of stakeholder and public engagement, will help the United States evolve toward a health care system that provides sustainable access to high-value care for all patients” [98].

  • Memorial Sloan Kettering Cancer Center’s (MSKCC) DrugAbacus: “DrugAbacus provides a way of thinking about how to price drugs. This interactive tool takes more than 50 cancer drugs and lets you compare the company’s price to one based on value” [97].

  • National Comprehensive Cancer Center Network (NCCN) Evidence Blocks: “The goal is to provide the health care provider and the patient information to make informed choices when selecting systemic therapies based upon measures related to treatment, supporting data, and cost” [101].

A novel comparative summary of the frameworks with respect to incorporation of AE and QoL specifically is provided here (Table 2). It is notable that AE/toxicity data (typically from published clinical trial data) are incorporated in all of the frameworks as a means of characterizing this key aspect of treatment choice.

Table 2 Comparison of five major value frameworks regarding the use of toxicity and adverse event approaches

Although the construct of these five frameworks and their intended audiences have been compared previously, this study further characterizes the current and future utility of these frameworks with respect to integration of AEs and patient reported outcome (PRO) information [102,103,104,105,106]. Specifically, this review addressed the following: What recommendations have been promoted to improve the quality and/or relevance of AE-related input data for value frameworks?

Search strategy

A systematic literature search was conducted using the following databases: PubMed, Web of Science, and CINAHL Plus with Full Text. Additional studies were identified through a manual search of references in relevant articles (snowballing). The search terms, exclusion criteria, and PRISMA diagram detailing the literature procurement and review strategy are available as Additional file 1: Tables S8-S9, and Additional file 3: Figure S2.


This review captures recommendations and perspectives from a total of 17 peer-reviewed publications [95, 102,103,104, 106,107,108,109,110,111,112,113,114,115,116,117,118]. The reviewed studies were almost evenly split between those including a narrative/qualitative comparison of different frameworks and those incorporating case study/quantitative comparisons across different value frameworks. With limited exceptions, all studies provided perspectives on opportunities to improve either the construct of the framework or some aspect of the input data.

The most commonly identified areas for improvement, in relation to the nature and quality of input data used to populate these frameworks, were divided into eight categories per Table 3. No one recommendation or modification to improve the relevance of the frameworks for informing patient QoL was cited by all the publications. However, the need for more robust and/or detailed safety and toxicity data inclusion in these frameworks was the most common recommendation identified.

Table 3 Literature-based Recommendations for Improvement of Inputs to Existing Frameworks

Additionally, several studies called for more overarching changes to clinical trial design with regard to patient inclusion criteria, duration, outcomes measures, and so forth [102, 110, 112, 115, 119]. The details of such modifications were not thoroughly addressed in these publications and are the subject of much discussion elsewhere, but they could have significant impact on the type of AE data generated in the future [120].

Although not the focus of this review, it is important to note that many of the publications also called for broad-based improvements in the design or use of the frameworks themselves. Specifically, enhanced clarity and transparency as to the intended audience for the framework outputs [103, 104, 110, 111, 121] and improved guidance to enhance reproducibility were common recommendations [107, 108, 111, 114, 119, 122].

Limitations of these reviews

These reviews have a number of limitations. Because this study sought to assess impact in a U.S. healthcare context, economically based health technology assessments (HTAs) as required in Europe and several other regions to assess the cost-benefit of novel therapies were not incorporated. HTA studies are numerous and relatively standardized in their approaches and assumptions. Although HTAs relate only to single-payer healthcare systems that do not match the current U.S. multi-payer profile, they could provide potentially useful sources for methodological comparisons. Because selected HTAs also include quality of life (QALY) assessment in their economic evaluation of the cost-benefit of the therapy they can also provide a resource in this regard for financial valuation-focused queries.

Similarly, the term “comparative effectiveness” was intentionally excluded in order to exclude the “comparative effectiveness research” literature. Although these studies do sometimes include cost estimates of AEs, their focus is exclusively on the differential/comparator between two similar therapies and thus the total cost of AEs (the focus of this review) is rarely measured [67]. Prior systematic reviews of AE effect cost assessments have noted this limitation in the use of comparative effectiveness studies and thus they were excluded from this search. This review also did not compare the absolute value of reported costs across studies because of the variable drugs, study designs, timescales, and patient populations assessed.

Additionally, the breadth of U.S.-based studies in this review provides an opportunity to characterize a diverse range of methods, but it also means that comparison across studies at a granular level is limited. Future studies might focus on a single drug class or cost assessment approach to allow for more focused cross-study comparison of input data and conclusions. Additionally, more comprehensive insights into methodological and data input assumptions across these studies could be gleaned by review of key underlying studies cited by the studies reviewed here.

Synthesis of results of literature evaluations

Characterizing the risk:benefit profile of an antineoplastic therapy requires integration of a complex and heterogenous mix of pharmacologic, economic, actuarial, ethical, and sociologic factors. The complementary literature searches described here illustrate progress toward this integration. However, with respect to use and integration of AE information, several common themes and areas for improvement were identified. These broad areas of commonality and their relevance to the field of cardio-oncology are discussed in detail below.

Challenges in use of clinician-reported adverse event data derived from randomized controlled trials

In both the purely economic and integrated value framework approaches reviewed here, Phase II–III RCTs serve as the primary source of data on the incidence of treatment-related adverse effects. Almost 70% of the cost studies reviewed here (and even more of the value frameworks) rely heavily or exclusively on RCT data to inform toxicity/safety. RCTs are accepted, well-controlled studies with defined inclusion criteria and dosing and monitoring strategies. However, many of the studies reviewed here noted the limitations of RCTs for purposes of providing pragmatic patient decision support (i.e., high internal validity but low external validity). These limitations include the following:

  • Population Mismatch. Populations engaged in RCTs tend to be “healthier” and with fewer comorbidities than the average patient population on the therapy [123]. The frequency and severity of AEs in the clinical trial population may be under-representative of AE incidence and severity in the broader patient population and may thus lead to an underestimation of overall cost and incidence burden [124].

  • Severity Skew. The type of AEs recorded in Phase II/III trials specifically have been reported to skew toward a focus on only high-grade (grade 3 or 4) toxicities, pool toxicities of varying severity, include both quantitative and qualitative evaluations, and/or misgrade toxicities [125, 126]. Thus, it is possible that a significant pool of AE data may be systematically excluded from these evaluations. Even when lower-grade toxicities are reported in RCTs, this review demonstrates an almost exclusive (~ 80%) use of the high-grade AE data for purposes of cost modeling or in value frameworks. The ASCO framework was recently revised to allow for incorporation of grade 1 and 2 AEs if they occur at sufficient frequency [95]. Given the tendency to under-report low-grade AEs in trials and published concerns about “unclear reporting of lower-grade toxicities,” the potential for these endpoints to usefully inform patient/clinician choice via integrative tools is limited at present [127,128,129].

  • Limited Duration. The timeframe of study in an RCT provides a limited window (months to ~ 4 years) for capture of treatment-related effects. This limitation is of particular concern in the cardio-oncology arena as many relevant cardiac AEs do not manifest for many years after the closure of therapy and/or persist for many years after therapy has been completed. As such, RCTs may provide an incomplete picture of impact.

  • Uncertain Naming Conventions. The evolving nature of the CTCAE ontology used to record and grade AEs creates a “moving target.” The number of terms has expanded by a factor of 4 in the last 20 years. Thus, the version of CTCAE (or other ontology) can have a significant impact on the nature, naming, and overall reported incidence of AEs used in cost evaluation studies. Several clinical specialty areas that address common antineoplastic treatment-related AEs (e.g., rheumatology) have developed their own AE ontologies and grades to reflect the more nuanced perspective of a specialist [130]. Experts have noted that the recording of AEs by non-specialists can result in non-specific calls such as ‘cardiac-general’ with no detail about the nature of the cardiotoxicity [127]. Without additional and well-sourced granularity in these toxicity reports, it will be challenging for regulators, clinicians, and patients to appropriately calibrate expectations and supportive care needs. Future AE incidence burden evaluations would benefit from a thorough characterization of the ontological and inclusion/exclusion framework that guided the capture of their core input data and its potential impact on outcomes.

  • Repurposing. RCTs are designed for evaluation by regulatory scientists for purposes of drug approval decision making. Although they are routinely used in this manner, these studies are not specifically designed to generate information to be used by clinicians or patients with regard to individual therapeutic or supportive care pathways.

Alternatives to the use of clinician-reported adverse event data derived from randomized controlled trials

Although of many of the studies reviewed here identified one or more shortcomings of RCT-derived AE data, only three (17%) of the publications on value frameworks proposed the future incorporation of data from sources outside of an RCT setting [102,103,104]. Similarly, only 18% of the cost models used data sources outside of RCTs for AEs [72, 74, 75, 87, 88]. This trend points to a simple fact: while it is relatively easy to identify weaknesses in the RCT as a data source for informing patient value-choices, the identification of viable alternatives or complements is quite challenging. As illustrated in Fig. 2, the articles reviewed here identified potential opportunities for improvement via 1) increased use of PRO measures; and/or 2) increased use of observational/surveillance/EHR data sets.

Fig. 2
figure 2

Summary of recommendations from literature review for improving adverse event data relevance in value frameworks

Patient-reported outcomes and quality of life metrics

As patients and clinicians seek both enhanced progression-free survival as well as positive QoL, the inclusion of PRO data and/or QoL metrics into the valuation (economic or otherwise) of antineoplastic therapy regimes has gained prominence in recent years. The sources of data on QoL in the studies in this review included Markov model-based utility factors derived from EuroQol 5-D surveys, direct patient surveys that collected data on quality metrics, and clinician judgement on impact of patient QoL. The ICER and ESMO frameworks include QoL through incorporation of a QALY metric. ASCO uses palliation of symptoms and treatment-free intervals as a proxy for QoL measures. However, the relevance of these input data are uncertain, as some of the QoL adjustment factors used in these economic evaluations were derived from assessments conducted as many as 20 years ago and some included undocumented “value judgements” based on clinician experience [77, 90, 99, 131]. Additionally, the way in which these data were integrated into the value assessments described in this review varied from probabilistic modeling approaches to awarding of ad hoc “bonus points.” It is beyond the scope of this review to assess the relative strengths and weakness of each of these approaches. However, this review does reveal that the practice of including PRO or QoL metrics into integrated value assessments relating AEs and treatment choice is neither systematic nor standardized.

Undoubtedly, this is a challenging space. The tools and perspectives on the extent to which QoL or PROs can or should be leveraged routinely in trials or clinical practice are evolving rapidly [132]. We also know that there are often disparities between patient and clinical perspectives on the scope and nature of AE burden [133]. Thus, the future use of tools to assess AEs from the perspective of the patient may provide novel insights into the overall physical, logistical, and financial burden of antineoplastic therapy. As patients or survivors experiencing cardiac AEs associated with their curative therapy are likely to experience impacts on their ability to work, exercise, engage in recreation, etc., QoL considerations are expected to be of significant relevance to therapy adherence and outcome.

Increased use of observational/surveillance/electronic health record data sets

Collectively, the publications reviewed here offered very limited recommendations for or examples of incorporating AE data from sources other than RCTs. The few prospective or patient-database driven economic evaluation studies in this review appear to provide a clearer picture of the frequency and nature of AEs, although the less controlled setting can make an estimation of treatment-attributable costs more challenging [72, 74, 75, 87, 88]. None of the value frameworks utilize non-RCT data this time. This phenomenon reflects the “gold standard” status of RCTs for driving drug safety and efficacy decisions and lack of standards for use of other data sources. Increasingly, the potential for observational studies and large-scale healthcare databases to provide reliable data on a broad range of patient adherence practices, outcomes measures, and polypharmacy/comorbidity situations has been recognized [134,135,136]. Future developments in this arena will require a thoughtful confrontation of the tension between uncontrolled data derived directly from patient care settings and the value of nuanced and realistic representation of patient experiences.

The costs

A detailed discussion of cost estimation methodology is not the focus of this review. However, the link between value decisions, cost calculations, and AE-related impacts is an important component of this discussion. Some economic evaluation studies reviewed here attempted to include all treatment-related costs that the author could identify (drug cost, hospital cost, doctor visits, monitoring and testing, over-the-counter drugs, administration fees, lost work cost, caregiver costs, future employment potential costs, etc.; e.g., Sorensen et al., 2012), whereas others addressed only the primary cost of treating the AE in a hospital setting [72]. Cost metrics used within the current value framework structures were equally variable but are largely restricted to cost of the drug and/or primary treatment visits. Many of the analyses reviewed here specifically recommended that future iterations of these models should incorporate the full cost of care including AEs [104, 108, 116, 117, 137] This recommendation, while sound on its face, begs the questions of what constitutes the burden of antineoplastic therapy-related AEs, who carries these burdens, and how broadly should costs be captured. It also speaks to the importance of transparent discussion regarding the stakeholders to whom the value assessment is intended to apply.

The patients

Patients are at the core of all of the value discussions and treatment choices described here. Surprisingly, nearly a third of the economic valuation studies reviewed here failed to provide clear demographic information on either the patient population that constituted their input data or the patient population to whom their model/valuation was intended to characterize or both [80, 82, 86, 89, 100, 111, 114]. In fact, none of the primary value frameworks described in Table 2 or any of the publications about these frameworks (as reviewed here) included a discussion of patient demographics other than a focus on patients with a specific disease. Even for those studies where the patient population was clearly defined, there were sometimes disconnects between the target population and the patient group that served as primary data on AEs, QoL metrics, and/or cost estimations. For example, ~ 40% of the studies reviewed here utilized cost data from Medicare, although only ~ 20% characterized their study population as older than age 60. Such disconnects may be inevitable given the limited availability of data in this space. However, the relevance of frameworks for informing patient choice and treatment decisions could be enhanced with greater clarity around these limitations and their potential impacts on the way in which AE-related impacts are synthesized and subsequently interpreted.

Improving the use of AEs to inform assessments of treatment impact on patients & survivors

We began these reviews by questioning whether current data capture and integration systems are improving in their ability to anticipate and proactively support AEs such as treatment-related cardiotoxicity. On the one hand, these reviews reveal a growing focus on the interests of patients, payers, clinicians, regulators, and drug developers in making informed choices around the use of antineoplastic therapies that enhance progression-free survival with minimal impact on overall QoL. However, it is also clear that our current means to assess and synthesize the scale and impact of this burden on patients, survivors, and the system at large remains insufficient in many ways. Much of the input data used in current integrative efforts to describe AE incidence and severity is of limited or unclear relevance for the diverse, long-lived, and comorbid patient population that is rapidly growing around the world. We observed an overall lack of clarity around how to best use AE data to inform cancer patient care and cancer therapy safety assessment – particularly in long term cancer therapy and survivorship contexts of relevance to the cardio-oncology field. This uncertainty suggests that we may continue to struggle with the proactive identification of treatment-related cardiac (as well as other) effects that take years to fully manifest or to be potentiated. Indeed, our ability to effectively capture the scope and impact of relatively rapid onset (but not highly severe) AEs appears limited as well.

Moving forward, limitations around the data, as well as the implications of those limitations in terms of how they may affect the patient’s lived experience and quality of life should be made more evident to patients, clinicians, and other stakeholders when possible. This communication is essential in supporting more person-centered cancer care planning and shared decision making that helps align treatments with patients’ values and preferences. While such information may not immediately influence treatment decisions, it can have important downstream effects. First, it can seed critical exchanges around supportive care, quality of life priorities and expectations, and health monitoring. These exchanges have the potential to positively influence patient outcomes. Second, as patients and their family caregivers become more educated about AE’s, they have the opportunity to leverage their influence as primary stakeholders to demand improvements that prioritize information quality, access to appropriate supportive care and monitoring, and AE-reducing treatment options in the future for maximizing both quantity and quality of life.


The current breadth of approaches for integrating AE, QoL, efficacy, and cost can be viewed as a signal of the public health and cancer communities’ commitment to and investment in these issues. However, the continued reinvention of these approaches also suggests that the complement of current efforts may not be adequately synergistic or fit for purpose. While the development of the cardio-oncology field has offered some important visibility to the impact of treatment-related AE on patient survival and QoL, it does not appear to be associated with any broad systemic transformations in AE-relevant data collection or use. The biomedical community has made incredible progress in treatments resulting in step-changes in progression free survival. However, the biomedical and public health communities have not adequately transitioned to AE-related information collection and use frameworks that fully aligns with this increasing therapeutic efficacy and survivorship. Future research and associated action is needed to ensure that understandable and reliable information about long term outcomes can be made more relevant and accessible for a growing population of long-term survivors and the stakeholders who seek to support them.


  1. Lipshultz SE, Cochran TR, Franco VI, Miller TL. Treatment-related cardiotoxicity in survivors of childhood cancer. Nat Rev Clin Oncol [Internet]. 2013[cited 2018 Mar 26];10(12):697–710. Available from:

  2. Lipshultz SE, Herman EH. Anthracycline cardiotoxicity: the importance of horizontally integrating pre-clinical and clinical research. Cardiovasc Res [Internet]. 2018[cited 2018 Mar 26];114(2):205–209. Available from:

  3. Bansal N, Amdani S, Lipshultz ER, Lipshultz SE. Chemotherapy-induced cardiotoxicity in children. Expert Opin Drug Metab Toxicol [Internet]. 2017 [cited 2018 Mar 26];13(8):817–832. Available from:

  4. Barac A, Murtagh G, Carver JR, Chen MH, Freeman AM, Herrmann J, et al. Cardiovascular health of patients with cancer and cancer survivors: a roadmap to the next level. Vol. 65. Washington, DC: Elsevier USA; 2015. p. 2739–46.

  5. Mehta LS, Watson KE, Barac A, Beckie TM, Bittner V, Cruz-Flores S, et al. Cardiovascular disease and breast Cancer: where these entities intersect: a scientific statement from the American Heart Association. Circulation [Internet]. 2018 Feb 20 [cited 2018 Mar 26];137(8):e30–e66. Available from:

  6. Robison LL, Armstrong GT, Boice JD, Chow EJ, Davies SM, Donaldson SS, et al. The Childhood Cancer Survivor Study: a National Cancer Institute-supported resource for outcome and intervention research. J Clin Oncol [Internet]. 2009 [cited 2016 Apr 20];27(14):2308–2318. Available from:

  7. Armstrong GT, Kawashima T, Leisenring W, Stratton K, Stovall M, Hudson MM, et al. Aging and Risk of Severe, Disabling, Life-Threatening, and Fatal Events in the Childhood Cancer Survivor Study. J Clin Oncol [Internet]. 2014[cited 2016 May 2];32(12):1218–1227. Available from:

  8. Milling L, Zhang Y, Irvine DJ. Delivering safer immunotherapies for cancer. Adv Drug Deliv Rev [Internet]. 2017[cited 2018 Mar 26];114:79–101. Available from:

  9. Jain V, Bahia J, Mohebtash M, Barac A. Cardiovascular Complications Associated With Novel Cancer Immunotherapies. Curr Treat Options Cardio Med [Internet]. 2017 [cited 2018 Mar 26];19. Available from:

  10. Howlader N, Noone AM, Krapcho M, Miller D, Bishop K, Altekruse SF, Kosary CL, Yu M, Ruhl J, Tatalovich Z, Mariotto A, Lewis DR, Chen HS, Feuer EJ, Cronin KA (eds). to the SEER web site A 2016. SEER Cancer Statistics Review, 1975–2013, National Cancer Institute.

  11. Edwards BK, Noone A-M, Mariotto AB, Simard EP, Boscoe FP, Henley SJ, et al. Annual report to the nation on the status of cancer, 1975–2010, featuring prevalence of comorbidity and impact on survival among persons with lung, colorectal, breast, or prostate cancer. Cancer[Internet]. 2014[cited 2017 Feb 24];120(9):1290–1314. Available from:

  12. Cleeland CS, Allen JD, Roberts SA, Brell JM, Giralt SA, Khakoo AY, et al. Reducing the toxicity of cancer therapy: recognizing needs, taking action. Nat Rev Clin Oncol [Internet]. 2012 Jul 3 [cited 2016 Apr 20];9(8):471–478. Available from:

  13. Berridge BR, Pettit SD, Sarazan RD. Opportunities to meet clinical cardio-oncology needs with new approaches to non-clinical safety assessment. Prog Pediatr Cardiol. 2014;36(1–2):31–2.

    Article  Google Scholar 

  14. Pettit SD, Lipshultz SE, Cleeland CS, Roberts S, Davis M, Berridge BR, et al. Enhancing quality of life as a goal for anticancer therapeutics. Sci Transl Med. 2016;8(344):344ed9.

    Article  PubMed  CAS  Google Scholar 

  15. Bennett S, Pigott A, Beller EM, Haines T, Meredith P, Delaney C. Educational interventions for the management of cancer-related fatigue in adults. In: Bennett S, editor. Cochrane Database of Systematic Reviews [Internet]. Chichester: John Wiley & Sons, Ltd; 2016 [cited 2017 Feb 24]. Available from:

  16. de Golian E, Kwong BY, Swetter SM, Pugliese SB. Cutaneous Complications of Targeted Melanoma Therapy. Curr Treat Options Oncol [Internet]. 2016 Nov 19 [cited 2017 Feb 24];17(11):57. Available from:

  17. Kumar V, Chaudhary N, Garg M, Floudas CS, Soni P, Chandra AB. Current Diagnosis and Management of Immune Related Adverse Events (irAEs) Induced by Immune Checkpoint Inhibitor Therapy. Front Pharmacol [Internet]. 2017[cited 2017 Feb 26];8:49. Available from:

  18. Macdonald JB, Macdonald B, Golitz LE, LoRusso P, Sekulic A, Agha R, et al. Cutaneous adverse effects of targeted therapies: Part I: Inhibitors of the cellular membrane. J Am Acad Dermatol [Internet]. 2015[cited 2017 Feb 5];72(2):203–218; quiz 219–20. Available from:

  19. Ocean A, Vahdat L. Chemotherapy-induced peripheral neuropathy: pathogenesis and emerging therapies. Support Care Cancer [Internet]. 2004 Jul 16 [cited 2016 Nov 17];12(9):619–625. Available from:

  20. Santoni M, Conti A, De Giorgi U, Iacovelli R, Pantano F, Burattini L, et al. Risk of gastrointestinal events with sorafenib, sunitinib and pazopanib in patients with solid tumors: A systematic review and meta-analysis of clinical trials. Int J Cancer [Internet]. 2014[cited 2017 Feb 24];135(4):763–773. Available from:

  21. Speck RM, Sammel MD, Farrar JT, Hennessy S, Mao JJ, Stineman MG, et al. Impact of Chemotherapy-Induced Peripheral Neuropathy on Treatment Delivery in Nonmetastatic Breast Cancer J Oncol Pract [Internet]. 2013[cited 2016 Nov 17];9(5):e234–e240. Available from:

  22. Castellanos EH, Chen SC, Drexler H, Horn L. Making the grade: the impact of low-grade toxicities on patient preference for treatment with novel agents. JNCCN J Natl Compr Cancer Netw [Internet]. 2015 Dec [cited 2017 Jan 26];13(12):1490–1495. Available from:

  23. Fitzner K, Oteng-Mensah F, Donley P, Heckinger EAF. Safety of Cancer Therapies: At What Cost? Popul Health Manag [Internet]. 2017;00(00):pop.2016.0097. Available from:

  24. Yang GS, Kim HJ, Griffith KA, Zhu S, Dorsey SG, Renn CL. Interventions for the treatment of aromatase inhibitor-associated arthralgia in breast Cancer survivors: a systematic review and meta-analysis. Cancer Nurs. 2016;00(0):1–16.

    Article  Google Scholar 

  25. Henriksen PA. Anthracycline cardiotoxicity: an update on mechanisms, monitoring and prevention Heart [Internet]. 2017 [cited 2018 Apr 12];heartjnl-2017-312103. Available from:

  26. Kottschade L, Brys A, Peikert T, Ryder M, Raffals L, Brewer J, et al. A multidisciplinary approach to toxicity management of modern immune checkpoint inhibitors in cancer therapy. MELANOMA Res [Internet]. 2016[cited 2017 Feb 25];26(5):469–480. Available from:

  27. Lee I, Garland SN, DeMichele A, Farrar JT, Im E-O, Mao JJ. A cross-sectional survey of pain catastrophising and acupuncture use among breast cancer survivors. Acupunct Med [Internet]. 2017 Mar [cited 2017 Feb 25];35(1):38–43. Available from:

  28. Wallner M, Köck-Hódi S, Booze S, White K, Mayer H. Nursing Management of Cutaneous Toxicities From Epidermal Growth Factor Receptor Inhibitors. Clin J Oncol Nurs [Internet]. 2016[cited 2017 Feb 25];20(5):529–536. Available from:

  29. Silver JK, Baima J, Mayer RS. Impairment-driven cancer rehabilitation: An essential component of quality care and survivorship. CA Cancer J Clin [Internet]. 2013[cited 2018 May 21];63(5):295–317. Available from:

  30. Silver JK, Raj VS, Fu JB, Wisotzky EM, Smith SR, Kirch RA. Cancer rehabilitation and palliative care: critical components in the delivery of high-quality oncology services. Support Care Cancer [Internet]. 2015[cited 2018 May 21];23(12):3633–3643. Available from:

  31. Kuderer NM, Wolff AC. Enhancing Therapeutic Decision Making When Options Abound: Toxicities Matter. J Clin Oncol [Internet]. 2014 1 [cited 2016 Sep 10];32(19):1990–1993. Available from:

  32. Eichler H-G, Pignatti F, Flamion B, Leufkens H, Breckenridge A. Balancing early market access to new drugs with the need for benefit/risk data: a mounting dilemma. Nat Rev Drug Discov [Internet]. 2008 Oct 12 [cited 2017 Feb 5];7(10):818–826. Available from:

  33. Mak IW, Evaniew N, Ghert M. Lost in translation: animal models and clinical trials in cancer treatment. Am J Transl Res [Internet]. 2014 [cited 2017 Mar 3];6(2):114–118. Available from:

  34. Basch E, Deal AM, Kris MG, Scher HI, Hudis CA, Sabbatini P, et al. Symptom Monitoring With Patient-Reported Outcomes During Routine Cancer Treatment: A Randomized Controlled Trial. J Clin Oncol [Internet]. 2016 Feb 20 [cited 2016 Jul 14];34(6):557–565. Available from:

  35. Redig AJ, Jänne PA. Basket trials and the evolution of clinical trial design in an era of genomic medicine. TL - 33. J Clin Oncol [Internet]. 2015;33 VN-r(9):975–977. Available from:

  36. Woodcock J, Ware JH, Miller PW, Mcmurray JJ V, Harrington DP, Drazen JM. Clinical Trials Series. 2016 [cited 2017 Mar 5];22. Available from:

  37. Fiore LD, Lavori PW. Integrating Randomized Comparative Effectiveness Research with Patient Care. Drazen JM, Harrington DP, McMurray JJV, Ware JH, Woodcock J, editors. N Engl J Med [Internet]. 2016[cited 2017 Mar 5];374(22):2152–2158. Available from:

  38. Dambach DM, Simpson NE, Jones TW, Brennan RJ, Pazdur R, Palmby TR. Nonclinical evaluations of small-molecule oncology drugs: integration into clinical dose optimization and toxicity management. Clin Cancer Res. 2016;22(11):2618–22.

    Article  PubMed  CAS  Google Scholar 

  39. Woolf CJ. Overcoming obstacles to developing new analgesics. Nat Med [Internet]. 2010;16(11):1241–7. Available from:

  40. Sherman RE, Anderson SA, Dal Pan GJ, Gray GW, Gross T, Hunter NL, et al. Real-World Evidence — What Is It and What Can It Tell Us? N Engl J Med [Internet]. 2016;375(23):2293–2297. Available from:

  41. Sherman RE, Davies KM, Robb MA, Hunter NL, Califf RM. Accelerating development of scientific evidence for medical products within the existing US regulatory framework. Nat Publ Gr [Internet]. 2017 [cited 2017 Sep 23]; Available from:

  42. Califf RM, Robb MA, Bindman AB, Briggs JP, Collins FS, Conway PH, et al. Transforming Evidence Generation to Support Health and Health Care Decisions N Engl J Med [Internet]. 2016;375(24):2395–2400. Available from:

  43. Medicine I of. The Learning Healthcare System [Internet]. Washington, D.C.: National Academies Press; 2007 [cited 2017 Jan 31]. Available from:

  44. Weinfurt KP, Hernandez AF, Coronado GD, Debar LL, Dember LM, Green BB, et al. Pragmatic clinical trials embedded in healthcare systems: generalizable lessons from the NIH Collaboratory. BMC Med Res Methodol. 2017;17:144.

    Article  PubMed  PubMed Central  Google Scholar 

  45. PCORnet, the National Patient-Centered Clinical Research Network - PCORnet [Internet]. [cited 2017 Sep 23]. Available from:

  46. Million Veteran Program (MVP) [Internet]. [cited 2017 Sep 23]. Available from:

  47. National Institute for Health and Care Excellence. Medicines adherence: involving patients in decisions about prescribed medicines and supporting adherence | Guidance and guidelines | NICE [Internet]. NICE; 2009 [cited 2017 Sep 17]. Available from:

  48. Atherton PJ, Smith T, Singh JA, Huntington J, Diekmann BB, Huschka M, et al. The relation between cancer patient treatment decision-making roles and quality of life. Cancer [Internet]. 2013 Jun 15 [cited 2017 Sep 13];119(12):2342–2349. Available from:

  49. Advisory Board Company Cancer Patient Preferences Explorer | The Advisory Board Company [Internet]. [cited 2017 Sep 13]. Available from:

  50. Zucca A, Sanson-Fisher R, Waller A, Carey M. Patient-centred care: making cancer treatment centres accountable. Support Care Cancer [Internet]. 2014 [cited 2017 Sep 14];22:1989–1997. Available from:

  51. Hare JI, Lammers T, Ashford MB, Puri S, Storm G, Barry ST. Challenges and strategies in anti-cancer nanomedicine development: Anindustry perspective. Adv Drug Deliv Rev. 2017;108:25–38.

    Article  PubMed  CAS  Google Scholar 

  52. Barry MJ, Edgman-Levitan S. Shared Decision Making — The Pinnacle of Patient-Centered Care. 2012 Mar 29 [cited 2017 Sep 17];(9):780–1. Available from:

  53. Meropol NJ, Egleston BL, Buzaglo JS, Benson AB, Cegala DJ, Diefenbach MA, et al. Cancer patient preferences for quality and length of life. Cancer [Internet]. 2008 Dec 15 [cited 2017 Sep 13];113(12):3459–3466. Available from:

  54. Singh S, Butow P, Charles M, Martin H N Tattersall. Shared decision making in oncology: assessing oncologist behaviour in consultations in which adjuvant therapy is considered after primary surgical treatment. 2010 [cited 2017 Sep 12];13:244–257. Available from:

  55. Shingler SL, Bennett BM, Cramer JA, Towse A, Twelves C, Lloyd AJ. Treatment preference, adherence and outcomes in patients with cancer: literature review and development of a theoretical model. Curr Med Res Opin. 2014.

  56. Hoerger M, Epstein RM, Winters PC, Fiscella K, Duberstein PR, Gramling R, et al. Values and options in cancer care (VOICE): study design and rationale for a patient-centered communication and decision-making intervention for physicians, patients with advanced cancer, and their caregivers. BMC Cancer [Internet]. 2013 [cited 2017 Sep 17];13:188. Available from:

  57. Yabroff KR, Lund J, Kepka D, Mariotto A. Economic burden of cancer in the United States: estimates, projections, and future research. Cancer Epidemiol Biomarkers Prev [Internet]. 2011 Oct [cited 2017 Mar 3];20(10):2006–2014. Available from:

  58. Wong FL, Bhatia S, Landier W, Francisco L, Leisenring W, Hudson MM, et al. Cost-Effectiveness of the Children’s Oncology Group Long-Term Follow-up Screening Guidelines for Childhood Cancer Survivors at Risk for Treatment-Related Heart Failure Ann Intern Med [Internet]. 2014 May 20 [cited 2018 Apr 12];160(10):672. Available from:

  59. McKee AE, Farrell AT, Pazdur R, Woodcock J. The role of the U.S. Food and Drug Administration review process: clinical trial endpoints in oncology. Oncologist [Internet]. 2010 [cited 2017 Mar 5];15 Suppl 1(Supplement 1):13–18. Available from:

  60. Siddiqui M, Vincent Rajkumar S. The High Cost of Cancer Drugs and What We Can Do About It. JMCP [Internet]. 2012 [cited 2017 Mar 5];87:935–943. Available from:

  61. Fleck LM and Danis M. How Should Therapeutic Decisions about Expensive Drugs Be Made in Imperfect Environments? AMA J Ethic [Internet]. 2017 1 [cited 2017 Aug 9];19(2):147–156. Available from:

  62. Kumar P, Moy B. The cost of cancer care--balancing our duties to patients versus society: are they mutually exclusive? Oncologist [Internet]. 2013 [cited 2017 Aug 9];18(4):347–349. Available from:

  63. Marty M, Espié M, Llombart A, Monnier A, Rapoport BL, Stahalova V. Multicenter randomized phase III study of the cardioprotective effect of dexrazoxane (Cardioxane®) in advanced/metastatic breast cancer patients treated with anthracycline-based chemotherapy. Ann Oncol [Internet]. 2006 1 [cited 2018 Apr 12];17(4):614–622. Available from:

  64. Armstrong GT, Ross JD. Late cardiotoxicity in aging adult survivors of childhood cancer. Prog Pediatr Cardiol [Internet]. 2014 Sep 1 [cited 2018 Apr 12];36(1–2):19–26. Available from:

  65. Limat S, Demesmay K, Fagnoni P, Voillat L, Bernard Y, Deconinck E, et al. Cost Effectiveness of Cardioprotective Strategies in Patients with Aggressive Non-Hodgkin’s Lymphoma. Clin Drug Investig [Internet]. 2005 [cited 2018 Apr 12];25(11):719–29. Available from:

  66. Nolan MT, Wang Y, Pathan F, Marwick TH. Abstract 16276: Cost-Effectiveness Analysis of Dexrazoxane in Preventing Long-Term Cardiac Dysfunction in Paediatric Patients With Haematological Malignancies. Circulation [Internet]. 2016 [cited 2018 12];134(Suppl 1). Available from:

  67. Pearce A, Haas M, Viney R. Are the true impacts of adverse events considered in economic models of antineoplastic drugs? A systematic review. Appl Health Econ Health Policy [Internet]. 2013;11(6):619–37. Available from:

    Article  Google Scholar 

  68. Ayvaci MUS, Shi J, Alagoz O, Lubner SJ. Cost-effectiveness of adjuvant FOLFOX and 5FU/LV chemotherapy for patients with stage II colon cancer. Med Decis Mak. 2013;33(4):521–32.

    Article  Google Scholar 

  69. Bajaj PS, Veenstra DL, Goertz H-P, Carlson JJ. Targeted erlotinib for first-line treatment of advanced non-small cell lung cancer: a budget impact analysis. J Med Econ [Internet]. 2014;17(8):538–546. Available from:

  70. Bilir SP, Ma Q, Zhao Z, Wehler E, Munakata J, Barber B. Economic Burden of Toxicities Associated with Treating Metastatic Melanoma in the United States. Am Heal drug benefits [Internet]. 2016 [cited 2017 Jan 15];9(4):203–213. Available from:

  71. Bristow RE, Santillan A, Salani R, Diaz-Montes TP, Giuntoli RL 2nd, Meisner BC, et al. Intraperitoneal cisplatin and paclitaxel versus intravenous carboplatin and paclitaxel chemotherapy for stage III ovarian cancer: a cost-effectiveness analysis. Gynecol Oncol. 2007;106(3):476–81.

    Article  PubMed  CAS  Google Scholar 

  72. Burke TA, Wisniewski T, Ernst FR. Resource utilization and costs associated with chemotherapy-induced nausea and vomiting (CINV) following highly or moderately emetogenic chemotherapy administered in the US outpatient hospital setting. Support Care Cancer [Internet]. 2011 [cited 2017 Mar 28];19:131–40. Available from:

  73. Burudpakdee C, Zhao Z, Munakata J, Gao S, Trochlil K, Barber B. Economic burden of toxicities associated with metastatic colorectal cancer treatment regimens containing monoclonal antibodies. J Med Econ [Internet]. 2012;15(2):371–377. Available from:

  74. Chu E, Schulman KL, Zelt S, Song X. Costs associated with complications are lower with Capecitabine than with 5-fluorouracil in patients with colorectal Cancer. Cancer. 2009;115(7):1412–23..

    Article  PubMed  CAS  Google Scholar 

  75. Craver C, Gayle J, Balu S, Buchner D. Original article Clinical and economic burden of chemotherapy-induced nausea and vomiting among patients with cancer in a hospital outpatient setting in the United States. J Med Econ [Internet]. 2011 [cited 2017 Mar 28];14(1):87–98. Available from:

  76. Dalton HJ, Yu X, Hu L, Kapp DS, Benjamin I, Monk BJ, et al. An economic analysis of dose dense weekly paclitaxel plus carboplatin versus every-3-week paclitaxel plus carboplatin in the treatment of advanced ovarian cancer. Gynecol Oncol. 2012;124(2):199–204.

    Article  PubMed  Google Scholar 

  77. Delea TE, Amdahl J, Diaz J, Nakhaipour HR, Hackshaw MD. Cost-effectiveness of pazopanib versus sunitinib for renal cancer in the United States. J Manag care Spec Pharm. 2015;21(1):46–54, 54a-b.

    Article  PubMed  Google Scholar 

  78. Goldstein DA, Chen Q, Ayer T, Howard DH, Lipscomb J, Harvey RD, et al. Cost effectiveness analysis of pharmacokinetically-guided 5-fluorouracil in folfox chemotherapy for metastatic colorectal cancer. Clin Colorectal Cancer [Internet]. 2014;13(4):219–225. Available from:

  79. Goldstein DA, Krishna K, Flowers CR, El-Rayes BF, Bekaii-Saab T, Noonan AM. Cost description of chemotherapy regimens for the treatment of metastatic pancreas cancer. Med Oncol [Internet]. 2016;33(5):48. Available from:

  80. Goulart B, Ramsey S. A trial-based assessment of the cost-utility of Bevacizumab and chemotherapy versus chemotherapy alone for advanced non-small cell lung cancer. Value Heal [Internet] 2011;14(6):836–845. Available from:

  81. Haiderali A, Menditto L, Good M, Teitelbaum A, Wegner J. Impact on daily functioning and indirect/direct costs associated with chemotherapy-induced nausea and vomiting (CINV) in a US population. Support Care Cancer [Internet]. 2011 [cited 2017 Mar 28];19:843–851. Available from:

  82. Havrilesky LJ, Secord AA, Kulasingam S, Myers E. Management of platinum-sensitive recurrent ovarian cancer: a cost-effectiveness analysis. Gynecol Oncol. 2007;107(2):211–8.

    Article  PubMed  Google Scholar 

  83. Hess LM, Rajan N, Winfree K, Davey P, Ball M, Knox H, et al. Cost analyses in the US and Japan: a cross-country comparative analysis applied to the PRONOUNCE trial in non-squamous non-small cell lung Cancer. Adv Ther. 2015;32(12):1248–62.

    Article  PubMed  CAS  PubMed Central  Google Scholar 

  84. Kowal-Podmore S, Munakata J, Tencer T, Smith TW. Economic burden of toxicities associated with salvage treatment in advanced and metastatic breast cancer. VALUE Heal. 2008;11(6):A479.

    Article  Google Scholar 

  85. Kurian AW, Thompson RN, Gaw AF, Arai S, Ortiz R, Garber AM. A cost-effectiveness analysis of adjuvant trastuzumab regimens in early HER2/neu-positive breast cancer. J Clin Oncol [Internet]. 2007;25(6):634–641. Available from:

  86. Niraula S, Amir E, Vera-Badillo F, Seruga B, Ocana A, Tannock IF. Risk of Incremental Toxicities and Associated Costs of New Anticancer Drugs: A Meta-Analysis. J Clin Oncol [Internet]. 2014 Nov 10 [cited 2016 Apr 20];32(32):3634–3642. Available from:

  87. Rajan SS, Carpenter WR, Stearns SC, Lyman GH. Short-term costs associated with primary prophylactic G-CSF use during chemotherapy. Am J Manag Care. 2013;19(2):150–9.

    PubMed  Google Scholar 

  88. Tina Shih Y-C, Xu Y, Elting LS. Costs of uncontrolled chemotherapy-induced nausea and vomiting among working-age cancer patients receiving highly or moderately emetogenic chemotherapy. Cancer [Internet]. 2007 Aug 1 [cited 2017 Mar 28];110(3):678–685. Available from:

  89. Sorensen SV, Goh JW, Pan F, Chen C, Yardley D, Martin M, et al. Incidence-based Cost-of-illness model for metastatic breast cancer in the United States. Int J Technol Assess Health Care. 2012;28(1):12–21.

    Article  PubMed  Google Scholar 

  90. Stopeck A, Rader M, Henry D, Danese M, Halperin M, Cong Z, et al. Cost-effectiveness of denosumab vs zoledronic acid for prevention of skeletal-related events in patients with solid tumors and bone metastases in the United States. J Med Econ [Internet]. 2012;15(4):712–723. Available from:

  91. Ting J, Tien Ho P, Xiang P, Sugay A, Abdel-Sattar M, Wilson L. Cost-effectiveness and value of information of Erlotinib, Afatinib, and Cisplatin-Pemetrexed for first-line treatment of advanced EGFR mutation-positive non-small-cell lung Cancer in the United States. Value Heal. 2015;18(6):774–82.

    Article  Google Scholar 

  92. Usmani SZ, Cavenagh JD, Belch AR, Hulin C, Basu S, White D, et al. Cost-effectiveness of lenalidomide plus dexamethasone vs bortezomib plus melphalan and prednisone in transplant-ineligible US patients with newly-diagnosed multiple myeloma. J Med Econ [Internet]. 2016;19(3):243–258. Available from:

  93. Wong YN, Meropol NJ, Speier W, Sargent D, Goldberg RM, Beck JR, et al. Cost implications of new treatments for advanced colorectal cancer. Cancer [Internet]. 2009;115(10):2081–2091. Available from:

  94. Xie J, Diener M, Sorg R, Wu EQ, Namjoshi M. Cost-effectiveness of denosumab compared with zoledronic acid in patients with breast cancer and bone metastases. Clin Breast Cancer. 2012;12(4):247–58.

    Article  PubMed  Google Scholar 

  95. Schnipper LE, Davidson NE, Wollins DS, Blayney DW, Dicker AP, Ganz PA, et al. Updating the American Society of Clinical Oncology Value Framework: Revisions and Reflections in Response to Comments Received. J Clin Oncol [Internet]. 2016 Aug 20 [cited 2017 May 27];34(24):2925–2934. Available from:

  96. ESMO. ESMO Magnitude of Clinical Benefit Scale | Evaluation Forms | ESMO [Internet]. 2017 [cited 2017 Jul 3]. Available from:

  97. MSKCC. DrugAbacus - Drug Pricing Lab [Internet]. 2017 [cited 2017 Jul 3]. Available from:

  98. ICER. ICER Value Assessment Framework – ICER [Internet]. 2017 [cited 2017 Jul 3]. Available from:

  99. NCCN. NCCN Clinical Practice Guidelines in Oncology (NCCN Guidelines) with NCCN Evidence Blocks [Internet]. 2017 [cited 2017 Jul 3]. Available from:

  100. ESMO. ESMO Magnitude of Clinical Benefit Scale. 2017.

    Google Scholar 

  101. NCCN. National Comprehensive Cancer Center Network Evidence Blocks. 2017.

    Google Scholar 

  102. Allen JD, Stewart MD, Roberts SA, Sigal E V. The Value of Addressing Patient Preferences. Value Heal [Internet]. 2017 Feb [cited 2017 May 23];20(2):283–285. Available from:

  103. Basch E. Toward a patient-centered value framework in oncology. [Internet]. JAMA American Medical Association; May 17, 2016 p. 2073–2074. Available from:

  104. Chandra A, Shafrin J, Dhawan R. Utility of Cancer Value Frameworks for Patients, Payers, and Physicians. JAMA [Internet]. 2016 May 17 [cited 2017 May 23];315(19):2069. Available from:

  105. Schnipper LE, Bastian A. New Frameworks to Assess Value of Cancer Care: Strengths and Limitations. Oncologist [Internet]. 2016 Jun [cited 2017 Jun 9];21(6):654–658. Available from:

  106. Subramanian, Ram and Schorr K. Musings on value frameworks in cancer. Eff Res [Internet]. 2016 [cited 2017 May 23];5(5):437–439. Available from:

  107. Kenneth Evans W, Cheung MC, K-W Chan K. Measuring value and benefit–a matter of perspective. Lancet Oncol [Internet]. 2017 [cited 2017 Jul 2];18:839–840. Available from:

  108. Mandelblatt JS, Ramsey SD, Lieu TA, Phelps CE. Evaluating Frameworks That Provide Value Measures for Health Care Interventions. VALUE Heal [Internet]. 2017 Feb [cited 2017 May 23];20(2):185–192. Available from:

  109. Bentley TGK, Cohen JT, Elkin EB, Huynh J, Mukherjea A, Neville TH, et al. Measuring the value of new drugs: validity and reliability of 4 ValueAssessment frameworks in the oncology setting. J Manag CARE Spec Pharm. 2017;23(6, S):S34–48.

    Article  PubMed  Google Scholar 

  110. Booth CM, Del Paggio JC. Approvals in 2016: questioning the clinical benefit of anticancer therapies. Nat Rev Clin Oncol [Internet]. 2017 Feb 20 [cited 2017 Jul 2];14(3):135–6. Available from:

  111. Cohen JT, Anderson JE, Neumann PJ. Three Sets of Case Studies Suggest Logic and Consistency Challenges with Value Frameworks. [cited 2017 May 23]; Available from:

  112. Del Paggio JC, Sullivan R, Schrag D, Hopman WM, Azariah B, Pramesh CS, et al. Delivery of meaningful cancer care: a retrospective cohort study assessing cost and benefit with the ASCO and ESMO frameworks. [Internet]. 2017 [cited 2017 Jul 2].

  113. Shah-Manek B, Galanto JS, Candidate P. Value Frameworks for the Patient-Provider Interaction: A Comparison of the ASCO Value Framework Versus NCCN Evidence Blocks in Determining Value in Oncology. JMCP Suppl to J Manag Care Spec Pharm [Internet]. 2017 [cited 2017 2];23(6). Available from:

  114. Westrich K, Buelt L, Dubois RW. Why Value Framework Assessments Arrive at Different Conclusions: A Multiple Myeloma Case Study. S28 Suppl to J Manag Care Spec Pharm [Internet]. 2017 [cited 2017 May 24];23(6). Available from:

  115. Jim HSL, McLeod HL. American Society of Clinical Oncology value framework: importance of accurate toxicity data. J Clin Oncol. 2017;35(10):1133+.

    Article  PubMed  Google Scholar 

  116. Miller TP, Aplenc R, Schnipper LE, Schilsky RL, Jim HSL, McLeod HL. American Society of Clinical Oncology value framework: importance of accurate toxicity data. Reply J Clin Oncol. 2017;35(10):1133.

    Article  Google Scholar 

  117. Reginald Waldeck A, Botteman MF, White RE, van Hout BA. The Importance of Economic Perspective and Quantitative Approaches in Oncology Value Frameworks of Drug Selection and Shared Decision Making. S6 Suppl to J Manag Care Spec Pharm [Internet]. 2017 [cited 2017 Jul 2];23(6). Available from:

  118. Yu PP. Challenges in Measuring Cost and Value in Oncology: Making It Personal. [cited 2017 Jul 2]; Available from:

  119. Bentley TGK, Cohen JT, Elkin EB, Huynh J, Mukherjea A, Neville TH, et al. Measuring the Value of New Drugs: Validity and Reliability of 4 Value Assessment Frameworks in the Oncology Setting. J Manag care Spec Pharm [Internet]. 2017 Jun [cited 2017 Jul 2];23(6–a Suppl):S34–S48. Available from:

  120. Bhatt DL, Mehta C. Adaptive Designs for Clinical Trials Drazen JM, Harrington DP, McMurray JJV, Ware JH, Woodcock J, editors. N Engl J Med [Internet]. 2016 Jul 7 [cited 2017 Jul 9];375(1):65–74. Available from:

  121. Waldeck AR, Botteman MF, White RE, vanHout BA. The importance of economic perspective and quantitative approaches inOncology value frameworks of drug selection and shared decision making. J Manag CARE Spec Pharm. 2017;23(6, S):S6–12.

    Article  PubMed  Google Scholar 

  122. Shah-Manek B, Galanto JS, Nguyen H, Ignoffo R. Value frameworks for the patient-provider interaction: a comparison ofthe ASCO value framework versus NCCN evidence blocks in DeterminingValue in oncology. J Manag CARE Spec Pharm. 2017;23(6, S):S13–20..

    Article  PubMed  Google Scholar 

  123. Martin K, Bégaud B, Latry P, Miremont-Salamé G, Fourrier A, Moore N. Differences between clinical trials and postmarketing use. Br J Clin Pharmacol [Internet]. 2004 Jan [cited 2017 Apr 2];57(1):86–92. Available from:

  124. asco. Clinical trial subjects compared to "real world" patients: Generalizability of renal cell carcinoma trials. | 2014 ASCO Annual Meeting | Abstracts | Meeting Library [Internet]. 2014 [cited 2017 Apr 2]. Available from:

  125. Zhang X, Zhang Y, Ye X, Guo X, Zhang T, He J. Overview of phase IV clinical trials for postmarket drug safety surveillance: a status report from the registry. BMJ Open [Internet]. 2016 Nov 23 [cited 2017 Feb 5];6(11):e010643. Available from:

  126. Peron J, Maillet D, Gan HK, Chen EX, You B. Adherence to CONSORT Adverse Event Reporting Guidelines in Randomized Clinical Trials Evaluating Systemic Cancer Therapy: A Systematic Review. J Clin Oncol [Internet]. 2013 1 [cited 2016 Aug 23];31(31):3957–3963. Available from:

  127. Zhang S, Liang F, Tannock I, Peron J, Pond G, Gan H, et al. Use and misuse of common terminology criteria for adverse events in cancer clinical trials. BMC Cancer [Internet]. 2016 Dec 4 [cited 2016 Aug 23];16(1):392. Available from:

  128. Miller TP, Li Y, Kavcic M, Troxel AB, Huang Y-S V., Sung L, et al. Accuracy of Adverse Event Ascertainment in Clinical Trials for Pediatric Acute Myeloid Leukemia. J Clin Oncol [Internet]. 2016 May [cited 2017 Jul 9];34(13):1537–1543. Available from:

  129. Shah-Manek B, Galanto JS, Nguyen H, Ignoffo R. Value Frameworks for the Patient-Provider Interaction: A Comparison of the ASCO Value Framework Versus NCCN Evidence Blocks in Determining Value in Oncology. J Manag care Spec Pharm [Internet]. 2017 Jun [cited 2017 Jul 2];23(6–a Suppl):S13–S20. Available from:

  130. Calabrese C, Kirchner E, Kontzias K, Velcheti V, Calabrese LH. Rheumatic immune-related adverse events of checkpoint therapy for cancer: case series of a new nosological entity. RMD Open [Internet]. 2017;3(1):e000412. Available from:

  131. Havrilesky LJ, Broadwater G, Davis DM, Nolte KC, Barnett JC, Myers ER, et al. Determination of quality of life-related utilities for health states relevant to ovarian cancer diagnosis and treatment. Gynecol Oncol. 2009;113(2):216–20.

    Article  PubMed  PubMed Central  Google Scholar 

  132. NCI’s Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE): Selecting Items and Assessment Frequency for Cancer Trials SEVENTH ANNUAL PATIENT-REPORTED OUTCOME (PRO) CONSORTIUM WORKSHOP. [cited 2017 Mar 31]; Available from:

  133. Basch E, Geoghegan C, Coons SJ, Gnanasakthy A, Slagle AF, Papadopoulos EJ, et al. Patient-Reported Outcomes in Cancer Drug Development and US Regulatory Review. JAMA Oncol [Internet]. 2015 Jun 1 [cited 2016 Dec 2];1(3):375. Available from:

  134. Mahajan R. Real world data: additional source for making clinical decisions. Int J Appl basic Med Res [Internet]. 2015 [cited 2017 Apr 2];5(2):82. Available from:

  135. Balicer RD, Afek A. Digital health nation: Israel’s global big data innovation hub. Lancet [Internet]. 2017;6736(17):8–10. Available from:

    Google Scholar 

  136. Fiore L, Brophy M, Ferguson R, Shannon C, Turek S, Pierce-Murray K, et al. Data Sharing, Clinical Trials, and Biomarkers in Precision Oncology: Challenges, Opportunities, and Programs at the Department of Veterans Affairs. Clin Pharmacol Ther [Internet]. 2017 May 1 [cited 2017 Jul 12];101(5):586–589. Available from:

  137. Del Paggio JC. Deconstructing Clinical Trials?Help From Oncology Value Frameworks. JAMA Oncol [Internet]. 2017 1 [cited 2017 Jul 2]; Available from:

  138. Kaufman HL, Atkins MB, Dicker AP, Jim HS, Garrison LP, Herbst RS, et al. Value of Cancer Immunotherapy Summit at 2016. [cited 2017 Jul 2]; Available from:

Download references


The authors acknowledge the support, guidance, and encouragement of Drs. Ethan Basch, Steven Lipshultz, Kristen Hassmiller Lich, Pam Silberman, and Lara Lorenzetti.


No funding was provided to conduct this work. The authors received no funding to conduct this research and it was conducted as background research in support of Syril Pettit’s dissertation under the mentorship of Dr. Ethan Basch, UNC Gillings School of Global Public Health.

Availability of data and materials

All information used in this review is available per the citations included in the reference list.

Author information

Authors and Affiliations



SP led the primary study design, conducted the literature review, primary data analysis, and synthesis of information for this manuscript. RK contributed expertise, editorial input, and design input to help ensure that the study would be of relevance to patients and long-term survivors. She collaborated on concept conception for focus and scope of literature review and assisted with manuscript development and revisions. Both authors read and approved the final manuscript.

Corresponding author

Correspondence to Syril D. Pettit.

Ethics declarations

Ethics approval and consent to participate

No novel human or animal studies were conducted for this paper thus ethical statements in this regard are not applicable to this paper – all information included here is from previously published and publicly accessible sources. Consent to participate is not applicable.

Consent for publication

The authors offer consent for publication of this manuscript.

Competing interests

Syril Pettit and Rebecca Kirch declare they have no competing interests. In addition to her role as a student at the UNC Gillings School of Global Public Health, Syril Pettit is also an employee of the non-profit Health and Environmental Sciences Institute (HESI). This work was not conducted as a term of her employment at HESI and she received no funding support from HESI. No conflicts are declared.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional files

Additional file 1:

Table S1. Search terms for literature review. Table S2. Inclusion and exclusion criteria. Table S3. Cost assessment methodologies. Table S4. Defining costs. Table S5. Nature/frequency of treatment-related adverse events. Table S6. Incorporation of “quality of life”. Table S7. Defining the population. Table S8. Search strategy. Table S9. Inclusion/exclusion criteria. (DOCX 138 kb)

Additional file 2:

Figure S1 PRISMA diagram demonstrating the part I literature evaluation and exclusion process. (JPG 100 kb)

Additional file 3:

Figure S2 PRISMA diagram demonstrating the part II literature evaluation and exclusion process. (JPG 72 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pettit, S.D., Kirch, R. Do current approaches to assessing therapy related adverse events align with the needs of long-term cancer patients and survivors?. Cardio-Oncology 4, 5 (2018).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: