Advertisement

Why Are We Still Creating Individual Case Safety Reports?

      Abstract

      The history of drug safety monitoring, or pharmacovigilance, has been an interesting one. Despite many and ongoing changes, it has typically been characterized by a rather slow-moving and reactive progression. Pharmacovigilance has always lagged behind other fields and industries and has been slow to adapt to new approaches. The main aspect holding it back has been a focus on the administrative and adherence side of creating individual case safety reports (ICSRs) and distributing these reports to the various stakeholders per strict regulatory requirements. Now, in 2018, we are more behind the curve than ever, and the field seems to be at a breaking point, calling for urgent and drastic changes. The question at hand is whether in this era of an abundance of electronically available data and technological advancements, which allow the application of automation, this process still makes sense. Is there still a place for creating and redistributing ICSRs from marketed use in a current, state-of-the-art safety system? Artificial intelligence, deep machine learning, and related technologies are already in place in many other industries. Swift and rigorous change is necessary for the discipline of pharmacovigilance to keep up with what is happening in the world at large.

      Key words

      Introduction

      This commentary aims to stimulate critical thinking around the current standards that rule drug safety monitoring and reporting, especially the aspect of so-called individual case safety reports (ICSRs) for marketed products and the associated regulatory requirements of expedited reporting. Given the major developments in technological capabilities, especially during the past decade, there may be more effective and efficient manners to identify and report new safety signals rather than the traditional creation, packaging, and distribution of ICSRs.
      The aims of a drug safety monitoring or pharmacovigilance system is to identify adverse effects of therapeutics as soon as possible and minimize their occurrence and associated impact and to do so during the entire life cycle of the product. The World Health Organization (WHO) defines pharmacovigilance as the science and activities that relate to the detection, assessment, understanding, and prevention of adverse effects or any other drug-related problem.
      • Meyboom R.
      Pharmacovigilance in a changing world.
      It covers a comprehensive set of activities from clinical trial data review, literature evaluation, individual case reporting, and aggregate reporting to safety surveillance and signaling and risk management.
      The foundation of pharmacovigilance has been the reporting of suspected adverse reactions by prescribers, other health care professionals, and patients.
      • Dal Pan G.
      Ongoing challenges in pharmacovigilance.
      Pharmacovigilance is held to high regulatory standards of quality, accuracy, and timeliness. The current system puts legal obligations on pharmaceutical companies to create an ISCR from each adverse event associated with any of their marketed products (ie, any untoward occurrence in a patient taking any of the company's products) that they become aware of from any source and report qualifying cases (serious, unexpected) to regulators in all territories where the product is available, worldwide, within 15 calendar days. This obligation requires a process of gathering adverse events and actively searching follow-up information to get full understanding of the case, entry of such adverse events in a database, review of the available data, coding the events with standard medical terms, and classifying them into serious versus nonserious, expected versus not expected, and related versus not related. Adverse events can come from various sources: events that occur in clinical trials; events reported spontaneously by prescribers, other health care professionals, or patients; events reported in the scientific literature; and increasingly events identified from patient support programs or even discussions and comments on social media.
      Much of the efforts in pharmacovigilance are focused on identifying, collecting, evaluating, and transforming relevant data into usable and shareable safety reports. The entire process typically results in a significant administrative burden for all involved, resulting in many organizations searching for enhanced efficiencies (eg, by outsourcing to low-cost external parties for bulk processing and/or automation of parts of the work flow).
      With the numbers of individual cases to be processed continuing to steadily increase,

      Oracle Research White Paper: Addressing the data challenges of pharmacovigilance: https://go.oracle.com/LP=67881?elqCampaignId=144621. Accessed 17 August 2018.

      the current approach of creation of individual case reports may become unsustainable. More importantly, given that the increased volumes mostly pertain to less well-documented, nonserious, known, and common adverse events, the signal to noise ratio is decreasing, potentially defeating the purpose of the system.
      Pharmacovigilance relies on good information; advances in data availability and analytical capabilities have changed considerably.
      • Dal Pan G.
      Ongoing challenges in pharmacovigilance.
      Although the traditional postmarketing safety reporting was an excellent and effective approach at its inception, a time when data availability was scarce and sharing not so easy, one has to rethink the current approach in a time where data are abundant and relatively easily accessible. Given the significant general technological developments and digitization of health care, a critical reassessment is long overdue, and some breakthrough thinking may be required.

      Review of History—Usefulness of ICSR Reporting Over Time

      The function of pharmacovigilance has typically and consistently taken a rather conservative approach, slowly evolving but never really becoming revolutionized. The basic principles and requirements of individual case reporting for marketed products have not changed materially since their inception in the 1960s. The main aspect that is holding this field back has been a focus on the administrative and adherence side of creating ICSRs and submitting them per strict regulatory requirements. This continued focus on and concerns about identifying sufficient data to work them into ICSRs is taking much of the(not so abundant) resources in pharmacovigilance. In addition, considerable effort is put into packaging these ICSRs, sending them to other parties around the globe, such as regulators and partners, who then in turn agitate over them, sometimes reprocessing, reassessing, and even redistributing them.
      The current pharmacovigilance system and regulations are basically resulting from the thalidomide disaster,
      • Campbell J.E.
      • Gossell-Williams M.
      • Lee M.G.
      A review of Pharmacovigilance.
      • Grootheest K van
      The dawn of pharmacovigilance.
      during which the marked adverse effects of the drug resulted in significant deformities in children whose mothers had taken the drug during pregnancy. The connection with drug use and this epidemic of limb defects was made by an Australian obstetrician and a German pediatrician based on their observations of an increase in deformed infants being born in their practice to mothers who had taken thalidomide during pregnancy. However, this association was not uncovered before the drug had been on the market for 2 years and at least 10,000 children were affected in 46 countries (only 3000 of whom are still alive).
      • Campbell J.E.
      • Gossell-Williams M.
      • Lee M.G.
      A review of Pharmacovigilance.
      Of course, in reaction (and rightfully so), regulations and systems were created that would ensure that individual cases of adverse events happening anywhere in the world would be identified and relayed to central control units to allow such signals, connections, and causal associations between drug exposure and untoward effects to be uncovered earlier and more systematically to prevent similar drug-induced disasters from happening again. Systematic attention was given to structured and regulated gathering of adverse event data, governments started to organize national pharmacovigilance centers, and subsequently the WHO international drug monitoring program was established to collect, analyze and distribute adverse event data through collaboration with these national centers.
      • Meyboom R.
      Pharmacovigilance in a changing world.
      • Grootheest K van
      The dawn of pharmacovigilance.
      To enforce much better control and oversight to prevent such a situation from ever happening again, pharmacovigilance ended up being the highly regulated and workflow-dependent function it is to this day.
      Although it is well understood why pharmacovigilance is highly regulated, this certainly puts limitations on the extent to which innovative approaches can be applied. Experimentation, essential for breaking through the traditional way of doing things, is virtually unheard of for people working in pharmacovigilance. No company is looking for trouble or nonadherence in this area; therefore, most pharmaceutical organizations do not want to disrupt the status quo. Adherence with the regulatory requirements as they are is typically one of the corporate goals for all companies. This is a sad truth because most recognize that not all aspects of the regulations make sense, actually do not always serve patient safety, and are grossly disharmonious across the globe.
      Some of the challenges are that the regulatory bodies are not frontrunners and that the regulations governing pharmacovigilance typically are impossibly hard to change because they are embedded in each country's and territory's governmental system (eg, US Food and Drug Administration [FDA] 21 CFR §310.305, §314.80, §314.98, §600.80, European Union Directive 2001/83/EC, European Medicines Agency guidelines on good pharmacovigilance practices, Japan's Pharmaceutical and Medical Devices Act REF3x, various country specific similar regulations).
      As a result, pharmacovigilance is held back in its much needed progression. The pharmacovigilance regulations are based on a historic reality that no longer holds true; it is based on paucity of data, not the abundance of data we face in today's world. In the ancient (pre-1990) world, where the internet had yet to fully emerge and there was limited to no automation in health care systems, much of the health care information was held in individual physician offices on paper. In this world, patients were relatively passive and uninformed and hardly an active part of the treatment decision making. Identification of possible adverse effects was based on observations in individual patients; it was hard for patients or health care professionals to make associations between similar unexpected events happening after drug exposure in different patients in different geographic locations.
      In the meantime, however, the world and how data and information are gathered and dealt with in general has notable changed, whereas the regulations and how one completes ICSRs has not.
      • Price J.
      Pharmacovigilance in crisis: drug safety at a crossroads.

      The Disconnect

      The question at hand is whether, in this era of abundance of electronically available data and technological advancements, there is still a place for creating and distributing ICSRs in a current, state-of-the-art safety system. The volume of cases and safety databases continue to increase in size and, in some cases, scope, and with that the demand for data analysis to identify new signals increases proportionality, if not exponentially, whereas investments and resources have not increased in proportion to this explosion of available data.
      • Dal Pan G.
      Ongoing challenges in pharmacovigilance.
      Simultaneously, there has been, and is, an unprecedented development in and maturing of breakthrough technologies, including, among others, wearables and sensors, the internet, big data, and advanced analytics, robotics, and artificial intelligence. Pharmacovigilance organizations have not been among the first to adopt innovative data science tools and techniques, even as most leaders acknowledge that traditional analytical methods, such as ICSRs, no longer fit the purpose. However, demands and expectations to meet these requirements have increased not relaxed over the years.
      The reality is that in today's world much of the safety profile information is already available in an electronically accessible manner somewhere. Much, if not all, of health- and drug use–related information is stored in one or multiple databases: with health insurers, with health care professionals, or at health institutions. In addition, there are electronic sources that contain additional safety profile information, such as various registries, patient support groups, chat rooms, and social media. The information is just not formatted as an ICSR per se and stored as such in a company's or regulator's database. The current challenge is that these various sources are disparate and not always easily accessible for pharmacovigilance departments within pharmaceutical companies. However, given the evolving capabilities of automation and artificial intelligence, more active and automated use of these data sources for understanding of the safety profile does not seem an unattainable goal. In fact, it is a very compelling prospect.
      The world of pharmacovigilance is actually still behaving as if it is living in a world where the internet, social media, big data, electronic health records, smart search tools, automation, and artificial intelligence, with the capabilities to screen and analyze large data sets, do not exist. With data having become ubiquitous
      • Beninger P.
      • Ibara M.A.
      Pharmacovigilance and biomedical informatics: a model for future development.
      and the drive for data transparency ever increasing, why do we hold on to the requirement that each case has to physically exist in the company's database and in not only one but in every one of the regulatory databases? Granted, Europe made a good move with EudraVigilance, centralizing at least a cluster of countries of the many around the globe. Unfortunately, in a way, this progressive approach was in a sense undone with the implementation of the EudraVigilance Data Analysis System in November 2017, requiring companies to go back into this central database, filter out their company's cases, and enter them in the company's own database.
      Meanwhile all pharmacovigilance organizations are struggling to keep up with increasing volumes of cases to be processed, with typically limited budgets and head counts assigned by a company's management to this task. Every organization is putting various smart approaches in place to contain cost and resources for case processing. Outsourcing, especially to low-cost territories, is still the most common approach, but automation of the case processing workflow is of increasing interest. Efforts are under way to digitize the world of pharmacovigilance; partial automation of workflow is a well-recognized and feasible option to reduce workload and errors of case processing. Some more innovative pharmacovigilance departments are piloting various aspects of workflow automation and applying cognitive automation,
      • Chen Y.
      • Argentinis E.
      • Weber G.
      IBM Watson: how cognitive computing can Be applied to big data challenges in life sciences research.
      but many more are still waiting to see whether it will deliver on its promise, and especially smaller organizations ask themselves if the effort and investment are worth it. Various organizations are also putting efforts into higher-level automation, with artificial intelligence to extract relevant pieces of information from unstructured data and turn it into structured data and even machine learning to teach a computer to make case assessments and causal associations. With the full application of such digitalization, there is the potential for entirely touchless case processing.
      However, none of these information technology solutions address the underlying problem but instead only treat the symptoms. Processing faster and more effectively and producing a bigger volume of data are not necessarily the right answer to the issue. Many of the data sources do not necessarily require translation into an individual case report to be able to inform us. What is needed is faster and more effective access to (relevant) information not to just more data. Automating existing practices does not mean truly revolutionizing the discipline of pharmacovigilance. In addition, currently not all available data are optimally used to help inform use about the pharmacovigilance of a product. This expansion of available data sources disrupts the presumptions underlying the existing regulatory system for pharmacovigilance
      • Beninger P.
      • Ibara M.A.
      Pharmacovigilance and biomedical informatics: a model for future development.
      (eg, social media and large health care data sets, but also wearables, sensors, and apps).
      The underlying question that needs to be asked is, “Why are we still creating all these ICSRs?” It seems there has not been time and room for pharmacovigilance professionals and thought leaders to take such a step back and return to the drawing board to figure out how to best perform true pharmacovigilance in the current environment.

      The Future Pharmacovigilance System

      So, let's take such step back here and now and try answering the question, “Given the world we live in, what would an optimal pharmacovigilance system look like?” It is obvious that a current state-of-the-art pharmacovigilance system would need to optimally protect patients from unnecessary harm, be very proactive and predictive, take patient perspectives into account, and be able to perform rigorous and ongoing benefit-risk assessments that are meaningful for all stakeholders and can be translated in useful information to support treatment decisions for prescribers, patients, and other stakeholders. This means the pharmacovigilance system would need to monitor on an ongoing basis all available data on the use of the product in real life, with a system that constantly looks for new associations and patterns, is able to recognize normal disease pattern from unusual new effects associated with drug use, and deduct if such association is linked to specific circumstances in the patient (eg, age, dose, concomitant medication, comorbidities). An ideal system should be one single unified system not multiple disconnected ones. It would be a system that breaks down current barriers between information verticals with the various stakeholders. It would be centrally maintained, cloud based, and accessible to all. There is some idealistic talk about one single pharmacovigilance database shared across industry and with regulators. To arrive at such a place, there are many regulatory, legal, political, and emotional hurdles to be mastered, but theoretically and technically it would be feasible to get such a single pharmacovigilance source. Basic and robotic process automation, cognitive automation, block chain, artificial intelligence, and machine learning can manage most if not all of the current partially manually performed workflow process and could be used to feed cases (semi) automatically from all sources into a central single pharmacovigilance database.
      Although such a single global pharmacovigilance database would certainly reflect enormous progress, it still would still be based on the traditional idea of packaging and timely distribution of ISCRs rather than better insights into safety profiles. It would still be data-based case reporting instead of an information-based narrative. With the revolutionary digital developments, some breakthrough thinking should be done in the field of pharmacovigilance instead of and before just applying new tools on old concepts.
      For optimal insights, it may even be unwanted to reduce the data to a standard case and in doing so potentially lose some of the details available in the original data source. In addition, there may be no need to do so either. If there is a way to let go of the traditional thinking of cases, there may be other and better ways to serve the above described goal for pharmacovigilance. Instead of normalizing data by extracting cases from various sources, the ideal future safety system would be read directly from the original source by an artificial intelligence engine with large-scale data analytics capabilities. Now practically, how would such a system work? A pull versus push system may be the way to go: Instead of pushing the adverse event information out of the individual pharmacovigilance databases as cases, smart analytics would pull information by reading the data in their original habitat and combine the patterns and associations, presenting the live, real-time results in a dashboard. In a way, parts of such an approach can be compared with the changed model for the FDA's safety surveillance with the Sentinel initiative, applying a distributed data infrastructure approach, giving FDA rapid and secure access to electronic health care data for almost 200 million patients from multiple data partners. Instead of creating data sets from each of these sources and merging them for analyses, the (same) analytics are applied to the various data sources, and the results are combined.
      A leap would need to be made to deploy pattern recognition techniques, artificial intelligence, and machine learning on sources currently not touched that much by pharmacovigilance; very smart machines would be needed to access, read, and deduct information from various data sources, such as electronic medical records, claims data sets, literature, registries, and maybe even social media. This safety system would continuously read, evaluate, and analyze real-world data sources and apply various methods to evaluate and identify patterns and trends and determine whether new insights can be derived from the available data. It would apply natural language processing and other technologies to correctly deduct the relevant information from unstructured data and narratives in social and other media. Such an artificial intelligence engine for data identification, validity checking, analysis, and normalization, let's call it the Safety Analytics Knowledge Center (SAKC), would be the single source of the truth about the safety profile of any compound.
      The analytics engine of such a pharmacovigilance system would most likely, at least in the foreseeable future, have to be supported by human intelligence. Again, to avoid the current duplicative and nonharmonized ways, ideally this human intelligence is centralized in a single, global, independent group, which could be called the Safety Analytics Smarts Team (SAST). Such a team would work with the data and analytics coming from the SAKC and be tasked with helping the machines not only to learn but also to provide validation and the ultimate medical judgment call on the interpretation and clinical effect of the findings. Longer term it is conceivable to have cognitive automation in SAKCs cover the full spectrum without an SAST. The validated information would have to be made readily available in a consistent and transparent manner, with visualization tools and language understandable for all stakeholders.
      Such a system would be able to have ongoing insights in all information available in various external databases. It would provide real-time information on the use of the product in question and similar products, the patient population, patterns of use, and natural history of disease. These insights are then to be used for the various decision making associated with drug development, prescription and use: on the scientific and business levels to determine viability of further development of new compounds, on the population level for regulatory decision making, and on the individual prescriber and patient levels to inform treatment decision making.
      This electronic resource (SAKC) would provide easily accessible dashboards and visuals and as such would replace all (currently suboptimal) risk communication approaches to provide patients and practitioners with useful and actionable information about the tolerability of medicines
      • Dal Pan G.
      Ongoing challenges in pharmacovigilance.
      ; it would take the place of patient leaflets, package inserts, and medication guides and would even be equipped with mechanisms to push out more urgent safety alerts to smartphones via the SAKC app. This central safety information hub would also allow easy comparison with alternative treatment options. One of the beauties of such a central tool with one version of the truth available to all is that it would empower patients and prescribers to have full insight in the risks and tradeoffs of therapeutic options, especially if a modeling tool could be added to the dashboard that allows for entry of various levels of individual risk tolerance. This way tradeoffs and benefit-risk decision making can be performed in a more conscious and transparent manner.
      An approach such as the one described above means that in such a future system there is no room or need for extensive pharmacovigilance departments within every pharmaceutical company, regulator, or service provider. Mentioned organizations probably still want to have a safety insights and intelligence group embedded in their structure to review information from the central tool to determine what it means for new compounds, new targets, and existing compounds under development and on the market.
      One can argue whether there remains utility for a spontaneous database somewhere. If there is a place for one, it would at most be the earlier mentioned single one, where anyone can directly report suspected adverse events. It would need to be a repository for truly spontaneously reported events by patients, health care professionals, and others. In contrast to current pharmacovigilance databases, it would not include solicited cases, cases more actively filtered out from, for example, patient support programs, because such sources have proven of limited value for identifying new pharmacovigilance data.

      Donzanti B: Evaluating adverse events from patient support and market research programs: proposed best practices and regulatory changes; http://info.exlevents.com/rs/195-NER-971/images/BruceDonzanti%20.pdf. Accessed 15 August 2018.

      In addition to the points earlier made, such data are already available in an electronic format and would therefore be picked up by the SAKC tool automatically. A future remaining pure spontaneous database would be one of the data sources for the hypothetical SAKC tool.
      This future pharmacovigilance system sounds like a bit of a utopia if balanced against how pharmacovigilance is being managed currently. However, if compared with the information technology solutions currently available in other fields, such a technologically different approach to pharmacovigilance would not be that far-fetched. It is hard to predict what really will be the breaking point that will trigger regulators and biopharmaceutical companies to let go of individual case processing and truly revolutionize; in all likelihood, it will not be imminent. However, as increasing case volumes become unmanageable for both regulators and companies and the signal to noise ratio becomes too low, a revisiting of the situation and an overhaul of regulations and accountabilities will ultimately result. An unwanted but not unrealistic scenario that could expedite such a transformation is if a new safety issue becomes apparent that is not identified through the traditional system in a timely manner, especially if it turns out the evidence was sitting elsewhere, beyond ICSRs.
      All this leaves us with the (very) big question of how to get there: how to get from the current state to the required future state. Given the regulatory constraints, it is not realistic to expect any single body (be it a regulator or a pharmaceutical company) to be able to make such a transformation by itself. It requires a dedicated, passionate collaboration, supported by at least some of the larger regulatory bodies and a comprehensive representation from pharmaceutical companies. It calls for a strong partnership among the pharmaceutical industry, technology providers, and regulators. A natural reaction is to wonder whether existing bodies such as the International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use or TransCelerate will be able to drive such change. Indeed, one of the TransCelerate intiatives is about intelligent automation opportunities for pharmacovigilance. Such a paradigm shift is not likely to happen through a gentle and slow consensus building because what is required to get to a contemporary system that ensures adequate (and manageable) pharmacovigilance will put all stakeholders outside their comfort zones, for various reasons. More likely, a team driving such revolution in the field of pharmacovigilance has to be a different gathering than the typical one, one that has global support or even a global mandate and can let go of the current thinking about pharmacovigilance. It may very well not be possible to drive such change from within because such breakthrough thinking may not come from the current pharmacovigilance experts because they are more or less traditionalists. It may be very well necessary to bring in people from different industries who are much farther along in adopting information technology solutions, for example, industries such as banking and financial services, retail, manufacturing and supply chain, transportation, which can bring in concepts of process and cognitive automation, natural language processing, machine learning, and artificial intelligence.
      Such a future safety system may not be exactly where the discipline of pharmacovigilance is going; there are obviously many other scenarios, but some out-of-the-box thinking is urgently needed to ensure that pharmacovigilance will be able to provide meaningful patient safety information because continuing traditional practices may not serve that purpose.
      There is a need to return to the drawing board and put renewed efforts toward finding the essence of pharmacovigilance, which is not about creating ICSRs but about ensuring that patients are optimally protected from adverse effects, with optimal treatment choices being made based on complete and transparent benefit-risk information.

      Conclusion

      At a time of increasing case volumes and an abundance of electronically available data and technological advancements, there may no longer be a place for manually creating and redistributing ICSRs. However, new technologies must be applied wisely to the field of pharmacovigilance, and systems should not simply automate what has been previously collected without taking some time and effort to thoroughly think through what is truly needed in today's world.
      To get a pharmacovigilance system to focus on the patient rather than on regulations and procedures, there is a need to move away from ICSR creation, reporting, and adherence toward optimal analyses and communication of real-time insights into the risks associated with the use of medicinal products. There is a clear need for a new business model. Swift and rigorous change is necessary to create a new ecosystem for the discipline of pharmacovigilance to keep pace with developments in information technology and ensure a new, fit-for-purpose pharmacovigilance system for the world we live in.
      These are exiting times because cognitive automation holds much promise for the future of pharmacovigilance.
      • Dal Pan G.
      Ongoing challenges in pharmacovigilance.
      It seems that the discipline of pharmacovigilance is at an inflection point, with new approaches and technologies offering great potential. With the convergence of expanding data sources, increasing case volumes, and digital revolution, pharmacovigilance is poised for disruption.

      References

      1. https://www.who-umc.org/global-pharmacovigilance/global-pharmacovigilance/glossary/. Accessed 8 August 2018.

        • Meyboom R.
        Pharmacovigilance in a changing world.
        Klinicka formakologie a farmacie. 2011; 25: 102-111
        • Dal Pan G.
        Ongoing challenges in pharmacovigilance.
        Drug Saf. 2014; 37: 1-8
      2. Oracle Research White Paper: Addressing the data challenges of pharmacovigilance: https://go.oracle.com/LP=67881?elqCampaignId=144621. Accessed 17 August 2018.

        • Campbell J.E.
        • Gossell-Williams M.
        • Lee M.G.
        A review of Pharmacovigilance.
        W Indian Med J. 2014; 63: 771-774
        • Grootheest K van
        The dawn of pharmacovigilance.
        Int J Pharmaceut Med. 2003; 17: 195-200
      3. https://en.wikipedia.org/wiki/Thalidomide. Accessed 8 August 2018.

        • Price J.
        Pharmacovigilance in crisis: drug safety at a crossroads.
        Clin Ther. 2018; 40: 790-797
        • Beninger P.
        • Ibara M.A.
        Pharmacovigilance and biomedical informatics: a model for future development.
        Clin Ther. 2016; 38: 2514-2525
      4. https://eudravigilance.ema.europa.eu/human/index.asp. Accessed 15 August 2018.

      5. http://www.ema.europa.eu/docs/en_GB/document_library/Other/2017/07/WC500230934.pdf. Accessed 15 August 2018.

        • Chen Y.
        • Argentinis E.
        • Weber G.
        IBM Watson: how cognitive computing can Be applied to big data challenges in life sciences research.
        Clin Ther. 2016; 38: 688-701
      6. https://www.fiercebiotech.com/medical-devices/ibm-watson-celgene-partner-cloud-based-patient-safety-monitoring-tool. Accessed 8 August 2018.

      7. https://www.fda.gov/safety/fdassentinelinitiative/ucm149340.htm. Accessed 8 August 2018.

      8. Donzanti B: Evaluating adverse events from patient support and market research programs: proposed best practices and regulatory changes; http://info.exlevents.com/rs/195-NER-971/images/BruceDonzanti%20.pdf. Accessed 15 August 2018.

      9. http://www.ich.org/home.html. Accessed 16 August 2018.

      10. http://www.transceleratebiopharmainc.com/. Accessed 16 August 2018.

      11. http://www.transceleratebiopharmainc.com/initiatives/intelligent-automation-opportunities-pharmacovigilance/. Accessed 16 August 2018.