Implementing Comparative Effectiveness Research in Primary Care

  1. 1.  Department of Academic Family Medicine, University of Saskatchewan

Abstract

Introduction: Despite the increasing uptake of electronic medical record (EMR) software in Primary Care, there has been little effort to date to utilize this software to conduct pragmatic comparative effectiveness research (CER) trials in Primary Care.

Objectives: The primary objective of the study was to design an implementation framework composed of key self-reflective questions and a prototype patient recruitment interface to aid in CER studies in Primary Care using current-generation EMR products.

Research Questions: What is the current state of EMR usage for CER in Primary Care? What are the barriers (technological, methodological, ethical and practical) to implementing CER in Primary Care?

Methods: We incorporated selected key stakeholders in discussions to improve on an initial CER framework prototype and “sham” EMR module for patient recruitment. We iterated on both after discussions with each participant. Participants included researchers with an interest in Primary Care research, technical representatives of EMR vendors, and Family Physicians.

Results: There was little familiarity and no apparent impetus from the vendor to collaborate in this type of research. There is a common theme of frustration from researchers directed at the difficulty in access EMR databases from a large field of vendors. From the clinician side, physicians are generally reluctant to participate in CER research without effective compensation for time spent. Patient recruitment interfaces should be designed to be as simple and straightforward as possible.

Conclusion: There are currently multiple barriers to conducting EMR-enabled research in Primary Care. The largest and most important barrier is the lack of effective IT infrastructure to support this type of research. Although this type of research is overall more cost-effective, there are significant upfront costs in creating the initial study infrastructure that private vendors are unlikely to bear themselves. Ideally, government would step forward and implement the backend infrastructure with which EMR vendors can interface to help enable this type of research. In the future, researchers will need to clearly outline the business case for vendors to participate in Primary Care research.

Introduction

Electronic Medical Record (EMR) use in Primary Care in Canada has exploded in popularity over the last decade, from 2.09% in 2004 to 79.7% in 2014 [College of Family Physicians of Canada, Canadian Medical Association, & Royal College of Physicians and Surgeons of Canada (2014)]. While this has resulted in improved clinical efficiencies and patient care [Canada Health Infoway (2013)], unfortunately these improvements have not been paralleled in the realm of Primary Care research. One particular type of Primary Care research that extensive EMR use can enable is comparative effectiveness research(CER): a subset of patient-oriented pragmatic research that examines available therapeutic options among typical patient populations to determine relevant health outcomes [Concato, Peduzzi, Huang, O’Leary, & Kupersmith (2010)]. Current generation EMRs can enable this on a broad scale, as well as allowing researchers to focus on patient-centred outcomes such as quality-adjusted life-years (QALYs) and mortality. For example, this type of research could be used to economically compare the efficacy of pharmaceutical classes with many similar options (known as “me too” drugs). The six statin medications currently on the market in Canada are an exemplar for this study as comparative efficacy data, especially on patient-centred outcomes is either non-existent or not publicly available.

The purpose of this study was to develop a self-reflective framework to guide clinicians, researchers, and technical implementers in developing prospective CER research studies in Primary Care, and overcoming barriers to implementation and use. To help us develop this framework, we collaborated with a selected group of family physicians, researchers, and EMR implementers. This framework is intended to be used to develop large-scale, long-term, pragmatic comparative effectiveness studies in Family Practice in Canada. To create a focus for the discussions with participants involving realistic patient scenarios, we used the scenario comparing the current statin medications on the market in Canada(atorvastatin (Lipitor), fluvastatin (Lescol), lovastatin (Mevacor),pravastatin (Pravachol), rosuvastatin (Crestor), and simvastatin(Zocor)) as an example upon which to base discussions. It is important to note that, although statins were used for the purposes of practical discussion, this framework is intended to be much more broadly applicable.

Background

Explanatory studies attempt to measure “the benefit a treatment produces under ideal conditions, often using carefully defined subjects in a research clinic” [Roland & Torgerson (1998)]. In contrast to explanatory studies, patient-centred, pragmatic studies are important in the world of Primary Care in that they attempt to measure effectiveness: the benefit of a particular treatment in routine clinical practice. The design of a pragmatic study reflects variations in real world use of a treatment between patients and aims to maximize the external validity(or generalizability) of an intervention [Godwin et al. (2003)].

Pragmatic, patient-centred studies are traditionally expensive to run. These studies must necessarily include a representative sample of real world patients: this generally means recruiting patients and physicians in a variety of locations and practice settings and circumstances, adding to the running costs of any potential study. Paper chart reviews for the large sample size of patients required are expensive but often necessary to gather sufficient data, and each patient’s contact information needs to be maintained and updated in order for outcome surveys to be performed.

With ever increasing utilization of EMRs in Primary Care in Canada, the prospect of running cost-effective pragmatic comparative effectiveness research (CER) with patient-centred endpoints arises. Vickers & Scardino (2009) described what they called a “clinically-integrated randomized trial” that leveraged EMR’s strengths to conduct these sorts of studies at lower cost. While this has been described in the literature, and there have been a few EMR-enabled studies conducted to date [Ford et al. (2007; Bener, Dogan, Barakat, & Al-Hamaq, 2014; Libby et al., 2010; Masica et al., 2013; J. Warren et al., 2012)], no pragmatic comparative study to date has measured a patient-centred outcome as a primary endpoint. Bener et al. (2014) conducted a CER study using EMR and pharmacy records in Primary Care to look at the effectiveness of four of the most commonly prescribed statins in a diabetic patient population. Although promising for their use of the combination of EMR and pharmacy records, this study disappointed, only looked at lipid reduction and not morbidity/mortality data or follow-up longer than a two year period.

An example of where Primary Care, EMR-enabled research might take us, is the Canadian Primary Care Sentinel Surveillance Network(CPCSSN) [Birtwhistle et al. (2009)]. The CPCSSN provides research access to EMR data for patients of 752 “sentinels”, or participating physicians, across Canada, but it records observational data only (i.e. no interventions). It also de-identifies data before research access, so making links to outside data sources (such as pharmacy or Bureau of Statistics records) is not possible.

Also in the U.S., Fiks et al. (2012) have been working on observational CER in pediatrics, with a network of 160+ attending physicians and 200,000+ pediatric patients. Pediatrics as a specialty would, in particular, benefit from pragmatic CER studies, as much of the evidence for pediatric therapies has been extrapolated from adult patients.

A description of EMR-enabled, pragmatic CER studies and the potential benefits to patients and physicians, is described in Vickers & Scardino (2009):

“[...] take the case of a patient with newly diagnosed depression. The patient’s doctor recommends antidepressant medication, and explains why this might help. The doctor then states that although all prescription antidepressants are known to work, exactly which is most effective is unknown, and proposes that the patient takes part in a randomized trial in which the drug is chosen at random. If the patient agrees, he or she would complete a depression questionnaire. The doctor, or office assistant, would then access a secure trial website and enter a limited amount of basic patient data, which would include any co-morbidities, age and contact details, as well as the results of the depression questionnaire. The trial database would then randomize the patient and present a ‘results screen’ showing the allocated drug. This drug is then prescribed in the normal way, with the patient collecting the drug at the local pharmacy. The patient is then sent an email at 6 and 12 weeks with links to the depression questionnaire and a brief questionnaire about side effects. The trial might also contact the patient at long-term follow-up, say 12–24 months, and include questions about depression, drug compliance and use of other therapies. The results of the questionnaire would be sent to the patient’s doctor so that he or she could adjust the patient’s care appropriately, for example, by changing treatment if response was poor (indeed, one advantage for doctors to participate in the trial might be that such a system might be made available to all their patients, whether or not they took part). Outcomes such as hospitalization or death – important in a comparison of say, different statin drugs – could be obtained by computerized linkage to national databases.”

All of this could be done at a cost substantially less than traditional studies. Once the study infrastructure (such as linkages to national databases, vendor development of EMR modules, and clinician recruitment) is in place, the marginal cost for each additional patient in a study or multiple studies would be negligible. This would enable researchers to work with very large sample sizes, helping to eliminate patient preference as a source of bias [Vickers & Scardino (2009)].

Background on Statin use in Primary Care

The first statin, Lovastatin, came on the market in 1987, based on clinical studies that showed a 25–35% reduction in LDL levels and a reduction in frequency of MI of 25–30% [Endo (2010)]. This first statin was approved without any studies having been completed that looked at patient-centred evidence (such as having a measurable effect on quality-adjusted life-years (QALYs) or mortality). It was only when subsequently developed “me too” statins were on the market that post-marketing studies such as the Heart Protection Study and the Scandinavian Simvastatin Survival Study (4S) looked at patient-centred outcomes (namely, reduced all-cause mortality) [Pedersen et al. (1994; Heart Protection Study Collaborative Group, 2002)].

Unfortunately, the comparative or head-to-head evidence for these medications isn’t nearly as robust. Pharmaceutical companies rarely have the incentive to do proper long-term studies looking at patient-centred outcomes, but rather look for marketable “bullet points” of data [Resnik (2007)]. A prime example of this type of study is the ECLIPSE study, comparing Crestor (rosuvastatin) to Lipitor (atorvastatin). This study was run with just over 1000 patients for a period of 24 weeks, with the primary endpoint being the number of patients who achieved an LDL below target at the 24 week mark [O. Faergeman et al. (2008)]. Clearly the endpoint in this study falls short of showing any patient-centred results, but does give Crestor a bullet point for marketing purposes: “In a clinical trial versus Lipitor, CRESTOR got more high-risk patients’ bad (LDL) cholesterol to a goal of under 100mg/dL.” [AstraZeneca (n.d.)]

We would be remiss if we didn’t point out that the statins as a pharmaceutical class are not unique in the market forces that bring “me too” medications to market. Another low-hanging fruit would be the six proton pump inhibitors (PPIs) on the market in Canada, with no head-to-head studies available to date, and and comparative effectiveness data that relies solely on a meta-analysis of highly heterogeneous study data [Gerson (2007)].

Methods

Research Questions

What is the current state of EMR usage for CER in Primary Care? What are the barriers (technological, methodological, ethical and practical) to implementing CER in Primary Care?

Data Gathering

In order to provide a base of evidence upon which to hold discussions, an English-language literature review was conducted using keywords related to the overlap between CER and Primary Care: Google Scholar: “[barriers to entry] ‘comparative effective research’ in Primary Care”; PubMed: “(comparative effectiveness research OR comparative effectiveness) AND (Primary Care OR general practice)” and “(comparative effectiveness research OR comparative effectiveness) AND (Primary Care OR general practice) AND (emr OR ehr)”: resulted in 20 relevant results.

Action Planning

Based on the literature review, we came up with a preliminary technical implementation in a “faux” EMR that we could use to guide the discussion with participants. We also developed a series of questions and talking points relating to the design of a hypothetical CER research study. We conducted sequential discussions and iterated on the on both the question framework and technical implementation after each interview.

From this, we improved our technical implementation and came up with a framework of steps and self-reflection questions that researchers could use in developing a CER study.

Results

Data Analysis and Results

We interviewed 2 researchers with an interest in CER studies, 3 practicing Primary Care Clinicians, and 1 technical implementer / EMR vendor. Discussions were held individually, and from this a framework and sham user interface were developed. We questioned participants on their current knowledge of research (and specifically CER) in Primary Care, as well as gathered a list of perceived barriers to implementation and potential solutions where appropriate. We used this data to iterate on our framework and technical implementation. Although organized into three distinct categories, the questions as framed are intended to be answered collaboratively by all involved parties.

Final Proposed Framework of Self-Reflective Questions

Clinicians:

          How are clinicians recruited to the study?

         Does this study include physicians from a research network, or individual clinicians / practices?

         Consider that each additional EMR that is used by clinicians can add significantly to the cost of the study.

          From the clinician and researcher perspective, there was universal call for a national backend EMR infrastructure to aid in conducting CER studies in Primary care.

          A common theme with clinicians is that “time is money”. What are the proposed incentives for clinicians to participate?

         Examples include:

          feedback of interim study results to physicians to help inform clinical decision making

          comparative patient performance and outcome data for clinicians to compare to their colleagues

          monetary incentives, i.e. reimbursement for time spent recruiting patients

          Validity of EMR data:

         Is the data that the researchers wish to study in the EMR likely to be accurate for patient population being studied? (i.e. are the researchers capturing discrete, generally accurate data such as blood pressures, or data with poor inter-rater reliability, such a record of acute abdominal pain [Mills et al. (2009)])

          “Me too” medications, although usually from the same class, often have different suggested initial dose ranges. How will clinicians know what the initial dosing range for each medication is?

         Clinicians recruiting for the study need to either know what those initial doses are, be willing to look them up, or need to have them listed after randomization in the EMR module.

Technical Implementer / EMR Vendor:

          Vendors require an incentive or impetus to engage or enable research utilizing their proprietary database infrastructure:

         Vendors want to know what benefits are available for participation. What can the researchers / research group offer them: perhaps intangible benefits such as acknowledgement, or a publicized vendor / research group partnership.

          How does the EMR backend infrastructure or database architecture need to be changed to accommodate CER studies?

         Who will fund vendors for required database changes? Once accommodated, further changes should be minimal: should the cost for the vendor work be amortized over more than one future study?

         For each particular EMR product, what discrete data fields are currently captured, and how does this compare to what the researchers would like to look at?

          For studies that don’t require access to EMR data (such as studies strictly looking at Provincial / National data stores, such as Bureau of Statistics mortality data), can a standalone module be built to support the randomization of patients, thus negating the need to engage EMR vendors?

          For data mining purposes, structured data fields may be required. Are these already present in the EMR? What fields and/or data would be required by the researchers? (i.e. blood pressure measurements)

          Due to the intended interventional nature of the targeted CER studies, patient consent needs to be recorded:

         Is the EMR implementation capable of capturing a patient’s digital signature? If not, a mechanism for uploading and storing the patient’s paper consent form is required.

          Where will the study data (including study start data, such as demographics) be stored?

         If the EMR vendor is responsible for storage, do they charge for this?

          If the researchers have access to their own data store, how does this data store get populated? Are the researchers expected to pull the data, or does the vendor push the data to the researchers? On what schedule does this occur? (i.e. weekly, continuously)

          What funding and/or infrastructure is available from government to help share study setup costs?

Researchers:

          A recurrent theme here was frustration at the difficulty of accessing EMR databases, and the general unwillingness of EMR vendors to participate in research activities.

          For medication studies, is there a cost difference to patients between medications such that a significant patient preference (or clinician preference) bias is liable to occur?

         Are the cost differences, if they exist, significant enough to raise ethical concerns around patient access and equity?

          EMR vendors commonly charge per-patient for access to their database. Is there an ethical duty to pass along these fees to the patients whose data is being sold?

          When the patient is being consented, it’s important for this type of study to state that there may be not direct benefit to the patient.

          What will the patient inclusion/exclusion criteria be for this study, and is this criteria easy to assess for each patient?

         Keep in mind that pragmatic CER studies are most effective when the inclusion criteria are kept as broad as possible to include the broadest range of real-life clinical scenarios.

          Research data infrastructure may need to be hosted in Canada to ensure that health information privacy is maintained. How will the researchers ensure that data does not leave national borders, and if it does that it does so in an encrypted format?

          How will researchers gain access to 3rd party data stores potentially required for outcome data?

         If the researchers are to access Bureau of Statistics or hospital records for data such as mortality or hospital admissions, consent needs to be obtained at the time of study enrollment.

         Pharmacy records are useful for monitoring patient compliance and also for identifying a non-compliant patient group to use as a control group. Are prescribing records electronically recorded in a central database? How would you get access?

          If patient follow-up is desired in the form of an exit survey, do researchers have ongoing access to up-to-date patient contact information for the patient?

          For medication studies, a potential control group would be available in those patients who are randomized to a medication but then are non-compliant:

         Does the EMR record a reason for medication non-compliance? Is that reason typically recorded in an accurate and timely fashion for the clinicians and patient populations participating in the study?

Technical Implementation of Patient Recruitment Interface

The technical implementation of a patient recruitment interface was built into a “sham” EMR system, and used during discussions with study participants. Screenshots from this interface are included in Figures 1 through 5 (all patient and physician names included are fictitious). A common theme from all participants was to keep the interface as simple and straightforward as possible.

 

Figure 1: The medication module with autocomplete functionality for filling in drug name

Figure 2: Once a medication is selected in the autocomplete list, a textbox pops up immediately below letting the clinician know that this medication (and this patient) may be eligible to enroll in the research study.

Figure 3: If the clinician and patient decide to consider enrollment, clicking on the textbox brings them to an enrollment screen with some more information and a link to the patient consent form (downloadable PDF).

 

Figure 4: Once the patient has signed the consent form, the physician proceeds and the patient is randomized by the system, and the randomization result recorded in the patient's electronic chart.

Figure 5: The patient's current study enrollments are recorded and displayed in the sidebar for future reference.

Discussion

Previous studies have shown that leveraging EMRs in Primary Care for the conduction of comparative effectiveness studies is a cost-effective way of producing high quality, patient-centered evidence independent of industry and the associated risk of bias. Although industry studies tend to meet higher standards of data quality than independent studies, they are also less likely to be published, more likely to produce favourable results, and more likely to have conclusions that do not match the data [Dunn and Coiera (2014)]. This is clearly worthwhile research that should be occurring on a wider scale, and in speaking with our study participants we identified a number of current barriers to the performance of CER studies in Primary Care.

The largest and most important barrier is the lack of effective IT infrastructure to support this type of research. This point was a common thread throughout our discussions with our researcher and clinician participants, in particular there is frustration at a lack of a centralized backend infrastructure for electronic medical records, either provincially or nationally. Possibly due to healthcare being a provincial responsibility in Canada, EMR vendors are highly fragmented, with little support for a nationwide technical infrastructure upon which EMR vendors could build a product and differentiate their services (what Cresswell et al. (2012) calls a “middle-out” approach). A representative quote from one of our participants goes thusly: ‘”if we were more coordinated in doing this, we'd be able to build something where we have the potential to do these types of studies, but right now we're heading in the wrong direction.” The U.K. is taking a more “top-down” approach, developing a national EMR infrastructure including patient- and clinician-facing interfaces. Furthermore, a “bottom-up” approach, as Canada has so far undertaken, can also be effective as long as semantic integrity is mandated and maintained between systems [Courtney (2013)]. So far in Canada, there has been no impetus from private industry or government to maintain any sort of interoperability or semantic integrity between the many EMR systems available. Although this type of research is overall more cost-effective, there are significant upfront costs (to be born by vendors unless alternative funding arrangements are organized) in creating the initial study infrastructure [Dunn and Coiera (2014)]. Ideally we would like to see government act to build a standardized, central backend infrastructure upon which EMR vendors could interface. Alternatively, we would like to government update provincial EMR specifications to include requirements for interoperability and guarantees of semantic integrity when transferring between EMR vendors. Either of these course of action would decrease friction for researchers in gathering research data from EMR databases, with the surrogate benefit of reducing vendor lock-in for clinicians and hospitals.

A second key barrier to the performance of CER studies in Primary Care is an apparent lack of familiarity with and little impetus for EMR vendors to participate in research activities. Our discussions supported the idea that currently, EMR vendors are focused solely on enabling day-to-day clinical work rather than research activities—the business case for participating in research does not currently exist. There are significant expenses required in setting up the infrastructure to enable Primary Care research, and this will likely require government support. We would like to see government become more involved on this front, and in particular we would like to see provincial EMR specifications include requirements to provide a standardized way for researchers to apply for and access, data relevant to studies that researchers conduct in Primary Care.

So far in North America, only large, diverse but integrated systems have had the ability to overcome the barriers to CER studies in Primary Care. The Veterans Administration (VA) is an exemplar of a system that is enabled and supportive of Primary Care CER research, and Fiore et al. (2011) has shown what can be accomplished in such a setting, and we hope to see government policy changes in Canada that can help further CER in Primary Care.

Acknowledgements

We would like to thank Dr. Vivian Ramsden, Dr. Aaron Prystupa and Shari McKay for advice and support throughout this project. We would like to thank Dr. Marshall Godwin and Dr. Richard Birtwhistle for pre-publication review of our manuscript.

Sources of Support

No financial support was provided for this research. This research was presented in a preliminary format at the University of Saskatchewan Family Medicine Resident Research Day, 2015, for which travel was reimbursed to the researchers by the Department of Academic Family Medicine. Dr. Bouchard has no conflict of interest to disclose. Dr. Shah has no conflict of interest to disclose.

References

1.     AstraZeneca. (n.d.). Clinical Trial Results. Retrieved from http://www.crestor.com/about-crestor/crestor-clinical-studies.html

2.     Bener, a, Dogan, M., Barakat, L., & Al-Hamaq, a O. (2014). Comparison of efficacy, safety, and cost-effectiveness of various statins in dyslipidemic diabetic patients. Indian J.Pharmacol., 46(1998-3751 (Electronic)), 88–93. DOI: 10.4103/0253-7613.125184

3.     Birtwhistle, R., Keshavjee, K., Lambert-Lanning, A., Godwin, M., Greiver, M., Manca, D., & Lagacé, C. (2009). Building a pan-Canadian primary care sentinel surveillance network: initial development and moving forward. Journal of the American Board of Family Medicine : JABFM, 22(4), 412–422. DOI: 10.3122/jabfm.2009.04.090081

4.     Canada Health Infoway. (2013). The Emerging Benefits of Electronic Medical Record Use in Community-Based Care. Retrieved from https://www.infoway-inforoute.ca/en/component/edocman/resources/reports/benefits-evalu\nation/1224-the-emerging-benefits-of-electronic-medical-record-use-in-community-based-care-full-report

5.     College of Family Physicians of Canada, Canadian Medical Association, & Royal College of Physicians and Surgeons of Canada. (2014). National Physician Survey. Retrieved from http://nationalphysiciansurvey.ca/

6.     Concato, J., Peduzzi, P., Huang, G. D., O’Leary, T. J., & Kupersmith, J. (2010). Comparative effectiveness research: what kind of studies do we need? Journal of Investigative Medicine : The Official Publication of the American Federation for Clinical Research, 58(6), 764–769. DOI: 10.231/JIM.0b013e3181e3d2af

7.     Courtney, K. L. (2013). Enabling Health and Healthcare Through ICT: Available, Tailored, and Closer (p. 399). IOS Press. Retrieved from https://books.google.com/books?hl=en\&lr=\&id=uuZNnyYWK2EC\&pgis=1

8.     Cresswell, K. M., Worth, A., & Sheikh, A. (2012). Comparative case study investigating sociotechnical processes of change in the context of a national electronic health record implementation. Health Informatics Journal, 18(4), 251–70. DOI: 10.1177/1460458212445399

9.     Dunn, A. G., & Coiera, E. (2014). Should comparative effectiveness research ignore industry-funded data? Journal of Comparative Effectiveness Research, 3(4), 317–320. DOI: 10.2217/cer.14.31

10.  Endo, A. (2010). A historical perspective on the discovery of statins. Proceedings of the Japan Academy. Series B, Physical and Biological Sciences, 86(5), 484–93. DOI: 10.2183/pjab.86.484

11.  Faergeman, O., Hill, L., Windler, E., Wiklund, O., Asmar, R., Duffield, E., & Sosef, F. (2008). Efficacy and tolerability of rosuvastatin and atorvastatin when force-titrated in patients with primary hypercholesterolemia: results from the ECLIPSE study. Cardiology, 111(4), 219–28. DOI: 10.1159/000127442

12.  Fiks, A. G., Grundmeier, R. W., Margolis, B., Bell, L. M., Steffes, J., Massey, J., & Wasserman, R. C. (2012). Comparative effectiveness research using the electronic medical record: an emerging area of investigation in pediatric primary care. The Journal of Pediatrics, 160(5), 719–24. DOI: 10.1016/j.jpeds.2012.01.039

13.  Fiore, L. D., Brophy, M., Ferguson, R. E., D’Avolio, L., Hermos, J. A., Lew, R. A., … Lavori, P. W. (2011). A point-of-care clinical trial comparing insulin administered using a sliding scale versus a weight-based regimen. Clinical Trials (London, England), 8(2), 183–95. DOI: 10.1177/1740774511398368

14.  Ford, I., Murray, H., Packard, C. J., Shepherd, J., Macfarlane, P. W., & Cobbe, S. M. (2007). Long-term follow-up of the West of Scotland Coronary Prevention Study. The New England Journal of Medicine, 357(15), 1477–86. DOI: 10.1056/NEJMoa065994

15.  Gerson, L. B. (2007). Are all proton pump inhibitors created equal? Gastroenterology, 132(4), 1622–4. DOI: 10.1053/j.gastro.2007.03.011

16.  Godwin, M., Ruhland, L., Casson, I., MacDonald, S., Delva, D., Birtwhistle, R., … Seguin, R. (2003). Pragmatic controlled clinical trials in primary care: the struggle between external and internal validity. BMC Medical Research Methodology, 3(1), 28. DOI: 10.1186/1471-2288-3-28

17.  Heart Protection Study Collaborative Group. (2002). MRC/BHF Heart Protection Study of Cholesterol Lowering with Simvastatin in 20 536 High-Risk Individual: A Randomised Placebo-Controlled Trial. Lancet, 360(9326), 7–22. DOI: 10.1016/S0140-6736(02)09327-3

18.  Libby, A. M., Pace, W., Bryan, C., Anderson, H. O., Ellis, S. L., Allen, R. R., … Valuck, R. J. (2010). Comparative effectiveness research in DARTNet primary care practices: point of care data collection on hypoglycemia and over-the-counter and herbal use among patients diagnosed with diabetes. Medical Care, 48(6 Suppl), S39–44. DOI: 10.1097/MLR.0b013e3181ddc7b0

19.  Masica, A. L., Ewen, E., Daoud, Y. A., Cheng, D., Franceschini, N., Kudyakov, R. E., … West, S. L. (2013). Comparative effectiveness research using electronic health records: impacts of oral antidiabetic drugs on the development of chronic kidney disease. Pharmacoepidemiology and Drug Safety, 22(4), 413–22. DOI: 10.1002/pds.3413

20.  Mills, A. M., Dean, A. J., Shofer, F. S., Hollander, J. E., McCusker, C. M., Keutmann, M. K., & Chen, E. H. (2009). Inter-rater reliability of historical data collected by non-medical research assistants and physicians in patients with acute abdominal pain. The Western Journal of Emergency Medicine, 10(1), 30–6. Retrieved from http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2672296

21.  Pedersen, T., Kjekshus, J., Berg, K., Haghfelt, T., Faergeman, O., Faergeman, G., … Scandinavian Simvastatin Survival Study Group. (1994). Randomised trial of cholesterol lowering in 4444 patients with coronary heart disease: the Scandinavian Simvastatin Survival Study (4S). Lancet, 344(8934), 1383–1389. DOI: 10.1016/j.atherosclerosissup.2004.08.027

22.  Resnik, D. B. (2007). Beyond post-marketing research and MedWatch: Long-term studies of drug risks. Drug Design, Development and Therapy, 1, 1–5. Retrieved from http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2763348

23.  Roland, M., & Torgerson, D. J. (1998). Understanding controlled trials: What are pragmatic trials? BMJ, 316(7127), 285–285. DOI: 10.1136/bmj.316.7127.285

24.  Vickers, A. J., & Scardino, P. T. (2009). The clinically-integrated randomized trial: proposed novel method for conducting large trials at low cost. Trials, 10(1), 14. DOI: 10.1186/1745-6215-10-14

25.  Warren, J., Kennelly, J., Warren, D., Elley, C. R., Wai, K. C., Manukia, M., … Robinson, E. (2012). Using the general practice EMR for improving blood pressure medication adherence. Studies in Health Technology and Informatics, 178, 228–34. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/22797046

Additional Assets

Showing 1 Reviews

  • Placeholder
    Belita Kin
    0

    Your blog inspired me when I read it. I love reading blogs. I don't excite always by reading blog posts. But in your blog I find something that implies in blogging. Anyway, you’re unquestionably a great blogger and you have a skill to make your reader to keep reading your blog again and again. Keep up the magnificent work. Continue moving the people!

    Visit : essay writing service reviews


License

This article and its reviews are distributed under the terms of the Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and redistribution in any medium, provided that the original author and source are credited.