Measuring the Influence of Commercial Entities in the Twitter backchannels of medical conferences: The #MICEproject

  • Tejas Desai (@nephondemand) 1 2
  • Vibhu Dhingra (@doc_dhingra) 2
  • Afreen Shariff (@ashartwit) 3
  • Aabid Shariff (@aabidshariff) 4
  • Edgar Lerma (@edgarvlermamd) 5
  • Parteek Singla 6
  • Swapnil Kachare 2
  • Zoheb Syed 7
  • Deeba Minhas 8
  • Ryan Madanick (@RyanMadanickMD) 9
  • Xiangming Fang (@Xiangming_Fang) 2
  1. 1.  Nephrology On-Demand, Online @ http://goo.gl/tfSAQT
  2. 2.  East Carolina University, Greenville, North Carolina
  3. 3.  Duke University, Durham, North Carolina
  4. 4.  Monsanto, Raleigh, North Carolina
  5. 5.  University of Illinois at Chicago College of Medicine, Chicago, Illinois
  6. 6.  Barnes Jewish Hospital, St. Louis, Missouri
  7. 7.  College of William and Mary, Yorktown, Virginia
  8. 8.  Cedars-Sinai Medical Center, Los Angeles, California
  9. 9.  University of North Carolina, Chapel Hill, North Carolina

Abstract

Twitter backchannels are increasingly popular at medical conferences.  A variety of user groups, including healthcare providers and third party entities (e.g., pharmaceutical or medical device companies) use these backchannels to communicate with one another.  These backchannels are unregulated and can allow third party commercial entities to exert an equal or greater amount of influence than healthcare providers.  Third parties can use this influence to promote their products or services instead of sharing unbiased, evidence-based information.  In the #MICEproject we quantified the influence that third party commercial entities had in 13 major medical conferences.

INTRODUCTION

Medical conference organizers (CO) must strike a balance with commercial entities (a/k/a 3rd party commercial entities; e.g., pharmaceutical companies and device manufacturers).  Third parties are needed to offset the cost of many national scientific meetings and provide valuable information about the latest developments in the field (1,2).  Concurrently, COs must mitigate “detailing”: the process in which third parties have direct and unregulated access to conference attendees (learners) (3,4).  COs have reached this balance in live conferences by: 1) not allowing third parties to select speakers at plenary and other sessions, 2) not allowing third parties to pass out literature in-and-around classrooms, and 3) restricting learner access to third parties to one geographic location ("exhibition hall") and only during specific periods of time that do not conflict with other scientific sessions (2).  Theoretically, these safety mechanisms allow a learner to experience a live medical conference without ever exposing him/herself to a third party.

 

This model has not been replicated in the increasingly popular Twitter backchannels (5,6).  Backchannels are online social media streams (typically Twitter streams) that allow learners, COs, and third parties to share information with each other (5,6).  Backchannels are considered an excellent way to enhance a live conference and an increasing number of medical conferences are incorporating them into their annual meetings (7-18).  To date, medical conference backchannels are unregulated; this allows third parties direct access to learners that they cannot achieve at a live conference.  Detailing on Twitter exposes learners to third parties and facilitates the transfer of biased information in an environment that does not have established safety mechanisms in place (1,3,4).  Theoretically, third parties can exert a greater influence over learners through Twitter detailing.

In this investigation, we quantified the degree of influence that third parties have in the Twitter backchannels of thirteen prominent medical conferences from 2011-2013.  We measured influence in three different ways.  In the unregulated realm of Twitter backchannels, we hypothesized that third parties are as influential as any other group.

METHODS

Data Set

We identified five medical societies that promoted the Twitter backchannels of their respective annual meetings.  These societies assigned a conference-specific hashtag for each backchannel and registered each with the Healthcare Hashtag Project (HHP; symplur.com).  The HHP is an independent organization that archives, and makes available for research, tweets of various healthcare-related Twitter backchannels.  The HHP is a comprehensive database of all tweets authored at a particular scientific conference.  We queried the HHP database for all conference-specific tweets using the pre-assigned hashtags.  Table 1 shows the thirteen conferences that were included in the data set.  We collected a) date and time of tweet, b) Twitter username of the tweet author, c) content of the tweet, and d) Twitter username(s) of individuals/organizations mentioned within the body of a tweet (@mentions).

Table 1. Baseline Data.

Medical Society / Conference Organizer(s)

Conference Name

Dates

Hashtag

Healthcare Hashtag Project URL (shortened)

Tweets (No.)

American College of Cardiology*

 

 

 

 

 

 

 

 

 

 

8505

 

2012 Annual Meeting

3/24/2012 - 3/27/2012

#acc12

http://goo.gl/Os3f0C

2951

 

2013 Annual Meeting

3/9/2013 - 3/11/2013

#acc13

http://goo.gl/nnwEPK

5554

American Society of Nephrology

 

 

 

 

4299

 

Kidney Week 2011

11/8/2011 - 11/13/2011

#kidneywk11

http://goo.gl/m3UvNm

583

 

Kidney Week 2012

10/30/2012 - 11/4/2012

#kidneywk12

http://goo.gl/ni5CC6

1137

 

Kidney Week 2013

11/5/2013 - 11/10/2013

#kidneywk13

http://goo.gl/8vog3n

2579

American Society of Clinical Oncology

 

 

 

 

31991

 

2011 Annual Meeting

6/3/2011 - 6/7/2011

#asco11

http://goo.gl/GX3svS

7531

 

2012 Annual Meeting

6/1/2012 - 6/5/2012

#asco12

http://goo.gl/HIky9X

9555

 

2013 Annual Meeting

5/31/2013 - 6/4/2013

#asco13

http://goo.gl/IOS4pf

14905

American Gastroenterological Association and American Society for Gastrointestinal Endoscopy and American Association for the Study of Liver Diseases and The Society for the Surgery of the Alimentary Tract

 

 

 

 

5123

 

Digestive Disease Week 2011

5/7/2011 - 5/10/2011

#ddw11

http://goo.gl/xpwZpY

1199

 

Digestive Disease Week 2012

5/19/2012 - 5/22/2012

#ddw12

http://goo.gl/fgM3di

1720

 

Digestive Disease Week 2013

5/18/2013 - 5/21/2013

#ddw13

http://goo.gl/QmL66C

2204

American Academy of Dermatology*

 

 

 

 

1241

 

2012 Annual Meeting

3/16/2012 - 3/20/2012

#aad12

http://goo.gl/ZrcGOj

585

 

2013 Annual Meeting

3/1/2013 - 3/5/2013

#aad13

http://goo.gl/BeCl23

656

*Conference Twitter backchannel for the 2011 meeting was not registered with the HHS & unavailable for analysis

 

Categorization of Account Holders

In order to quantify the influence that third parties exert in Twitter backchannels, we categorized every tweet author/account holder mentioned within the body of a tweet (@mentions) into one of four categories:  a) healthcare provider (HCP), b) third party commercial entity (third party), c) unclear identity, and d) none of the above.  Table 2 defines each category and provides a representative example.  We used each account holder’s Twitter profile to ascertain under which category that account should be.  Categorization was done from January to April 2014.  We did not perform additional Internet searches (e.g., Facebook or Google search) of accounts categorized as “unclear”.  None of the investigators contacted any of the account holders, in any venue, to determine their identity. 

To ensure inter-rater reliability when categorizing Twitter account holders, we performed a Light’s kappa statistic on a different set of previously published Twitter data (19).  The Light’s kappa score was 0.72.

Table 2.  User categories and examples.

Category

Definition

Representative Example

Twitter Profile

Healthcare Provider

Individual or organization whose primary purpose is to disseminate medical information or provide clinical care for patients

@nephondemand

Tejas Desai, MD.  Creator of Nephrology On-Demand & Kidney Konnection & Nephrology Fellowship Director @ ECU. I conduct research in social media & medicine & program iOS Apps

3rd Party Commercial Entity

Organization or individual representing an organization whose primary purpose is to provide a product or service to medical professionals and/or patients

@mmsholdings

MMS Holdings Inc.  MMS Holdings Inc. is a global niche pharmaceutical service organization that focuses on regulatory submission support for the pharma and biotech industries.

None of the Above

Individuals or organizations that are unrelated to healthcare or the purpose of the scientific meeting

@RdgTerminalMkt

The Reading Terminal Market - Since 1893

Unclear Identity

Individual or organizations whose Twitter profile was vague or empty

@KhaliqWhy

Khaliq. Seeking Knowledge

 

Measuring Influence of Account Holders

We assessed Twitter influence by three different methods.  In the first method, we measured the number of distinct account holders per category that authored at least one tweet in one of the 13 conferences analyzed.  We defined a high Twitter influence as that category with the largest number of account holders. 

In the second method, we measured the total number of tweets authored by account holders in each category.  We calculated the tweet:author ratio by dividing the number of tweets composed by the total number of authors within a particular category.  We ascribed the greatest Twitter influence to that category with the highest ratio.

In the third method, we measured influence by calculating the PageRank of any account holder that was mentioned (@mentions) in the body of a tweet.  Originally developed by Page, Brin, Motwani and Winograd, the PageRank is a link-based algorithm and considered by Williams, Baldwin, and Rubel to be the best measure of social media influence (20-23).  As described by Abdullah, in the PageRank “a link from a page to another page is understood as a recommendation and the status of the recommender is important” (24).  A webpage, to which many others are linked, is considered an influential webpage and is given a high PageRank (20,24).  Its PageRank increases even more when the linking webpages are influential as well (i.e., have their own high PageRanks) (20,24-26).  Similarly, a Twitter account that is mentioned (@mentions) many times and/or mentioned by other influential Twitter accounts will, itself, appropriately receive a high PageRank (28).  Indeed a number of investigators, including Abdullah, Kwak et al and Bakshy et al, have successfully adopted the PageRank to accurately measure Twitter influence using @mentions (23,24,27-30).  The PageRank of @mentions is also known as the “Influence Index” and is used by the independent research firm Twitalyzer to measure one’s Twitter influence (31).  It is also the preferred method of measuring influence by Evan Williams, co-founder of Twitter (31). 

Privacy Considerations for Account Holders

The tweets collected from the HHP contained identifying information or links to such information.  The same identifying information is freely available to the general public through the Library of Congress (32,33).  Twitter’s Terms and Conditions warn account holders of the public nature of tweets, specifically, “what you say on Twitter may be viewed all around the world instantly” (12).  Perhaps because such identifying information is freely accessible, prior investigators have not requested approval from their local institutional review boards (13,23,24,27-30).  Currently there are no expectations for researchers to gain approval from any external agency (government, Twitter, or others) to research Twitter data (34).  In many investigations, including our own, researchers have adopted the “distance principle”, explained by Buchanan et al (35).  Given that our investigation was an observation of data in the public space and did not involve direct interaction with any account holder, the “distance principle”, along with the precedent set forth by previous investigators, supported our belief that external committee review was unwarranted (34-37). 

Nevertheless, the identifying information within each tweet was as critical to our investigation as our ethical use of it.  Therefore, we designed our methods in accordance with the United States Department of Homeland Security’s 2012 Menlo Report – a guide for investigators performing “communication technology research” (38).  We also designed our methods to conform to the British Psychological Society’s guidelines for “Internet-mediated research” (39).  Finally, we complied with the six ethics guidelines recommended by Rivers and Lewis when analyzing “big data” (34).  Our adherence to these strict and established guidelines satisfied our professional sense of duty to maintain the privacy of the account holders whose Twitter activities comprised our data set.

Statistical Considerations

We calculated frequencies per category for: 1) number of Twitter accounts that authored tweets, 2) number of Twitter accounts that were mentioned within a tweet, 3) number of tweets composed.  We performed chi-square tests to compare these data using JMP Pro version 10.0.0.  We calculated PageRank using the NodeXL plugin (https://nodexl.codeplex.com) for Microsoft Excel 2013.  Median and interquartile ranges for the PageRank were calculated and compared using the Kruskal-Wallis test.  Each group needed to have at least 8671 @mentions in order to have achieved an 80% power to detect a 0.2 difference in PageRank.  In order to mitigate any future concern about the lack of reproducibility of our results, we 1) did not perform subgroup analyses of Twitter influence by conference and 2) followed recent guidelines that make “classical hypothesis testing more congruent with evidence thresholds for Bayesian tests” (40).  As a result, the significance level was set at p < 0.005 (40).

This investigation conforms to STROBE guidelines for observational research and SAMPL guidelines for statistical reporting (41,42).

RESULTS

Baseline Data

We collected 51159 tweets, authored by 8778 Twitter account holders, in 13 conferences, sponsored by 5 medical societies, from 2011 to 2013 (Table 1).  Our data set represents 94.6% of tweets and 78.1% of authors in the HHP.  The remaining data was either lost during the extraction process from the HHP or could not be parsed correctly by the software we used.  The largest number of tweets and authors was in the 2013 American Society of Clinical Oncology’s annual meeting (15120 and 3156, respectively). 

Twitter Influence by Number of Authors

Nearly 61% of the authors had a Twitter profile that identified them (Table 3).  In this group, there were 2173 (25%) healthcare providers and 1575 (18%) third party entities (p < 0.0001).  The largest group of authors could not be identified (3412; 39%; p < 0.0001). 

Twitter Influence by Tweet:Author Ratio

Despite being the greatest number of authors, those with unclear identities did not compose the greatest number of tweets (Table 3).  The tweet:author ratio for unidentified Twitter account holders was only 3.7.  Healthcare providers composed 19503 tweets and had a tweet:author ratio greater than that of third party entities (8.98 versus 6.93 tweets per author; p < 0.0001).   

Twitter Influence by PageRank of @mentions

In our data set, a total of 3316 Twitter accounts were mentioned a total of 39997 times (Table 3).  Healthcare providers were mentioned nearly 46% of the time, while third party commercial entities were mentioned less than 20% of the time.  The sum total of @mentions in the healthcare provider and third party categories was 26175: 1.5 times greater than the 17341 @mentions needed to achieve 80% power.  The median PageRank for healthcare providers was the highest amongst the four categories.  However, there was no statistical difference between it and the median PageRank for third party commercial entities (0.797 versus 0.761, respectively; p 0.175). 

Table 3.  Measures of Twitter Influence.

Measure of Influence

Third Party Commercial Entity

Healthcare Provider

None of the Above

Unclear Identity

Total Authors

1575

2173

1617

3413

Total Tweets

10916

19503

8105

12635

Tweets:Author Ratio

6.931

8.975

5.012

3.702

 

 

 

 

 

Unique @mentions

683

864

772

997

Total @mentions

7834

18341

8011

5811

PageRank

 

 

 

 

10th Percentile

0.313

0.316

0.304

0.297

25th Percentile

0.425

0.441

0.405

0.401

Median

0.761

0.797

0.677

0.591

75th Percentile

1.391

1.546

1.124

1.005

90th Percentile

3.103

3.089

2.158

1.921

 

DISCUSSION

Third Party Influence

Third party commercial entities had a statistically similar PageRank as healthcare providers (0.761 versus 0.797, respectively) despite having significantly fewer authors (1575 versus 2173, respectively) and significantly less Twitter activity (6.931 versus 8.975 tweets/author, respectively).  This suggests that third parties are equally influential in the Twitter backchannels of scientific meetings as healthcare providers; a parity that is difficult to achieve in live conferences.  Admittedly, there are no investigations that measure third party influence at live conferences.  Perhaps the lack of data is due to conference organizers’ financial reliance on third parties to sponsor their conferences.  In 2009, third parties gave close to $850 million dollars of sponsorships to various medical conferences (1).  In 2011, 75% of conference organizers received third party financial support (2).  Third parties provide printed and digital conference materials, travel grants, and meals gratis.  This financial dependence may preclude any scientific study of third party influence at live conferences.  Nevertheless, conference organizers mitigate third party influence by geographically isolating third parties, curtailing their “hours of operation”, and independently selecting topics and speakers for the conference agenda (2). 

Conference organizers do not depend on the financial support of third parties to maintain active Twitter backchannels.  Creating and registering a conference-specific hashtag and generating messages on Twitter are free.  Yet not one of the eight conference organizers (in any of the 13 conferences studied) implemented any safeguards to limit third party “detailing” (3).  As a former third party representative, Ahari outlined eight forms of detailing used by third parties to influence individuals (3).  All eight can be easily adapted to work in Twitter backchannels.  Indeed any message from a third party is more likely to place a favorable bias on that party’s product/service than unprejudiced evidence-based medicine (1).

Jalali, Wood, and others have suggested that conference organizers learn how their respective Twitter backchannels are being used/misused in order curtail the influence that third parties have within them (16,28,43).  Our study is the first to elucidate this use/misuse by various groups.  Second, more must be done to establish guidelines for third party activities in Twitter backchannels.  There are plenty of well-intentioned recommendations on the use of Twitter by healthcare providers and conference organizers (17,44).  There are no comparable recommendations for third parties or their interactions with HCPs/COs (45).  Both the Pew Charitable Trusts and American Medical Student Association discuss how COs can mitigate conflicts of interest (COI), but neither offer specific guidelines in managing COIs within social media streams (1,46,47).  Therefore, the investigators of this study recommend the following activities to bring the medical community closer to such guidelines:

·      Conference organizers should publicly state in their Twitter backchannel that third party entities should declare themselves as such in their respective Twitter profiles (33)

·      Conference organizers should insist that third parties compose tweets that disseminate scientific facts and not solicitations for products/services

·      If third parties wish to solicit for a product/service, they should include an additional hashtag in the body of their tweet (e.g., #ad) to allow participants within the backchannel to filter out such tweets

·      Conference organizers should encourage third parties to restrict their Twitter activity to coincide with their live “hours of operation”

·      Conference organizers should task independent individuals/groups to annually measure the PageRanks for each Twitter account mentioned (@mentions) within their conference-specific hashtag

·      Conference organizers should target third party accounts with abnormally high PageRanks for further education about best-practices within their respective Twitter backchannel

These recommendations would align third party activities in Twitter backchannels with their activities at live conferences.  We believe they are reasonable: neither burdensome to conference organizers nor offensive to third party commercial entities.  Compliance can be measured yearly through PageRank assessments, as performed in this investigation, with targeted re-education to those third parties that require additional assistance. 

PageRank versus other measures of Twitter influence

The PageRank of @mentions has been used by a number of Twitter researchers and is considered the closest estimation of Twitter influence (21-23,27,29,30).  Indeed even commercial research firms, such as SEOmoz and Twitalyzer, use the PageRank of @mentions to measure Twitter influence for their clients (31,48).  As Bray and Peters have indicated, mentioning someone in one’s tweet represents a major commitment to that person (48).  The more a person is mentioned, the more they effect the conversation and the greater the influence they exert (28,31,48).

A common misperception is that the number of followers or impressions (which equals the product of the number of followers and tweets composed) is an accurate measure influence.  Any metric that uses the number of followers and/or tweets often results in false calculations of influence (49).  Bots can artificially inflate the number of tweets composed, causing the impressions to be misleadingly elevated.  Moreover, the number of followers or impressions excludes any interaction between participants.  Perhaps for these reasons impressions and the number of followers are considered “vanity” metrics: easy to calculate but of little value in measuring one’s Twitter influence (48).

Twitter researchers do not perform content analyses to measure influence (23).  Neither this investigation nor the studies referenced analyzed tweet content to measure influence.  Cha et al has mathematically analyzed various metrics to measure Twitter influence and concluded that the PageRank of @mentions was one of the best ways to do so (29).

Unclear Identities on Twitter

There were 3413 Twitter accounts that could not be identified because their Twitter profiles were vague or empty.  These accounts generated only 24.7% of the total tweets analyzed.  We consciously avoided using alternative methods to identify these accounts.  In accordance with recommendations by Farnan and McKee, we assumed that account holders with vague profiles wanted to remain anonymous (36,50).  To respect these wishes, we did not contact any author or perform additional Internet searches to ascertain their identities (34,35).

Strengths

Perhaps the greatest strength of this investigation is its breadth (13 conferences sponsored by 8 medical societies) and depth (51159 tweets).  Chaudhry et al conducted an analysis of 12644 tweets from 2 conferences sponsored by one medical society while Jalali et al analyzed 10937 tweets from 4 conferences sponsored by as many medical societies. (13,51).  We measured Twitter influence by calculating the PageRank of @mentions – the recommended metric by a number of researchers, commercial research firms, and the co-founder of Twitter (21-23,27,29-31,48).  We conformed to three well-established sets of guidelines for conducting Internet-based research and respected the privacy of those users who wanted to remain anonymous (1,34-36,38,50).  Finally, and to the best of our ability we have reported our findings in accordance with two sets of research-reporting guidelines (41,42). 

CONCLUSION

Third party commercial entities exert an equal influence as healthcare providers in the Twitter backchannels of medical conferences.  Without safety mechanisms in place, Twitter backchannels can devolve into a venue for the spread of biased information rather than evidence-based medical knowledge, as that seen at lives conference.  Continuing to measure the influence that third parties exert can help conference organizers develop reasonable guidelines for Twitter backchannel activity.

REFERENCES

1.  Austad KE, Avorn J, Franklin JM, Kowal MK, Campbell EG, Kesselhelm AS.  Changing Interactions between Physician Trainees and the Pharmaceutical Industry: A National Survey.  Journal of General Internal Medicine.  2013; 28(8):  1064-1071.

2.  Lo B, Ott C.  What is the Enemy in CME, Conflicts of Interest or Bias?  Journal of the American Medical Association.  2013; 310(10):  1019-1020.

3.  Fugh-Berman A, Ahari S.  Following the Script:  How Drug Reps Make Friends and Influence Doctors.  PLoS Medicine. 2007; 4(4): e150.

4.  Millenson ML.  Getting doctors to say yes to drugs: The cost and quality of impact of drug company marketing to physicians.  Blue Cross Blue Shield Association. 2003.  Available at http://www.thedcasite.com/Kickbacks/Getting_doctors_to_say_yes_to_drugs.pdf.  Accessed on 29 October 2013.

5.  7 Things You Should Know About:  Backchannel Communication.  Educause. 2010.  Available at http://www.educause.edu/library/resources/7-things-you-should-know-about-backchannel-c\nommunication. Accessed on 29 October 2013.

6.  Coiera E.  Social networks, social media, and social diseases.  British Medical Journal.  2013; 346:  f3007.

7.  Djuricich AM, Zee-Cheng JE.  Live tweeting in medicine: “Tweeting the meeting”.  International Review of Psychiatry.  2015; 27(2): 133-139.

8.  Shariff AI, Fang X, Desai T.  Using Social Media to Create a Professional Network Between Physician-Trainees and the American Society of Nephrology.  Advances in Chronic Kidney Disease.  2013; 20(4): 357-363.

9.  Desai T, Shariff A, Shariff A, Kats M, Fang X, Christiano C, Ferris M.  Tweeting the Meeting:  An In-Depth Analysis of Twitter Activity at Kidney Week 2011.  PLoS One. 2012; 7(7): e40253.

10.  O’Conner A, Jackson L, Goldsmith L, Skirton H.  Can I get a retweet please?  Health research recruitment and the Twittersphere.  Journal of Advanced Nursing.  2014; 70(3): 599-609.

11.  Mysko C.  An exploration of medical students’ views on using Twitter for medical education.  Technical Report, Newcastle University.  2013.  Available at http://twitdoc.com/2BY1.  Accessed on 29 October 2013.

12.  Neuhaus F, Webmoor T.  Agile Ethics for Massified Research and Visualization. Information, Communication, and Society.  2012; 15(1): 43-65.

13.  Chaudhry A, Glode M, Gillman M, Miller RS.  Trends in Twitter Use by Physicians at the American Society of Cilnical Oncology Annual Meeting, 2010 and 2011. Journal of Oncology Practice. 2012; 8(3): 173-178.

14.  Fenny N, Dimov V, Wolf R.  Rapid Growth of Twitter Use by Allergists and Immunologists for Professional Purposes.  Annals of Allergy Asthma and Immunology.  2012; 109(5): A125.  Available at http://bit.ly/WDujw

15.  Hajar Z, Clauson KA, Jacobs RJ, Melomed YL.  Leveraging social media in pharmacy:  analysis of pharmacists engagement and use of Twitter.  Presented at the 47th American Society of Health-Systems Pharmacists Midyear Meeting and Exhibition. 2012; Available at http://www.ashpmedia.org/mcm12/documents/Onsite Program Book.pdf.  Accessed on 29 October 2013.

16.  Jalali A, Wood TJ.  Tweeting during conferences: educational or just another distraction.  Medical Education. 2013; 47: 1129-1130.

17.  Chretien KC, Kind T.  Social Media and Clinical Care:  Ethical, Professional and Social Implications.  Circulation.  2013; 127: 1413-1421.

18.  Sugawara Y, Marimatsu H, Hozawa A, Shao L, Otani K, Fukao A.  Cancer patients on Twitter: a novel patient community on social media.  BMC Research Notes. 2012; 5: 699.  Available at http://www.biomedcentral.com/1756-0500/699.

19.  Desai T, Patwardhan M, Coore H.  Factors that contribute to social media influence within an Internal Medicine Twitter learning community.  F1000Research. 2014; 3(120): 1-9.

20.  Page L, Brin S, Motwani R, Winograd T.  The PageRank Citation Ranking:  Bringing Order to the Web.  Techincal Report, Stanford InfoLab.  1999.  Available at http://ilpubs.stanford.edu:8090/422/.  Accessed on 29 October 2013.

21.  Williams S.  Is that all there is?  A literature review and potential approach to measuring influence in social media.  The Institute for Public Relations 16th International PR Research Conference.  2013.  Available at http://www.instituteforpr.org/is-that-all-there-is-a-literature-review-and-potential-a\npproach-to-measuring-influence-in-social-media/.  Accessed on 29 October 2013.

22.  Baldwin M.  How to: Measure online influence.  Mashable. 2009.  Available at http://mashable.com/2009/03/02/measuring-online-influence.  Accessed on 29 October 2013.

23.  Bakshy E, Mason WA, Hofman JM, Watts DJ.  Everyone’s an Influencer:  Quantifying Influence on Twitter.  Proceedings of the fourth ACM International Conference on Web Search and Data Mining.  2011; 65-74.  Available at http://dl.acm.org/citation.cfm?id=1935845.  Accessed on 29 October 2013.

24.  Abdullah IB.  Incremental PageRank for Twitter Data Using Hadoop.  Techincal Report, University of Edinburgh.  2010.  Available at http://homepages.inf.ed.ac.uk/miles/msc-projects/abdullah.pdf.  Accessed on 29 October 2013.

25.  Langville AN, Meyer CD.  Google’s PageRank and Beyond: The Science of Search Engine Readings.  2006.  Princeton University Press, Princeton, NJ.

26.  Rogers I.  The Google PageRank Algorithm and How It Works.  Available at http://www.cs.princeton.edu/~chazelle/courses/BIB/pagerank.htm.  Accessed 15 May 2015.

27.  Kwak J, Lee C, Park H, Moon S.  What is twitter, a social network or a news media?  Proceedings of the 19th International Conference on World Wide Web.  2010; 591-600.  Available at http://dl.acm.org/citation.cfm?id=1772751.  Accessed on 29 October 2013.

28.  Ye S, Wu SF.  Measuring Message Propagation and Social Influence on Twitter.com.  Proceedings of the Second International Conference on Social Informatics. 2010; 216-231.  Available at http://dl.acm.org/citation.cfm?id=1929342. Accessed on 29 October 2013.

29.  Cha M, Haddadi H, Benevenuto F, Gummadi K.  Measuring User Influence in Twitter:  The Million Follower Fallacy.  Proceedings of the Fourth International AAAI Conference on Weblogs and Social Media.  2010.  Available at http://aaai.org/ocs/index.php/ICWSM/ICWSM10/paper/viewPaper/1538.  Accessed on 29 October 2013.

30.  Weng J, Lim EP, Jiang J, He Q.  Twitterrank:  finding topic-sensitive influential twitterers.  Proceedings of the third ACM International Conference on Web Search and Data Mining.  2010; 261-270.  Available at http://dl.acm.org/citation.cfm?id=1718520.  Accessed on 29 October 2013.

31.  Leonhardt D.  A Better Way to Measure Twitter Influence.  New York Times. 2011.  Available at http://6thfloor.blogs.nytimes.com/2011/03/24/a-better-way-to-measure-twitter-influence\n/.  Accessed on 29 October 2013.

32.  Halavais A.  Open up online research.  Nature. 2011; 480:174-175.

33.  Forgie SE, Duff JP, Ross S.  Twelve tips for using Twitter as a learning tool in medical education.  Medical Teacher. 2013; 35: 8-14.

34.  Rivers CM, Lewis BL.  Ethical research standards in a world of big data.  F1000Research.  2014; 3: 38.

35.  Buchanan E, Aycock J, Dexter S, Dittrich D, Hvizdak E.  Computer Science Security Research and Human Subjects:  Emerging Considerations for Research Ethics Boards.  Journal of Empirical Research on Human Research Ethics.  2011; 6(2): 71-83.

36.  Farnan JM.  Connectivity and Consent: Does Posting Imply Participation?  The American Journal of Bioethics.  2014; 14(10): 62-63.

37.  Volpe R, Blackall G, Gree, M, George D, Baker M, Kauffman G.  Googling a patient: Should health care professionals “google” their patients?  Hasting Center Report. 2013.  Available at http://www.thehastingscenter.org/Publications/HCR/Detail.aspx?id=6537.  Accessed on 29 October 2013.

38.  The Menlo Report:  Ethical Principles Guiding Information and Communication Technology Research.  US Department of Homeland Security.  2012.  Available at http://www.caida.org/publications/papers/2012/menlo_report_actual_formatted/.  Accessed on 29 October 2013.

39.  Ethics Guidelines for Internet-mediated Research.  The British Psychological Society.  2013.  Available at http://www.bps.org.uk/system/files/Public files/inf206-guidelines-for-internet-mediate\nd-research.pdf.  Accessed on 29 October 2013.

40.  Johnson VE.  Revising standards for statistical evidence.  Proceedings of the National Academy of Sciences. 2013; 110(48): 19313-19317.

41.  von Elm E, Altman DG, Egger M, Pocock SJ, Eotzsche PC, Vandenbroucke JP.  The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: Guidelines for Reporting Observational Studies. PLoS Medicine. 2007; 4(10): 1623–1627.

42.  Lang TA, Altman DG.  Basic Statistical Reporting for Articles Published in the Biomedical Journals:  The “Statistical Analyses and Methods in the Published Literature” or The “SAMPL Guidelines”. September 2013;  Available at http://www.equator-network.org/wp-content/uploads/2013/03/SAMPL-Guidelines-3-13-13.pdf.  Accessed 29 October 2013.

43.  Mehrotra N, Dimov V.  Tweeting the Meeting:  Investigating Twitter Activity at the 2012 AAAAI Conference.  Journal of Allergy and Clinical Immunology. 2013; 131(2): AB34.  Available at http://www.jacionline.org/article/S0091-6749(12)02805-9/abstract

44.  DeCamp M, Koenig TW, Chisolm MS.  Social Media and Physicians’ Online Identity Crisis.  Journal of the American Medical Association.  2013; 310(6): 581-582.

45.  Guidance for Industry Fulfilling Regulatory Requirements for Postmarketing Submissions of Interactive Promotional Media for Prescription Human and Animal Drugs and Biologics:  Draft Guidance.  US Department of Health and Human Services. 2014. Available at http://www.fda.gov/downloads/drugs/guidancecomplianceregulatoryinformation/guidances/u\ncm381352.pdf.  Accessed on 29 October 2013.

46.  Carlat D, Bergh S, Carroll S.  Conflict-of-Interest Policies for Academic Medical Centers:  Recommendations for Best Practices.  The Pew Charitable Trusts.  2013.  Available at http://www.pewtrusts.org/en/research-and-analysis/reports/0001/01/01/conflictsofintere\nst-policies-for-academic-medical-centers.  Accessed on 29 October 2013.

47.  The AMSA Scorecard 2014:  Methodology.  Accessed on 18 May 2015.  Available at http://teaching-hospitals.amsascorecard.org/methodology.

48.  Bray P, Peters M.  Social Authority:  Our Measure of Twitter Influence.  Moz. 2013.  Available at https://moz.com/blog/social-authority.  Accessed 29 October 2013.

49.  Rubel S.  Google’s PageRank is best way to rate online influence.  Advertising Age. 2008; 79(43): 42.

50.  McKee R.  Ethical issues in using social media for health and health care research.  Health Policy.  2013; 110(2-3): 298-301.

51.  Jalali A, Wood TJ.  Analyzing Online Impact of Canadian Conference of Medical Education through Tweets.  Education in Medicine Journal. 2013; 5(3): e110.

 

Additional Assets

Reviews

Showing 7 Reviews

  • Placeholder
    Chi Chu
    Confidence in paper
    Quality of figures
    Quality of writing
    Originality of work
    3

    Very original and timely topic...thought provoking as it's difficult to say how much of a problem commercial entity influence on social media will be going forward, and what measures will ultimately be effective at managing 3rd party influences in the social media sphere rather than in a physical space. (I suspect it will be something more than simply telling 3rd parties how they ought to behave.)

    It is particularly nice to see the introduction of elsewhere validated/studied analytical social media tools used here to help gain insight from existing data in the medical world. I think these tools will only be seen more as the role of social media increases in healthcare and medical education. I expect this is the beginning of a domain of research exploring the interaction of social media and healthcare/medical education, and so researchers as well as consumers of that research will need to acquire familiarity with these analytical tools and techniques.

    I'm wondering, were there social media researchers among the authors? Either way I think it'd be helpful to seek additional feedback/critique from those with expertise in social media research as well to comment or make suggestions on the methodology.

    This review has 1 comments. Click to view.
    • 333
      Tejas Desai

      Thank you for your review and comments Dr. Chu. As you know, the authors of this investigation make up the Nephrology On-Demand Research Team. Our team is comprised of full-time physicians, medical trainees, and professionals who are interested in quantitative research in 1) medicine and social media and 2) online medical education . Our team's research experience with the former include:

      -- Factors that contribute to social media influence within an Internal Medicine Twitter learning community (http://f1000research.com/articles/3-120)

      -- Is Content Really King? An objective analysis of the public’s response to medical videos on YouTube (http://dx.plos.org/10.1371/journal.pone.0082469)

      -- Using Social Media to Create a Professional Network between Physician-trainees and the American Society of Nephrology (http://www.ncbi.nlm.nih.gov/pubmed/23809288)

      -- Assessing a nephrology-focused YouTube channel's potential to educate healthcare providers (http://www.ncbi.nlm.nih.gov/pubmed/22460179)

      -- Introducing the Global Medical Community to the information presented at local scientific conferences through nephrology blogs (http://www.ncbi.nlm.nih.gov/pubmed/24555021/)

      -- Tweeting the Meeting: An in-depth analysis of Twitter activity at Kidney Week 2011 (http://www.ncbi.nlm.nih.gov/pubmed/22792254)

      -- Twitter use as a platform for rapid dissemination of informative content from Digestive Disease Week is increasing. Gastroenterology (https://www.scribd.com/doc/222326417/Twitter-at-Digestive-Disease-Week)

      -- Analyzing the Role of YouTube as a Platform for Political Conversation: A Case-study of Same-sex Marriage Referendums in North Carolina and Washington (https://www.scribd.com/doc/260429201/Role-of-YouTube-in-the-Same-Sex-Referendums-of-North-Carolina-and-Washington)

      I hope this list provides the necessary perspective about the authors' experiences and skill sets that went into performing the current investigation. Thanks‼️

  • Placeholder
    francesco iannuzzella
    Confidence in paper
    Quality of figures
    Quality of writing
    Originality of work
    3

    An interesting and well-conducted analysis, focusing on an emerging ethical issue which could limit reliabily of Social media as an effective e-learning tool.

    This review has 1 comments. Click to view.
    • 333
      Tejas Desai

      Thank you Dr. Iannuzzella. We are hopeful that healthcare professionals recognize the possibility of Twitter backchannels of medical conferences being "misused" by commercial entities and help us take steps to establish best/better practice guidelines for pharma/industry Twitter activity in these backchannels.

  • Img 20150729 155406 animation 360
    Joshua Nicholson
    Confidence in paper
    Quality of writing
    Originality of work
    2

    The study by Desai et. al examines the relationship between commercial entities and conference participants at medical conferences on Twitter. This is an important area of study as Twitter and other social media streams become more prominent at conferences, both for attendees and observers (those following the social media streams).

    I think the suggestions for monitoring commercial influence on Twitter are reasonable, however implementation is likely a difficult task. Specifically, getting commercial entities to label their own tweets with #ad or having an organizer moderate such tweets seems difficult due to the number of tweets and the unappealing nature of labeling your tweets with #ad. While there may be some difficulties in implementation, discussion around these suggestions should be had and Desai et. al raise good points for further discussion.

    It would be interesting to see if the influence of commercial entities was somehow related to the size of the field in terms of market (ie are conferences devoted to cancer dominated by commercial entities more so than conferences where there is less money in the field?).

    Overall, this is a great piece and will become more and more important as social media becomes more and more prevalent.

    COI: I am the founder of The Winnower and helped with formatting issues on this paper. I also have tweeted during scientific conferences on Twitter to offer The Winnower as a service to scholars, therefore I could be seen as a commercial entity and may be biased in this regard.

  • Placeholder
    David Goldfarb
    2

    This is an admirable attempt to look at whether commercial enterprises are offering biased tweets during medical conferences. The authors clearly demonstrate that a certain proportion of promotional material is being transmitted during these conferences. The strengths of the work include the large number of tweets and contributors that are scrutinized, while a weakness in a similar vein is that the authors discuss is they inability to classify all of the contemporaneous tweets. Among the tweeters that are identified, no disclosures or conflicts of interest statements are of course available. The authors gamely suggest that regulations proffered by the conference organizers might be able to restrain these commercial enterprises. While a noble goal which I endorse, instituting a set of parental controls like this would seem to be as likely to be effective as reining in Uber or AirBnb. That is not to say that it is not worth attempting. Every time someone measures the influence of medical advertising it turns out to be hugely consequential "or they wouldn't be spending that much money". So the authors are pioneers in trying to quantitate this phenomenon and suggest that it is one with monitoring. It can only grow in magnitude and in influence.
    Disclosure: I evolved from 10 years of absolute chastity, refusing to eat pizza or Chinese food sponsored by pharma, to being a consultant to, and recipient of honoraria from, a number of pharma enterprises. It's a challenging and critical issue about which I have complex feelings and thoughts.
    ----David S. Goldfarb MD, FASN
    Professor of Medicine & Physiology,
    NYU School of Medicine and NYU Langone Medical Center

    This review has 1 comments. Click to view.
    • 333
      Tejas Desai

      Thanks for your comments Dr. Goldfarb.

      I think a lot of attention has been focused on "enforcement" of our 6 recommendations rather than seeing our recommendations as the beginnings of best practice guidelines for Twitter backchannel activity/use. We (the authors) admit that none of the conference organizers (COs) can enforce any rule on a Twitter backchannel. Just like in practice, where COs make recommendations for best medical practices or release position papers on a particular subject (neither of which are "enforceable"), a set of best (or better) practice guidelines for Twitter use might go a long way in "cleaning up" the Twitter backchannels upon which an increasing number of healthcare providers rely to gain new medical information.

  • Placeholder
    Vivek Jha
    Confidence in paper
    Quality of writing
    Originality of work
    2

    For me, reading this paper was educative. Despite being moderate user of social media, I had never given serious thought to the issue of how this tool could be potentially exploited by the pharma and device companies for surrogate detailing. The authors have done well to highlight it and perform a till date the broadest and numerically largest comparative analysis of twitter use by medical professional vs the commercial entities and presenting some interesting findings. The analysis had adhered to the required standard, and suggests that we should indeed be aware of this potentially insidious activity. Thy also present some recommendations which conference organises and healthcare workers will do to consider. The next steps might be to see how the professionals respond to these messages. Well done to the authors for this!

    This review has 1 comments. Click to view.
    • 333
      Tejas Desai

      Thank you Dr. Jha. We are hoping that one of the conference organizers establishes a set of "guidelines" for pharma/industry to follow. Then, we can repeat our analyses and determine if voluntary guidelines have any effect in "purifying" the Twitter backchannels of medical conferences. That would be an even more informative investigation.

  • Placeholder
    francesco iannuzzella
    Confidence in paper
    Quality of figures
    Quality of writing
    Originality of work
    1

    An interesting analysis on a very original topic. It raises new concerns about reliabilty of social media as an effective e-learning tool. Reliable and well-presented data.
    NO COI

  • Placeholder
    Len Starnes
    Confidence in paper
    Quality of figures
    Quality of writing
    Originality of work
    -1

    This is an extremely timely and well-presented analysis, coming during a period of increasing debate concerning the future of medical society conferences and their transition to hybrid physical-virtual events. It’s a topic I have been following and discussing since early 2013:
    http://www.slideshare.net/lenstarnes/future-of-med-confs-final-11-dec-2013 .

    My concerns are neither with the rigour of the methodology nor with the statistical analysis, far more with definitions, context and above all, the pragmatic realities of social media.
    Firstly, the use of the prejorative term ‘backchannel’ for conference hashtags implies some form of covert communications, which in my opinion they are not. Medical conference hashtags are overt, open to all, and often ‘owned’ my multiple organizations; the European Society of Cardiology and the European Song Contest make amusing bedfellows.

    Secondly, the term ‘detailing’ is used carte blanche to describe tweets from all participating commercial entities. Yet detailing is specifically defined as a product-selling situation, a descriptor that is not truly applicable to the majority of biopharma or medtech tweets. In most instances the regulatory environment forbids this. Inviting attendees to drop by a stand for a smoothie may indeed constitute #boothspam but it’s not strictly speaking ‘detailing’. This is where I would like to have seen more insight, and with it balance, into the qualitative nature of commercial-entity tweets.

    The third aspect of definitions concerns hashtag participants which the analysis simplifies to HCPs, commercial entities (biopharmas and medtechs), and unknowns. Right now we are witnessing participants falling into at least eight categories: HCPs, medical societies, patients and patient organizations, biopharmas and medtechs, medical media, communications agencies, management consultancies, and unknowns.

    However, it is the context of conference Twitter hashtags that the analysis neglects. My observation of the evolving formats of medical society conferences (see presentation link above) indicates that multiple digital/social channels, not just Twitter, are being used to host conversations before, during and after events. Hence its influence should be evaluated in the context of conversations taking place on other platforms, an issue that highlights the ever-valid question: what is the absolute, incremental influence of a digital/social channel?

    Finally, inspiring medical societies to implement all six recommendations may prove to be difficult, impossible, or counterproductive, although the first one – using transparent Twitter bios – is one that I would unequivocally support. The problem lies in the attempt to ‘control’ a social media platform that, on the contrary, demands medical societies relinquish control and embrace open dialogue with all stakeholders. It may not be easy, but if the track record of Twitter hashtag use in other disciplines is a meaningful guide, then open policies will be the inevitable future.

    Disclosure:
    I am currently a digital healthcare consultant with the following client base: biopharmas, medtechs, HCPs’ social networks, medical societies, management consultancies, health start-ups, venture capitalists, and government health authorities.

License

This article and its reviews are distributed under the terms of the Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and redistribution in any medium, provided that the original author and source are credited.