Social Media Analytics for R&D: a Catalan Vision

Xavier Lasauca i Cisa (@xavierlasauca) is the person in charge of Knowledge Management and Information Technologies on R&D (Directorate General for Research, Ministry of Economy and Knowledge, Government of Catalonia).

In this post I review how institutions that are part of the Catalan R&D environment make use of social media and I describe the benefits of this approach. I also discuss the metrics used by the Catalan Administration to evaluate and measure the impact of the government’s presence in this area and their benefits for the public.

Image: Research.cat 2.0, by Maricel Saball (CC BY 3.0), adapted from My social networks

Introduction

The Directorate General for Research is the unit of the Generalitat de Catalunya (Government of Catalonia) responsible for promoting science and technology research centres, planning training and career development of researchers, promoting Catalan participation in national, European and international research programs, and designing actions on science communication and dissemination in Catalonia, among other functions. This unit, along with the Directorate General of Universities, is part of the Secretariat for Universities and Research, which at the same time is part of the Ministry of Economy and Knowledge, headed by Minister Andreu Mas-Colell. The parallels with the British political system lie in the fact that the Directorate General for Research is the equivalent to the Government Office for Science within the Department for Business Innovation and Skills.

As the person responsible for Knowledge Management and ICT on R&D, I am in charge of the management of R&D computer applications at the Directorate General for Research, of the technical coordination of the research website of the Ministry of Economy and Knowledge, and of an electronic newsletter (RECERCAT). I am also the person responsible, in conjunction with the Communication department of the Secretariat, for the administration of the Directorate General for Research profiles on social media (Twitter, Facebook, Flickr, etc.). In addition, I maintain a personal blog (“L’ase quàntic” or “The quantum donkey“) where I write about innovation in Public Administration, the use of social media in universities and research, the Open Galaxy (Open Access, Open Science, Open Data, Open Courseware…) and the issue of women in science, among others.

This article focuses on the use of social media by the units within the departments of the Catalan Government (specifically the Secretariat for Universities and Research), research centres, large research support infrastructures and the reference networks in Catalonia. I would like to thank Professor Miquel Duran, from the University of Girona, for his support in the preparation of data on the number of Twitter, Klout and Kred followers of the organisations analyzed during the second week of October this year.

A General Overview of the Catalan R&D System

The Catalan public R&D system is primarily composed of universities, research centres, large research support infrastructures, hospitals, science and technology parks, networks of reference and research groups.

The central topics in science policies applied in Catalonia in recent years are, on the one hand, talent attraction and retention, with excellence and internationalization as their benchmarks (a good example of this line of action is ICREA, Catalan Institution for Research and Advanced Studies), and on the other hand, a sustained increase in research funding, with the bulk of the spending allocated to research structures, both research centres and large facilities (such as the Alba synchrotron light facility or the MareNostrum supercomputer).

A good sign of the health status of the Catalan scientific system is that, if we consider that the size of the population in Catalonia represents 1.5% of the EU-27, the system has managed to attract 2.2% of the financing available from the European Union Seventh Framework Programme, and has obtained 3.4% of European Research Council (ERC) grants. Another relevant fact is that 2.9% of scientific publications in the EU-27 have been written by Catalan researchers. You can find these data and more information on the Catalan research system in the article by the Secretary for Universities and Research of the Catalan Government, Antoni Castellà, published in the issue 1 of the journal Global Scientia.

Institutional support

Three social media accounts are being managed from the Secretariat for Universities and Research: the Directorate General for Research account (@recercat), the Directorate General of Universities (@universitatscat) and the Secretariat for Universities and Research (@coneixementcat).

The Twitter account of the Directorate General for Research is used to disseminate the scholarships and research grants funded by the unit, as well as the publications of the institution (for example, the most important news published in the newsletter RECERCAT). It is also used to post news and updates from the web, to promote the scientific dissemination activities from the unit and from the Recerca en acció (Research in action) website, as well as events, awards, scholarships, publications and other information from the system related agents. Apart from this public information service, the Twitter account also serves to promote government action (with links to press releases) and to share institutional statements from events or interviews of policymakers.

The institutional account management of the Secretariat for Universities and Research, as well as of other departments of the Catalan government, is based on the Style and usage guide of the Government of Catalonia’s social networks, produced by the General Directorate for Citizen Services and Publicity (GDCSP), at the Ministry of the Presidency of the Government of Catalonia. This publication establishes common guidelines for a consistent presence of the Government of Catalonia on social networks and lists the different social media utilities, their various uses, the purpose of each network, recommendations for an appropriate and productive presence, and criteria for finding the best communicative style for each tool.

One of the most important chapters in the guide is dedicated to metrics, an essential tool to monitor the activity that is being done and to assess and measure the impact, in this case, of the presence of the Administration in this environment and the benefits it represents for citizens. Metric indicators are based on the following key concepts:

Dialogue: measures the degree of dialogue that the Government of Catalonia maintains with citizens on different social networks.

Reach: information on the distribution of the Government of Catalonia contents to the people who are part of the social network.

Action: indicates whether the content shared on the networks promotes activity.

Interaction: shows the global relationship between an account and its audience.

Acceptance (Applause): quantifies the degree of satisfaction.

For each of these key concepts, the indicators shown in Table 1 (List of indicators for Twitter and Facebook) and Table 2 (List of indicators for YouTube, Flickr and Slideshare) are used:

Concept Twitter Facebook

Audience Followers Friends

Tweets sent Entries

Interactions

Mentions Comments

Retweets (RT) Shares

Clicks to links Likes

Interest

Dialogue Mentionts/tweets Comments/entries

Reach RT/tweets Shares/entries

Action Clicks to links/tweets

Applause

Likes/entries

Interactions (Mentions+RT)/tweets (Comments+shares+likes)/entries

Commitment

Dialogue Mentions/followers Comments/friends

Reach RT/followers Shares/friends

Action Clicks to links/followers

Applause

Likes/friends

Interactions (Mentions+RT)/followers (Comments+shares+likes)/friends

Table 1: List of indicators for Twitter and Facebook

Tool Indicator

Youtube

Total number of videos uploaded

Videos uploaded during the month

Number of views of all the videos uploaded

Visits to the channel

Subscribers

Flickr

Total number of photos published

Photos published during the month

Number of views of all the photos published

Slideshare

Total number of presentations and documents published

Presentations and documents published during one month

Number of downloads of all the presentations and documents published

Number of visits of all the presentations and documents published

Table 2: List of indicators for Youtube, Flickr and Slideshare

In order to facilitate a better interpretation of the metrics, the GDCSP prepares a quarterly report that shows the evolution of these indicators graphically and sends it to each of the units responsible for corporate social media accounts. These reports help the units to evaluate the effectiveness of their activity on social media and to consider whether the previously defined objectives are being achieved. In addition,

the information obtained can serve as a basis for predicting future actions and planning campaigns. After all, assessment in the Administration must serve to identify public policies that work, knowing the impact and to what extent it is attributable to the intervention of Public Administration. Table 3 shows the number of Twitter, Facebook and YouTube followers of the institutional accounts of the Universities and Research areas in the Catalan Government, as well as the Klout and Kred reputation indices:

Secretariat for Universities and Research accounts Twitter FB YT Klout Kred

Directorate General for Universities (@universitatscat) 3,300 859

52 718 5

Directorate General for Research (@recercat) 2,970 538

53 697 5

Secretariat for Universities and Research (@coneixementcat)

1,467

49 656 4

Research in action (@RecercaenAccio) 1,438

87 49 616 3

Table 3: Social Analytics for Institutional Twitter Accounts of Secretariat for Universities and Research of Catalan Government

The Twitter and Facebook accounts of the Universities area lead the classification ahead of the Research accounts, probably because their target audience is considerably larger. The Twitter account of the Knowledge area of the Ministry of Economy and Knowledge of the Government of Catalonia ranks third, whereas the account of the science dissemination website Research in action closes the classification.

Research Centres in Catalonia: Increasingly Intensive Use of the Social Web

As regards research centres in Catalonia, it has to be mentioned that the CERCA Institute is the Government of Catalonia’s technical service and its means for supervising, supporting and facilitating the activities of the 47 research centres in the CERCA system. These research centres are independent entities with their own independent legal status, partially-financed by the Government of Catalonia (which provides them with stable funding through programme contracts) and their main aim is excellence in scientific research. They follow a private sector management model that is totally flexible and based on multi-year activity programmes within the framework of a strategic plan and ex-post supervision that respects the autonomy of each centre.

The aim of this model is to encourage co-ordination and strategic co-operation between centres, to improve the positioning, visibility and impact of the research carried out and to facilitate communication between public and private agents. To illustrate the efficiency of the system, out of the 60 ERC Starting Grants awarded in Catalonia during the 2007-2012 period, 34 were awarded to researchers from the CERCA centres (56%), whereas in the case of the ERC Advanced Grants, the percentage rises to 63% (19 out of 30) for the 2008-2011 period.

Out of all the 47 CERCA centres, 25 use social media tools, primarily Twitter, as part of their communication strategy. Table 4 summarises the most important indicators of their presence on social media:

CERCA centres accounts Twitter Facebook YouTube Klout Kred

1 IJC – Josep Carreras Leukemia Research Institute

2,344 44,758 245 50 730 3

2 i2CAT – Internet and Digital Innovation in Catalonia

1,503

44 638 4

3 CTFC – Forest Technology Centre of Catalonia

1,195 957 21 49 650 3

4 CREAF – Centre for Ecological Research and Forestry Applications

1,191

51 681 5

5 IPHES – Catalan Institute for Human Palaeoecology and Social Evolution

1,097 1,048 29 49 677 5

6 IGTP – Health Sciences Research Institute of the Germans Trias i Pujol Foundation

923

50 667 5

7 CRG – Centre for Genomic Regulation

905 449 58 50 675 6

8 IDIBELL – Bellvitge Biomedical Research Institute

777 234 3 52 685 5

9 IDIBAPS – August Pi i Sunyer Biomedical Research Institute

772 185

43 597 2

10 ISGlobal-CRESIB-Barcelona Centre for International Health Research

758 353

52 694 6

11 IMIM – Municipal Institute for Medical Research Hospital del Mar

744

24 44 583 3

12 VHIR – Vall d’Hebron Research Institute

734 301 62 47 644 3

13 ICCC – Catalan Institute of Cardiovascular Sciences

617 217

39 595 4

14 ICIQ – Institute of Chemical Research of Catalonia

499 428 12 45 600 4

15 IRB Barcelona – Institute for Research in Biomedicine

295 448 13 43 576 3

16 IRSI-CAIXA – Institute for AIDS Research

283

46 597 5

17 ICP – Catalan Institute of Palaeontology Miquel Crusafont

258 2,683 20 42 545 3

18 CVC – Computer Vision Center 252 93

43 577 4

19 ICFO – Institute of Photonic Sciences 228 197

45 619 3

20 IMPPC – Institute of Predictive and 70

31 434 2

Personalized Cancer Medicine

21 IC3 – Catalan Climate Sciences Institute

33 218

31 351 2

22 CTTC – Catalan Telecommunications Technology Centre

29 12 1 25 344 1

23 CMR[B] – Centre of Regenerative Medicine in Barcelona

21 21

52 0

24 IBEC – Institute for Bioengineering of Catalonia

265 7

25 CReSA – Centre for Animal Health Research

6

Table 4: Social Analytics for Institutional Twitter Accounts Provided by CERCA centres

As for the number of Twitter followers, the Josep Carreras Foundation, on which the Josep Carreras Leukaemia Research Institute depends, leads the account classification with 2.344 followers. At a certain distance, and above 1.000 followers, we find the i2CAT Foundation (Internet and Digital Innovation in Catalonia), the Forest Technology Centre of Catalonia and the Centre for Ecological Research and Forestry Applications.

Regarding the José Carreras Foundation, which also tops the rankings on Facebook (over 44.000 followers) and YouTube (with 245 subscribers), it should be noted that the Foundation probably generates a very significant number of emotional supporters, which may not occur in most other centres.

In the case of Facebook, 18 CERCA centres are present in this network. Apart from the aforementioned first position, the second one goes to the Catalan Institute of Palaeontology Miquel Crusafont, and the third is for the Catalan Institute for Human Palaeoecology and Social Evolution, both of them with over 1000 followers.

About YouTube, the top channels in number of subscribers correspond to the Josep Carreras Foundation, the Center for Genomic Regulation and the Vall d’Hebron Research Institute. As we can see, there is a wide variety of fields of knowledge regarding the top positions of the various social media.

As regards to reputation indices, the Bellvitge Biomedical Research Institute and the Barcelona Institute for Global Health (ISGlobal-CRESIB) lead the Klout ranking (Klout 52), and there are four centres over 50: the Centre for Ecological Research and Forestry Applications, the Center for Genomic Regulation, the Health Sciences Research Institute of the Germans Trias i Pujol Foundation, and the Josep Carreras Leukaemia Research Institute. Interestingly, when analyzing the Kred index substantial variations are not observed with respect to the centres that occupy the top six ranking positions of the Klout index, except for the entry of the Catalan Institute of Human Palaeoecology and Social Evolution into the Top 6, which moves the Health Sciences Research Institute of the Germans Trias i Pujol Foundation up to the seventh position.

Apart from the 47 CERCA centres, Catalonia has 21 centres from the Spanish National Research Council (CSIC), which are public state-owned agencies. Among these research centres, we wish to highlight the Artificial Intelligence Research Institute (IIIA), with 369 followers on Twitter, the Institute of Materials Science of Barcelona (ICMAB), with 306 followers, and the Institute of Robotics and Industrial Computing (IRII), with 144 followers.

Large research support infrastructures

Large research support infrastructures require large investments for their construction and maintenance, with the aim to advance cutting-edge experimental science. Catalonia has basically two major infrastructures: the Alba synchrotron light facility at the CELLS Consortium and the MareNostrum supercomputer at the Barcelona Supercomputing Center – Centro Nacional de Supercomputación (BSC-CNS).

These major research support infrastructures in Catalan territory are mainly consortia participated by the Government of Catalonia, the Spanish State and other organizations that take a minority stake. Apart from the two major infrastructures mentioned above, there are up to 10 other major research support infrastructures. Only five out of these 12 structures are present on social media as shown in Table 5:

Catalan large infrastructures accounts

Twitter Facebook YouTube Klout Kred

Barcelona Supercomputing Center (BSC-CNS)

385 236 11 45 597 4

Center for Scientific and Academic Services of Catalonia (CESCA)

158 45 0 31 523 3

Ebre Observatory 99 137

38 494 2

National Centre for Genomic Analysis (CNAG)

64

40 395 3

Montsec Observatory

940

Table 5: Social Analytics for Institutional Twitter Accounts Provided by Catalan Large Infrastructures

The Barcelona Supercomputing Center – Centro Nacional de Supercomputación (BSC-CNS) ranks first in the number of Twitter followers, while the Montsec Astronomical Observatory leads the Facebook network.

Reference networks of R&D and innovation

Reference networks of R&D and innovation consist of a series of groups from different institutions that carry out research and innovation projects, and other activities collaboratively. These groups have common goals and the networks aim at promoting collaborative, interdisciplinary and multidisciplinary work, as well as the optimization of infrastructure and R&D and innovation facilities in Catalonia. Four out of the eight reference networks are present on Twitter, as shown in Table 6, with the

Reference Network of R&D and innovation on Theoretical and Computational Chemistry leading the classification:

Catalan reference networks accounts Twitter Facebook Klout Kred

Reference Network of R&D&I on Theoretical and Computational Chemistry (XRQTC)

117

35 516 4

Catalan Biotechnology Reference Network (XRB) 112

41 498 4

Reference Network of R&D&I on Aquaculture (XRAq)

63 74 23 396 3

Reference Network of R&D&I on Food Technology (XaRTA)

39

17 220 1

Table 6: Social Analytics for Institutional Twitter Accounts Provided by Catalan Reference Networks

How Can We Measure the Reputation of a Research Network?

Is the number of Twitter followers a good indicator of the presence of an institution on the net? In my blog, I regularly analyze the presence of research structures in Catalonia on social media, based on the number of followers. I realized that this indicator may not be a sufficiently complete indicator, so I decided to introduce also the Klout and Kred indicators, in line with the analysis of Professor Miquel Duran, an expert in analyzing metrics in universities of the Catalan-speaking territories, and Brian Kelly, with his detailed analysis on the presence of the UK Russell Group universities (note the latter also includes indicators such as Peerindex or Twitalyzer). Both Klout and Kred provide complementary and useful information in order to assess the impact of bidirectional communication.

Klout is a social networking service that measures influence using data points from Twitter, such as the size of a person’s network, the content created and how other people interact with that content. This analysis is also done on data taken from Facebook, Google+, Linkedin, and other sites. Klout creates profiles on individuals and assigns them scores ranging from 1 to 100. Despite being criticized because of its opacity, this service has become quite popular and I believe it is a good complement.

Another interesting measure of influence is Kred. Unlike Klout, Kred provides a fully transparent view of the actions that compose any user’s score and it is the only influence measure to openly publish its algorithm. Kred’s scoring system, which is based on Twitter profiles, is composed of two scores: Influence measures a user’s ability to inspire action from others like retweeting, replies or new follows and it is scored on a 1000 point scale. Outreach reflects generosity in engaging with others and helping them spread their message and it is scored on a 10 point scale. Outreach score is cumulative and always increases, and it is measured on Twitter by your retweets, replies and mentions of others.

The Importance of Being Present on Social Media

In late May 2012, I gave a presentation at the University of Barcelona on dissemination using Web 2.0 (Com divulgar en el web 2.0 / Cómo divulgar en el web 2.0). The workshop aimed to provide tools and strategies for scientific knowledge dissemination to researchers and other agents linked to R&D and innovation system, so that they could be in a better position to spread the object of their research.

Although my remarks focused on the importance of having a blog and a Twitter account to disseminate research, I finally mentioned other instruments that could contribute to it such as repositories (Slideshare, YouTube, Flickr) or social networking tools (ResearchGate, LinkedIn, Google+). During the talk it was mentioned that, while Facebook has been considered a very suitable network for personal rather than professional purposes, currently a trend has been detected among young people to use this network to disseminate research. Therefore, it should be considered in future studies.

Twitter and Facebook are the social networking tools where research structures in Catalonia are mostly present, although with slightly different communication strategies. The centres use these media mainly to disseminate research and, in some cases, to make dissemination activities organized by the centre more widely available and to engage with the public, even as a teaching support. Moreover, these tools are often used to post vacancies at the institution. According to Raül Toran, science writer at the Bellvitge Biomedical Research Institute (IDIBELL), generally the topics that are forwarded mostly are the ones related to cutting-edge research and job vacancies at the institution.

Social media at the research centres are basically managed by the Communication Departments at the same institution although in most centres, some researchers use social media mostly for personal rather than professional activity. Cinta S. Bellmunt, Head of Communication at the Catalan Institute of Human Paleoecology and Social Evolution (IPHES) states that the researchers in the centre are aware of the value and visibility that social networks provide to research, because quite often I have to refer questions that arise in the group to them, so they realize there is movement, interrelation. To follow up on the impact of the communication strategy of the centre, tools such as Hootsuite and Tweetdeck are used.

A good example of the impact of the dissemination activity of social media by the centres is to be found in the increasing traffic to their websites, as well as an increase in the number of job applications for possible vacancies. Inevitably, this communication activity results in a continuous increase in the number of Twitter and Facebook followers. As far as readers’ preferences is concerned, it is highly variable and it depends on the centres: in the case of the IDIBELL, there is more interaction via Twitter (direct messages, mentions, RT, etc.) than via Facebook (Likes or comments), while in the case of IPHES the situation is reversed.

As regards privacy, there is growing awareness that knowledge must flow, but with precaution in order not to affect the privacy of others, respecting authorship and quoting the source of what is being communicated.

Open social network tools such as Diaspora or identi.ca, are still little known in Catalonia. In contrast, a growing increase in Catalan researchers in the ResearchGate network has been detected.

In summary, we could say that research structures in Catalonia are consolidating and increasing their presence on social media, especially on Twitter and Facebook, which has become part of their communication strategy, increasing visibility. To disseminate the research that is being carried out and to approach society are the main goals. In addition, communication units are progressively incorporating metrics tracking tools designed to evaluate and measure the impact of the communication activity and its benefits to the target audience. And good news is that research, often funded with public money, engages with the whole of society.

This article was published in Brian Kelly’s UK Web Focus blog as a guest post (thank you, Brian!) on 5th November 2012, and built on previous posts on this blog which had described use of social media in the UK higher education sector, including posts on Social Analytics for Institutional Twitter Accounts Provided by the 24 Russell Group Universities, Use of Facebook by Russell Group Universities and Links to Social Media Sites on Russell Group University Home Pages. The post aimed to provide a wider view on approaches to use of social media and evaluation of its impact beyond the UK.

The post was also published in the run-up to the Spot-On London (SOLO12) conference which included sessions on Assessing social media impact, Altmetrics beyond the Numbers and Using Twitter as a Means of Effective Science Engagement. The post was discussed in the workshop Assessing Social Media Impact by Alan Cann (University of Leicester) (video of the workshop).

Link to Catalan translation of the post