Skip to main content
  • Research article
  • Open access
  • Published:

Identification of sex-associated network patterns in Vaccine-Adverse Event Association Network in VAERS

Abstract

Background

Vaccines are one of the most important public health successes in last century. Besides effectiveness in reducing the morbidity and mortality from many infectious diseases, a successful vaccine program also requires a rigorous assessment on their safety. Due to the limitations of adverse event (AE) data from clinical trials and post-approval surveillance systems, novel computational approaches are needed to organize, visualize, and analyze such high-dimensional complex data.

Results

In this paper, we proposed a network-based approach to investigate the vaccine-AE association network from the Vaccine AE Reporting System (VAERS) data. Statistical summary was calculated using the VAERS raw data and represented in the Resource Description Framework (RDF). The RDF graph was leveraged for network analysis. Specifically, we compared network properties of (1) vaccine - adverse event association network based on reports collected over a 23 year period as well as each year; and (2) sex-specific vaccine-adverse event association network. We observed that (1) network diameter and average path length don’t change dramatically over a 23-year period, while the average node degree of these networks changes due to the different number of reports during different periods of time; (2) vaccine - adverse event associations derived from different sexes show sex-associated patterns in sex-specific vaccine-AE association networks.

Conclusions

We have developed a network-based approach to investigate the vaccine-AE association network from the VAERS data. To our knowledge, this is the first time that a network-based approach was used to identify sex-specific association patterns in a spontaneous reporting system database. Due to unique limitations of such passive surveillance systems, our proposed network-based approaches have the potential to summarize and analyze the associations in passive surveillance systems by (1) identifying nodes of importance, irrespective of whether they are disproportionally reported; (2) providing guidance on sex-specific recommendations in personalized vaccinology.

Background

Vaccines are one of the most cost-effective public health interventions to date, leading to at least 95–99 % decrease of most vaccine-preventable diseases in the United States [1]. While their benefits far overweigh their risks and costs, vaccines are accompanied with specific adverse events (AEs). Assessment of vaccine safety usually starts at the pre-approval stage, when information about AEs is collected during Phase I-IV of clinical trials. However, there are several limitations of such information. First, clinical trials usually have small sample sizes which are insufficient to detect rare AEs. Second, clinical trials are usually carried out in well-defined, homogeneous populations within relatively short follow-up periods, which may limit the generalizability of their effect in all populations. Therefore, the complete safety profiles associated with a vaccine cannot be fully established only through clinical trials. Post-approval surveillance of vaccine AEs is needed to assess the vaccine safety throughout its life on the market.

The Vaccine AE Reporting System (VAERS) is a passive surveillance system to monitor vaccine safety after the administration of vaccines licensed in the United States [2]. The VAERS is co-managed by the United States Food and Drug Administration (FDA) and the Centers for Disease Control and Prevention (CDC). By the end of 2013, the VAERS contains more than 200,000 reports in total, including 72 vaccine types and 7368 reported symptoms/AEs. However, there are several limitations we need consider in the analyses of spontaneous reporting systems such as VAERS, including lack of verification of reported diagnoses, lack of consistent diagnostic criteria for all cases with a given diagnosis, wide range of data quality, underreporting, inadequate denominator data, and absence of an unvaccinated control group [3]. To address some of these limitations, various data mining approaches have been developed to identify potential signals in the data [4]. Most of these approaches focus on disproportionality of reporting, which aims to identify conditions that comprise a larger proportion of reported events for a given vaccine, compared to other vaccines in the same reporting system [3]. However, such disproportionality methods still have difficulties to identify potential vaccine-AE associations due to the limitations of VAERS data. In Bate et al. 2009 [5], the authors suggested that a single drug-AE should be analyzed in the context of all drug-AE associations. Harpez et al. proposed a clustering approach to identify drug groups that were reported to have same AEs [6]. However, this approach didn’t account for all co-administered drugs and co-occurring AEs. Since VAERS receives more than 14,000 reports every year, there is a pressing need to develop novel approaches to organize these high-dimensional VAERS data and identify potential vaccine-AE associations.

In recent years, network analysis emerges as a very promising approach for simultaneous representation of complex high dimensional data. Specifically, these network-based computational approaches gained popularity and have become a new paradigm to investigate associations among biological entities (e.g., drugs, diseases, and genes). Applications of these approaches include drug repositioning [7, 8], disease gene prioritization [911], and identification of disease relationships [12, 13]. These network analysis approaches are usually developed based on the observations from real-world networks. First, most real-world networks (e.g., WWW network, protein-protein interaction network, and social network) are not randomly organized but are driven by preferential attachment and growth (e.g., some nodes have more connections than others). Such networks are called “Scale-free” networks. In the “scale-free” network, the most highly connected nodes are called “hub’ nodes. Second, most real world networks are modular, comprised of small, densely connected groups of nodes. Network analysis metrics and algorithms have been designed to identify network hub nodes and modules in a scale-free network. Ball and Botsis proposed a network-based approach to aid visualization of patterns in VAERS data that a medical expert might recognize as clinically important [14]. In our previous work, we developed a network analysis approach to dentify vaccine-related networks and their underlying structural information from PubMed literature abstracts, which were consistent with that captured by the Vaccine Ontology (VO) [15]. The modular structure and hub nodes of these vaccine networks reveal important unidentified knowledge critical to biomedical research and public health and to generate testable hypotheses for future experimental verification.

In this paper, we proposed a network-based approach to investigate the vaccine-AE association network from VAERS data. First, we extracted and represented data summarized from VAERS database using Resource Description Framework (RDF). We calculated overall proportional reporting ratio (PRR), yearly PRR and sex-specific PRR for each vaccine-AE association in the VAERS. We then applied a series of network approaches to the network consisting of significant vaccine-AE associations (i.e., PRR > 1). Specifically, we compared network properties of (1) vaccine-AE association network based on reports collected over a 23 year period as well as each year; (2) sex-specific vaccine-AE association network. We observed that (1) network diameter and average path length don’t change dramatically over a 23-year period, while the average node degree of these networks changes due to the different number of reports during different period of time; (2) vaccine-AE associations derived from different sexes show sex-associated patterns in sex-specific vaccine-AE association networks.

The rest of the paper is organized as follows. In Section Materials and methods, we introduce our methodology on data collection, summarization, representation, and analysis. In Section Results, we present the result of our study. In Section Discussions, we discuss the potential scientific contributions of this study. In Section Conclusions and future work, we conclude the paper and discuss future directions.

Materials and methods

In this section, we first describe the data resources and preprocessing method in this work. We then introduce our proposed network-based approach for investigating vaccine-related associations derived from VAERS. Figure 1 illustrates the steps of the proposed approach.

Fig. 1
figure 1

Overview of the proposed study. VAERS: Vaccine AE Reporting System; RDF: Resource Description Framework; VAE: Vaccine-AE; PRR: Proportional Reporting Ratio

VAERS database

We downloaded raw data from the VAERS system in comma-separated value (CSV) format (https://vaers.hhs.gov/data/data). All the data from 1990–2013 was loaded to a mySQL relational database for further processing. The VAERS database contains three tables: Data, Data sources and preprocessing Symptom, and Vaccine. The Data table contains general information about each report including VAERS report ID, date the report was received, the state patient was in, age and sex of the patient, and detailed description of the symptom (e.g., if the symptom was life threatening, if the patient in the report died and if-so the date of death, if the patient ever attend the ER for treatment, and if so, how many days was the patient administered at the hospital.). The Symptom table contains a list of symptom terms (MedDRA terms) involved in the report. Completed information about one report can be jointed from the three tables using VAERS ID. The Vaccine table includes information about the vaccine administered to the patient such as vaccine manufacturer, type of vaccine, dosage of the vaccine, vaccination route, vaccination site, and vaccination name.

Statistical summary of VAERS data

As we discussed above, the VAERS is a spontaneous reporting system which contains unverified reports with inconsistent data quality. Symptoms reported occurring after vaccination do not necessarily indicate a causality association with the vaccine. Therefore, we used statistical methods to summarize meta-level features of vaccine-symptom pairs. For each vaccine-symptom pair, we calculated the following features (1) the number of reports that contains the pair; (2) the number of reports that contains the pair each year; (3) the demographic distribution among the reports that contain the pair (total and yearly) grouped by gender and age groups; and (4) overall proportional reporting ratio (PRR) and yearly PRRs [16]. A PRR is the ratio between the frequency with which a specific symptom (e.g., AE) occurs for a vaccine of interest (relative to all symptoms reported for the vaccine) and the frequency with which the same symptom occurs for all vaccines reported to the VAERS (relative to all symptoms for all vaccines reported to VAERS) [3]. A PRR greater than 1 suggests that the post-vaccination symptom (AE) is more commonly observed for individuals administrated with the particular vaccine, relative to all other vaccines reported to the VAERS.

The overall PRR ratio of a vaccine (V) and a symptom (S) association was calculated by (Numreports for V that containsS /Numall thereports for V )/(Numtotalreports that contains S/Numtotal reports in VAERS).

The yearly PRR ratio of a vaccine (V) and a symptom (S) association in Year (Y) was calculated by (Numreports for V that contains Sin year Y/Numall the reports for V in Year Y )/(Numtotalreports that contains S in Year Y /Numtotalreports in VAERS in Year Y ).

The sex-specific PRR ratio or a vaccine (V) and a symptom (S) association in Gender (G) was calculated by (Numreports for V that contains S forpatient with G /Numall thereports for V for patient with G )/(Numtotalreports that contains S for patient with G /Numtotalreports in VAERS for patient with G ).

RDF conversion

The statistical summary introduced in the previous section was stored in a relational database and converted to the Resource Description Framework (RDF) format. We have introduced detailed information about how to represent vaccine symptom pairs with meta-information in RDF and our vision on linking heterogeneous vaccine-related data sets using linked data approach in our previous work [17].

Figure 2 shows the meta-level RDF graph representation of a vaccine symptom association. Each unique association (vaccine-symptom pair) has an unique identifier. The corresponding vaccine, symptom, demographic distribution, and PRR values are also represented in RDF. SPARQL queries can be conducted to retrieve useful information for network analysis which we will introduce in the next section.

Fig. 2
figure 2

Sample RDF graph representation of vaccine AE association

Network analysis

The analysis of network properties was performed using the “Network Analyzer” plugin in Cytoscape [18]. Cytoscape is an open-source platform for integration, visualization, and analysis of biological networks. Its functionalities can be extended through Cytoscape plugins. Scientists from different research fields have contributed more than 160 useful plugins so far. These comprehensive features allow us to perform thorough network-level analyses, visualization of our association tables, and integration with other biological networks in the future. In this study, the average node degree, average path length, and network diameter of one network was calculated.

The vaccine-AE association network is a bipartite network, which consists of interactions between two different types of nodes (X-type and Y-type), with edges connecting only nodes of different types. To calculate the similarity of one type of nodes (e.g., X-type nodes) based on their interactions with another type of nodes (e.g., Y-type nodes), the Pearson correlation coefficient (PCC) was employed as an association index. We assume that node A and B are X-type nodes, and the PCC between node A and B is calculated by

$$ PC{C}_{AB}=\frac{\left|N(A)\cap \mathrm{N}(B)|\cdot {\mathrm{n}}_y-\left|N(A)\right|\cdot \Big|N(B)\right|}{\sqrt{\left|N(A)\right|\cdot \Big|N(B)\Big|\cdot \left({n}_y-\left|N(A)\right|\right)\cdot \left({n}_y-\left|N(B)\right|\right)}} $$
(1)

where A and B are nodes of same type, N(A) and N(B) are their total number of interactions with A and B, N(A)∩N(B) is the total number of Y-type nodes that interact with both A and B, and n y is the total the total number of Y-type nodes in the network. A PCC of 1 indicates a perfect overlap, 0 corresponds to the number of shared interactors expected by chance and −1 depicts perfect anti-correlation.

The hierarchical clustering analysis is used to identify the similarities among vaccines using their association indexes. Both the heatmap of the dendrogram are used to visualize the clustering results. The clustering analysis and visualization of vaccine-AE association network was performed using the GAIN tool [19].

Results

Overview of the results

Overall, we extracted 2,346,367 vaccine-AE associations from the VAERS system, with 83,148 distinct associations. We defined that a vaccine-AE association is significant if PRR for this association is greater than 1. Among all vaccine-AE associations reported in the VAERS, we identified 277,698 vaccine-AE associations, 53,795 of which have overall PRR greater than 1 between 1990 and 2013. We also investigated yearly PRRs of these associations. For one specific year, we define that a vaccine-AE association is significant if the yearly PRR is greater than 1. Table 1 presents the numbers of significant associations for each year (N link Column).

Table 1 General characteristics of the networks

Based on the significant yearly or overall associations, we further investigated these association networks using different network properties. Table 1 presents the general characteristics of the overall association network as well as yearly-significant association networks, including average node degree, average path length and network diameter. This demonstrates that vaccine-AE network is dense network, with any given node connected to all other nodes through an average of approximately two other nodes and a maximum of 5–6 nodes. It is explained partly by that many vaccines are co-administered. However, given that there are more AEs than vaccines in the network, it is plausible that many AEs were reported together. Another interesting observation was that across 23 years, the average path length and network diameter for yearly vaccine-AE association networks don’t change dramatically. It indicated that in most cases, it is relatively common that two vaccines sharing one AE or two AEs associate with one vaccine in the network. On the other hand, the average node degree of these networks changes over time, partly due to the increasing number of reports received from 1990 to 2013 (Table 1). All the network information in Table 1 were presented in Additional file 1.

Different AE association patterns in different genders

We further investigated whether vaccine-AE associations are different between genders. We constructed sex-specific vaccine-AE association networks by computing the PRR based on reports only from female/male populations. There are 49,616 and 51,578 significant vaccine-AE associations (i.e., PRR > 1) in female and male populations, respectively. The network properties of these two sex-specific association networks are similar with overall association network (Table 1). We clustered the vaccines based on their association indexes calculated by their associations with AEs. In Fig. 3a and b, we observed different similarity patterns in female (Fig. 3a and male (Fig. 3b). For instance, HBHEPB, ROTH1, PNC13, DTAP IPVHIB, PNC, ROTHB5, DTAPHEPBIP, HIBV, DTAP, and IPV were clustered together based on their associations with adverse events in the female population. Besides most of the vaccines that were grouped in the female population, we also found four more vaccines in the same group in the male population, including PNC, HEP, VARCEL, and MMR. Similarly, while DIPHIB, DTP, and OPV were tightly clustered in the male population, RV was also grouped in this cluster in the female population. The dendrograms indicate the same differences between two populations (Fig. 3c and d). These results indicate that there are indeed sex-specific reponse differences after vaccine injection.

Fig. 3
figure 3

Comparison of vaccine similarity in different sexes. a Hierarchical analysis of vaccines based on association information in female reports; b Hierarchical analysis of vaccines based on association information in male reports; c Dendrogram of vaccine similarity in female reports; d Dendrogram of vaccine similarity in male reports

We also compared whether pairs of vaccines with similar association profiles in female-specific association network are also similar in male-specific network. In Fig. 4, the distributions of PCC indexes are different in two populations, indicating that there are some sex-specific associations in both populations, although majority of association relationships can be identified in both populations. Specifically, there were more vaccine pairs showing high similairties in the male population than in the female population. The underlying mechanisms need further investigation using other types of biological data, such as genomic, metablomic, and proteomic level measurement data.

Fig. 4
figure 4

Density plot of PCC association indexes in female vs male populations (red: female; blue: male)

Discussions

Most vaccine-preventable diseases have declined in the United States by at least 95–99 % [20, 21]. However, vaccines are pharmaceutical products that carry risks. Certain biomarkers or individual variations could implicate different vaccine responses, which are essential for precision medicine. Identifying these associations is critical to vaccine safety, which reassures public acceptance of vaccines. One way to address this question is the post-approval surveillance of vaccine AEs. For instance, the VAERS is a passive surveillance system to monitor vaccine safety after the administration of vaccines licensed in the United States [2]. Such surveillance data can complement the original safety evaluation data generated from the clinical trial phases and provide more comprehensive safety assessment in a much larger population. We are one of the first research groups that investigates sex-specific vaccine-AE association patterns by integrating traditional statistical signal detection and network analysis approaches. Our findings indicated that saftety signals present different patterns in female and male population. This is consistent with previous studies in the vaccine community [22, 23]. With high-throughput technology advances such as next generation sequencing, transcriptomics, epigenetics, proteomics, and new computational approaches to interpreting big data, we expect a better understanding of associations and mechanisms of vaccine AEs and immunogenicity. Network analysis approaches is one of the promising stratetigies to integrate such heterogeneous “big data”, leading to a more personalized or individual approach to vaccine practice in the near future.

Conclusions and future work

In this paper, we proposed a network-based approach to investigate the vaccine-AE association network from VAERS. The results indicated that (1) network diameter and average path length of vaccine-AE association networks don’t change dramatically over a 23-year period, while the average node degree of these networks changes due to the different number of reports during different period of time; (2) vaccine-AE associations derived from different genders show sex-associated patterns in sex-specific vaccine-AE association networks. To our knowledge, this is the first time that a network-based approach has been used to identify sex-specific association patterns in a spontaneous reporting system database. Due to unique limitations of such passive surveillance systems, network-based approaches have the potential to (1) identify nodes of importance, irrespective of whether they are disproportionally reported; (2) provide guidance on sex-specific recommendations in personalized vaccinology.

Extensions of this work include: (1) integration of other spontaneous reporting system databases (e.g., the European Adverse events following immunization (AEFI) system) to construct more complete vaccine-AE association networks; (2) incorporation of other complementary public databases such as Semantic MEDLINE [24]; (3) development of advanced network-based approaches taking the PRR values into account; (4) investigation of other types of data mining methods to assess the significance of vaccine-AE associations; (5) focused investigation of examples based on the network parameters; and (6) identification of sex-specific subnetwork patterns of AE correlation networks.

References

  1. Poland GA, Levine MM, Clemens JD. Developing the next generation of vaccinologists. Vaccine. 2010;28(52):8227–8.

    Article  Google Scholar 

  2. The U.S. FDA/CDC Vaccine Adverse Event Reporting System (VAERS) [http://vaers.hhs.gov/index]

  3. Banks D, Woo EJ, Burwen DR, Perucci P, Braun MM, Ball R. Comparing data mining methods on the VAERS database. Pharmacoepidemiol Drug Saf. 2005;14(9):601–9.

    Article  Google Scholar 

  4. Harpaz R, DuMouchel W, Shah NH, Madigan D, Ryan P, Friedman C. Novel data-mining methodologies for adverse drug event discovery and analysis. Clin Pharmacol Ther. 2012;91(6):1010–21.

    Article  Google Scholar 

  5. Bate A, Evans SJ. Quantitative signal detection using spontaneous ADR reporting. Pharmacoepidemiol Drug Saf. 2009;18(6):427–36.

    Article  Google Scholar 

  6. Harpaz R, Perez H, Chase HS, Rabadan R, Hripcsak G, Friedman C. Biclustering of adverse drug events in the FDA's spontaneous reporting system. Clin Pharmacol Ther. 2011;89(2):243–50.

    Article  Google Scholar 

  7. Arrell DK, Terzic A. Network systems biology for drug discovery. Clin Pharmacol Ther. 2010;88(1):120–5.

    Article  Google Scholar 

  8. Dudley JT, Deshpande T, Butte AJ. Exploiting drug-disease relationships for computational drug repositioning. Brief Bioinfo. 2011;12(4):303–11.

    Article  Google Scholar 

  9. Piro RM, Di Cunto F. Computational approaches to disease-gene prediction: rationale, classification and successes. Febs J. 2012;279(5):678–96.

    Article  Google Scholar 

  10. Kohler S, Bauer S, Horn D, Robinson PN. Walking the interactome for prioritization of candidate disease genes. Am J Hum Genet. 2008;82(4):949–58.

    Article  Google Scholar 

  11. Chen J, Aronow BJ, Jegga AG. Disease candidate gene identification and prioritization using protein interaction networks. BMC Bioinfo. 2009;10:73.

    Article  Google Scholar 

  12. Goh KI, Cusick ME, Valle D, Childs B, Vidal M, Barabasi AL. The human disease network. P Natl Acad Sci USA. 2007;104(21):8685–90.

    Article  Google Scholar 

  13. Suthram S, Dudley JT, Chiang AP, Chen R, Hastie TJ, Butte AJ. Network-based elucidation of human disease similarities reveals common functional modules enriched for pluripotent drug targets. PLoS Comput Biol. 2010;6(2):e1000662.

    Article  Google Scholar 

  14. Ball R, Botsis T. Can network analysis improve pattern recognition among adverse events following immunization reported to VAERS? Clin Pharmacol Ther. 2011;90(2):271–8.

    Article  Google Scholar 

  15. Zhang Y, Tao C, He Y, Kanjamala P, Liu H. Network-based analysis of vaccine-related associations reveals consistent knowledge with the vaccine ontology. J Biomed Semantics. 2013;4(1):33.

    Article  Google Scholar 

  16. Rothman KJ, Lanes S, Sacks ST. The reporting odds ratio and its advantages over the proportional reporting ratio. Pharmacoepidemiol Drug Saf. 2004;13(8):519–23.

    Article  Google Scholar 

  17. Tao C, Wu P, Zhang Y: Linked Vaccine Adverse Event Data Representation from VAERS for Biomedical Informatics Research. 18th Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD 2014): 2014; Tainan, Taiwan

  18. Smoot ME, Ono K, Ruscheinski J, Wang PL, Ideker T. Cytoscape 2.8: new features for data integration and network visualization. Bioinformat. 2011;27(3):431–2.

    Article  Google Scholar 

  19. Fuxman Bass JI, Diallo A, Nelson J, Soto JM, Myers CL, Walhout AJ. Using networks to measure similarity between genes: association index selection. Nat Methods. 2013;10(12):1169–76.

    Article  Google Scholar 

  20. Zhou F. Updated economic evaluation of the routine childhood immunization schedule in the United State. In: 45th National Immunization Conference; Washington DC. 2011.

  21. Poland GA, Ovsyannikova IG, Jacobson RM. Adversomics: the emerging field of vaccine adverse event immunogenetics. Pediatr Infect Dis J. 2009;28(5):431–2.

    Article  Google Scholar 

  22. French KM, Barnabas RV, Lehtinen M, Kontula O, Pukkala E, Dillner J, et al. Strategies for the introduction of human papillomavirus vaccination: modelling the optimum age- and sex-specific pattern of vaccination in Finland. Br J Cancer. 2007;96(3):514–8.

    Article  Google Scholar 

  23. Aaby P, Jensen H, Samb B, Cisse B, Sodemann M, Jakobsen M, et al. Differences in female–male mortality after high-titre measles vaccine and association with subsequent vaccination with diphtheria-tetanus-pertussis and inactivated poliovirus: reanalysis of West African studies. Lancet. 2003;361(9376):2183–8.

    Article  Google Scholar 

  24. Tao C, Zhang Y, Jiang G, Bouamrane M-M, Chute CG: Optimizing semantic MEDLINE for translational science studies using semantic web technologies. Proceedings of the 2nd international workshop on Managing interoperability and compleXity in health systems; Maui, Hawaii, USA. 2389683: ACM 2012: 53–58.

Download references

Acknowledgements

This project was supported by the National Cancer Institute grant P30 CA 134274–04 to the University of Maryland Baltimore, and the National Library of Medicine of the National Institutes of Health under Award Number R01LM011829 to C.T.

Author information

Authors and Affiliations

Authors

Corresponding authors

Correspondence to Yuji Zhang or Cui Tao.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

YZ and CT led the study design and analysis, and drafted the manuscript. PY extracted associations from VAERS database and performed statitistical analysis. YL converted the VAERS information into RDF format. All authors read and approved the final manuscript.

Additional file

Additional file 1:

Network file containing all the networks presented in Table  1 . (CYS 13422 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Y., Wu, P., Luo, Y. et al. Identification of sex-associated network patterns in Vaccine-Adverse Event Association Network in VAERS. J Biomed Semant 6, 33 (2015). https://doi.org/10.1186/s13326-015-0032-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13326-015-0032-2

Keywords