Paige M. Alfonzo B.A., M.S., Teresa J. Sakraida, PhD, RN and
Marie Hastings-Tolsma, PhD, CNM, FACNM
Alfonzo, P.M., Sakraida, T. J., & Hastings-Tolsma, M. (2014). Bibliometrics: Visualizing the Impact of Nursing Research. Online Journal of Nursing Informatics (OJNI), vol. 18(1), Available at http://ojni.org/issues/?p=3093
Nurse scientists commonly weigh the impact of their work on the discipline of nursing as well as within the larger healthcare arena. Bibliometrics, a statistical method used in citation and content analysis, is a quantitative approach for calculating output and for analyzing value and merit of scientific output. Bibliometric mapping is a method for visually representing bibliometric data. A synthesis between creative design and information visualization, bibliometric mapping highlights the impact of given research on a discipline and has the potential to foster increased data comprehension. Widely used in the field of information science, bibliometrics has received less attention in nursing and healthcare. This paper describes the methodological considerations for bibliometrics, software that could be considered for citation analyses, and an exemplar that shows the visual richness of bibliometric mapping. Recommendations are made for facilitating bibliometric analyses.
Keywords: scientometrics, bibliometric mapping, infographics, data visualization, nursing theory
Graphic visual representation of complex information or data so that it can be easily and quickly understood is the focus of infographics or visual network analysis (Smiciklas, 2012). A relatively new field of study, infographics involves both design and narrative (Franchi, 2013) while masterfully connecting the viewer to the data (Spiegelhalter, Pearson, & Short, 2011) making large datasets or datasets that are more difficult to grasp more readily consumable (Vincent, Hastings-Tolsma, & Effken, 2010). One type of infographics, known as bibliometric mapping offers a means to evaluate academic output as publication and citation information for parameters of a particular field using statistical methods (Moed, 2005; Van Raan, 2004). This paper specifies methodological considerations for bibliometric mapping, identifies strengths and limitations, and offers recommendations for use in nursing science, nursing informatics, and nursing education. A novel exemplar shows the visual richness of bibliometric mapping.
Academic libraries have services for information gathering and curriculum support, and newer trends of research support that include bibliometric support and data management for a global research knowledge base (Corrall, Kennan, & Afzal, 2013). While citation analysis has been around since the 1960s with development of the Science Citation Index, early work was performed manually and precluded determination of findings to the scale of the visualization (Reed, 1995). Originally targeted to identify relationships between citations, analysis of citations has expanded to support the analysis and assessment of research output at individual, departmental, and institutional levels. Coined by Pritchard in 1969, bibliometrics has most frequently been used in information and library science though there is wide utility in nursing and for other disciplines (Pritchard, 1969).
The combined use of methodologies that give information on different aspects of scienti?c output is generally recommended (Van Leeuwen, Visser, Moed, Nederhof, & Van Raan, 2003). Thus, bibliometric methods are found in a research evaluation toolkit that includes case study analysis, peer review, economic rate-of-return analyses and surveys, and consultations (Ismail, Edward, Sonja, & Grant, 2009). Bibliometrics offer informative insights along four dimensions of measurement (Narin, Olivastro, & Stevens, 1994): (1) scientific activity – article counts to show the volume of outputs in a given research field; (2) knowledge transfer – meaning that the citation process reflects the communication of knowledge within the scientific community and provides an indirect measure of research quality; (3) linkage – assessment of links between individuals and research fields to indicate the social and cognitive networks of scientific research; and, (4) citation analysis – as a proxy for one or more dimensions of the quality of scientific output.
Bibliometrics moves beyond weighing publication volume as the strict definition of productivity (e.g. h-indices, single indicator of quantity and impact of scienti?c output) to demonstrating the outcomes from a highly selective publication approach, scientific collaboration, and interdisciplinary team science (Ismail et al., 2009). As such, bibliometrics is meaningful for early career researchers since their effort may have reduced publication volume.
Bibliometrics offers several opportunities that support research and scholarship purposes. For example, research impact measures for investigator profile statements in grant applications, output comparisons for benchmarking with peer groups of other institutions (Ball & Tunger, 2006), observing accomplishments in science (Heinze, 2013), finding the university’s most published papers, and using quality indicator calculation to trend scientific topics over time, for the most cited researchers (Corrall et al., 2013; Costas & Bordons, 2007), and analyzing publishing patterns and usage data to inform an institution of scholarship output to promote planning (Amos et al., 2012). The outcomes of science may be readily mapped to observe state of the science shifts to translational research (Yao, Peng-Hui, Ma, Yao, & Zhang, 2013) and to evaluate the impact of scholarly work by individuals and research teams (Ismail et al., 2009; Yao et al., 2013). For shaping career development, bibliometric methods can support job applications and map output for publication planning (Ismail et al., 2009). A visual bibliometric profile of achievements for promotion and tenure, referred to as tenure metrics fits well with the growing trend of an e-dossier (Hendrix, 2010).
Bibliometrics augments nursing’s ability to showcase its works of scholarship in research, practice and education. Bibliometric analysis and mapping has been used to observe the impact of nursing publications as contributions to the knowledge gained in clinical practice (Goode et al., 2013), mapping research utilization literature (Estabrooks, Winther, & Derksen, 2004), conducting citation analysis of maternal-child literature (Oermann, Blair, Kowalewski, Wilmes, & Nordstrom, 2007), and noting the articles most frequently cited in nursing (Wong, Tam, Wong, & Cheung, 2013). Incorporating bibliometric methods into nursing’s science agenda is crucial to demonstrating field impact.
The bibliometric methodology requires developing a business-like plan that considers critical factors of clear project aims and task timeline (Sakraida, D’Amico, & Thibault, 2010). The plan should include an orientation of bibliometric mapping for all team members. Table 1 displays elements to consider in planning for bibliometric mapping. These elements include partnership with a librarian, clear identification of the goal and steps for conducting citation and content analysis, and identifying the type of bibliometrics to be used and the requisite software for mapping and analyzing the data.
Academic librarians are essential to the success of bibliometric projects. Their evolving role continues to expand to include application expertise in infographic programs and system programming (Corrall et al., 2013). The librarian can guide the researcher to the best database that fits the projects aims, assist with keywords and the selection of other search parameters such as Boolean operators and truncation additions. When establishing the partnership, the initial discussions are commonly focused upon the project scope. Discussion topics include using select databases for the citation search, publication type, timeframe, and geographic parameters. A trial run to test the selected search parameters may be indicated.
Information visualization offers the opportunity to broaden the exposure of research findings. The visual makes information much more understandable. As such, information visualization like bibliometric mapping considers the concepts of aesthetics and design in the viewer experience and seeks to stimulate thought about the substantive content as a wider issue. This notion of design appeal has the potential to engage the viewer, making the information more understandable. Careful detailing of the analysis is required to explain the selected data parameters that are being visually emphasized through use of color, word size, circle size, and circle proximity.
Bibliometric mapping is an approach with a distinct emphasis that differs from traditional statistical graphics. A larger number of variables can be included with bibliometric mapping through use of proximity (x, y axis), color, size, and labeling, than can be demonstrated with traditional statistical graphics. Traditional graphics provide an overview of the data, conveying a sense of scale and dataset complexity, and flexible display (e.g. pie charts, bar graphs) (Gelman & Unwin, 2013). Statistical graphs show effective and precise display of numerical data in order to understand patterns in an applied problem, but may have less visual appeal (Gelman & Unwin, 2013). From a statistical perspective an infographic display can specify its numerical foundation with other statistical displays, e.g. a frequency table.
There are several steps involved in conducting citation and content analysis (see Table 1). Bibliometric analysis first requires identification of a clear focus of investigation. This focus might be an author, research report, or topic such as physical activity. After a focus is determined, decisions will need to be made regarding search strategy and which databases will be trolled to cull relevant citations (e.g., keywords, title, database, subject or classification). Once relevant citations have been identified, bibliometrics can be used to analyze the data. Once bibliometric calculations have been performed they can then be converted into mapping files to visualize the relationships that have been targeted. This could include who has published on a particular topic and the country or university where the publication emanated, chronology, and the impact of the work. Further, key authors can be identified, as well as which scholarly works have been of greatest significance as determined by secondary referencing and number of citations. Finally, researchers can use bibliometric mapping to ascertain how the work of different researchers covaries. Once citations have been identified, analysis can be conducted using one of several bibliometric software packages.
Citation analyses has become more accessible to researchers due to the availability of a variety of software packages, many of them freeware or open source (see Table 2). With use of these software packages, bibliographic information can be prepared and measured for impact analysis and to trend scientific topics over time (Ismail et al., 2009). This information includes author(s), titles, dates of publication, references, author(s) affiliated institution, among others. Once the desired data has been selected, it must be prepared and calculated in order to create a bibliometric map. Research Information System (RIS) file formats used in citation management software programs can be saved as a text file format for analysis. Text file formats are recognized by most bibliometric software programs preparing data for ease of import and analysis. Once data have been calculated, mapping software can be used to convert it into files that can be mapped for visualization.
Bibliometric mapping allows for the representation of information in ways which make relationships more obvious and easier to understand and can lead to new insight and discovery. While there are a variety of maps which could be created, the most common bibliometric maps implement multidimensional scaling to graphically represent networked relationships with large datasets. The map includes several different components including nodes (circles), node weight (size), node networked relationship clustering (color and proximity), and label (text). Nodes as the concepts of interest are displayed as colored circles from smaller (less in frequency) to larger sizes (greater in frequency). The networked relationship shows the linkages between nodes either by proximity on the x, y axis or by a drawn line. The node color indicates the cluster or group with which they are associated. Clustering shows the dimension of similarity to other nodes in the display. Networks that show node linkage by proximity place nodes with the strongest linkage in close proximity to demonstrate the strongest relationships. Nodes found in more than one networked relationship can be shown in the same color to demonstrate additional linked relationships. Clustering can also be used to represent a variable independent of a network relationship such as co-authorships, year, etc. Each node is given a label derived from the text content being analyzed. Labels become larger as the size of the node increases and becomes smaller as the size of the node becomes smaller. Visually, the size of the node shows impact (the bigger the node the more impact).
VOSviewer (http://www.vosviewer.com/), a bibliometric program, uses proximity to show relationships between nodes. Nodes that are closer demonstrate a stronger relationship; the farther apart, the weaker the relationship. VOSviewer also has zooming capabilities and combined with proximity relationships help combat node and label overlap of large datasets. Other bibliometric mapping software, like Pajek (http://pajek.imfm.si/doku.php) and SPSS (IBM, Inc.), use lines between nodes to denote a direct relationship. Pajek can also show this relationship by selecting a map layout much like the one in VOSviewer. However, drawn lines and no zoom capability can result in overlap that can deter from the aesthetic aspects of the map and interpretation of the data (Van Eck & Waltman, 2007).
To demonstrate use of bibliometric mapping in nursing science, a project was conducted with the purpose of identifying the use of a select nursing theory (Roy’s Adaptation Model) in research conducted by masters and doctoral nursing students between 2000 and 2014. The Roy Adaptation Model is a grand theory developed in 1970 by Roy (Roy & Andrews, 2008). This project serves the discipline by identifying the impact of one nursing theory on research studies conducted by beginning nurse scientists.
The inclusion criteria for citation selection entailed the following: (1) Material type: doctoral dissertations; (2) Location: dissertations/theses from the United States (U.S.); (3) Institution: school/college of nursing affiliation; (4) Dissertation/theses type: related to the nursing profession, nursing practice, or nursing education; and (5) Dissertation/theses topic: related to nursing with Roy’s Adaptation Model specified in the dissertation/thesis abstract. Excluded were doctor of nursing practice projects/studies.
To achieve the project primary aim, the ProQuest Dissertation Abstracts International database was selected after consultation with the academic librarian, since it houses virtually all dissertations and theses from the U.S. The use of the ProQuest database offers a novel perspective. To date there are no known nursing publications that have used this database nor are there nursing reports of bibliometric mapping using the bibliographic coupling/coauthor analysis of cited references.
In consultation with an academic librarian, the research team learned the nuances of the database and how to conduct an advanced search using the selected key terms of Roy, Adaptation Model, and dissertation. Selecting the option of an advanced search using the key terms related to Roy’s Adaptation Theory, a total of 111 citations were found in the nursing classification. Each citation was reviewed to confirm use in dissertation/thesis research and institutional affiliation in the U.S. One citation was excluded since it was dissertation research from another discipline. A final total of 110 citations were entered into a spreadsheet for descriptive analyses to report frequency and percentages.
Data was prepared using Bibexcel; a freeware program. Bibexcel was chosen because it is specifically designed for bibliometric calculations and analyses. Bibexcel accepts multiple file format importation including RIS files allowing for flexibility of database choice. Bibexcel allows for versatility of file types to accommodate importing synchronicity with statistical programs like SPSS and Excel and bibliometric mapping software, e.g. VOSviewer and Pajek. Table 2 shows examples of the various programs or toolsets available for data preparation for bibliometric analysis.
For data to be calculated in a statistical software program it must first be converted into the required file types. Databases like Web of Science (ISI/Thompson) and Scopus (Elsevier) index bibliographic data, including cited references, and provide the option to export file formats as text files making it easier to import into a statistical software of choice. In these databases network comparison calculations can be performed that allow for direct import into bibliometric mapping software. Although other databases have an option for exportation as a text file, only a few have the cited references indexed and even fewer have the functionality to export the cited references. If this data is not available for export it must be manually modified in a text file according to the statistical program’s specific XML requirements. Manual coding can be very time consuming, thus making databases like Web of Science and Scopus desirable. Bibexcel prepares Web of Science and Scopus files as well as RIS, XML, Winspirs/Silverplatter, and EI files.
From ProQuest to Bibexcel Outfile. Selected data were exported from ProQuest as an RIS file and imported into Bibexcel for preparation. While there are databases that allow for exporting of citations that includes cited references, ProQuest does not have this function so manual coding of cited references was required. The RIS file was opened as a text file and cited references were manually entered into a cited reference (CD) field for each dissertation entry and this met the required XML format for Bibexcel. The fields come from Thompson Reuters Science Citation Index. Only cited references authored by Roy or that mentioned Roy or Adaptation Model in the journal article title, journal title, or book title were coded. Once the cited references were coded, the RIS file was imported into Bibexcel and then converted into a DOC file. (Select Misc menu. Next select convert to dialog format and lastly select convert from RIS format). The DOC file was then edited to the required XML format for Bibexcel and converted into a TX2 file (Select Edit doc file menu, then select Replace line feed with carriage return). An OUT file was then created from the TX2 file by selecting the TX2 file then clicking on Select field to be analyzed and choosing Any;separated field, typing CD into the Old Tag field, then clicking prep. The OUT file is vital to the mapping process since it performs all the bibliographic calculations (see Table 3).
From OUT File to CIT File. The first calculation performed was a frequency distribution. The frequency distribution was used to calculate the co-occurrence of the cited references and form the node size/weight in the bibliometric map. To calculate the frequency distribution the OUT file was selected and under the Frequency distribution list the item whole string was chosen. Next the Sort descending box was checked, and then Start was clicked. This resulted in a CIT file.
From CIT to co-occurrence (COC) file. The CIT file records the frequency of each cited reference entry (see Table 3). The files that had a minimum frequency of two were highlighted in the CIT file. With those files selected in The List box, co-occurrence then select units via list box was selected from the Analyze menu. This action deleted all the unwanted files (files with a frequency of one). With the selected entries viewable in The List box the OUT file was selected and then co-occurrence was clicked with make pairs via listbox chosen from the Analyze menu. This resulted in a COC file that included the number of times the references were cited with another reference (see Table 3). The COC file shows networked relationships in the bibliometric map.
From bibliometric calculation file to map files. Map files were created from the bibliometric calculation files. For this exemplar a NET and VEC file were created. The NET (network) file was created by selecting the COC file, and selecting Create net file for Pajek, VOSviewer, Mapequation, NetDraw, Unicet, etc from the Mapping menu (Note: Pajek is another bibliometric mapping program). The VEC (vector) file was created by selecting the CIT (reference frequency) file and selecting Create VEC file from the Mapping menu. Once the data were prepared and calculated, it was imported into VOSviewer for visual creation and analysis.
Bibliometric analysis showing distance based mapping was applied using VOSviewer; a freely available software program. Unlike most programs that are used for bibliometric mapping, “VOSviewer pays special attention to the graphical representation of bibliometric maps” (Van Eck & Waltman, 2009, p. 1). The functionality of VOSviewer is especially useful for displaying large bibliometric maps in an intuitive way with networked relationships represented by proximity. The variables for bibliographic mapping included frequency of each cited reference (taken from the VEC file in Bibexcel), network of cited references (taken from the NET file in Bibexcel), and link strength (calculated from the NET file in VOSviewer).
Bibliometric maps were created in VOSviewer by selecting Create under the Action tab, and selecting Create a map based on a network option. Next, the Pajek tab was selected and the NET and VEC files were uploaded. Link strength was calculated in VOSviewer from the NET file for cluster analysis. Cluster analysis grouped cited references with the strongest linkage. This added another dimension to the network analysis such that cited references with the strongest co-citation were mapped in closer proximity with each other. All cited references that were cited with another reference (co-citation) were mapped in the same cluster. Clusters were grouped by color.
In VOSviewer the map is interactive allowing the ability to hover over each node for cited reference specifics, the ability to zoom, and click and drag capabilities. The generated map can be saved as a map file and network file in VOSviewer. The files can be easily opened as a txt file in a plain text editor program and modified to change node labels, weight, and cluster. VOSviewer has four map modes: Label View, Density View, Cluster Density View, and Scatter View. For this exemplar Label View was chosen (see Figure 1). This view was selected as it displayed the concepts being examined in the co-citation analysis in the clearest way.
VOSviewer has the option to create a map from a text corpus. A text corpus is any group of text saved into a TXT file. A text corpus can consist of a portion of a document, such as the abstract, subject headings, or keywords or the entire document itself (Van Eck & Waltman, 2009). This is helpful in determining frequently used noun phrases within abstracts or full text articles without having to be manually performed or rely on expert judgment which can be subjective (Van Eck, 2011). VOSviewer selects noun phrases that consist of “more than one word… and removes unimportant adjectives such as first, many, new and some” (Van Eck, 2011, p. 31). For this exemplar a term map was created from the previously selected dissertation abstracts. The abstracts were extracted from the RIS file into a blank TXT file to create a term map. From the Action tab Create was selected. Create a map based on a text corpus was chosen and the TXT file was uploaded. A threshold of five was selected, meaning that each noun phrase within the text corpus had to appear at least five times in order to appear in the map. With the relatively small number of citations, a threshold of five was deemed adequate to display the co-word analysis. Of the 3611 terms found in the co-word analysis, 223 met the threshold of five, and 133 were deemed relevant in VOSviewer. VOSviewer uses a specific technique for selecting the most relevant noun phrases. “For each noun phrase, the distribution of (second-order) co-occurrences over all noun phrases is determined. This distribution is compared with the overall distribution of co-occurrences over noun phrases. The larger the difference between the two distributions (measured using the Kullback-Leibler distance), the higher the relevance of a noun phrase” (Van Eck & Waltman, 2011, pp. 1–2). Terms were then mapped by occurrence frequency and cluster and proximity network relationship (see Figure 2). Proximity and clustering of terms reflects their relatedness, with “each cluster being seen as a topic” (Van Eck & Waltman, 2011, p. 2).
The dissertations and theses citations (N =110) selected for analysis included doctoral dissertations with degrees of PhD, DNSc, DNS, and DSN (n = 84; 76%), masters theses with degrees of MS and MSN (n= 21; 20%), and final doctoral dissertations with degrees of EdD (n=5; 5%). The dissertation and master theses were comprised of the following categories: practice (n = 97; 88%), profession/system leadership (n=7; 6%) and nursing education (n=6; 5%).
The bibliometric mapping of the 110 citations that comprised the text corpus (see Figure 1) identified The Roy Adaptation Model: The Definitive Statement as being the most frequently cited reference (n= 47) and this reference was most commonly co-cited with Introduction to Nursing: An Adaptive Model. For the co-citation analysis, 75 citations met the criteria of having at least one co-citation and were included in the analysis.
Figure 2 displays the bibliometric mapping of concepts identified adaptation model (largest yellow circle) as the most frequently mentioned phrase followed by the concepts of difference, social support, and stress. Frequency figures can be viewed by hovering over the node in the interactive map (see Figure 2).
The exemplar in this bibliometric mapping project demonstrated the frequency of dissertation/thesis citations using the Roy’s Adaptation Model that was supplemented with co-citation analysis. Further, the mapping demonstrated the conceptual impact of the model.
Note: This map demonstrates the network of the Adaptation Model. The larger nodes (in violet) have the greatest impact and are found in close proximity to the major Adaptation Model node (in yellow), demonstrating similarity and a stronger relationship. Nodes with the same color fall into a common cluster. For the interactive version go to: http://www.vosviewer.com/vosviewer.php?map=http://library.umhb.edu/palfonzo/TermMap2.txt&network=http://library.umhb.edu/palfonzo/TermMap2NetworkMap.txt
The ProQuest database includes theses and dissertations from other disciplines that were also classified as nursing. It is also possible that dissertations which used Roy’s Adaptation Theory were missed particularly where keywords relevant to this study were not identified and/or the author’s abstract failed to mention Roy’s work. A final limitation is the temporal nature of this analysis. Bibliometric mapping provides an estimate of the value of the research to the field but the impact is limited to a specified point in time.
A systematic approach in bibliometric procedure is indicated for careful and accurate citation search and it is essential to partner with an academic librarian to select the databases that best suit the project. Not all databases use the same taxonomy nomenclature i.e. MeSH vs. CINAHL. Since keywords are commonly selected by authors at the time of manuscript submission, it is crucial for ease of citation retrieval in bibliometric analyses to consult with a librarian about keywords prior to submission.
Careful interpretation is indicated in bibliometric mapping. The figure requires understanding the meaning of the nodes and colors. When used for the purpose of tenure metrics, it is important to examine closely the bibliometric mapping presentation of citation impact. For instance, a widely cited research document may be cited for its negative impact to the literature. When using bibliometric mapping for career development and promotion, plan to also show the impact of scholarship with comparative data.
Bibliometric mapping is aesthetically pleasing to the visual senses and it jolts a cognitive response. One strength of this mapping is the immediate invitation to view the larger impact of and context of the subject of interest. The graphical display focuses your attention, evokes greater awareness, and inspires thinking about the subject, however this strength of seeing the greater picture does not necessarily facilitate a deeper understanding of the data (Gelman & Unwin, 2013). For this reason, bibliometric mapping procedure must be carefully documented with consideration given to presenting relevant data fairly (Gelman & Unwin, 2013). It is recommended that supplemental tables and graphical statistical displays (e.g. line charts, pie charts, etc.) be provided for greater explanatory detail.
Education programs today need to prepare students for newer ways of presenting data as necessary to their future work (Murtonen, Olkinuora, Tynjälä, & Lehtinen, 2008). We recommend that undergraduate and graduate curriculum designs be updated purposefully to promote long term retention of learning and transfer to situations (Halpern & Hakel, 2003; Walker, Golde, Jones, Bueschel, & Hutchings, 2008). Since nursing informatics emphasizes a core area focused on research methodologies to disseminate new knowledge into practice, it is important to consider bibliometrics as a useful exemplar. Specifically, curriculum should include how to use bibliometric analysis effectively and present data as figurative displays. Curriculum should also include a library orientation about databases and publication with attention to keyword selection. And lastly, curriculum should mandate use of a citation manager to organize references in preparation for publication and for the prospect of bibliometric mapping.
The expected use of static and grayscale or black and white graphics is a common limitation for displays in publication. As a means to overcome these limitations, journals with an online presence now offer the opportunity to link out to a figure/table or to a supplemental site. Such supplemental sites offer the opportunity for dynamic, interactive bibliometric mapping displays. It is timely given the ability to create vivid and colorful figures to render a call to all publishers to adjust their display requirements to accommodate the newest graphics like bibliometric mapping.
Bibliometric mapping offers an innovative and exciting strategy for presenting the impact of nursing research both within the field, as well as the larger healthcare arena. Documentation of such impact has the potential to make explicit the meaning and impact of work by nurse scientists Use of these techniques can serve to promote the development, design, and implementation of communication and information technology, thereby expanding vision and management in nursing informatics (Nursing Informatics, 2014).
It has often been said that “a picture is worth a thousand words.” Visualization of the data allows for invention of visual metaphors to present the targeted data. Such infographics have been referred to as infosthetics – the beauty of data visualization where form follows the data (Vande Moere, 2007). At a time when there is an explosion of knowledge, creative strategies to demonstrate use and impact of data are essential.
The authors would like to thank Benjamin Harnke, MLIS, Reference Librarian at the University of Colorado Denver Anschutz Medical Campus for his support and encouragement in the preparation of this work.
Amos, K., Mower, A., James, M. A., Weber, A., Yaffe, J., & Youngkin, M. (2012). Exploring publishing patterns at a large research university implications for library practice. Evidenced Based Library and Information Practice, 7(3), 32-50.
Ball, R., & Tunger, D. (2006). Bibliometric analysis- a new business area for information professionals in libraries? Support for scientifc research by perception and trend analysis. Scientometrics, 66, 561-577.
Corrall, S., Kennan, M. A., & Afzal, W. (2013). Bibliometrics and Research Data Management Services: Emerging Trends in Library Support for Research. Library Trends, 61(3), 636-674. doi: 10.1353/lib.2013.0005
Costas, R., & Bordons, M. (2007). The h-index: Advantages, limitations and its relation with other bibliometric indicators at the micro level Journal of Informetrics, 1, 193-203.
Franchi, F. (2013). Infographics [Blog post]. Retrieved from http://www.francescofranchi.com/projects/infographics
Gelman, A., & Unwin, A. (2013). Infovis and Statistical Graphics: Different Goals, Different Looks. Journal of Computational and Graphical Statistics, 22(1), 2-28. doi: 10.1080/10618600.2012.761137
Goode, C. J., McCarty, L. B., Fink, R. M., Oman, K. S., Makic, M. F., Krugman, M. E., & Traditi, L. (2013). Mapping the organization: a bibliometric analysis of nurses’ contributions to the literature. The Journal of Nursing Administration, 43(9), 481-487.
Halpern, D., & Hakel, M. (2003). Applying the science of learning to the university and beyond: Teaching for long-term retention and transfer. Change, 35(4), 36-41.
Heinze, T. (2013). Creative accomplishments in science: definition, theoretical considerations, examples from science history, and bibliometric findings. Scientometrics, 95(3), 927-940. doi: 10.1007/s11192-012-0848-9
Hendrix, D. (2010). Tenure metrics: bibliometric education and services for academic faculty. Medical Reference Services Quarterly, 29(2), 183-189.doi: 10.1080/02763861003723416
Ismail, S., Edward, N., Sonja, M., & Grant, J. (2009). Bibliometrics as a tool for supporting prospective R&D decision-making in the health sciences: Strengths, weaknesses and options for future development. Santa Monica, CA: RAND Corporation. Retrieved from http://www.rand.org/pubs/technical_reports/TR685
Moed, H. F. (2005). Citation analysis in research evaluation. Dordrecht: Springer.
Murtonen, M., Olkinuora, E., Tynjälä, P., & Lehtinen, E. (2008). “Do I need research skills in working life?”: University students’ motivation and difficulties in quantitative methods courses. Higher Education, 56(5), 599-612. doi: 10.1007/s10734-008-9113-9
Narin, F., Olivastro, D., & Stevens, K. A. (1994). Bibliometrics/theory, practice and problems. Evaluation Review, 18(1), 65-76.
Nursing informatics (2014). Retrieved from http://www.amia.org/programs/working-groups/nursing-informatics
Oermann, M. H., Blair, D. A., Kowalewski, K., Wilmes, N. A., & Nordstrom, C. K. (2007). Citation analysis of the maternal/child nursing literature. Pediatric Nursing, 33(5), 387-391.
Pritchard, A. (1969). Statistical bibliography or bibliometrics? Journal of Documentation, 25(4), 348-349.
Reed, K. L. (1995). Citation analysis of faculty publication: Beyond Science Citation Index and Social Science Citation Index. Bulletin of the Medical Library Association, 83(4), 503-508.
Roy, C., & Andrews, H. A. (2008). The Roy Adaptation Model (3rd ed.). Upper Saddle River, NJ: Prentice Hall.
Sakraida, T. J., D’Amico, J., & Thibault, E. (2010). Small grant management in health and behavioral sciences: Lessons learned. Applied Nursing Research, 23(3), 171-177. doi: 10.1016/j.apnr.2008.06.006
Smiciklas, M. (2012). The Power of Infographics Retrieved from http://ptgmedia.pearsoncmg.com/images/9780789749499/samplepages/0789749491.pdf
Spiegelhalter, D., Pearson, M., & Short, I. (2011). Visualizing uncertainty about the future. Science, 333(6048), 1393-1400. doi: 10.1126/science.1191181
Van Eck, N. J., & Waltman, L. (2007). Bibliometric mapping of the computational intelligence field. International Journal of Uncertainty, 15(5), 625-645.
Van Leeuwen, T. N., Visser, M. S., Moed, H. F., Nederhof, T. J., & Van Raan, A. F. J. (2003). The holy grail of science policy: Exploring and combining bibliometric tools in search of scienti?c excellence. Scientometrics, 57(2), 257-280.
Van Raan, A. F. J. (2004). Sleeping beauties in science. Scientometrics, 59, 467-472.
Vande Moere, A. (2007). Information Aesthetics. [online]. . http://infosthetics.com/
Vincent, D., Hastings-Tolsma, M., & Effken, J. (2010). Data visualization and large nursing datasets. Online Journal of Nursing Informatics (OJNI), 14(2). http:ojni.org/14_2/vincent.pdf
Walker, G. E., Golde, C. M., Jones, L., Bueschel, A. C., & Hutchings, P. (2008). The formation of scholars:Rethinking doctoral education for the twenty-first century. San Francisco, CA: Jossey-Bass.
Wong, E. L. Y., Tam, W. W. S., Wong, F. C. Y., & Cheung, A. W. L. (2013). Citation Classics in Nursing Journals: The Top 50 Most Frequently Cited Articles From 1956 to 2011. Nursing Research, 62(5), 344-351. doi: 10.1097/NNR.0b013e3182a2adff
Yao, Q., Peng-Hui, L., Ma, F. C., Yao, L., & Zhang, S. J. (2013). Global informetric perspective studies on translational medical research. BMC Medical Informatics and Decision Making, 13, 77. doi:10.1186/1472-6947-13-77.
There are no conflicts of interest.
Paige is Reference and Instruction Librarian at the University of Mary Hardin-Baylor in Belton, Texas where she provides research instruction to doctoral students and faculty, among others, utilizing a range of advanced Web applications. Her scholarship emphasizes strategies to promote advanced digital literacy skills among students and faculty. She has conducted interdisciplinary research related to the use of data visualization software.
Teresa is an Assistant Professor at the University of Colorado Denver, College of Nursing with educational expertise in public health, health policy, and health promotion. Dr. Sakraida studies transitions in disease burden and altered self-management by patients with type 2 diabetes and chronic kidney disease and employs tailored intervention for self-management support. Her interest in infographics and informatics led to collaborative projects that explore the impact of scientific and theoretical work in nursing.
Marie is a Professor at the University of Colorado Denver where she teaches in the nurse midwifery and doctoral nursing programs, as well as practices in the faculty nurse midwifery full-scope practice. Her research examines nurse midwifery outcomes where she has examined large datasets and data visualization methods. She has conducted research on data visualization with Ms. Alfonzo and Dr. Sakraida.