Google Scholar and the Continuing Education Literature

Howland, Jared L., Wright, Thomas C., Howell, Scott, and Dickson, Cody

Citation Information: Howland, Jared L., Wright, Thomas C., Howell, Scott, and Dickson, Cody (2009). “Google Scholar and the Continuing Education Literature” The Journal of Continuing Higher Education. 57(1): 35–39. DOI: 10.1080/07377360902806890 (requires subscription).

Abstract

The recent introduction of Google Scholar has renewed hope that someday a powerful research tool will bring continuing education literature more quickly, freely, and completely to one’s computer. The authors suggest that using Google Scholar with other traditional search methods will narrow the research gap between what is discoverable and available. They present results of an investigation in which the names of two scholars were submitted to research queries using traditional library databases and Google Scholar. While not all of the scholars’ academic publications were identified in the search, more were identified by Google Scholar than the other databases. However, other databases identified some that Google Scholar did not. It was evident from this informal analysis that utilizing Google Scholar with other traditional research methods adds value and discoverability in the search for relevant continuing education literature.

Introduction

When one of the co-authors (Howell, a continuing education administrator) recently checked in with his librarian colleagues and asked them if there was anything new to help him with his applied research, they suggested he consider Google Scholar – another derivative search product from Google Inc. He was surprised to hear two librarians recommend an Internet-based research tool as ubiquitous as Google. Did they not know that too much use of Google by students and professors would only threaten their work as librarians? He also thought that perhaps they were just being realistic and realized they might as well join other users, since they were not going to beat them. The librarian co-authors of this article may not agree with this last notion, but they agreed to help prepare this article to better inform continuing education administrators and researchers about the merits of Google Scholar.

Although co-author Howell had heard about Google Scholar, and even used it once or twice, he really had not considered it as a “best practice” until his librarian colleagues encouraged him to do so. His interest had been piqued by their serious recommendation. Much of this article is the story of what unfolded from the guided tour of Google Scholar by two librarians and their student assistant with this continuing education administrator. The purpose of this article is to not only introduce (or reintroduce) Google Scholar to the continuing education community but also to convince busy administrators and faculty that it is worth their time to use this tool as part of their applied research strategy. It can be used for everything from looking into a programming or student service question, to an extensive literature review on a relevant subject for the academic provost, to identifying a theoretical construct for the dissertation that needs to be completed after all these years. Why might Google Scholar be so especially suited for practical or theoretical research in the field of continuing education?

About Google Scholar

Google Scholar was introduced in November 2004 as a beta product in trial mode and retains this developmental status at the time of this writing some three-and-a-half-years later, suggesting that this product is undergoing constant development and refinement. According to the Google Scholar development team, the purpose of this research tool is to:

…provide a simple way to broadly search for scholarly literature. From one place, you can search across many disciplines and sources: peer-reviewed papers, theses, books, abstracts and articles, from academic publishers, professional societies, preprint repositories, universities and other scholarly organizations. Google Scholar helps you identify the most relevant research across the world of scholarly research (About Google Scholar, n.d.).

Google Scholar searches content across various scholarly Internet sites, including those of academic publishers, and ranks findings using a confidential relevancy algorithm. Google also works cooperatively with publishers and academicians to help them create metadata for their content so as to make it more identifiable by Google’s search engines. The development team has also made it possible for local libraries to team up with Google Scholar to help patrons conduct an integrated search of their resources as part of a larger academic search. The whole search process is user friendly and occurs quickly – usually in just a fraction of a second – allowing patrons to seamlessly identify relevant academic materials and access full text for materials licensed and reposed in the local library or freely available elsewhere in the academic community.

How to use Google Scholar

Google Scholar includes a host of helpful features to make queries user friendly and is accessed through a browser at the following URL: http://scholar.google.com/. One need not be a librarian with an MLS degree to maximize the search capability of this product available wherever there is internet connectivity. Google Scholar can generally determine what the user is looking for by using simple keywords. The internal logic is also sophisticated enough to suggest to the user other spellings or keyword choices whenever deemed appropriate. Google Scholar accommodates more advanced searching techniques, (e.g., exact phrases, specific author or publications names, time periods, and so on) by simply clicking the “Advanced Scholar Search” link (see Figure 1) to the right of the basic Google Scholar search button. Even more sophisticated searches are possible by using techniques discussed in an online tutorial prepared by the development team at http://www.googleguide.com/advanced_operators_reference.html.

Figure 1: (1) link to display only the most recent citations, (2) grouping of multiple versions of the same citation and (3) link to the resources that have cited this research

Once a user initiates a search, the results are returned with the most relevant information listed in descending order – most to least relevant. If the user desires a search by date rather than relevancy, he/she clicks the “Recent Articles” link (see Figure 1) underneath the search box at the top of the page for a search of academic materials from the past four years. At the bottom of the results page a list of key authors is also reported for the search just performed. Furthermore, for each academic publication listed, a citation count is noted immediately below, which becomes an indicator for how useful that particular publication has been to others in their own research. If Google Scholar finds the full text of a citation available in multiple locations, it will also indicate the number of online versions (see Figure 1) and link to those Web sites where the full text may be accessed.

By placing the most relevant and useful information at the top of the report, Google allows the user to quickly identify the seminal literature in the field. Then, by providing a link to the articles cited in the earlier search, Google Scholar acts as a gateway to accessing the content by directly linking to the full text or to the publisher from which the material may be ordered. Additionally, searches can be conducted in multiple languages, and bibliographic references for publications can be exported to five different reference software programs, (i.e., Bib TeX, EndNote, RefMan, RefWorks, and WenSianWang).

Why use Google Scholar?

Research in the field of continuing education has always been a challenge, since most of the journals in the field are not indexed or abstracted. Furthermore, a search of the Journal Citation Reports (JCR), a product of Thomson ISI (Institute for Scientific Information), reveals that none of the distance learning or continuing education journals are included in the more prestigious list of journals for which this company calculates “impact factors” and “immediacy indexes” to help librarians and database companies decide which journals to license. This suggests that the field’s literature is not as well indexed by academic clipping services nor as well represented in commercial library databases as other academic disciplines.

Unfortunately, much more literature in this field is extant than is identified and then retrieved as a result of incomplete and uneven representation of content in the competing databases licensed by libraries. It is no wonder that searching the literature has been so problematic for researchers in continuing education. A co-author, Howell, serves as a member of multiple editorial boards for journals in the field and has observed that literature reviews are typically weak and that authors all too frequently cite a “paucity of research” as part of their literature review. In some instances, his knowledge of the literature was such that he, as a reviewer, anonymously shared his own bibliography of relevant articles with manuscript authors and gently encouraged them to review it as part of their own revised literature review. Many of these authors probably wondered how the reviewer came up with this bibliography of relevant research when their own search yielded so much less. What the manuscript authors should really be saying when they cannot find sufficient literature in their review is that “We are not sure if there is more literature but, if there is, we sure can’t find it.” This has been our experience as well. What literature exists in this field is difficult to find using traditional search algorithms and existing databases.

Is Google Scholar Really that Effective?

After talking with his librarian colleagues and co-authors about how effective Google Scholar was, co-author Howell conducted his own Google Scholar query using his own scholarly publications as reference. He then contrasted these results with a query of six major education databases licensed by his university’s research library and listed in Table 1. His curriculum vitae shows 33 articles and edited books published in the past 10 years. Searching through the six education-related databases licensed by his university, he was able to discover 36% of his publications, contrasted with Google Scholar discovery of twice as many (73%). These summary results and more specific findings from the six databases, including full-text availability, are reported in Table 1.

As convincing as this result was, we thought it would be even more helpful to see how well Google Scholar held up using the publication record of a professional colleague with a longer and more decorated academic record. With the permission and cooperation of Dr. Joe Donaldson (aka Joe F. Donaldson for research query purposes), professor of higher and continuing education at the University of Missouri-Columbia, the authors used his curriculum vitae reaching back to 1978, with exactly 100 scholarly publications at the time of this writing, to conduct a comparison (Table 2). Each publication across six major library databases and Google Scholar was painstakingly checked.

As reported in Table 2, Google Scholar identified nearly half (46%) of Donaldson’s academic research over the past 30 years; the other six library databases (when combined) identified just over a third (38%) of his research. When Google Scholar results are contrasted singularly against each of the library database sources, the result is almost two times as effective as the ERIC database (46% vs. 28%) and 46 times more effective (46% vs. 1%) than the database (Academic OneFile) with the lowest return. Furthermore, Google Scholar identified 26 (out of 100) academic publications that the six library databases did not, whereas the databases found only three citations that Google Scholar did not find. Interestingly, Google Scholar and the library databases were more likely to identify the more recent publications than the earlier ones; they both turned in equally poor performances for materials published before 1990.

What is Other Research Saying About the Effectiveness of Google Scholar?

The value of Google Scholar is still hotly debated by the library community. From its inception, Google Scholar was viewed with both curiosity and skepticism (Brophy & Bawden, 2005; Gardner & Eng, 2005; Kesselman & Watstein, 2005) by librarians. It continues to undergo careful review with mixed results (Bakkalbasi, Bauer, Glover, & Wang, 2006; Kousha & Thelwall, 2007a/b; Neuhaus, Neuhaus, Asher, & Wrede, 2006; Robinson & Wusteman, 2007). However, more and more scholars – and practitioners too – are embracing Google Scholar as part of their own research strategy. The results are apparent to those who have watched the product mature over nearly the past four years that search algorithms are better, the availability of retrievable content improved, and that not-found-elsewhere scholarly content identified by this no-charge, ubiquitous search product is only a mouse click away. A recent study reports that “Google Scholar’s coverage of the…free Internet [is] markedly superior to coverage of restricted or fee-based Internet resources” (Neuhaus et al., 2006, p. 135). The study also concluded that its “[c]overage of open access journals, freely accessible databases, and single publisher databases is very strong” (p. 138).

More recently, some articles have appeared that suggest Google Scholar is moving toward making much of the scholarly literature available and easily discoverable (Pomerantz, 2006). The research conducted by two of these authors and recently reported on at the most recent American Library Association (ALA) conference has shown that Google Scholar yields results that are typically more scholarly than the results returned by individual library databases (Howland, Wright, Boughan, & Roberts, 2008) and this generalization applies to many disciplines, though the field of continuing education appears to especially benefit from using Google Scholar for reasons previously discussed. On balance, it seems that the library community, and more and more of the scholarly community, are beginning to acknowledge the benefits of including Google Scholar as part of their successful research strategy.

While Google Scholar appears to have remarkable research capacity, it is not without an inherent shortcoming. Since Google Scholar uses query software to parse citation information without librarian intervention, the relevancy results (used to rank findings) are usually overstated and suffer from what has come to be known as the Lake Wobegon effect – everything is above average or overestimated.

Publishers Collaborating with Google Scholar for Improved Discovery

What else can institutions and continuing education administrators do to ensure better and more customized coverage of their field’s extant literature? An easy first step is to invite journal and magazine editors to do their part by tagging or indexing their scholarly publications for the Google Scholar software crawlers to more easily find. Sharing any level of metadata about the academic content with Google Scholar will enhance the discoverability of material related to continuing education. It will also increase the accuracy of the citation information found within Google Scholar. The Google Scholar development team will provide support and direction once the editor (or representative) completes a very brief web form at http://www.google.com/support/scholar/bin/request.py. Even the editors of print-based only journals can help by placing online the Table of Contents so that Google Scholar search engines can recognize it and then direct researchers to the journal for more information on how to order the content. Because of Google Scholar’s relationship with many universities and scholarly libraries across the world, users may either access immediately the full text online or request the academic content through more traditional means.

Publishers that tag or index content for Google Scholar will make their material more discoverable and thereby increase accessibility to and interest in their publications. The increase in online traffic also affects statistics that librarians use to decide whether they should purchase, discontinue, or perpetuate licensing agreements with publishers (or those consortiums to which publishers belong). Whenever libraries and publishers work closely with Google Scholar, both benefit. Ultimately, researchers are able to conduct better research, and the field of continuing education, and the world around, improves as a result.

Conclusion

Research in continuing education looks to the day when not just a third or half of its research is easily (and freely) available to academic and practitioner alike, but all of it. The eventual goal is not only “no child left behind” but also “no good scholarship left behind.” If there is a paucity of literature on a topic, it should really be the case and not just because the researchers are unable to discover it.

While the recent introduction of Google Scholar has not become the research panacea everyone hoped it might become, it has challenged traditional research strategies; it has demonstrated that a powerful research tool will someday bring quickly, freely, and more completely the body of academic literature, including the field of continuing education, to one’s own computer. Google Scholar is certainly improving the research situation and giving the field renewed hope that the research gap will close between what is discovered and what is actually available.

References

1. About Google Scholar–(n.d.). Retrieved July 8, 2008, from http://scholar.google.com/intl/en/scholar/about.html
2. Bakkalbasi, N., Bauer, K., Glover, J. and Wang, L. (2006) Three options for citation tracking: Google Scholar, Scopus and Web of Science. Biomedical Digital Libraries 3:7, pp. 498–512. – Available at: http://www.bio-diglib.com/content/3/1/7
3. Brophy, J. and Bawden, D. (2005) Is Google enough? Comparison of an internet search engine with academic library resources. Perspectives 57:6, pp. 498–512.
4. Gardner, S. and Eng, S. (2005) Gaga over Google? Scholar in the social sciences. Library Hi Tech News 22:8, pp. 42–45.
5. Howland, J., Wright, T., Boughan, R. and Roberts, B. (2008) How scholarly is Google Scholar?: A comparison of Google Scholar to library databases pp. 65–71. Proceedings of the American Library Association, USA
6. Kesselman, M. and Watstein, S. B. (2005) Google Scholar and libraries: Point/Counterpoint. Reference Services Review 33:4, pp. 380–387.
7. Kousha, K. and Thelwall, M. (2007a) Google Scholar and Google Web/URL citations: A multi-discipline exploratory analysis. Journal of the American Society for Information Science and Technology 58:7, pp. 1055–1065.
8. Kousha, K. and Thelwall, M.(2007b) How is science cited on the web? A classification of Google unique web citations. Journal of the American Society for Information Science and Technology 58:11, pp. 1631–1644.
9. Neuhaus, C., Neuhaus, E., Asher, A. and Wrede, C. (2006) The depth and breadth of Google Scholar: An empirical study. Portal: Libraries and the Academy 6:2, pp. 127–141.
10. Pomerantz, J. (2006) Google Scholar and 100 percent availability of information. Information Technology and Libraries 25:1, pp. 52–56.
11. Robinson, M. L. and Wusteman, J. (2007) Putting Google Scholar to the test: A preliminary study. Program: Electronic Library & Information Systems 41:1, pp. 71–80.