October 27, 2014
Michael Bugeja, director,
Greenlee School of Journalism and Communication,
Iowa State University of Science and Technology
I have been appointed chair of a committee of the Association of Schools of Journalism and Mass Communication looking into the use of Academic Analytics at various institutions and whether its large database is representative of departments, schools and colleges of journalism and mass communication.
You can access the AA website at this URL: http://www.academicanalytics.com/
The company’s database includes more than 270,000 faculty members across the country. It ranks units by graduate program, department, discipline and individual, along with other functions. It documents traditional scholarship to serve its primary function of rating graduate programs, which includes scholarly publications, citations, funded research and awards. Its “About” website states its mission as supporting “university leaders as they strive for excellence,” guiding them “in understanding strengths and weaknesses, establishing standards, allocating resources, and monitoring performance.”
To be sure, we are not discouraging nor questioning subscribing to or using AA. Actually, many of its functions are useful and practical, especially its peer program comparisons. Its Internet functions are easy to learn, and the comparison charts—how your unit stacks up against others across the country—are savvy and well-designed.
However, a database is only as good as its data. Our role is to enrich it as best we can. To accomplish that, it is important to note that AA’s goal is to be inclusive but standards-oriented in its approach to compiling faculty productivity data.
Our committee includes colleagues from some of the finest institutions in our ASJMC family:
• Ann Brill, Dean, William Allen White School of Journalism and Mass Communication, University of Kansas
• Jerry Ceppos, Dean of the Manship School of Mass Communication, Louisiana State University
• Christopher Callahan, founding dean of the Walter Cronkite School of Journalism and Mass Communication, Arizona State University
• Jan Slater, Dean, College of Media, University of Illinois
• Paul Voakes, former dean, School of Journalism and Mass Communication, University of Colorado-Boulder
Ex-officio members are:
• Brad Rawlins, Dean, College of Communication, Arkansas State University
• Don Heider, Dean, School of Communication, Loyola University
• Jennifer McGill, Executive Director, Association for Education in Journalism and mass Communication
The charge to the committee is as follows:
In journalism and mass communication, what do we consider excellence in terms of creative work?
What are the awards that should be listed under journalism and mass communication in databases like this?
The committee was formed at the behest of Ann Brill who informed us that Academic Analytics was interested in collaborating with ASJMC, especially since the company is eager to hear how scholarship in journalism and law are measured.
“Even if your university is not subscribing to such a database,” she writes, “the national trends suggest that more and more, state universities have to establish their value in some way. If we aren’t proactive then we will have to live with whatever such databases use to compile data.”
The committee has been looking into Academic Analytics since July. At the Greenlee School of Journalism and Communication at Iowa State University, I have been working with my colleague, Jan Lauren Boyles—a presidential hire here with a specialty in big data—and have been analyzing the databank so that the ASJMC committee can make reasonable recommendations to enrich Academic Analytics so that it better documents professional disciplines like ours.
Our preliminary findings seem to suggest that Academic Analytics appears to pull a significant amount of data from conference proceedings, does not fully track foundation funding because this reportedly is not part of its overall mission (Knight, McCormick, etc.), omits creative awards (Peabody, Emmy, etc.) and trade publications (Columbia Journalism Review, Chronicle of Higher Education, etc.), overlooks many juried shows (photojournalism, multimedia, etc.), excludes several AEJMC division-related journals, and prioritizes single-author works (not fully indexing collaborative ventures).
Concerning awards, the category of “Communication and Communication Studies” includes 6 awards from the National Communication Association and 13 awards from the International Communication Association. It also includes one award for “Outstanding Educator” from the Public Relations Society of America. In the “Mass Communication and Media” award category, the AA list includes 5 ICA awards, 5 NCA awards, 1 National Association of Science Writers award, Guggenheim, MacArthur, Modern Language Association and Pulitzer awards.
AEJMC awards are not represented in any award category. (Note: The list of awards is much larger than presented above, including single awards such as a science writing honor by the American Institute of Physics.)
Currently ASJMC committee members are submitting to me individual lists of their own 10-20 peer-reviewed journals and awards that should be included in the database. I will combine those lists and then compare them to what is in the database. Any journal not on the list would be recommended to the company. However, only journals that meet AA’s standard of “peer review” will be included.
Each committee member also was asked to submit recommendations that better represent the quality and scope of our scholarship—creative, professional and research-related.
That report, when completed by March 2015, will be published on the ASJMC website.
Without speaking for the committee, I personally feel it is important for AEJMC, ASJMC and the Accrediting Council on Education in Journalism and Mass Communications to closely follow our results and to begin monitoring use of Academic Analytics on various campuses.
Some recommendations, such as including works of creativity and practice, likely will prove unsuccessful. We are told that those data would erode the graduate program ranking mission of the company. Thus, our organizations may want to supplement AA use by posting on our various association websites what counts as scholarship in advertising, journalism (all platforms) and public relations and by notifying provosts at ACEJMC institutions to visit our supplemental sites for a fuller analysis.
That said, I am optimistic that our committee will make useful recommendations to Academic Analytics. The company has a Board made up of faculty from various disciplines. It frequently convenes discipline specific committees to address issues like authorship and awards, and representatives have told me that the company is always willing to consider ideas for improvement.