Beyond the static text: multimedia interactivity in academic journal publishing in the humanities and social sciences (Not)
Four postulates of inertia
Despite the discourses that herald the innovation associated with Web 2.0 and the opportunities created by Open Source and Open Access computing(Miller & Berners-Lee, 2008), there is a simple truth. The web has not delivered an interactive environment for ongoing engagement with scholarly research publishing that uses and is enlivened by multimedia data. There are four major reasons for this failure to bridge the two dominant silos – on the one hand multimedia data with its possibilities for interactivity (as revealed for instance by but not limited to social networking on the web); and academic journal-based publishing, with its linear and traditionally constrained presentation of knowledge in “finished” blocks, albeit illustrated and hyperlinked.
These four elements can be summarised as follows:
a) a political economy of web publishing in which large multinational corporates seek to optimise returns by seeking ownership and control of intellectual property in discreet objects called “papers” assignable to the authorship of single individuals or groups, while they minimise expenditure on infrastructure within an increasingly Fordian production-line environment(Smith, 2005). Complex and interactive multimedia publishing is perceived to have a low return-to-investment ratio, and has dangers to IP regimes associated with its continuing and “unfinished” nature;
b) a system of hierarchies of journals (based on esteem) the value of which to publisher, editor and author is a function of their difficulty to enter, while the demand by author communities to be published in high-impact titles assumes an efficiency factor in which papers represent the “least-time-cost” form of publishing research outcomes;
c) the control of access to journals acts as a rationing system ostensibly determined by peer esteem, but in fact controlled by well-established and usually traditionalist disciplinary elites who for the most part have neither the skills, motivation nor the experience to referee or manage the publication of interactive multimedia projects;
d) younger scholars who might otherwise be innovators in the use of interactive digital media for publishing face disincentives both from the criteria used to assess for tenure and promotion (traditional journals, academic monographs) and the fact that older mentors have no experience in innovatory multimedia research and publishing and are unable to provide role models.
In this chapter I want to explore the case for each of these assertions, and then suggest a model for resolving this combination of factors that have hitherto been woven into a “shark-net” of resistance to innovation.
The political economy of web-publishing
In their analysis of the opportunities for publishing humanities and social sciences research in open source on-line journals in the UK, Heath Jubb and Robey (Heath, Jubb, & Robey, 2008) argue that the humanities and social sciences have been slow (compared with the physical and life sciences) to take up the opportunities offered by the new technologies. In preparing their case they explore the structure of journal publishing and in particular the development of open source and open access publishing. The expansion of the commercial publishing of journals has transformed the knowledge landscape, and has only been possible because of the knowledge management technologies underpinning them. However the consequences of the involvement in multinational corporations in the industry have been problematic; high status journals have enormous masthead value, and that value depends on the competition for entry among prospective authors. With the ever-greater institutionalization of quality and impact metrics within national university funding models(Weale, 2007), driven by governments working with neo-liberal economic models dependent on “objective” criteria, the intellectual property bound up in high-valued discipline-leading journals has become a source of serious struggle.
The complex economic base from which the intellectual property is generated includes many different forms of value creation. While the brain-work of the researcher/authors supplies the essential commodity, the value embedded in that commodity and the costs of realizing that value reveal the nub of the drive towards open-access journals. Most research published in journals in the humanities and social sciences is produced by scholars employed either as fulltime researchers or as combined teachers/researchers. Their salaries are paid by research grants or funds, and core funding to universities that is then allocated to “research time”. The labour value of the research is thus already covered by the state or by other corporate funds, and is not an item with which most publishers need to concern themselves. The editorial role taken on by academic editors and the labour carried out by referees of submitted articles are also not covered by the journal publisher. However then the process of realizing the value of the research in the market place begins – journals have to be sold to university libraries and research institutions. That is, for the research to be available, a cash exchange is necessary, and it is an exchange in which the purchasing parties have up until recently had very little real leverage. Moreover, the IP produced by academic labour and paid for by the state (ie the public), is not freely available but rather has to be purchased at considerable cost.
Publishers argue that they play a crucial role in managing the process and realizing the value embedded in the research. Only they have the capacity to program and support the production process and utilize economies of scale to create the underlying knowledge management structures that allow effective database searching and delivery of high quality products to users in a timely way. Some publishers recognize that they add value to something that already has value and accept the grey publishing pre-print environment where drafts are circulated; usually they seek to control the IP at the point they start to introduce “their” intellectual input via the editors and the reviewers. Moreover and more importantly for our purposes the Fordian production line approach that reduces all online journals produced by a publisher to basically the same format makes breakouts in creativity very unlikely and also costly. Most innovation focuses on the bibliographical and search functions, rather than interactive or creative functions.
The open source movement of course confronts these issues from outset to final presentation- and has received strong government support in countries like Australia, where the Australian Research Council now mandates the free availability of government-funded research through open repositories. The Heath et al. (2008) survey of open-access journals in the UK still notes that the Humanities and Social Sciences remain more dependent on print journals, and far less comfortable with journals that only appear in an on-line form. They ascribe this phenomenon to a number of important factors – the long half-life of research, especially in the humanities where the intellectual trajectory under review may stretch back decades being one, and the low general level of funding for HSS research and thus the long development time in the writing of journal papers being another. Even here they note the rapid take-up of the JSTOR archive by HSS researchers, despite their reluctance to publish in electronic forms.
So the political economy of current journal publishing means that large publishers are unlikely to innovate, while individual researchers are less equipped with economic resources to do so easily. Even in the open access community such innovation is not widespread, as the craft-based design driven innovation that is often thought necessary for HSS interactivity in publishing is not easily supported by most open-source systems.
Journal hierarchies, the publishing race and creative scholarship
Given the broader contextual constraints, how do the barriers between innovatory research and traditional publishing work in practice? Innovation in the use of interactive digital technologies in the gathering of data and the presentation of material is widespread. The UK Humanities computing schemes over the last decade have generated substantial databases of materials from ancient history through to sociology and political science. In their joint attempt to find a shared common ground between the various stakeholders in the process of academic publishing the Research Information Network (Research Information Network, 2007)argued that one of the key policy principles should be that“researchers, publishers and others are innovative in exploiting new technologies, new publication and dissemination models and a variety of media to communicate the results of research as widely, rapidly and effectively as possible”. Yet we have seen that the boundary forces that exist seem to militate against such an outcome.
A British Academy report on peer review, metrics and quality (Weale, 2007) was unimpressed by most “metric” based approaches, criticizing validity, reliability and the distorting affects such an approach might have on the quality of research and in particular, its direction. That is on my reading the Academy seemed to fear that more exploratory, challenging and quirky research would be underplayed, and publishers seeking high metric gradings would be drawn inevitably towards blander and more centre-ground research. Moreover in their discussion of innovation in particular, they were aware of innovative projects that had failed in mainstream journals, only to be picked up by “lesser” journals, and then becoming classics. However the Academy was mainly concerned with the role of peer review in traditional publishing environments, and did not address alternative and innovatory forms of research publishing.
Innovatory forms of research may include for instance the use of extensive ethnographic video recording (a traditional practice in anthropology, which has produced the filmography and the bibliography as parallel but distinct forms of publishing), the application of animation and programming to the manipulation of images and their interpretation, the development of interactive projects where the audience engagement changes the data and transforms the output through time(Dicks, mason, Coffey, & Atkinson, 2005).
Yet even where this research is very interactive and seeks to produce interactivity as an outcome, it is reported in a fairly flat and traditional way. The Arden project at Indiana University is a valuable example of this issue. The team draws on Edward Castranova’s “Synthetic Worlds Initiative” to build a Shakespearean space “that he could use as a platform for social-science experimentation while providing interesting and arguably useful content for players” (Castranova et al., 2008). It is an exciting and highly proficient project, and yet even though the article about it is produced as a PDF file by the IEEE MultiMedia journal, there is no sense of interactivity in the article – eg an embedded SWF file that clips some of the action and gives readers a sense of the feel of the experience. On the other hand IEEE MultiMedia began in 2008 to include a videoblog, where scholars can present and debate their perspectives. The first contributions are perhaps scholarly rather than creative, but no doubt here too there are possibilities for exploration(Editors, 2008).
Another project that works well online is Norie Neumark and Maria Miranda’s “Talking about the Weather” (Neumark & Miranda, 2006). It comes in many forms – including a series of installation and performance pieces. However its online version works as a mixture of blog, photographic display, exhibitions, YouTube videos, and essays, mostly backed by radiophonic pieces. It has both an artistic design problem that it seeks to solve, and a research question – what happens when you go into different societies seeking to have people simply breath into the microphone while being videoed, in the name of combating global warming? The project is published through SCANZ, a New Zealand artists site (Solar Circuit) based on a creative residency camp in 2006. Yet publishing an article in a refereed journal that managed to capture the experience of the site – other than by linking to it – would be very difficult.
It is clear though that with a metric-driven government mindset, and competition for prestigious positions in prestigious institutions, early and mid-career researchers have to be very tough about how they allocate their time. One journal that has recognized this problem Vectors, published out of the University of Southern California, has decided to “wrap” the interactive project within a series of texts, thus bridging the divide between creative project and journal article. These texts include the editors’ comments, the author and creative director’s statements, and a commentary field for audience engagement. Each journal issue is produced as a set of connected projects linked to a theme. Vectors (http://www.vectorsjournal.org/ ) seeks to bridge the humanities and social sciences, drawing in projects as diverse as interactive programmed onscreen games, photo/audio documentaries, political philosophy simulations and historical sociology.
The editors are closely connected with the HASTAC (pron.haystack http://hastac.org) ), which also gives access to blogs such as those of MIT Humanities Professor Henry Jenkins. Jenkins uses his blog to publish multimedia interviews he does with scholars using the web for innovatory explorations – including one about a YouTube course at a California college, run by Alex Juhasz in which YouTube becomes a publishing medium. (http://www.henryjenkins.org/2008/02/learning_from_youtube_an_inter.html)
These are extremely interesting and transformational publishing exercises, yet none can be encompassed by traditional journals. For the most part the innovation is being carried by scholars in fairly secure situations, who use their resources and influence to lever up the work of early career researchers. An attempt to advance these innovations has been instituted through the Digital Arts and Humanities site created by the Centre for eResearch at Kings College London (http://www.arts-humanities.net), and by the US National Endowment for the Humanities Office for Digital Humanities opened in March 2008 (http://www.neh.gov/ODH/ODHHome/tabid/36/Default.aspx).
Commenting on the development of these resources and the emergence of e-publishing HASTAC associate Cathy Davidson has asked:
Why use a new technology to address (badly, incompletely, and even falsely) one problem? Why not use a new technology to explore new possibilities?… And it should make us think about those things in terms of credentialing and the values we give to how we credential young scholars in a profession. …. Perhaps making a new kind of data base available in searchable form is even more valuable then yet another historical monograph that makes use of that data base? Maybe universities need to think about the kind of intellectual labor that goes into making such a data base available and usable. If we believe all Foucault has to tell us about what an archive is, then creating an archive is itself a form of argumentation and, indisputably, a major contribution to knowledge. That should constitute an argument for tenure at those institutions that collectively decide it should constitute tenure. (http://www.hastac.org/node/1257)
Power and control in academic publishing
A senior MIT computing academic asked me during a research interview on innovation in multimedia publishing what I saw as the problem. I spoke about gatekeepers and their reluctance to shift paradigms that had served them well. He said that the solution was clear – either go around them or wait till they die. It was an in-your-face comment that was designed to address the issues of institutional power and willingness to change. In many ways the “going around” has occurred – new journals, particularly those with an interdisciplinary bent, are opening regularly, both through established and new commercial publishers, and in independent exercises online by University departments and centres (eg Transforming Cultures eJournal (http://epress.lib.uts.edu.au/ojs/index.php/TfC) or professional associations. Many of the latter use Open Source programs – such as DSpace (http://dspace.org) based at MIT, for both journal and repository development. Others use development environments and content management systems such as Joomla (http://www.joomla.org) and even the blog–focused Drupal (http://drupal.org). As numerous reports have documented, the value of scholarly articles is in part derived from their validation by peers; yet with innovatory research and publishing pressing at the doors of the old established journals, there may not be sufficient or even any ranked peers capable of reviewing the new approaches.
Senior academics are not necessarily all that skilled at adapting their world-views to constantly changing circumstances; while a minority will be anxious to grasp new opportunities, the majority will rely on the corpus of practice built up in a discipline over generations. Even where the ambition to innovate exists there may be barriers that simply cannot be overcome. A classic example of this problem can be found in the leading journal American Historical Review (AHR). The AHR has an electronic projects section (http://www.indiana.edu/%7Eahr/elec-projects.html), with the majority of projects associated with the advocacy of the late Roy Rosenzweig of George Mason University. Rosenzweig had established the Center for History and New Media (http://chnm.gmu.edu) as a focal point for innovation in the presentation of historical analysis. The pivotal test for the journal would be how it handled the challenge of a project on the American Civil War by Thomas and Ayers from the University of Virginia Center for Digital History. In introducing their project, they note:
scholars have only begun to craft scholarship designed specifically for the electronic environment. In this article, we attempt to translate the fundamental components of professional scholarship-evidence, engagement with prior scholarship, and a scholarly argument-into forms that take advantage of the possibilities of electronic media.( http://www.vcdh.virginia.edu/AHR/)
The original concept, according to Rosenzweig (interview November 2004), was to incorporate the project as an electronic article into the journal. However two issues arose that made this impossible – the journal could not agree on criteria for refereeing the project, and there were no arms-distance referees with the composite capacities in both history and digital media. A compromise was struck – somewhat unsatisfactory – that the authors would write an academic article using the traditional genre and criteria of scholarship – and the journal would link to the Virginia website and list it under Electronic Projects(Thomas & Ayers, 2003). As of April 2008, there had not been a new electronic project AHR since 2005.
The young and the restless
By the second half of the 2000s, a new generation was moving through the entry level avenues in tertiary institutions. Young graduate students or recent PhDs were competing for tenure track positions in universities, often with personal histories of creative innovation in media arts, and aspirations to apply their “digital native” (http://www.digitalnative.org/Main_Page ) orientations to their new work environments. In a 2004 study of six new PhDs employed as assistant professors or lecturers in the humanities at leading north eastern US universities, I explored the factors in their situations that might explain the disjunction between their aspirations as creative scholars using digital media, and their behaviour as traditional authors of journal articles and monographs.
Two examples help illuminate the issues. Paula (not her real name) had been an ethnographic filmmaker who had undertaken anthropological fieldwork in Latin America. She had a large pool of videos of everyday and ritual life among her field community, and had made one short documentary analysing an aspect of her research. She then faced a critical decision – whether to use the material and develop an interactive research environment based on a digital video database that could be interrogated by a research community – or to process the data in traditional anthropological writing forms, and focus on getting academic tenure. As she reflected on her choice – to follow the tenure track – she identified four key factors that influenced her choice. Firstly, her mentor in her department counseled her to think about her career and the importance of impressing the gerontocracy that ruled the tenure committee. Secondly, there was no one in the hierarchy of the department who had any understanding of the potential of digital interactivity to whom she could turn for advice. Thirdly, none of the relevant journals at the time were able to consider anything outside the norm. Indeed it was not until October 2007 that the Open Anthropology project began, spurred by the advent of Web 2.0 (http://openanthropology.wordpress.com/categories/), while the first online open source anthropology journal was set up in February 2007 as the Museum Anthropology Review. It initially using WordPress then from February 2008 migrating to using Open Journal System (DSpace) through the University of Indiana Bloomington library (http://museumanthropology.net/; http://scholarworks.iu.edu/journals/index.php/mar/index). However it is thus far limited to publishing reviews, and does not make the full use of the multimedia capabilities of Web2.0. Finally, the technology that might have enabled Paula to more easily incorporate an interactive multimedia component into the concept and design of the paper was not available – instead she would need to invest significant time, develop high levels of design skills, and build a craft-based website, even if it were to be accepted by a journal outside her disciplinary field, such as Vectors.
Jacques (not his real name) teaches French at a nearby university to Paula. Jacques has the design and creative skills, and indeed was engaged by the university as a language and culture lecturer so that he could research and develop new interactive learning environments. These resources would be built around a dense data base of French accents, dialects and cultural minorities drawn from an extensive ethnographic video project that documented diversity in France and its territories. This research is potentially valuable as the basis for further explorations and testing of the idea of “French culture” in a period of significant social turmoil, and as serving a community of scholars. However Jacques, entrapped in a teaching only job, aspires to be a researcher and professor, so concentrates on research that can be published in journals and as a monograph. He hopes, he says, to return to this rich data at a later point, once he is embedded and tenured.
The inevitability of change: what will be the paradigm shift that makes it happen?
The argument about the failure of the academic journal in relation to engaged interactive multimedia-based publishing, as either commercial mainstream or collegial open source, is bolstered by the sense of transitional challenge identified by major institutions. A major US report (in part driven by Rosenzweig) (Rosenzweig)postulated that “many who work in the humanities and social sciences (HSS) have come to recognize that the knowledge practices in these disciplines are on the edge of some fundamental changes . . . the online world is a new cultural commonwealth (yet there are major constraints including) unsatisfactory tools, incomplete resources, and inadequate access” (ACLS, 2006).
There are the institutional barriers of gerontocratic power, an undeveloped corpus of review criteria, and limited technological capacity, compounded by economic forces seeking to reduce the costs and price of publishing. However I want to turn here to another issue, that of how the processes of production need to be transformed to enable creative HSS publishing to be more of an industrial process (akin to the wordprocessing and printing formula) and less one dominated by scarce craft skills (of web design, graphic complexity and interactive scripting). Clearly Web 2.0 opens up some possibilities here, particularly when linked with embedded video (such as YouTube, LiveLeak, CurrentTV), or with audio, or with flash movies and animations. USC history Professor Phil Ethington (see (Ethington, 2000) for an early exploration of the multimedia essay, albeit without interactive audience opportunities) has been a key innovator in the application of digital media to the study and communication of historical analyses, especially of urban spaces; Ethington has curated/edited a multimedia companion to the journal Urban History that utilises a spread of Flash animations to explore visual and aural dimensions of urban historical events and locations(Ethington, 2008)).
However enabling audience community engagement needs to go further than this. A model for such a process can be found in the Multimedia Interactive Research Environment (M.I.R.E.) based on the metamedia project at MIT. This essentially uses an XML shell with tags, notes, and commentaries, including codes drawn from the Dublin Core Metadata Initiative that enable effective open searching on the web. This shell is wrapped around every piece of data, which can then be retrieved, commented on and extended.
If we bring these challenges together it is possible to model a simple flow chart of the steps involved in a digital research and publishing project. The project named as MIRE – in reference to the murky terrain as well as an acronym for the key elements of interactivity, multimedia data and research – can be stepped out conceptually as is done in Diagram 1. The work-flow models, in skeletal form, the movement through time of an archetypal research project. This is an ongoing process where the publication is never “finished” but always open to continuing discussion, debate and elaboration, as occurs in a wiki environment currently.
Diagram 1: The MIRE Work-flow
The MIRE environment creates a work space where a researcher or group of researchers can create a project. The project begins by inviting the researchers to specify a broad research topic, either for individuals or for members of designated research team. The topic has associated metadata, including the research fields and links to the websites of the researchers. Then as the research data is generated (in multimedia forms) they build a digital repository of research materials (using programs such as Portfolio), which is tagged to reflect the research framework, while applying Dublin Core and Text Encoding Initiative templates, as well as more elaborate coding. As the project repository expands, it can be searched to build inductive theoretical concepts that can be formed into research questions – or indeed initial research questions can (and usually would) drive the development of the research materials in the DB. The repository grows in iterative cycles, as the search pathways are attached to the DB items. As the research process intensifies the researcher or research team would build commentaries and hypotheses around the data, which would remain attached for future researchers (and become part of the repository) should they wish to explore them.
The research pathways would be mapped visually, showing what materials were most useful in responding to specific questions, and the way in which different data items are conceptually related to each other. As the researchers begin to “write” their research results for publication, they would be able to choose one or more of a number of templates, that would produce “texts” suitable for publication in specific environments – online journals (such as the Electronic Journal of Sociology) or emerging e-journals and e-print solutions that incorporate peer review, more creative environments such as Vectors: Journal of Culture and Technology in a Dynamic Vernacular, web projects (e.g., American Historical Review under Rosenzweig’s influence) or refereed blog type publications. The publication would have embedded links to the digital repository used for the research, enabling readers to comment directly on research items, link in to full versions of original documents etc. held in other digital repositories, and view a range of materials (e.g., excerpts of video interviews on a common theme) that underpin the research. Over time another valuable benefit would flow, namely a set of operationalisable reference materials that enable peer reviewers to assess both the academic content and the effectiveness of the communication (as is currently done with the comprehensibility of the text and the interpretation of graphic materials).
The failure of current journal publishing to innovate in a way that utilises the range of possibilities inherent in digital environments is a consequence I suggest of two related factors. Firstly the political economy of publishing still constrains in an unfortunate and ultimately self-defeating way the capacity of journals to try out risky and experimental options in a systematic and well-resourced way. Secondly the industrial technologies are there in the multiple parts but have not yet been brought together, because the problem is usually conceived of as being one either of data collection and storage on the one hand and therefore an issue for museums or galleries, or on the other one of dissemination of completed research findings being the responsibility of publishers and libraries.
We need a journal publishing strategy that understands the scholar as a participant in a community of knowledge making and testing, in which there are many legitimate stakeholders, and in which truth/insight/understanding is a function of the interrogations we make of the data we choose. We need to move beyond the current impasse – the issue is which pathway might best take us around the blockages, with their economic, institutional, political and scholarly dimensions.
ACLS. (2006). Our Cultural Commonwealth: The report of the American Council of Learned Societies Commission on Cyberinfrastructure for the Humanities and Social Sciences. Washington: American Council of Learned Societies.
Brown, L., Griffiths, R., & Rascoff, M. (2007). University Publishing in the Digital Age.
Castranova, E., Ross, T., Bell, M., Falk, M., Cornell, R., & Haselton, M. (2008). Constructing Arden: Life Inside the Machine. IEEE Media Impact(January), 5.
Clarke, R., & Kingsly, D. (2007). ePublishing’s Impacts on Journals and Journal Articles. Journal of Internet Commerce, 6(4).
Dicks, B., Mason, B., Coffey, A., & Atkinson, P. (2005). Qualitative Research and Hypermedia – Ethnography for the Digital Age London: Sage.
Editors. (2008). Your Multimedia Community’s Video Blog, IEEE MultiMedia.
Ethington, P. (2000). Los Angeles and the Problem of Urban Historical Knowledge. American Historical Review, 105(5).
Ethington, P. (2008). “Urban Icons,” Multimedia Companion to Urban History. . Urban History.
Forte, M. (2008). The Library as Open Access Publisher, and Digital Publishing 2.0, Open Anthropology.
Heath, M., Jubb, M., & Robey, D. (2008). E-Publication and Open Access in the Arts and Humanities in the UK. Ariadne(54).
Jaschik, S. (2008). Abandoning Print, Not Peer Review, Insidehighered,com.
Kennedy, H. (2003). Technobiography: researching lives, online and off. Biography, 26(1), 120-142.
Khut, G. P. (2007). Cardiomorphologies: An Inner Journey through Art, IEEE MultiMedia (Vol. 14, pp. 5-7).
Miller, P., & Berners-Lee, T. (2008). Sir Tim Berners-Lee Talks with Talis about the Semantic Web, talis Podcasts.
Nellhaus, T. (2001). XML, TEI, and Digital Libraries in the Humanities portal: Libraries and the Academy 1(3), 257-277.
Neumark, N., & Miranda, M. (2006). Talking about the weather.
Research Information Network. (2007). Research and the Scholarly Communications Process: Towards Strategic Goals for Public Policy A Statement of Principles . London: Research Information Network.
Rosenzweig, R. (2007). Collaboration and the Cyberinfrastructure: Academic Collaboration with Museums and Libraries in the Digital Era, First Monday (Vol. 12).
Shah, N. (2008). From global village to global marketplace: Metaphorical descriptions of the global Internet. International Journal of Media and Cultural Politics, 4(1), 9-26.
Sharp, D., & Salomon, M. (2008). User-led Innovation: A New Framework for Co-creating Business and Social Value Smart Internet Technology CRC.
Slane, A. (2007). Democracy, Social Space, and the Internet. University of Toronto Law Journal, 57(1), 81-105.
Smith, J. (1999). The Deconstructed Journal – a new model for Academic Publishing, Learned Publishing (Vol. 12).
Smith, J. (2005). Reinventing Journal Publishing, Research Information.
Thomas, W., & Ayers, e. (2003). An Overview: The Differences Slavery Made: A Close Analysis of Two American Communities. The American Historical Review, 108(5).
Weale, A. (2007). Chapter 6: Metrics and Peer Review. In (chair) Peer Review: the challenges for the humanities and social sciences: The British Academy.