Henry Jenkins’ New Media Literacies class at USC has been a treasure-trove of readings and insights. Three recent articles covered in class — read alongside Jenkins’ own book, Convergence Culture, and his excellent MacArthur-funded New Media Literacies white paper — struck me as particularly essential for anyone who’s looking to build an understanding of what multimodal communication is and how transmedia relates to education, literacy and literature. Most of these readings can be found in various corners of the Web, but I’ve also posted them here (along with a brief gloss and anecdote) for those who are interested. They are:
- Gunther Kress, Literacy in the New Media Age (New York: Routledge), Chapter 4 “Literacy and Multimodality: A Theoretical Framework“
- Mimi Ito, “Technologies of the Childhood Imagination: Yugioh, Media Mixes, and Everyday Cultural Production”
- David Buckingham and Julian Sefton-Green, “Structure, Agency and Pedagogy in Children’s Media Culture,” in Joseph Tobin (ed.), Pikachu’s Global Adventure: The Rise and Fall of Pokemon (Durham: Duke University press, 2004)
Ito’s succinct article makes the case most directly: “technologies of the imagination populate even the most mundane corners of our lives,” (34) and, contrary to the fears of those who worry that new media threatens to compartmentalize and disembody, the media mix is in practice productive of a culture that is “extroverted and hypersocial, sociality augmented by a dense set of technologies, signifiers, and systems of exchange.” (32) Buckingham and Sefton-Green hammer the point home: skeptics ought to consider examples like the “striking contrast between the high levels of [multimodal reading, sociality and production] activity that characterize the Pokemon phenomenon and the passivity that increasingly suffuses our children’s schooling” (30); and who could disagree that banning such phenomena from the school yard would do anything other than increase their “forbidden appeal” and “prevent schools from building on the enthusiasms children possess”? (31)
Of course, we have a long way to go before these kinds of messages can establish a critical mass in institutional and creative practice. Last week, I attended an experimental literature conference and found that while many of the assembled authors and scholars were keen on experimenting with new media, few if any of them were open to a wholesale redefinition of what literature is/can be. (( …and here I’m thinking not in either/or terms but in both/and: the novel will always be around and will always be the best at doing those things that novels do best. But there are other kinds of literature lurking in the shadows, and that’s what I’m interested in here. )) The works presented would inevitably employ language — spoken or written — as their core expressive resource (unsurprising for a conference largely run and constituted by poets and English professors), which they would then back up with video, flash animations, sound effects, etc. The effect of this was to reduce any image, sound, interactive or procedural elements present in the works to subordinate “supporting” status, lending credence to the commonly-expressed concern that the use of new media “in” literature amounts to little more than gimmickry. As Kress argues, we need to not only shift our definition of text to include “any instance of communication in any mode or in any combination of modes, whether recorded or not,” (48), but also our concept of the role design plays in both reading and writing. “Design does not ask, ‘what was done before, how, for whom, with what?” Kress writes. Rather, Design asks, “What is needed now, in this one situation, with this configuration of purposes, aims, audience, and with these resources, and given my interests in this situation?” (49)
The easy analogy here is that of the early cinema, wherein fiction films were shot using the conventions of the proscenium drawn from the theatre. It was only after a thorough interrogation of the affordances of the camera and the film splicer that the cinema began to reveal itself as a space for artistic endeavor. That is, once filmmakers let go of the notion that the cinema should attempt to create the same experiences as earlier forms of narrative art, they found themselves liberated to discover the unique way of “speaking” that film affords. What complicates this analogy is that we now confront a dynamic multiplicity of media modes. Like Gardner’s multiply-intelligent children, not all authors are going to be able to work well across all media. But in an age of expanding definitions of words like “text,” “author,” and “reading,” creators of literature, as educators and thought leaders, need to ask themselves the questions Kress’s personified Design asks: “What is needed now…with these resources, and given my interests?” Intelligently using new media is not about adding bells and whistles or referencing the Web — rather, it’s about selecting the right mode (or modes) to express what it is you have to say.