Gameplay in Reality Ends Here takes place in every corner of its players’ lives, as they collect, share, trade, and combine game cards in order to generate creative prompts which are then used to guide the making of unique media artifacts and real-world events. By sharing the resulting creative works through the social media platform at the center of the game, players connect with one another across disciplinary and institutional boundaries and unlock customized “trailheads” leading to intimate and offbeat encounters with SCA alumni, artists, and other industry professionals. Launched in August of 2011, Reality Ends Here is having a tangible positive impact on the culture of the SCA, bridging the gaps between traditionally siloed disciplines and creating an atmosphere of artistic experimentation. Henry Jenkins describes the game as follows:

I’ve seen lots of discussion over the past few years about the potentials of using ARGs for pedagogical purposes. But, this is the first time I’ve seen such a large scale experiment in integrating ARG activities across an entire school to orient entering students to a program and to serve a range of instructional goals. The passion the game is motivating in USC students is palpable. And I can tell you that many of the faculty, who have gotten pulled into the game through one play mechanic or another, are feeling a real pride in their school for its willingness to embrace this kind of experimentation and innovation. (henryjenkins.org)

A Procedural Creative Prompting System

Reality Ends Here is driven by a card-based “procedural prompting system”: by sharing, trading, and combining cards, players create challenges within the constraints of a connectivity play mechanic.

As designers, we knew from the start that it was important that the challenges in our game come from the players, not us. We knew that a set of challenges curated “from on high” would take away many crucial aspects of agency and authorship from our players — and since those things are at the heart of the kind of creative and performative impulses that underly engagement with our game, we knew we needed to protect them. We also believe that players should author the challenges themselves because in our experience, doing so is an integral part of what’s fun and engaging about these kinds of games. In this sense, Reality Ends Here has a lot in common with other open-ended collaborative production games such as SF0 or Super Going.

On the other hand, we felt that a total lack of constraints could be hobbling to creativity, particularly for players who are not already ensconced in strong “maker” or DIY communities and practices. As Orson Welles famously said, “the enemy of art is the lack of limitations.” Brainstorming, story workshopping, or any kind of creative spitballing without clear constraints and anchors will often drift into outright confusion. To address this issue, we devised a simple card game that structures and limits creative brainstorming in a manner similar to a Tarot deck or an idea generator like Grow a Game. Here’s how it works:

Through this card interaction, players generate creative prompts of varying complexity based on the number of cards they manage to integrate into their Deal. These prompts then function somewhat like Fluxus “event scores”, guiding the actual creative production phase of the Deal by specifying the kinds of media to be used and a range of ideas, physical objects or places, and narrative or figural elements (collectively referred to as “properties”) that players must incorporate into their project.

With the complete set of cards numbering in the several hundreds, the number of possibile combinations is practically endless, limited only by the specific cards that a player or group of players has access to. Since players start the game with only 9 cards (7 random and 2 which they select themselves from a draw of 10), trading, pooling, and earning new cards through gameplay is essential. This face-to-face social interaction strengthens in-game player bonds and further accelerates creative serendipity and discovery.

Once players complete a project, they submit their media artifacts through the website, then “justify” their work on video, explaining how they satisfied the conditions of their Deal. All this material — including a clickable list of cards used in the Deal, the completed project, the justification video, and the list of those who collaborated on the project (including links to their profiles) — then appears live on the game site, sharable with the world.

Some highlights from the game so far include:

• A beautifully-constructed Character Artifacts Deal, featuring hand-made love letters, mock telegraphs, and typewritten correspondence with mysterious “redacted” passages.
• A hilarious short film about the game itself, produced by a collaboration between dozens of students.
• A comedic sendup of police procedurals.
• A poetic video game about a man’s struggle with his own shadow.
• A pitch-perfect music video featuring choreographed dance and smoke machines.
• A wide-ranging discussion thread about the Long Take card, including video samples, recommendations, and other kinds of knowledge sharing (all cards in the deck have their own page where discussions like this can take place).

The website also serves as a social networking platform for SCA students, faculty and alumni. All players have profiles on the site which aggregate all their Deal-making activity and status updates, along with displaying any photos they have submitted to the site. Profiles also include an evolving data visualization that is generated based on the kinds of Deals and activities that the player has been involved in.

While most of the site is publicly viewable, including player profiles, some of the social networking functionality is semi-private, primarily because we wanted to create a kind of exclusive workshopping space (which we’ve named The Bullpen after a historic cinema school workshop space here at USC) where players could feel free to brainstorm, ramble, and even trash-talk “behind the curtain.” Other features not immediately visible to non-players include the Leaderboard, which tracks scores on a weekly and overall basis in a variety of dimensions, the Card Lookup feature, which players can use to view and discuss individual cards in the archive, and the Members Directory, which players can search by name or keyword when looking for collaborators or new connections.

Informal, Optional, “Secret”

The game is not mandatory for SCA students. It’s not even openly publicized at the school. In fact, we’ve gone to lengths to try to keep it under the radar. The Reality Committee, the benevolent group behind this project, wouldn’t have it any other way. The game is meant to belong to the players, not the other way around. Players have discovered it on their own, picking up on clues we’ve left around the campus — clues hidden in old cameras, left near our mysterious flag which intermittently hangs off the third floor balcony, or hanging from LED throwies we’ve stuck to the underside of staircases. One by one or in groups, they have come to the Game Office, gone through the initiation rites, received their game cards and website logins, and started playing.

After 15 weeks of gameplay, we have over 120 players, 122 completed projects, and 4000+ comments, status updates, and photo posts on the website. Would we have had engaged all 180 of the freshman students if we had made the game mandatory? Certainly. But we doubt they would have been as committed, or that the game would mean as much to them. Students discover this game the same way they discover things like the college radio station. They hear about it, and if they like the sounds of it, they show up and pour their hearts into it. And since everyone who comes to the School of Cinematic Arts is in some way interested in making media — or at least talking about making media — the game has a natural pull.

Finally, many players have responded to the various layers of intrigue and mystery that surround the origins of the game itself and the strangely subversive tone of the game’s communications. These more “traditional” ARG elements augment the collaborative production game with puzzles and secret codes that lead to additional encounters with alumni, special cards and game powers, and impromptu excursions.

To learn more, check out the public face of the site here, or follow us on Twitter. If you’re interested in participating — or just in having a peek into what’s going down in the Bullpen — send an email to remotedevice@gmail.com or contact @remotedevice on Twitter.

• Cultivate a community in LA of people with a shared interest in creating compelling pervasive and environmental games, including outdoor games, street games, ARGs and any other form of gaming that takes place in public space
• Meet on a regular basis to discuss pervasive games and critique and playtest each others’ projects
• Bring pervasive and environmental gaming to the general public through events such as Come Out and Play and IndieCade as well as stand-alone game activities

Co-founders: Sarah Brin, Celia Pearce, Jeff Watson

Join our Facebook Group

Working in teams, players collaboratively pursue wealth and prosperity. However, success in the game depends on face-to-face interactions with a cross-section of visitors to the Lower Manhattan Financial District. Performing an act of citizen journalism, players engage individuals to ask how the financial crisis (and recovery) may have impacted their lives. The stories are captured via a voicemail system and uploaded onto the game’s website along with additional gameplay imagery gathered by the players, forming a visual and audio mosaic of stories highlighting the financial, emotional, and mental consequences of an unstable and unregulated financial system.

New York’s Wall Street exists as a vibrant physical community as well as a powerful symbol in the collective imagination of the world’s interconnected financial systems, and as such, CLEAN UP WALL STREET makes an intervention in both registers by performing play at its physical location and by interrupting its symbolic narrative. In part inspired and informed by Charles H. Ferguson’s 2010 Academy Award winning documentary INSIDE JOB, the game playfully distills machinations of the highly complex derivatives market and highlights the dire and very real ramifications of deregulation.

Download (PDF, 94.68KB)

Tier One: Discover

As users move through the city — in their cars, on foot, or via public transit — the application works in the background, checking their location against a database containing media artifacts, historical information and updates about upcoming events. Any items the user passes are stored in their application for later review and the user is notified with an “Inbox”-style push notification. Items can include images, video, audio, or links to relevant websites. Users may also search the database for items of particular interest.

Tier Two: Collect

Users who want to dig deeper into the collection will discover that as they physically go to specific locations related to Pacific Standard Time, they “unlock” special content that is not available to users who have not visited those locations. This content can include exclusive interviews with artists, real postcards that are automatically mailed to the user, and other unique items. By rewarding users for “checking in” at various sites, we motivate a deeper engagement with the exhibition and its related events.

Tier Three: Participate

By moving through the city and checking in at specific sites, users not only unlock fresh media content — they also open up event- and location-specific opportunities for participation. These activities, thematically linked to individual exhbits, artists, movements, and time periods, provide engaged users with the ability to take part in special events and media-making challenges. These challenges will lead to a special “climax event” at the end of the exhibition’s run.

In the spring of 2011, I redesigned and launched a new community site for the USC School of Cinematic Arts Interactive Media Division. The site is designed to foster serendipity and discovery among students and faculty, creating vectors for social connection and creative collaboration. In addition to traditional blogging functionality, the site also features extensive microblogging capabilities (including the ability to embed media artifacts such as videos and images in status updates), an achievements system geared toward encouraging participation and awareness of the history and practice of interactive media-making, and a member profile infrastructure that integrates numerous social media features.

Contemporary automobiles contain a wide range of sensors and communication devices that serve a variety of safety and performance purposes. The “Ambient Storytelling for Vehicle-Driver Interaction” project explores how these systems can be integrated with cloud computing and social media services to make possible new modes of driver-to-vehicle and driver-to-driver interaction and engagement. Specifically, this project proposes methods that:

• Deepen the relationship between driver and car by using aggregate sensor data, context and location sensing protocols, and social media feeds to develop a narrative “lifelog” for the vehicle and its driver(s);
• Use sensor and lifelog data to create cinematic augmentations for the driving experience, including generative soundscapes, single- and multiplayer games, and driver-specific guided tours;
• Develop the character or identity of the vehicle by employing usage metrics and driver feedback to modulate the rate, style, and mode of vehicle-to-driver communications; and,
• Visualize the ecosystem of lifelog and character across a multitude of device contexts both inside and outside the vehicle, with an emphasis on touchscreen and gestural interfaces.

White Paper: Ambient Storytelling for Vehicle-Driver Interaction

Download (DOC, 564.5KB)

“It is better to live in a state of impermanence than in one of finality.” Gaston Bachelard, The Poetics of Space

Much of the early part of my qualifying exam preparation process was consumed with trying to come up with flexible-yet-precise titles for my exam areas. Since all knowledge is interconnected, this is always going to be a big challenge; but in the context of interdisciplinary new media theory and practice, I’m going to go out on a limb and say that it’s extra-super-hard to establish and maintain boundaries. Distinguishing between productive overlap (e.g. “History and Theories of Participatory Culture and Art Practice” is a powerful complement to “Interaction Design for Social Media and Pervasive Computing”) and outright redundancy (e.g. “The Poetics of Collaboration” would work as a subset of “History and Theories of Participatory Culture and Art Practice,” but would double-up on a lot of material if it were conceived of as a separate exam area) requires a whole lot of iteration and reshuffling. Some texts that began in one exam area are now comfortably in the middle of another. Because of the inherent either/or nature of the exam area structure (“text A is either in research area X, or it isn’t”), I found that some texts that were relevant to two or three of my research areas were being arbitrarily forced to exist in just one. What I really longed for was a non-hierarchical relational database that would provide an interface to my research alongside the more traditional “three exam areas” approach. Such a database would evolve as my own thoughts evolve. Like the impermanent, always-ahead-of-us dream space proposed by Bachelard, such an information architecture’s dynamism and impermanence would have the capacity to energize and liberate the imagination through serendipitous connection and unexpected emergence.

This website (http://remotedevice.net) is an effort to model what something like that might look like. The site automatically aggregates my posts to Twitter, Tumblr, my personal blog, several collaborative blogs, and my RSS feed bookmarks. These artifacts are then automatically and/or manually tagged with keywords, categorized, and made discoverable through sitewide searches. Some posts are further tagged so as to appear in the archives for my main research areas. The goal is that this will make visible and usable (to me and to anyone curious about my work) the connections between materials across the totality of my research.

“American Music” is an iPhone and Android application that uses contextual geolocation data as a kind of musical notation, creating a generative soundtrack to accompany everything from cross-country trips to short drives to the mall. The application uses the often repetitive landscape of American roadways (gas stations, fast food franchises, auto dealerships, etc) as a means to develop musical themes and figures. Patterns emerge as the application passes geodata information about nearby businesses, attractions, landmarks, and traffic features through a series of sound synthesis modules. Specific places acquire unique “sound signatures” that are sequenced and layered by the user’s passage through space. The result is a dynamic musical representation of 21st century American landscapes.

Media artifacts embedded in the real-world become visible on the map as the player approaches them.

The installation begins with a brief presentation outlining current research and development in the field of locative media arts. Following this survey of prior art, I provide attendees with instructions on how to register for the activity via their cell phones (almost any cell phone will work, although participants with more advanced iPhone or Android handsets will have an extended experience). Postcards are also made available at relevant locations to invite anyone who misses the presentation to register, as well as drawing in any non-conference-goers who may be interested in participating.

After participants have registered, they receive SMS communications from No File’s central character. Participants can respond to and converse with this character in the manner of an interactive fiction chatbot. Based on these interactions, participants discover narrative objectives which can be further investigated only by moving to particular places in the physical conference space and its environs; once there, the system detects their location (either via GPS or cell-tower triangulation) and unlocks additional material, enabling the participants to delve deeper into the themes of loss, departure, and rebirth that constitute the emotional terrain of the text.

Crucially, participants are not beholden to completing their traversal of this text in a single sequence of time; rather, they “carry the story with them” as No File’s narrative unfolds ambiently, augmenting each participant’s natural movements through the conference space. Interacting with No File thus occurs at the fringes of the day-to-day activity of the conference: in spare moments prior to the beginning of a session, while waiting for the elevator, late at night on the way back to one’s room, and so on.

is this ARG? is a social media aggregator I’ve built to gather feeds from the ARG community/affinity. The site is kind of like an amped-up Twitter list, displaying feeds from blogs, Delicious accounts, and other open/publicly-available social media sources — including the Unfiction forums and even Wikipedia.

The idea is to amplify the potential vectors for collaboration and research among and across the ARG community (and provide a handy real-time research tool). Everything that gets posted across the web by the sources listed on the site’s sidebar is gathered and put into a single stream on the root page. I’ve also made it possible to view the individual output of people/sources on the list by clicking on their names. There’s an FAQ on the site, and I’m also working on some more features that will roll out soon, including a way of archiving everything, curating venn-diagram-like clusters of streams, and generating keyword clouds to help with browsing through past posts.

The inspiration for this came in part from something Brooke Thompson wrote in reference to the ARGdb project, something to the effect of wanting to find a way to bring together all the energies members of the community are putting into different web fora. This is an effort in that direction, and also a bit of a curatorial endeavor: for me, making this project is kind of like editing a magazine or journal of sorts, populated not by articles or essays but rather by voices.

What if there was a hidden world layered atop visible reality that could only be seen by certain people? What if you were one of those people?

Viz is a casual augmented reality game that enables players to discover and embed virtual objects — or “vobjects” ((yup, i just said vobjects.)) — in physical space by looking through the camera viewfinders of their mobile phones.

By discovering and embedding vobjects, players earn points, advancing their positions on local, national and international leaderboards. These points also allow players to unlock special add-ons, tricks and bonuses for their vobjects (see “Variations and Extensions,” below).

Pervasive & Persistent Social Gaming

Players who install the Viz application on their phone can expect an ambient “always on” play experience that exists within the flow of their everyday lives. Players engage with Viz when they have spare moments in public spaces. The application icon, which sits inobtrusively in the notification area of the player’s phone (see figures below), works in the background and changes in appearance depending on the proximity of vobjects (i.e., a quick glance at your phone’s desktop tells you all you need to know). If the player wants to score points by locating these objects, they can view GPS information and activate the “scanner” tool by simply clicking on the application icon. The game is meant to accompany players as they move through space, providing active and mobile individuals with a persistent low-intensity creative social gaming outlet.

Players

Because of its dependency on user-created content, Viz requires a sizable player base in the areas where it will be played. Like a certain social networking service or other open-ended collaborative production games such as SF0, Viz will begin in specific neighborhoods and cities before expanding to national or international distribution. The game will be released in limited numbers to geographically-constrained player groups, beginning with student beta-testers at urban post-secondary educational institutions. The targeted distribution of the game will expand from there, first to tech subcultures in relevant urban centers, then to various segments of early adopters and tastemakers across the Web. These initial user groups will “seed” the environments of their respective urban areas, ensuring that secondary waves of users have no trouble discovering and interacting with active vobjects.

Objective

The objective of the game is to accumulate points by a) discovering vobjects other players have embedded in the physical world; and, b) having vobjects that you have embedded discovered by others.

Setup/Interface

Viz has two main “active” modes: scanning and embedding. Players toggle between these modes via a simple button interface. The game also has a persistent “passive” mode. This mode informs players of nearby vobjects via inobtrusive notification bar icons on their phone’s desktop. A “library” mode gives the player access to game statistics and other information.

Figure 1: Passive mode

Subtle alerts, similar to those used to denote wi-fi range/signal strength, appear in the notification bar when players are within 500 yards of a vobject (see icon in upper right corner). By clicking on the alert, players can activate the Viz interface.

• The application can also be launched by clicking on its icon in the applications drawer.
• The rest of the user’s phone works the same as always: the game tries hard not to be intrusive.

Figure 2: Viz Main Screen

Viz’s main screen changes in various ways according to the location and context of the player.

• If the player is outdoors (pictured), a simple compass-like feature will point in the direction of the nearest vobject.
• If the player is indoors and within range of one or more vobjects, the main screen will provide the player with additional information about the number and nature of any vobjects in the immediate area.
• The tabs at the bottom of the main screen toggle between modes and settings.
• Hardware controls enable the user to exit the app at any time.

Figure 3: Map mode

Viz’s map mode shows the location of nearby vobjects (red pins) relative to the player (blue pin). By consulting this map, players can move to within scanning range of a vobject.

• Clicking on a vobject causes a text balloon to pop up indicating whether it has been captured by the player. If the player has already captured the vobject (vobjects can only be captured once — see “Rules,” below), they will be able to click on the balloon and view relevant comments and statistics. If the player has yet to capture the vobject, the balloon will merely say, “Unknown.”
• Players can opt-in to share their location with other players. Map mode will display any participating players as yellow pins.
• Players can zoom in and out and pan the map as in other applications.

Figure 4: Scan mode

Once players have reached the vobject’s GPS location, they can scan the immediate area for the vobject itself by switching Viz into scanning mode. Scanning mode uses the phone’s built-in camera functionality to turn the screen into a kind of window through which the player can look. The player finds the vobject by panning the camera around, examining visual features of their surroundings until they find the feature that contains the vobject. This feature is an image stored in the Viz database and streamed to the phone based on the player’s present location. Viz “locks on” to features by comparing the original embedded image to the image currently visible through the player’s phone. This effect is achieved through the use of image recognition and orientation systems like those recently developed by Nokia or Microsoft. Once players have locked on to a vobject, they can press the “Capture” button (the only button on the scanning screen) to add it to their library and collect points.

• Vibration feedback and various visual cues let the player know when they pan past a vobject.
• Image ghosting helps players to lock in the image.
• A standard dialog box pops up when the player successfully captures a vobject, informing them about how many points they have scored. Players can click on “more” to view a separate info screen containing statistics and comments about the vobject. This screen also contains a submission form enabling players to add their own comments. Once a player has captured a vobject, this info screen is accessible from their Library (see below).
• The “Capture” button appears grayed-out until the image is framed within the approximate boundaries of the original (embedded) image.

Figure 5: Embed mode

Once players have accumulated enough points to embed a vobject, they may do so by switching Viz into embed mode. In this mode, players can point their phone’s camera at an object in the physical world to which they wish to attach a vobject. By pressing the “Embed” button, players will expend points and attach a vobject linked to their profile to the physical object, building or landscape they see in their viewfinder. This vobject now becomes something for other players to find (and therefore a new way for the embedding player to earn points).

• Embedding a vobject is a simple point-and-shoot operation.
• Players can consult an “Image Legibility” reading while in embed mode to make sure that the vobject they are embedding will be easily capturable by anyone who finds it.
• The embed screen is only available to players who have enough points to embed a vobject. If a player attempts to switch into embed mode when they do not have enough points to embed a vobject, an alert box (not pictured) will inform them that they need to earn more points before they can complete this action.
• If a player has purchased or unlocked a power-up — for example, a bonus modifier of 10 points/scan — they will be asked if they would like to use this power-up on the vobject they are embedding. This process is discussed below in both “Rules” and “Variations and Extensions.”

Figure 6: The Library

Library mode enables players to view lists of the vobjects they have captured, the vobjects they have embedded, and the points the have earned. The screen also contains a button that launches a Settings window (not pictured) wherein players can adjust profile information.

• Two simple buttons allow players to toggle between lists of vobjects they have captured and vobjects they have embedded.
• Clicking on an individual vobject in a list opens a dialog containing additional information about that vobject, including how many times it has been scanned, when and where it was embedded, and who embedded it.
• Profile information can be modified in the “Settings” screen (not pictured).
• All lists are scrollable to conserve screen space.

Procedures/Controls

Read Map To locate vobjects, players will activate the GPS map function of Viz to help them get within range.

Capture Vobject Once players are near a vobject, they will scan the area using Viz’s camera function until they “see” the vobject superimposed over real space. They will then press the “Capture” button to collect the vobject and score points.

Embed Vobject To embed vobjects, players must first accumulate enough points to do so. Once they have the requisite points, players can activate Viz’s embedding tool. This tool enables players to embed a vobject in a particular location by pointing their phone’s camera at a real-world object, building or landscape. Once the player has selected the view to which they want to attach their object, they will press the “embed” button. Doing so subtracts points from the player’s account.

Check stats Players can read statistics about their own usage, how many points they have, and how many points they earn from embedded vobjects by switching to library mode.

Review Vobjects Library mode also enables players to view lists of the vobjects they have captured or embedded; by clicking on the vobject identifiers (GPS coordinates or city/postal code designators), players can view additional information about each vobject.

Unlock Power-ups Players can use points to unlock power-ups which modify the way the vobjects they embed tally scores and affect other players. The power-ups button in library mode accesses a list (not pictured) of power-ups, organized by price. Power-ups that players have purchased are displayed in this list with a star next to them.

Use Power-ups Players can attach a power-up to a vobject as they embed it in a physical place. If a player has any power-ups to use, they will be given the option to use them at the time of embedding.

Modify profile settings Players can change their username, home location and other information in a standard profile/settings dialog. This profile can also be modified via an online web interface.

Flow/Game States

See attached flow chart (pictured below).

Rules

Beyond the rules implied and specified above, readers should be aware of the following scoring rules and constraints:

• Whenever a vobject is discovered, points are awarded to both the discoverer and the embedder.
• Objects that are discovered on a regular basis score fewer points than objects that are only viewed occasionally. Players can thus adopt a variety of strategies for accumulating points: for example, player A might embed objects in obvious public spaces (e.g. tourist sites, popular cafes, etc) in order to get a large volume of low-value discoveries, while player B might embed fewer objects in more hard-to-find locations in order to get a small volume of high-value discoveries.
• Any objects that are not found within a period of thirty days will disappear. This constraint prevents players from embedding objects in overly obscure locations and adds an element of gambling to certain placement strategies — e.g. players must balance the risk of their object not being found at all against the potential gain of it being found only once or twice.
• Embedding vobjects (and unlocking extra functionality) “costs” points, motivating players to earn as much as possible.
• Vobjects may only be scanned once. Players may not scan their own Vobjects.

Variations

Power-ups

When embedding, players can opt to attach “power-ups” to their vobjects. These power-ups have a variety of effects on scoring and gameplay. Below are two proposed power-ups and their effects:

• Trap The Trap power-up causes the player who scans the vobject to lose 500 points. The embedder of the vobject receives these points in addition to those they collect for the vobject being scanned.
• Trade When a vobject imbued with the Trade power-up is scanned, the player who scans it becomes the owner. In exchange, the former owner of the vobject receives one of the scanning player’s vobjects (selected at random). If the scanning player has no embedded vobjects in play, the Trade power-up is lost.

Other

Finding and Leaving Traces

Viz is an inherently social game. Players get to experience the thrill of discovering secret objects left behind by other people, often in scenic or humorous locations. Interacting with these objects reveals additional layers of sociality: for example, upon discovering a vobject, players may choose to attach a voice or text “comment” to the item, which can be then read (and responded to) by the vobject’s embedder and any subsequent discoverers. Importantly, however, players don’t need to comment on vobjects in order to experience the social dimension of Viz — even if players ignore all but the most basic affordances of the game, they will engage in the fundamentally social activity of finding and leaving traces of passage. In this sense, embedded vobjects are reminiscent of the Inuit inuksuks (or inuksuit) found in the far North:

An inuksuk… is a man-made stone landmark or cairn, used by the Inuit, Inupiat, Kalaallit, Yupik, and other peoples of the Arctic region of North America, from Alaska to Greenland…

The inuksuk may have been used for navigation, as a point of reference, a marker for hunting grounds, or as a food cache. The word inuksuk means “something which acts for or performs the function of a person.” The word comes from the morphemes inuk (“person”) and -suk (“ersatz” or “substitute”)… (Wikipedia: Inuksuk)

Inspiration

This idea sprung from two sources. First, a whiteboard session with my colleagues Amanda Tasse and Lauren Fenton got me thinking about the issues involved in making an augmented reality game with a social dimension (our original idea was a kind of “truth or dare” casual mobile app wherein players could discover other players in proximate space via GPS or other location/context-sensing technologies and challenge them to do humiliating and/or dangerous things). This brainstorming session raised a few intriguing questions: What kinds of ambient AR gaming experiences would players reasonably want to layer atop their “real” lives? What motivation would they have to participate? And, crucially, what kinds of game mechanics can we create with AR that simply can’t be created any other way (ie, how can we avoid simply using AR as a gimmick)?

The second source of inspiration is largely a consequence of the first. In an effort to answer some of the questions raised during the whiteboard session with Amanda and Lauren, I started looking around the web for existing AR games and apps. I found plenty of cool examples. But the most thought-provoking discovery occurred when I came across Denno- Coil, a recent Japanese science fiction series about kids living in a world wherein AR has been seemlessly integrated into everyone’s lives via special eyeglasses and visors. The story and setting conceits of Denno Coil are a goldmine for AR developers. Here are a few notes I clipped from the Wikipedia article on the series [Accessed 20 September 2009] as I was developing my own game concept:

• [Users] access the virtual world through Internet-connected visors called denno- eyeglasses. This allows them to see virtual reality superimposed on objective reality. To visually confirm something as virtual, the children often lift their glasses from their eyes. The visors also work in conjunction with futuristic ear monitors placed behind the ear, which allows them to hear sounds from the virtual environment.
• Software tools are visually represented as tactile tools (e.g. a fishing rod) which occupy 3D virtual space and must be manipulated by hand. Metatags, which can be used to damage or enhance virtual objects with certain properties, are visually represented as o-fuda.
• Virtual objects such as pets cannot be recalled or reset; when a pet runs away, it must be chased and caught in 3D space. Virtual objects and pets are also susceptible to a form of “death” by data corruption or deletion.
• Fumie Hashimoto handles a software spray can loaded with so-called “black bug spray”.
• A virus on the run…leaves behind a (virtually) tangible, evaporating black trail of raw “denno- substance”.

See also: Serial Experiments Lain

If a building could talk, what would it say? How would it “feel” about the comings and goings of the people who use it every day? Would it be affected by their moods and desires? What kind of relationship would it have with its occupants if it could communicate with them somehow, and how would they respond? Perhaps most importantly, would such a feeling-and-talking building even be desirable?

In attempting to answer these whimsical questions, the Mobile and Environmental Media Lab (MEML) at the University of Southern California conceived of Million Story Building, an experimental design project exploring how location-specific mobile technology can add playful, imaginative and practical new layers to the relationship between a structure and its inhabitants.

Using the newly-constructed School of Cinematic Arts (SCA) Building as a test bed, the MEML team has designed a location-sensitive iPhone application (see screenshots below) that enables students and faculty to engage with their workplace in a variety of exciting new ways, from scanning Quick Response (QR) code glyphs mounted next to posters in the hallways in order to access and tag video clips from a central database, to leaving virtual messages for others to read in an Augmented Reality view of the building’s central courtyard.

Functions such as these, working together with networks of sensors, interactive plasma screens and web-based social media profiles, made it possible for iPhone-carrying building occupants to learn about SCA history, discover events and activities, share stories and updates of their own, enrich the school’s media archive, and participate in an Alternate Reality Game.

Furthermore, and crucially, users could come to expect increasingly customized communications, behaviors and interaction opportunities as a profile concerning their preferences, habits and interests was generated based on their usage of the system. The end result was a prototype for a personalized and self-renewing “ambient story” experience co-constructed by the collaboration between the occupants of a building and the building itself.

iPhone App Image Gallery

Background Notes

At its heart, Million Story Building is an effort to mobilize a range of storytelling and interaction tactics such that the occupants of the SCA building can experience a deeper and richer connection with their workspace and co-workers. Such an effort is not unique in the history of design; indeed, the practice of embedding story and play in physical space is almost as old as civilization itself – one recalls the Catholic Via dolorosa (Stations of the Cross), the Shingon Buddhist pilgrimage of Shikoku or the transit to the Temple of the Sun at Teotihuacan.

More recently, what Jill Walker-Rettberg calls “distributed narratives” have manifested themselves in public and institutional space through sticker art, coordinated graffiti campaigns, flash mobs and other interventions. Million Story Building emerges at the nexus of these practices and the new potentialities unleashed by the recent blossoming of mobile and ubiquitous computational technologies.

More Information

The Million Story Building series archive is an effort to gather together, enframe and contextualize key planning documents from the first phase of the project’s development. By browsing proposals, demo descriptions, annotated meeting notes and other development documents, readers can get a sense of the key concerns from which the project arose and the various components of the prototype application, both implemented and imagined.

Code & Notes

The Processing code is actually very simple. The program reads range data from the Arduino via the serial port, then triggers samples and visualizer elements according to the distances reported. Any set of samples could be used (for this version, I’m using 16 different short sound textures generated with Reason), and it’s easy to imagine how the code could be modified to send MIDI signals instead of triggering samples.

// SONAR TONEBANK

// 1.0
// Jeff Watson
// 27 April 2009

import processing.serial.*;
import ddf.minim.*;

// arduino variables

Serial myPort; // The serial port

// video variables

// minim variables

Minim minim;

AudioSample bell;
AudioSample bell2;
AudioSample bell3;
AudioSample bell4;
AudioSample bell5;
AudioSample bell6;
AudioSample bell7;
AudioSample bell8;
AudioSample bell9;
AudioSample bell10;
AudioSample bell11;
AudioSample bell12;
AudioSample bell13;
AudioSample bell14;
AudioSample bell15;
AudioSample bell16;

// visualizer variables

void setup () {

size(640, 480, P2D);
background(0);
noStroke();

minim = new Minim(this);
bell = minim.loadSample("F#3.wav", 2048);
bell2 = minim.loadSample("G#3.wav", 2048);
bell3 = minim.loadSample("A#3.wav", 2048);
bell4 = minim.loadSample("C#4.wav", 2048);
bell5 = minim.loadSample("D#4.wav", 2048);
bell6 = minim.loadSample("F#4.wav", 2048);
bell7 = minim.loadSample("G#4.wav", 2048);
bell8 = minim.loadSample("A#4.wav", 2048);
bell9 = minim.loadSample("C#5.wav", 2048);
bell10 = minim.loadSample("D#5.wav", 2048);
bell11 = minim.loadSample("F#5.wav", 2048);
bell12 = minim.loadSample("G#5.wav", 2048);
bell13 = minim.loadSample("A#5.wav", 2048);
bell14 = minim.loadSample("C#6.wav", 2048);
bell15 = minim.loadSample("D#6.wav", 2048);
bell16 = minim.loadSample("F#6.wav", 2048);

// List all the available serial ports
println(Serial.list());
// Open whatever port is the one you're using.
myPort = new Serial(this, Serial.list()[1], 9600);
// don't generate a serialEvent() unless you get a newline character:
myPort.bufferUntil('\n');
}

void draw () {
// everything happens in the serialEvent()
}

void serialEvent (Serial myPort) {
// get the ASCII string:
String inString = myPort.readStringUntil('\n');

if (inString != null) {
// trim off any whitespace:
inString = trim(inString);
// convert to an int and map to the screen height:
float inByte = float(inString);
int VideoMod = int(inString);

println(VideoMod);

// 1st Octave

if ( VideoMod < 5 )
{
bell.trigger();
delay(33);
fill(0,0,255);
rect(1,0,128,480);
}

else if ( VideoMod < 8 )
{
bell2.trigger();
delay(33);
fill(13,82,122);
rect(129,0,128,480);
}

else if ( VideoMod < 11 )
{
bell3.trigger();
delay(33);
fill(0,0,255);
rect(257,0,128,480);
}

else if ( VideoMod < 14 )
{
bell4.trigger();
delay(33);
fill(13,82,122);
rect(385,0,128,480);
}

else if ( VideoMod < 17 )
{
bell5.trigger();
delay(33);
fill(0,0,255);
rect(513,0,128,480);
}

// 2nd Octave

else if ( VideoMod < 20 )
{
bell6.trigger();
delay(33);
fill(0,0,0);
rect(0,0,640,480);
fill(0,0,255,191);
rect(1,0,128,480);
}

else if ( VideoMod < 23 )
{
bell7.trigger();
delay(33);
fill(0,0,0);
rect(0,0,640,480);
fill(13,82,122,191);
rect(129,0,128,480);
}

else if ( VideoMod < 26 )
{
bell8.trigger();
delay(33);
fill(0,0,0);
rect(0,0,640,480);
fill(0,0,255,191);
rect(257,0,128,480);
}

else if ( VideoMod < 29 )
{
bell9.trigger();
delay(33);
fill(0,0,0);
rect(0,0,640,480);
fill(13,82,122,191);
rect(385,0,128,480);
}

else if ( VideoMod < 32 )
{
bell10.trigger();
delay(33);
fill(0,0,0);
rect(0,0,640,480);
fill(0,0,255,191);
rect(513,0,128,480);
}

// 3rd Octave

else if ( VideoMod < 35 )
{
bell11.trigger();
delay(33);
fill(0,0,0);
rect(0,100,640,380);
fill(0,0,255,127);
rect(1,0,128,480);
}

else if ( VideoMod < 38 )
{
bell12.trigger();
delay(33);
fill(0,0,0);
rect(0,100,640,380);
fill(13,82,122,191);
rect(129,0,128,480);
}

else if ( VideoMod < 41 )
{
bell13.trigger();
delay(33);
fill(0,0,0);
rect(0,100,640,380);
fill(0,0,255,191);
rect(257,0,128,480);
}

else if ( VideoMod < 44 )
{
bell14.trigger();
delay(33);
fill(0,0,0);
rect(0,100,640,380);
fill(13,82,122,191);
rect(385,0,128,480);
}

else if ( VideoMod < 47 )
{
bell15.trigger();
delay(33);
fill(0,0,0);
rect(0,100,640,380);
fill(0,0,255,191);
rect(513,0,128,480);
}

// Extra note clears screen

else if ( VideoMod < 50 )
{
bell16.trigger();
delay(33);
background(0);
}

else if ( VideoMod > 50 )
{
}

}
}

See also: Arduino-Graph

The Arduino code is also pretty straight-forward. All it does is translate the raw PING sonar data into easier-to-handle Imperial and metric measurements, which are then sent out the serial port to Processing.

int pingPin = 7;

void setup()
{
Serial.begin(9600);
}

void loop()
{
long duration, inches, cm;

// The PING))) is triggered by a HIGH pulse of 2 or more microseconds.
// We give a short LOW pulse beforehand to ensure a clean HIGH pulse.
pinMode(pingPin, OUTPUT);
digitalWrite(pingPin, LOW);
delayMicroseconds(2);
digitalWrite(pingPin, HIGH);
delayMicroseconds(5);
digitalWrite(pingPin, LOW);

// The same pin is used to read the signal from the PING))): a HIGH
// pulse whose duration is the time (in microseconds) from the sending
// of the ping to the reception of its echo off of an object.
pinMode(pingPin, INPUT);
duration = pulseIn(pingPin, HIGH);

// convert the time into a distance
inches = microsecondsToInches(duration);
cm = microsecondsToCentimeters(duration);

Serial.println(inches);

delay(100);
}

long microsecondsToInches(long microseconds)
{
// According to Parallax's datasheet for the PING))), there are
// 73.746 microseconds per inch (i.e. sound travels at 1130 feet per
// second). This gives the distance travelled by the ping, outbound
// and return, so we divide by 2 to get the distance of the obstacle.
// See: http://www.parallax.com/dl/docs/prod/acc/28015-PING-v1.3.pdf
return microseconds / 74 / 2;
}

long microsecondsToCentimeters(long microseconds)
{
// The speed of sound is 340 m/s or 29 microseconds per centimeter.
// The ping travels out and back, so to find the distance of the
// object we take half of the distance travelled.
return microseconds / 29 / 2;
}

See also: Arduino-Ping

During the early prototyping phase, I also made a simple Theremin using the PING sensor, based on Alberto Bietti’s excellent documentation. The piezo speaker and button on the current version of the project are remnants of this prototype; I’ve left them on there for now for demoing purposes.

This program was created using Processing and RiTA. The source material is a dream log I kept during the summer of 2008.

Click here to launch Dream_Jrnl in a separate window. Interact with the application by clicking, moving the mouse or typing.

Kcolc is a visual/textual timepiece suitable for display on billboards or other large public displays.

This program was created using Processing. Click here to launch Kcolc in a separate window.

S.T.I.F.L.E. is an interactive fiction about broken hearts, experimental pharmaceuticals and wandering philosophers. Written using Inform 7, Fall 2008.

Play here (requires Java): http://remotedevice.net/if/

Flash application screen Capture

The videos gathered from participants in the CityStory experiment were assembled into a digital archive. An interactive Flash application was written to display the videos and text snippets based on user-selected keyword tags. (A generative music soundtrack was also created on the fly by the application, but is not audible in the above screen capture.)

Project blog

Feature-length Screenplays

Marathon
The story of a remote northern Ontario community haunted by the ghosts of the German POW camp that once stood on its outskirts. (.pdf)

Astoria
A genre-bending story of a down-and-out boxing trainer who falls in love with a woman with a secret. (.pdf)

Short Screenplays

The Offer
A short-short about two prisoners with a Hitchcockian twist. Produced at the Canadian Film Centre. Directed by Eric Plummer. (.pdf)

Move 36
A short film about a chess-playing recluse, a phone sex operator and an old man’s legacy. Produced at the Canadian Film Centre. Directed by Sean Frewer. (.pdf)

Synopses

Project Portfolio
A selection of synopses and proposals for feature-length screenplays. (.pdf)

Screenplay Coverage

Coverage sample 1 (.pdf)
Coverage sample 2 (.pdf)
Coverage sample 3 (.pdf)

All material © Jeff Watson.

There are no limitations as to the subject matter, structure or intention of the films curated by the DiCE project. Categories such as “fiction”, “experimental” or “documentary” do not apply here. Participating filmmakers need only observe the following limitations:

• Standard length: Each DiCE film curated for this phase of the initiative will be precisely one hour in length. This is the full duration of the DV tape included in the DiCE FilmKit.
• No editing outside of camera: The editorial moment for a DiCE film occurs simultaneous to the moment of inscription. Filmmakers are not permitted to edit or modify their footage using any equipment exterior to the camera itself.
• Filmmakers may, if they wish, rewind or cue the tape within the camera to capture certain shots out of sequence.
• The completed DiCE film must be returned to Blue Blouse immediately after filming ends. The tape may not be tampered with or exchanged.
• If a tape breaks or otherwise becomes unusable before the one hour time limit has been reached, filmmakers should consider their DiCE film complete and submit it immediately.

Representative Project

“Bobby” is a short video created to give participating filmmakers a sense of the kind of work they could create using the DiCE constraints. The video, a documentary about a troubled yet optimistic mental health survivor, was shot over the course of two days in the Summer of 2007. Originally intended as a full-length DiCE project, the video was cut short after the cassette tape jammed in my camera. Significantly, the moment of the film’s termination, imposed upon the narrative by the DiCE ruleset (rule 5, above), is productive of a strangely satisfying ending.