Projects / Mentality

Mentality is a 3D knowledge-mapping system that uses semantic networks (or “mind maps”) created with a modified version of IHMC cMap Tools to procedurally generate information-rich virtual environments. These environments will enable users to navigate conceptual space using architectural metaphors.

Mentality is envisioned for use in educational and project-development settings as an information visualization tool; other potential applications include use in health care/recovery as an assistive mnemonic system for patients dealing with memory loss, architecture as a means for analyzing flow and connectivity between discretely-purposed spaces, blogging as a way to create personal spatial archives, social networking as a way to map and enable relations between friends and affinity groups, and game design as a level-authoring tool. Finally, Mentality points the way toward a more thorough implementation of the so-called “3D Web” by offering a critique of the architectural metaphors underlying current online virtual spaces.

Technical Outline

In its simplest form, Mentality is a mash-up of mind-mapping and computer-assisted design (CAD) software. The application renders a “room” for each node on a source mind map, and a “hallway” for every connection between nodes (and, ultimately, other mind maps). In addition to creating nodes and connections, users can attach media objects, such as images, 3D models, interactive widgets, audio or video clips, web pages or autonomous software agents to individual nodes for examination and interaction. The virtual space will be generated procedurally at runtime, meaning any additions to or modifications of the cMap file will be represented immediately. The system will accommodate simultaneous authorship and access by multiple users.

The initial implementation of Mentality is envisioned as a way of spatially visualizing the rhizomatic structures of semantic networks. To get a picture of how this might work, imagine a simple semantic network on the subject of American history. To create this network, one might begin by writing down the names of a few presidents, say, “Washington,” “Lincoln” and “Kennedy.” One might then connect all three of these nodes with a new node called “Presidents.” Going further, “Lincoln” could be connected to “Republicans,” “Kennedy” to “Democrats,” and “Washington” to “Revolutionaries.” This process could be iterated at length according to the needs and knowledge base of the user (indeed, it’s not that difficult to imagine a system that could do this automatically, by stripping such connections from an already highly-interconnected information archive such as Wikipedia, Facebook or the Web itself). Furthermore, the user can add media artifacts to each node. For example, the “Kennedy” node could contain an audio clip of Kennedy’s famous “To the Moon” speech, while the “Presidents” node could contain an image of the Presidential Seal.

As the user adds nodes, connections and media artifacts to the semantic network, this information is passed to Mentality, which procedurally generates a spatial representation of the network by finding the most efficient uninterrupted paths (“hallways”) between nodes (“rooms”) and constructing a legible architectural environment to render these connections. When the user changes the semantic network underpinning the structure, these changes will be reflected in real-time, meaning that while connections between nodes may remain unaltered, the specific spatial/architectural arrangement of the nodes and connections will be dynamic, changing according to the needs of the larger system of connectivity demanded by the knowledge base.

Since this architectural knowledge representation will be created in three dimensions, sloping and curved hallways, “staircases,” “ladders” and “chutes” will enable a high degree of one- and two-way connectivity between nodes while maintaining a coherent (although fanciful) spatiality. A variety of algorithms will be tested as the project evolves in order to explore different approaches to rendering this architectural model. For example, in cases of heavily-connected nodes wherein the number of required exit pathways exceeds the limitations of architectural coherence, certain pathways could be shared, branching off to separate nodes prior to termination; alternatively, room size could increase in direct proportion to the number of connections.

Despite the potentially massive architectural representations that such a system suggests, Mentality is intended to make the most of available graphics processing memory and RAM, while leaving a relatively small footprint on data storage and network bandwidth. Because the virtual environment in Mentality is generated at runtime based on an algorithmic interpretation of mind map data, the representation itself is not dependent on huge amounts of storage space or bandwidth; all that is required is the map of connections between nodes and the set of rules governing how to display such a map, meaning the primary bandwidth/storage-intensive elements are the media artifacts within each node. In this sense, the system is akin to a web browser: it is an interpreter of information, rather than an archive. The textures and lighting that flesh out Mentality’s virtual spaces could also be generated procedurally, using techniques similar to those developed by .theprodukkt. Recent industrial and academic work in procedural world and texture generation suggest that this is a promising line of development.

Clearly, successful implementation of Mentality depends on being able to overcome many significant technical challenges. The project will require collaborative work with a diverse range of programmers, cognitive scientists, designers and artists. By working towards the creation of a prototype system, I hope to explore the range of possibilities, problems and theoretical implications presented by procedural spatial representation, and in so doing, generate new questions and objectives for future research. Some of my current thoughts on the implications of Mentality in the development of Massively Multiplayer Online (MMO) gamespaces can be found in the “Discussion” section below.

Discussion: Procedural MMO Gamespaces

The fluid adaptive knowledge spaces proposed by Mentality present a direct challenge to contemporary virtual spaces that use architectural metaphors as a way of systematizing information and organizing communication. Second Life, for example, uses architectural metaphors in an effort to create what proponents describe as the “3D Web.” But if systems like Second Life are at least partially intended to point the way toward a truly “spatial” Web, then for the most part, they fall short of the mark.

Despite recent online and ubicomp shifts toward dynamic data collection, display and representation, information in social MMO gamespaces such as Second Life is presented in a static, hard-coded manner: while certain aspects of the architecture are dynamic – virtual screens, for example, might grab XML feeds to display changing content – the architecture itself is decidedly non-dynamic. Buildings are fixed in space and layout unless their owners engage in the laborious process of redesigning them and/or spend (real) money to relocate. Nowhere in Second Life is there a significant computational feedback between the information content of the virtual spaces and/or their owners and the organization of the spaces themselves. In this sense, Second Life is akin to the early days of the Web: a network of static, hand-coded sites updated only when the author sees fit to do so.

Furthermore, and crucially, the spatial connections between sites in Second LIfe are limited by the location of their initial instantiation: once users set up their virtual homestead, their immediate neighborhood is fixed. The resulting irony is that the geographical metaphor supposedly guiding Second Life becomes increasingly irrelevant over time, as even the most fundamental affordance of the Web – the ability to create hyperlinks – is not adequately represented using the spatial metaphor: if your friends aren’t located in an immediately proximate zone, you’re going to have to “teleport” to see them (whereas in a fully-implemented spatial metaphor, affinity and social networks would be represented architecturally, with proximity reflecting both usage and density of connections). As such, Second Life often takes on a ghostly, abandoned and untraveled feel, much like a selection of rarely-updated websites left over from the mid-1990s.

Like Second Life, the central question asked by Mentality concerns the utility of virtual spaces in communication and representation. But while Second Life suggests that these spaces must adhere to a certain baseline of spatial verisimilitude in order to be usable (ie, the virtual world must be represented as a fixed and contiguous landmass), Mentality asks for the reexamination of such limitations.

A hypothetical MMO virtual world based on Mentality would be a dynamic self-organizing space, shifting form based on context and the connections created by its users. Virtual “neighborhoods” would be organized not on the basis of property acquisition and fixed geography, but rather in terms of affinity networks and taste fabrics. Proximity would be mapped to density of connections and usage patterns; geography and architecture would thus shift to accommodate new connections and activities on a highly contextual basis. How would users react to such an ephemeral and transitory spatial representation? Would this kind of dynamic, contextually-generated environment be of greater utility than geographically static worlds that mimic the physical and economic structures of the real world? Or would it just be confusing? And finally, how would the system navigate and reconcile the multiplicity of contexts implicit in a massively multiplayer gamespace?

Investigating these and other questions raised by the notion of procedural MMO gamespace generation is a core objective of the Mentality project. In concert with several other current projects addressing this issue – from Georgia Tech’s Charbitat to Will Wright’s Spore – Mentality is intended to assist in the creation of a vision for the next generation of networked virtual spaces.