From Augmentation to Articulation: (Hyper)linking the Locations of Public Writing

Jacob Greene, University of Florida

(Published April 4, 2017)

 

“. . . [r]hetorical power can be activated by those who are equipped to articulate available openings in discourse.”
–Sharon Crowley, Toward a Civil Discourse

“A trout, having snatched at a hook but having had the good luck to escape with a rip in his jaw, may even show by his wiliness thereafter that he can revise his critical appraisals.”
–Kenneth Burke, Permanence and Change

According to Gunner Brammer’s October 12th, 2015 post on the Flymen Fishing Company blog, if you really want to catch trout, then you need to be using articulated streamers. Unlike “dry flies,” which rest on top of the water, streamers glide just under the water’s surface where their flexible joints can “articulate” the serpentine motion of small baitfish such as minnows. As Brammer notes, fishing with an articulated streamer is not as simple as purchasing “a streamer with an articulation joint.” Rather, it is a process of constructing temporary material relationships (between factors like streamer length, joint tension, and knot types) such that the streamer “will articulate” as fish food.

However, articulation does not simply emerge out of the streamer’s potential to articulate. The streamer does not actually “articulate” until it begins to move through the water, thereby (per)forming an expressive, interpretable sign for the trout. In other words, the articulated streamer leverages its agency as a minnow through the actions of the angler, and the angler leverages her agency as person-eating-fish-for-dinner through the material rhetoric of the streamer. Moreover, for both streamer and angler, this rhetorical action (“minnow-ness”) only emerges when it is articulated through a confluence of already-present environmental variables from the color of the riverbed to the current of the water to the “wiliness” of the trout (Burke 5). Following Cheryl Geisler’s report on the Alliance of Rhetoric Societies’ conception of rhetorical agency, we might say that although both angler and streamer hold the “capacity” of minnow-ness, they can only act on this agential capacity by entering into a co-productive relationship with the various environmental constraints and affordances (e.g., current, wind patterns, etc.) that serve as the “point of articulation” for this emergent interaction between human, bait, trout, and river (Gaonkar qtd. in Geisler 10).

In disguising herself behind the very movement through which her presence is made known to the fish, the angler’s role is similar to the analogy drawn by Bruno Latour between the work of the scientist and the work of the stage manager. Latour writes that scientist and stage manager alike work toward “disappearance” so their respective audiences will focus on the work itself rather than their role in producing it (Pandora’s Hope 135). In all three cases—fishing, scientific work, stage managing—human actors augment (or “add” to) a situation to participate in an emergent rhetorical act alongside (and through) other humans, nonhumans, and environments. Although engaging in any rhetorical act requires augmenting a situation in some way—writing an essay, designing a poster, reeling in a fishing line—this “additional” rhetorical element is not solely responsible for all of the changes that might emerge as a result of its addition. Rather, this act participates within an emergent network of potential relations and rhetorical interactions. Consequently, agency is “dispersed, as a series of articulated networks” across the human and nonhuman actors inter-acting in the situation (Wells qtd. in Geisler 11).

This article claims that creators of location-based public writing projects should focus less on how a location can be “augmented” and more on how it can be “articulated.” I draw on articulation theory to outline a rhetorical practice for designing location-based, augmented reality experiences with mobile technologies.1 Whereas “augmentation” ascribes to a rhetoric of addition-as-improvement, “articulation,” I argue, encourages us to approach location-based AR design as a process of creating and identifying emergent connections between the digital content of the application and its material surroundings. Popular applications of AR construe it as a technology for enhancing the user’s environment by 1) overlaying digital objects that represent physical objects and 2) overlaying real-time data streams that enhance the user’s ability to interact with her immediate surroundings. Such applications are certainly useful, beneficial, and potentially lifesaving.2 However, they also perpetuate a valuation of AR content primarily in terms of human agency, thereby neglecting the distributed agential forces embedded within one’s environment. As a way of demonstrating alternative trajectories for AR technology, I turn to a discussion of how artists are leveraging AR to articulate new perceptions of public spaces, thereby modeling emerging genres of digital counter-discourse. Finally, I demonstrate how articulation operates as a rhetorical design strategy within the context of a location-based, mobile AR application that I co-created for use within the SeaWorld-Orlando amusement park. The “SeeWorld” mobile app allows visitors to access digital overlays detailing the park’s efforts to disguise the hazards of marine captivity behind a rhetoric of entertainment and conservation. From surveying the amusement park to creating multimedia content for the application overlays, SeeWorld demonstrates how articulating a location through AR is less an effect of any particular technology and more of a design strategy that must be actively engaged throughout the entire creation process.

The term “augmented reality,” though firmly established within our cultural lexicon, overstates the importance of digital content being added at the expense of the environment from which it co-produces its rhetorical efficacy. Like fishing with an articulated streamer, mobile AR is less a technology for imposing rhetorical agency onto a location and more a practice of co-creating rhetorical agency alongside one’s material environment. When a writer “augments” a location, she merely adds digital content to the user’s experience of the location; however, when a writer “articulates” a location, she discovers and creates relationships between the human and non-human entities already circulating within it as potential rhetorical actors.

Articulating Articulation: From Cultural Studies to Rhetorical Theory

“(In the Latourian parlor, we worry about the furniture as much as the conversation.)”
–Paul Lynch and Nathaniel Rivers, Thinking with Bruno Latour in Rhetoric and Composition

Colloquially, to “be articulate” often means to speak (or write) with a clarity of thought and style, and it is often used as a synonym for “eloquent.”3 For example, if I am having trouble conveying a particularly complex idea, I might say that “I know what I want to say, I just can’t quite articulate it.” But what do we mean when we say that a piece of writing needs to be “articulated” more clearly? In this sense, “articulate” seems to refer to how a writer’s words, phrases, and sentences work together for rhetorical effect. Indeed, to say that a text is clearly articulated often implies that it is persuasive or at least impactful in some way. I might say that a writer presented a particularly “well-articulated” overview of a concept or topic even if their text did not “add” anything new to the conversation. In everyday speech, then, “articulation” seems to refer less to what we are saying than to how we are saying it. In a linguistic sense, then, articulation is the process of creating temporary, contingent connections in language through which familiar ideas begin to emerge as something new, thereby endowing them with a newfound communicability and rhetorical efficacy.

The verb “to articulate” also has a more material connotation. According to the Oxford English Dictionary, anything “united by a joint” is articulated. To “articulate” something means to endow it with moveable, flexible joints through which it can maneuver more adeptly within physical space. Thus, at a material level, articulation refers to a flexible connection between two or more distinct objects or elements. A truck is in an “articulated” relationship to the trailer it is towing just as an angler “articulates” the flexible joints of a lure as she reels in her line. This overlap between linguistic and technological domains reveals the extent to which “articulation” refers to the process of creating connections in general, whether through language or material objects.

In the late twentieth century, cultural studies scholars such as Stuart Hall began to draw on Ernesto Laclau’s notion of “articulation” as a method for creating “non-necessary links” among different social and cultural discourses (e.g., class, race, gender, religion, etc.) (Slack 120-124). In an interview with Lawrence Grossberg, Hall describes articulation as “a linkage which is not necessary” (53). For Hall, an “articulated” connection is possible but not “essential.” As a cultural studies methodology, then, articulation is less interested in reducing social phenomena to their essential characteristics than it is in generating novel connections among seemingly disparate groups, cultural discourses, and ideologies as an attempt to open potential avenues for political formation and social change.

For writing and rhetoric scholars, one of the more common definitions of articulation comes from Ernesto Laclau and Chantal Mouffe, who define it as “any practice establishing a relation among elements such that their identity is modified as a result” (105). In Toward a Civil Discourse: Rhetoric and Fundamentalism, Sharon Crowley draws on Laclau and Mouffe’s theory of articulation to describe how rhetors can create “openings” in discourses and ideologies (52). For Crowley, articulation describes a model of “ethical rhetorical exchange” in which the goal is not to “shut down argumentative possibilities but to generate all the positions that are available and articulable in a given moment and situation” (56). Similarly, in Image Politics: The New Rhetoric of Environmental Activism, Kevin DeLuca points to Laclau and Mouffe’s theory of articulation as a process of “linking” discursive elements “such that they can be understood as being spoken anew” (38). For both Crowley and DeLuca, “articulation” serves as an active practice that rhetors engage to open up alternate discursive pathways into entrenched public arguments and ideologies.

Indeed, theorists from a range of disciplines draw upon articulation as a term for describing the act of creating temporary links or “associations” among seemingly disparate entities. In his work in science studies, Bruno Latour describes articulation as a method for tracing how human and nonhuman “actors” co-produce scientific knowledge. In Pandora’s Hope: Essays on the Reality of Science Studies, Latour describes articulation as a method for linking together “occasions” for actors “to enter into contact” (141). Latour is interested in such “occasions” for how they allow previously disconnected entities to “modify their definitions over the course of an event” (141). Following the work of two scientists researching a section of the Amazon rainforest, Latour writes that as the elements of the rainforest are scaled down and set in close relation to one another, they form new, emergent relations:

In the calm and cool office the botanist who patiently arranges the leaves is able to discern emerging patterns that no predecessor could see . . . Scattered through time and space, these leaves would never have met without her redistributing their traits into new combinations. (38, emphasis added)

By extrapolating the forest into a portable grid and filing forest leaves into manila folders, the scientists create “occasions for interaction” between previously disconnected phenomena (141). This process is not an imposition of human agency onto the environment but rather a mutually constitutive engagement with the material “constraints and affordances of the jungle” (Prenosil 101). Indeed, Latour is quick to point out that articulation is not the “privilege of a human mind surrounded by mute things” but rather an activity “in which many kinds of [human and nonhuman] entities can participate” (142).

In his book Reassembling the Social: An Introduction to Actor-Network-Theory, Latour draws upon the concept of articulation to explain how actors inter-act within physical locations to produce a sense of place. Latour writes that places become “articulated” through “the transported presence of places into other ones” (194). He uses a lecture hall to demonstrate how locations are, to an extent, already “scripted” for us before we inhabit them. A teacher walking in on the first day of class, for instance, confronts rows of tiered seats and a large podium projecting out into the crowd. These nonhuman objects work together to “place” the teacher’s actions toward specific institutional ends (i.e., lecturing). Entering such a “scene,” writes Latour, gives you the feeling that “most of what you need to act is already in place” (195). By “placing” a location, nonhuman actors mediate human agencies. Although the architects who designed the lecture hall are no longer present, the desks, blackboards, and walls that currently inhabit this space “carry their action in absentia” (195).

Throughout his work, Latour emphasizes the dexterity of “articulation” as a method for tracing the mutually constitutive network of associations between human and nonhuman actors. However, these networks are not created ex nihilo out of the scientist’s work nor do they magically emerge from an objective, “natural” world. The process of tracing networks produces networks. According to Thomas Rickert’s interpretation of actor-network-theory, a “network” is not an entity waiting to be discovered but a process waiting to unfold: “Not network, says Latour, but worknet—one does the work to establish what the network will have been” (138). Rickert’s approach to “worknet” is useful for understanding how “adding” elements to something—a discourse, a science experiment, etc.— “transforms the very framework for understanding” what that thing even is in the first place (143). In his chapter “The Whole of the Moon: Latour, Context, and the Problem of Holism,” Rickert references a 1922 science experiment described in Karen Barad’s Meeting the Universe Halfway. In the experiment, two scientists accidentally prove electron spin (which was not the focus of their research) thanks to the sulfur fumes from one of the scientist’s cheap cigarettes. Although the sulfur smoke was certainly a vital, nonhuman actor in the experiment, this discovery still required the scientists’ work tracing the smoke’s role in reshaping the bounds of the experiment itself. For Rickert, this “indicates that an assemblage needs more to be understood than the incorporation of added elements that make a difference” (143, emphasis added). Whereas the cigarette was initially “plasma” (i.e., a seemingly insignificant element of the experiment) it ultimately became a full-fledged actor. As Rickert demonstrates, when we “add” elements, we must account for the fundamentally emergent and transformative effects they initiate. Similarly, when we “augment” a location, we must pay attention to how these “added elements” operate as co-producers alongside the network of relations within it.

In their introduction to Thinking with Bruno Latour in Rhetoric and Composition, Paul Lynch and Nathaniel Rivers draw upon Latour’s theories to understand how the phenomenon of writing operates as a mutually-constitutive process taking place between and among human and nonhuman actors. Lynch and Rivers note that re-distributing agency among and between nonhuman “actors” like objects, texts, and locations is troubling to a discipline like rhetoric and composition that likes to “hold people accountable” (and teaches its students to hold others accountable) for their rhetorical actions (5). However, in redistributing agency among nonhuman actors, humans are not deprived of agential capacity. Rather, as Lynch and Rivers claim, agency should be understood as “a partially discernible—though mysterious—network” emerging out of the inter-actions of humans and nonhumans. Similar to how the scientists articulate the leaves to “discern emerging patterns” about the rainforest, writers articulate things, places, people, and events to produce and discover knowledge about the world. Although writing produces contingent associations (i.e., the associations could be traced differently), this does not necessarily entail such associations are not real and active; it just means that they are not determinative, or as Hall states, “essential for all time” (53). As Lynch and Rivers note, the fact that knowledge is articulated through language does not make it “any less true” (9). Indeed, as Latour writes in his discussion of the various technologies through which scientific knowledge is produced—methodologies, microscopes, graphs, writing—the scientist is not obscuring an objective world by applying such “filters” to it; rather, it is through these very filters that the scientist’s view of the world becomes “clearer” (137).  Similarly, every time we write, argue, exhort, critique, or explain, we articulate a reality that is inseparable from (but not utterly determined by) the human and nonhuman actors within our rhetorical situations.

In his article “Articulation Theory: A Discursive Grounding for Rhetorical Practice,” Kevin Deluca describes how “articulation” opens alternative frameworks of deliberation by creating “antagonisms” that “point to the limit of a discourse” (336). DeLuca writes that antagonisms arise when a taken-for-granted discourse or ideology (e.g., “The American Dream”) is called into question by linking it to elements (e.g., minorities, immigrants, etc.) for which it doesn’t account. Antagonisms seek to expose relations as social, or non-natural, and thus capable of being changed. DeLuca extends this idea in his book Image Politics: The New Rhetoric of Environmental Activism, where he describes how antagonisms can be produced through “image events.” DeLuca describes how organizations like Greenpeace and Earth First! began to create performative “stunts” (e.g., tying themselves to trees, blocking whaling ships, etc.) intended for circulation through tele-visual media to raise awareness and support for environmental issues. DeLuca writes that image events are “critique through spectacle;” they operate “in the territory of the system but outside the sense-making rules or the lines on the grid of intelligibility of the system” (20-22). Image events are not reducible to the conscious rhetorical actions of the protestors or their intentions for how such “events” will circulate across media. In one example, DeLuca describes how a Greenpeace protest against whaling vessels produced an “image-event” when the whaling ship decided to shoot its harpoon at the whale despite the proximity of the protestors. As footage and images of the event began to circulate, public discourse about the ethics of whaling began to coalesce around the image of the protesters standing in the whalers’ line of fire. This event, or rather its circulation as image-based media, created an antagonism within the public’s conceptualization of the whale as both majestic animal and economic resource.

As DeLuca points out, merely exposing differences and “contradictions” is insufficient as a method for inducing social change or opening alternative discursive pathways. Rather, it is “through rhetorical practices” taken up by both human and nonhuman actors that contradictions become “antagonisms” capable of re-articulation (“Articulation Theory” 343). When rhetors account for the nonhuman elements of a discourse, they are able to more fully engage their “available means” for effecting social change. When we write, we do not create rhetorical elements from scratch; rather, we draw upon the existing (human and nonhuman) elements that are already circulating as part of the discourse with which we are engaging. Although a rhetor might “augment” a conversation with a new piece of information, “articulation” forces us to consider how this “added element” is insufficient in itself to evoke change. Articulation encourages us to approach public discourse less as a practice of adding something to it (e.g., data, statistics, evidence, etc.) and more as a practice of working with what is already an important part of the discussion. This is not to discount the value of additional information. For instance, new data and scientific research are incredibly valuable in the public conversation about global climate change. However, merely “augmenting” the public conversation about global climate change with more research has been shown to be insufficient as a rhetorical move “to carry the day politically” (342).

From its usage within cultural studies scholarship to its application within rhetorical theory, “articulation” serves as a useful theoretical framework for understanding how temporary, contingent relations create emergent patterns of meaning. In my next section, I describe how the theories of articulation outlined here offer writing and rhetoric scholars an alternative way of conceptualizing AR as a location-based writing technology. Unlike its commercial trajectory, which configures AR primarily as a technology for spatial improvement, artists’ and activists’ trajectories for AR configure it as a technology for spatial intervention.

Articulating Augmentation: From Improvement to Intervention

“Augmentation of reality . . . believes in the division between representations and that which they represent, thus obscuring the performative nature of people, things and their surroundings.”
–Laura Lotti, “Through the Augmenting Glass”

In January 2015, Google announced they were ending funding for their wearable augmented reality device, “Glass.” Although Google has since clarified their position, stating that they are simply getting the device “ready for users,” this announcement marked a major setback for AR’s emergence as a mass medium (Vincent). As Douglas Eyman cautioned the field of computers and writing back in 2011, any prediction about future changes to communication practices based on contemporary technologies must be “tempered by an awareness of the kinds of infrastructure and social changes such technologies will require” (Walker 328). In the case of Google Glass, and perhaps wearable computing in general, some of the most unanticipated “social changes” were related to public concerns about privacy. Indeed, it was not uncommon for “Glass Explorers”—public beta-testers of the device—to be met with vehement, and sometimes even violent, reactions from those wary of Glass’s surreptitious recording capabilities (Suba).

Although privacy concerns certainly contributed to Google’s decision to discontinue research into Glass, issues related to the rhetorical function of the device itself might have been even more of a determining factor. According to MIT’s technology review editor, Rachel Metz, Glass essentially did everything we expect from our smartphones (web browsing, texting, navigation, phone calls, etc.); it just “didn’t do any of them all that well.” Metz writes that this problem could have been overcome if the functionality of Glass actually did the “amazing things” it claimed rather than working like a smartphone you wear on your face. By not promoting compelling interactions between physical space and digital content, Glass failed to capitalize on the unique rhetorical affordances of augmented reality. Although Glass may have incorporated AR content into the design of its interface, most applications did not encourage the user to interact in any meaningful way with the physical layer, and the novelty of digital content simply “existing” within physical space was not a compelling enough reason to engage with the device.

In Understanding Augmented Reality: Concepts and Applications, Alan B. Craig defines “augmented reality” as any medium “in which information is added to the physical world” (15). Unlike “virtual reality,” which immerses the user in a completely digital world, AR integrates digital and physical elements into the user experience. Craig contextualizes AR technologies within an evolutionary narrative of humanity’s ability to “augment” our physical environment through the development of more advanced technologies: from primitive tools to complex symbol systems (2-5). This conception of AR as technological improvement is widespread among futurists and technology evangelists, and, as a result, the term “augmented reality” is quickly becoming synonymous with digital enhancements to physical space.

Technically, “augmented reality” typically refers to any technology or combination of technologies capable of overlaying visual information within physical space (16).4 Craig argues that AR can break from the derisive labels leveled at AR devices like Google Glass “only by exploiting the affordances” of which it is uniquely capable as a medium (151). Instead of seeing AR as a technology for accomplishing activities typically done through a smartphone (e.g., texting, web browsing, social media, etc.), technology companies are beginning to configure AR more as a task-specific technology.5 Although there are a variety of industries and applications for these task-specific AR applications, I break them into two general categories: proxy and context-aware AR.

Proxy AR applications configure digital content as a placeholder, or proxy, for physical objects. Used primarily within commercial industries that sell physical products, proxy AR applications overlay realistic digital versions of objects within physical space. Whereas print and television media allow consumers to imagine their interaction with products not physically present, proxy AR makes it possible for consumers to simulate this interaction. Marketing expert Rebecca Borison writes that companies who use proxy applications effectively leverage AR by showcasing “the functionality of [their] brand” rather than merely linking to a website through a Quick Response (QR) code.

Fig. 1. Example proxy AR application from Marxent Labs

Although proxy applications are certainly useful, they inadvertently perpetuate a representation of AR content as subordinate to the physical objects they represent. Within this representationalist trajectory, AR becomes a medium for maintaining established relationships among the elements of a physical space (e.g., people as consumers, objects as products) rather than a medium for articulating new relationships between them. For the commercial release of its Kinect AR-motion capture technology, Microsoft created a retail scenario video showcasing a Kinect application that exemplifies both proxy and context-aware AR.6   In the video, customers walk in front of a Kinect mirror as their bodies are overlaid with digital hats, sunglasses, and dresses. The video centers on a woman who wanders over to the Kinect mirror and stares approvingly at her reflection as it is augmented with a new dress. The woman does not do anything to indicate that she has given the Kinect permission to augment her reflection. The AR software depicted in the video modulates product options for each customer based off gender designations decided by the Kinect. In other words, the Kinect attempts to “identify” customers’ genders so proxy AR clothing can be more precisely targeted to them. The application exposes the degree to which the naturalized space of the clothing store works to stabilize gender identity for profit and, in the process, reveals how gender identification can be (super)imposed upon individuals by a location. However, as Sidney I. Dobrin points out, any attempt to “order a space to a particular end” in no way guarantees that this “order” will be “recognized or obeyed by all who enter that space” (17). Indeed, any gender-nonconforming customers confronted with such an AR experience might invoke their right to “disobey” the gender identification assigned by Microsoft’s augmentation of reality.

Fig. 2. "Kinect for Windows Retail Clothing Scenario" screenshot

As John Tinnell points out, to say that a technology “augments” reality implies the presence of a “universal, stable reality” capable of being augmented in the first place (“All the World’s a Link”). In the case of the Kinect mirror, the digital clothing is configured as a mere representation of the physical clothing that it represents and will eventually be replaced by. Moreover, to draw the customer’s attention to the physical verisimilitude of the digital clothing, the physical space reflected back in the mirror is configured as a stable backdrop which the user ignores when engaging with the application’s content. Representationalist AR applications maintain the user’s preconception of her physical environment because to disrupt it would be to simultaneously disrupt the spatial stability that makes it possible to “augment” the environment in the first place. In such a scenario, “reality” functions as a stable backdrop that is capable of change only insofar as a new element is “added” to it. Such applications create a narrow representation of what constitutes an AR experience by construing physical space as a stable backdrop for human-oriented actions.

However, digital artists are beginning to explore alternative trajectories for augmented reality by leveraging this emerging technology to articulate new relationships among the rhetorical elements of public and private spaces. These born-digital artworks incorporate physical space as a key rhetorical component of the user’s experience. As these projects demonstrate, AR can not only improve the user’s experience of a space but also can point out undisclosed connections between seemingly disparate social, cultural, and environmental phenomena.

Using AR technologies as an interventionist practice first emerged in the work of a group of digital artists known as Manifest.AR.7  Digital artist Tamiko Thiel’s mobile AR project “Clouding Green” is accessible at the corporate headquarters of ten major technology companies in Silicon Valley. Drawing from a 2012 report from Greenpeace revealing that cloud-computing was expanding at a rapid rate with little thought put into its impact on the environment, Thiel’s application overlays digital green and gray clouds over the buildings to visualize the carbon footprint of each company’s cloud-computing services. By articulating elements that have been ignored and/or actively repressed from the rhetorical production of these locations, “Clouding Green” articulates a visual antagonism between the environmentally-friendly rhetoric of “cloud computing” and its dirty, material reality.

Fig.3. Tamiko Thiel's "Clouding Green" application at Twitter headquarters in California.

As Thiel’s project demonstrates, one of the primary affordances of AR is the potential to establish relations between experientially disconnected phenomena. Although her project’s overt critique is directed at the technology companies, Thiel’s application also serves to expose the user’s connection to the issue of cloud computing and carbon emissions. Because her work can only be accessed through an internet-connected mobile device, the user must rely on the same technology (cloud computing) that the project critiques. As the user looks through her iPhone at the large, gray clouds swirling over Twitter’s headquarters, the various rhetorical elements present within this space suddenly link up in new ways that force the user to articulate her own complicity within the environmental concerns raised by Thiel’s project.

Many interventionist AR projects take an approach similar to Thiel’s by linking the user’s personal actions to spatially and temporally distant public issues. Mark Skwarek, one of the founding members of Manifest.AR, created an AR intervention accessible by scanning any British Petroleum logo. His project, “The Leak in Your Hometown,” was created in response to BP’s complicity in the Deepwater Horizon oil spill—thereby re-articulating the relationship between the user’s private actions (getting gas) and a distant public disaster (the Gulf oil spill). Skwarek’s project functions as what Gregory Ulmer refers to as an “abject monument:” it articulates the public’s perception of the Deepwater Horizon oil spill so that it is no longer a random accident but rather a structurally necessary sacrifice for maintaining a particular cultural value (i.e., cheap gas) (57).

Fig. 4. "The Leak in Your Hometown," Photo courtesy of Mark Skwarek.

John Craig Freeman’s AR intervention “Border Memorial: Frontera de los Muertos” addresses the issue of rising immigrant death rates at the U.S.-Mexico border. Freeman’s application generates a digital calaca—a traditional Mexican wood carving commemorating the death of a loved one—at the precise GPS coordinates of each recorded migrant death. According to Freeman, “Border Memorial” seeks to invoke the affective weight of the migrant death toll by allowing users “to visualize the scope of the loss of life” that occurs invisibly at the U.S.-Mexico border each year. In doing so, the calacas create an important link between a social value (cheap immigrant labor) and the sacrifices that sustain it (immigrant deaths). Although the geo-located skeletons “augment” the space of the border wall, they are not the sole rhetorical agents within the critique. The digital calacas work alongside the harsh material rhetoric of this space (hot, dusty, dry, etc.), and they create a juxtapositional framework, which allows the user to articulate her physical experience of the border within the context of the more than 6,000 people who have died attempting to cross it.

Fig.5. "Border Memorial: Frontera de los Muertos," Photo courtesy of John Craig Freeman.

 

As a rhetorical practice, articulation seeks to reinvigorate the complexity of a space by placing the human and nonhuman elements circulating within it in new, emergent relationships. Raúl Sánchez defines emergence as “modes of action that develop over time and as the result of recurrent and multiple exchanges between and among actors” (29). Articulating reality through location-based public writing projects means crafting “emergent” relations between the location, the AR content, and the users who are likely to engage with it. Creating location-based augmented reality, then, is not only “adding” to a stable conception of a space; it is an ongoing rhetorical process of crafting flexible, contingent links “between and among actors” who are already part of it.

Articulating Location: From Technological Effect to Design Strategy

“It is not only that individual words shift meaning given their context within a sentence, but also that words shift meaning given their embodied context and their physical location in the world”
–Andrea Lunsford, “Writing, Technologies, and the Fifth Canon”

In his chapter “Techno-Geographic Interfaces: Layers of Text and Agency in Mobile Augmented Reality,” John Tinnell traces the history of ubiquitous computing alongside efforts to untether human agency from the stationary constraints of the personal computing (PC) era. As an alternative to PC-era logic, Tinnell argues that AR is poised to initiate a “broader ecology of digital interaction whereby multimedia becomes structured by non-human activity” (70). Building on Gilbert Simondon’s concept of “techno-geographic milieus,” Tinnell describes how context-aware AR produces “techno-geographic information,” or information that maintains a “direct relationship to the present activities occurring at that particular location” (77). Unlike AR applications that merely provide access to web-based information, such as a QR code attached to an historical marker in a local park, “techno-geographic information” (or what I call “context-aware AR”) generates content based on real-time analysis of “geographical flows” within the user’s immediate surroundings (72). Because context-aware AR applications necessarily draw upon the user’s immediate surroundings to generate content, they distribute writerly agency out into the environment. For Tinnell, AR is a posthuman writing technology; the production of its textual content is inseparable from the physical location in which it is generated. A context-aware AR clothing advertisement, for instance, would be able to analyze current weather conditions and nearby social media feeds to determine if it should display a digital raincoat or tank top.

As Tinnell points out, writers who create location-based AR cannot ignore the material rhetoric embedded within the location: “from the standpoint of production, one must learn to write and design with the accidents of the sensible” (82). For Tinnell, the emergence of techno-geographic information transforms the role of the user’s material environment from “a standing-reserve . . . acted upon by an already constituted subject” to a “constitutive element” in the production of “perception, memory, and decision-making” (73). However, as Tinnell makes clear, the production of techno-geographic information is not reducible to highly technical, context-aware AR technologies; this content operates as an integral rhetorical element throughout the “design process” itself (78).

Along with Tinnell, I argue that designing location-based AR is not reducible to context-aware AR. In seeing ourselves as “co-producers” of location-based AR experiences, however, there is a danger that we might conflate writerly agency with context-aware AR technologies. In doing so, we downplay the rhetorical significance of “static” multimedia content—images, videos, 3D models, etc. Although context-aware technologies are certainly one way in which writers can articulate an environment, I argue that “articulation” as a model for designing location-based AR is less an effect of any particular technology and more of a design strategy. As I hope to demonstrate through my discussion of a mobile AR application designed for SeaWorld-Orlando, creators of location-based AR must allow the rhetorical elements in the location (human and nonhuman) to shape each stage of the creation process, from the selection of augmentable areas to the creation of multimedia overlays.

At Trace Innovation, a digital humanities initiative in the University of Florida English Department, we support the development and publication of “Augmented Reality Criticisms” (ARCs). ARCs are mobile applications that provide critical and/or educational digital content within the context of a physical location, image, or text. Co-created by Sidney I. Dobrin, Melissa Bianchi, and myself, the SeeWorld ARC works to re-articulate visitors’ perceptions of specific marine life with the aim of drawing them into a broader public conversation taking place about SeaWorld and the ethics of marine captivity.8 

Creating a location-based AR application is not a linear process. Like any writing project, the various stages of planning, designing, and building the application often overlap and/or occur simultaneously as new developments within one stage force changes in another. This process requires a keen rhetorical eye for how a confluence of human and nonhuman elements (e.g., the animals in the park, the software used to design the app, park visitors, Florida’s tourist economy, etc.) co-produce the application alongside its human “creators.” Without a doubt, the process of planning and designing the SeeWorld ARC revealed how “articulating” a location with AR requires sustained consideration for how the location itself is operating as a significant rhetorical actor within each phase of the application’s design.

The process of “articulating” the space of SeaWorld-Orlando began the moment we started “mapping out” the application itself. In our initial meetings, email threads, and planning documents, we created a rough outline of what we wanted the application to do, how it would work, and what kind of content it would allow the user to access. We decided that the application content should be short, straightforward, and non-condemnatory; we wanted to, as Sharon Crowley writes, “articulate available openings” in discourse about SeaWorld and marine captivity in general, not close them down through divisive, accusatory rhetorics (52).

Once we had a good grasp of the general purpose and function of the application, we travelled to Orlando where we (literally) mapped out the specific locations where augmented reality content would appear throughout the park. SeaWorld-Orlando is broken up into eight main sections designed to guide visitors in a clockwise direction through the park’s main attractions. This layout encourages visitors to start in the Sea of Shallows (dolphins, stingrays, and turtles) and end in the Sea of Power where the park’s final Orca show starts at 5:00 pm, thereby encouraging guests to wrap up their day before the park’s 6:00 pm closing time.

Fig.6. The SeeWorld ARC overlays interactive virtual buttons on the park map to help users locate augmented locations. Photo by the author.

The spatio-temporal structure of SeaWorld-Orlando contributed a vital piece of “techno-geographic information” to our knowledge of how guests move through the park and informed the overall structure of the application interface. Beginning at the park’s entrance, we decided to augment six areas of the park, focusing most of our augmentations on popular exhibits where we noticed guests linger for longer periods of time, such as the stingray petting pool, the dolphin viewing area, and the Orca exhibit. These areas are typically full of informational texts, which not only serve as useful trigger images but also encourage visitors to slow down their pace within that section of the park.9  Because these areas are more popular, we also hoped that other visitors might notice people engaging in the ARC and inquire about the project. We also had to pay attention to how the user’s immediate physical surroundings would impact their experience of the application content, such as noise or light interferences. We noticed that the dolphin viewing area was smaller and more confined than other exhibits, so any noise from a user’s device would likely draw attention. We leveraged this acoustic facet of the user’s physical environment to place all our dolphin critiques in this area, not only because it is a popular place for visitors to remain for extended periods of time but also because the audio overlays would likely draw attention to those engaging in the ARC and hopefully spread more awareness about the project.

“Articulating” a location is a two-way process: the rhetorical actors embedded within the location (the park layout, tourists, exhibit designers, etc.) influence the type of content that can be accessed within it as well as the manner in which it can be accessed. From the mapping phase of the design process, we were forced to consider how our plans for the mobile application had to work alongside SeaWorld’s physical design of the park and how visitors were likely to move through it. As Latour notes in Reassembling the Social, when “acting” within a physical location, “most of what we need to act is already in place” (195). As writers of location-based experiences, we cannot ignore these geographic scripts; rather, our goal is to re-articulate them so they link up in new ways for the user.

The AR overlays for the SeeWorld application are short videos accessed by scanning the park’s existing signage such as informational texts about specific animals or SeaWorld logos scattered through the park.10  There were a few reasons why we decided to use short video overlays as opposed to other media types such as images, web links, or 3D models. First, we figured that video content creates a more engaging contrast to the park’s physical signage than would images or web links. In addition, it is much simpler to push updates to streaming video content than complex 3D animations. Designers of AR content can update streaming video without updating the entire application.  Lastly, the videos allow us to include both audio and captioned text so our overlays can be accessible to visitors with visual or hearing impairments.

Using “static” video overlays does not mean that our application is not “context-aware.” The AR content of the application still articulates new relations between the human and nonhuman elements (marine animals, tourists, park informational texts, etc.) circulating within different locations of the park. For instance, as we began work on our video content, we wanted our overlays to directly respond to the rhetoric of the physical signs being augmented and not just create a general critique about SeaWorld or the hazards of marine captivity. The image in Figure 7 is one of the trigger images located in the dolphin viewing area.11 This sign attempts to dispel any concerns about the scarring that is common among dolphins held in captivity as well as those in the wild. Moreover, the playful posture of the three dolphins in the image reinforces the playful, carefree attitude that SeaWorld wants the viewers to feel about the dolphins swimming in the large pool behind them. Indeed, the elements within the underwater viewing area craft a powerful rhetorical message, which seeks to persuade visitors that SeaWorld’s dolphins live happy, fulfilling lives within captivity.

Fig.7. Example augmentation from the SeeWorld ARC located in the underwater dolphin viewing area. Scan the image above with the Aurasma app.

While addressing visitors’ superficial concerns about the dolphins’ physical appearance, the sign glosses over more pressing social concerns for dolphins held in captivity. As the video overlay explains, captive dolphins cannot participate in the same social, cultural, and environmental interactions as their wild counterparts. The video overlay does not contradict the information in the sign put up by SeaWorld. Rather, it establishes an “antagonism” between the viewer’s experience in the dolphin viewing area and the information provided by the video. Kevin DeLuca explains how “the act of linking in a particular articulation” creates a rhetorical framework that allows elements to be “spoken anew” (Image Politics 38). When the viewer scans the image and the video begins to play, the various “elements” within the underwater viewing area—calm music, swimming dolphins, reassuring information, playful images—begin to re-articulate for the viewer from “natural” to “artificial” elements as they are juxtaposed with information about wild dolphins.

Articulating locations through AR technologies provides another avenue for writing and rhetoric scholars to extend their teaching and research practices into what Christian Weisser terms “public writing in context” (95). For Weisser, public writing in context occurs when writers engage with communities who are not merely participants within an abstract public sphere (e.g., people reading letters to the editor) but rather potential agents of social change. By articulating contested locations like SeaWorld, writers participate in shaping public perceptions about that space as well as the public issues that circulate within it. Like any writing project, it’s impossible to predict the degree of public impact that a text is going to have on an issue, argument, or location. However, considering the massive popularity of mobile devices and predicted advances in wearable technologies, it seems likely that location-based AR is only going to become more popular in the coming years. Thus, even if location-based AR projects “fail” to have a social or political impact in the present, they are still valuable experimental testing grounds for an emerging medium that seems poised to fundamentally reshape how we create and access digital content with physical locations. Although the public impact of current location-based AR projects may not carry the same rhetorical heft as they might if AR were more prevalent, we can prepare for (and perhaps even actively shape) the digital genres of location-based writing that are beginning to emerge alongside an impending era of ubiquitous computing.

As a method for creating location-based AR, articulation is an ongoing, recursive design strategy through which the human and nonhuman elements of a location co-produce emergent rhetorical experiences for both designer and user. Although context-aware AR technologies are able to generate dynamic, real-time data about the user’s environment, they are not the only way to “articulate” a location. For static AR content to engage with the material constraints and affordances of its users’ surroundings, AR designers must pay attention to how the location itself is articulating the available means of persuasion embedded within the location.

Conclusion

Locative writing technologies such as augmented reality hold great potential in reshaping our approach to writing within and about contested places. Through AR, writers have the capacity to participate in a genre of public writing that exposes alternative perspectives on a location, potentially providing an avenue of representation for those (human and nonhuman) voices who are not being heard. Here, I have called for a turn to articulation theory for how it emphasizes the creation of emergent, identity-modifying relationships within the existing rhetorical elements of a location. Although commercial trajectories of AR configure it as a technology for enhancing a space, I demonstrated how digital artists are re-configuring AR as a digital genre for intervening into spaces and re-articulating the user’s relationship to disparate social and cultural phenomena. Building from the work of these digital artists, I described my own involvement in a location-based public writing project at SeaWorld-Orlando to argue that “articulation” is less a technological effect and more a strategy that rhetors must engage throughout the entire design process. Through this, I hope to provide a generative model to inspire other scholars to engage in their own location-based AR projects in their teaching and research practices.

The field of digital rhetoric is uniquely situated to take on the challenge of experimenting with ways in which location-based writing technologies like AR can be used to write within and about physical objects, texts, and locations. As Catherine Braun points out, digital media work must be actively “cultivated” if it is going to become a viable form of scholarly production. Braun argues that the production of digital projects should not be restricted to students in multimodal writing courses but rather “a site to reconceptualize the work of scholarship” (5). By designing and building location-based AR applications, writing and rhetoric scholars not only provide generative models for what it means to “write” a space, but they simultaneously take up an active role in shaping the rhetorical future of this emerging medium and the new genres of mobile writing it makes possible.

• 1. I use the term “augmented reality” (AR hereafter) to refer to computer-generated, multimodal data overlaid in registration with a live camera view of the physical world.
• 2. Indeed, these kinds of “context-aware” AR applications, as I term them later, are highly valued within jobs where people must quickly evaluate and make decisions about their physical surroundings. For instance, engineers in Switzerland have created prototype helmets outfitted with an AR optical display that allows firefighters to view digital representations of surface temperatures so they know to avoid certain areas of a burning building. These types of AR applications are also becoming more common within medical contexts by providing surgeons with real-time navigational assistance during complex procedures (“Scopis Augmented Reality”).
• 3. Of course, “being articulate” is culturally relative and is often directed against those who do not act, speak, or write in a manner appropriate to the dominant modes of a particular discourse community. In the United States, for instance, a person is often only considered “articulate” if she or he speaks according to the conventions of standard American English—a dialect synonymous with the white, upper-middle class. As a result, the term “articulate” often carries micro-aggressive connotations when directed at minorities or marginalized communities (e.g., “You’re so articulate!”). In this sense, to “articulate” is not just something that you do; it is something that you inhabit at a social level, and what it means to “be” articulate can shift according to culturally contingent markers of race, class, gender, orientation, and/or ethnicity.
• 4. In 1997, computer scientist Ronald Azuma established AR as a visual medium when he defined it as any technology that “allows the user to see the real world, with virtual objects superimposed upon or composited with the real world” (2). Although the term “augmented reality” (or “mixed-reality” as it is sometimes called) is often conflated with GPS-based applications (e.g., Google Maps) or even digital technology in general, I use the term to refer more specifically to computer-generated visual data overlaid in registration with a live camera view of a physical space. Although this definition excludes other emerging technologies that overlay computerized data onto physical space (such as holograms), it does have the advantage of specifying the technological and rhetorical framework that serves as the exigence for this article: a growing population of internet-connected, mobile device users (see Zakrzewski).
• 5. Within the last few years, task-specific AR applications have exploded in popularity. Companies such as Lowe’s and IKEA are currently developing mobile AR applications that allow customers to view new countertops and couches within the physical space of their homes simply by looking through the camera view of a tablet or smartphone. In addition, the car manufacturer Ford recently released an augmented reality app that overlays visual aids on top of a car engine to demonstrate simple maintenance procedures such as checking oil or refilling washer fluid (Liszewski).
• 6. I use the term “context-aware” AR to refer to applications that generate AR content based on real-time data inputs from the user’s immediate surroundings. In Everyware: The Dawning Age of Ubiquitous Computing, Adam Greenfield points out that computers will increasingly operate only according to parameters that are “appropriate to our location and context” (1). Context-aware AR applications engage most explicitly with this aspect of ubiquitous computing by creating content unique to the specific time and place in which it is generated.
• 7. Manifest.AR connect their interventionist work to the Situationist art movement of the 1950s (Skwarek, 8). The Situationists practiced a technique they referred to as “detournement” (French for “diversion”) in which they would appropriate cultural materials such as logos and advertisements as sites of critique. Contemporary iterations of detournement can be seen in the work of “culture jamming” groups like Adbusters and the Billboard Liberation Front as well as the work of street artists like Banksy and Shepard Fairey. This type of public art activism is typically unsanctioned, highly visual, and aims to intervene into the social and political conditions of viewers’ everyday lives.
• 8. SeaWorld-Orlando was the focus of the 2013 documentary Blackfish, which followed the death of one of their trainers (Dawn Brancheau) during a live Orca performance. Since the release of Blackfish, SeaWorld has faced strong public backlash as former trainers have come forward to recount a history of abuse and negligence within SeaWorld’s facilities.
• 9. Unlike geo-based AR, which uses GPS coordinates to determine the location of digital content within physical space, vision-based AR relies on visual markers known as “trigger images” (such as a street sign or public advertisement) to orient digital content. There are a variety of free, vision-based augmented reality creation platforms available online, such as Aurasma Studio and Vuforia. Aurasma is a mobile application that allows users to create content for specific accounts that other users can then follow. Vuforia is a software development kit (SDK) that can be used to create standalone applications for the Google Play and Apple app stores. For more information about the Vuforia SDK, see Greene.
• 10. As counter-augmentations become more accessible and ubiquitous, it is possible that target sites (such as SeaWorld) will claim that these augmentations violate property and/or image copyright laws by overlaying unofficial information on top of private property and copyrighted images. However, Brian Wassom, one of the few legal scholars working in augmented reality law, argues that augmented reality criticism operates as a form of “critical consumer speech” and should be protected under the First Amendment (122-24).
• 11. To access the example augmentation, download the free Aurasma application to your mobile device and follow the channel “counter publics.” Then, scan the image in figure 7.