How does UX design fit into an augmented reality and 360 video world?

Jon Reed Profile picture for user jreed August 9, 2016
My recent forays into augmented reality got me thinking - how should we factor these new technologies into UX design? Jeff Piazza of Behavior Design has some answers. Here's what he told me about the readiness of VR, AR, and 360 video, and what he's learning from current projects.

We know this much about UX design: it's about engagement. It's about earning sustained attention. The problem? Enterprise UX is changing faster than our collective design chops. Just when we get a handle on mobile UX, we get a slew of wearables to consider, along with virtual and augmented reality.

But we're not designing for tomorrow; we're designing for today. So which of these trends should enterprises sandbox now? To get a handle, I talked with Jeff Piazza, co-founder of Behavior Design in New York City. Piazza's team has already started working with clients on next-gen UX.

Behavior Design does exactly that with their client base, providing design services to a number of industries, including technology (Red Hat) media (The New York Times) and entertainment (HBO).

Form factors are changing - interactive design is the constant

So how do we take interactive design and apply it to new formats? That's what everyone is trying to figure out. Piazza:

We're always looking at how can we take those ideas that we've been designing as interaction behaviors, and apply those to emerging media.

Even experts are grappling with new mediums:

It seems like we're in this experimental time. You see that a lot with 360 video, where filmmakers are just trying to figure out, "Well, how do I tell a story when I can't put my crew anywhere?" Or Google's latest virtual reality news, which they announced at Google I/O.

To figure out which UX is going to stick, you have to experiment:

We've just trying to put out prototypes constantly. I'm trying to figure out where user interactions are happening. I think that's what's really exciting to figure out.

Piazza believes we are looking for the right "language for interaction":

We need to get to that next level of experimentation to figure out what's the right language for interaction. That will help drive what the content will be, which will then pull in the right users. It's not going to be anything unless the content is interesting.

Virtual reality - an adoption gut check

So how do these trends look in the real world? Piazza doesn't think virtual reality (VR) is ready for broad adoption yet. He sees experimentation - and marketing VR for sex appeal:

It feels like there's a lot of brands out there that have been experimenting with VR on the advertising level, but in a way it's more of like, "Hey, and we do VR." It's not integral in terms of part of their model of selling their business.

If VR has traction now, it's for "short term" or "concise" experiences: "VR might be giving you a sense of what a car can look like, or what a college tour could feel like." Piazza can see VR as an extension of a brand's web site before too long:

When VR starts to mature a little bit more, you'll start to see more opportunities for websites to add that piece as part of their responsive platform. A mobile-ready, responsive web site plus VR experience would all be part of their digital package.

Augmented reality - enterprise potentials

Piazza defines AR as "The combination of a synthetic or artificial three-dimensional experience applied to the real world." It's the ability to fuse 3D - and visual data overlays - into a real world context. Two AR initiatives Piazza is tracking: Magic Leap and Microsoft Hololens:

These two stand out as being able to say, "Okay, I have these glasses I can see through. I can see co-workers, I can see in environments that are not completely closed off. Then I'm going to apply some sort of computer graphic element that I can interact with that will give me additional information, that will enhance what I'm doing."

Piazza cited Meta, a secretive AR startup that recently, according to Forbes, "dazzled" a TED crowd (not the highest bar, granted). Piazza says Meta is talking about possibly eliminating the need for computer monitors for their development team: "They're saying, "We can use this as the next level of information delivery." He also cited Atheer, an industrial AR company that uses AR goggles in conjunction with the Internet of Things":

That's what we're really excited about. We think AR will eventually change how we interact with these systems... People who are checking, let's say, industrial machinery, making sure that's working correctly. Generally, you have to open it up and then figure out what's wrong with it or do a check. Ideally in the future, sensors will be built in and you can use your glasses that will then read directly, send information to the glasses as you're walking through.

Real world projects, and getting started

Intriguing - but not exactly mainstream adoption either. So how should companies approach this? Some if it is industry-based. Example: the travel and real estate industries are ideal for the "show and tell" aspects of AR or VR.

Piazza's advice for getting started? Figure out the stories you want to tell first. Then the technology comes into play. Technology includes your audience: have they adopted the tools/goggles/headsets needed to consume? Piazza:

What exactly do we want to say? If it is truly building a prototype for an interactive application, maybe you do need something a little more heavy duty, more in the high end Oculus vibe... If they don't have [the ideal equipment] now, then you're left with, "Okay, what do we have?" Maybe it's your 360 video experiences, and telling stories that way. Maybe it's a mobile play, which we can still do, and still benefit clients, because there are still good stories to tell in that platform - as long as you're doing it in a way that makes sense.

Industry nuances are a huge factor:

We say, "The important thing here is to start with understanding what your market is." Whether they are in the automotive market or the technology market, our client should be aware of what their competitors are doing.

Each industry has unique areas where UX tech can be exploited:

If it's something like the banking or financial services market, which is really slow to adopt, this is something which takes a longer time frame. We help them find unique areas where we can find efficiencies or greater productivity.

Piazza used collaboration as an example. If a client has a communication issue, focusing UX design on better collaboration can impact productivity across departments. So has Behavior Design taken a client live on an augmented reality type of project?

We're working with a large technology partner right now, developing the framework of interaction behaviors for different business needs. The thing I can tell you at this point: it's about manipulating and searching data in an AR environment. We're finishing up that prototype now; we'll be sharing that with our more data driven and tech-focused clients shortly.

So does that mean the existing searching tools are inadequate?

We see friction in the same areas across different clients. When you have lots of information that you are working with every day, either as a practitioner or a manager, you have a hard time navigating it; you have a hard time searching it... We're building systems to make that experience better... It's really an experiment just to say, "Okay. How can we do exactly what we're doing online on this platform and does that yield a better result? More efficient result?" Ultimately, that's really what our goal is. To improve efficiency, to get better insights, better knowledge, from the information that we're interacting with.

And are they doing anything with 360 video?

We're talking to a couple of clients about it now, including a university. but it would be a bit different than what YouVisit does. What we're trying to do is tell stories using 360 video in particular, by connecting to the "brand pillars" of the universities. We'd like to tie some of those ideas to stories of people who interact at those universities; we think could be a really interesting experience.

My (quick) take

We're clearly in the early stages here, but not too early to reframe our UX thinking, and, as Piazza advocates, experiment. In particular, entry-level 360 video gear is not expensive for prototyping; I recently kicked tires on some local footage taken with a $400 Ricoh 360 unit, compatible with YouTube (editing expenses are not cheap, however). Down the road, I'll look to interview a go-live customer from Behavior Design - once there are results to consider.

Image credit - Feature image - Man wearing virtual reality goggles. © Sergey -

Disclosure - Diginomica has no financial ties to Behavior Design. I was approached by their PR, and the topic fit into my UX and recent AR pieces.

A grey colored placeholder image