Imagine seeing the prop details of Stephen Colbert’s shelves (@StephenAtHome), Jimmy Fallon’s talk show desk (@JimmyFallon), or even participate in a roundtable with them without having to endure the hours of long lines in New York City.

Or, finding new ways to hold intimate conversations with people, learning about how to communicate with others, while maintaining complete anonymity.

That’s the promise of virtual reality and avatars. Virtual reality (VR) technologies can be a new way we socialize, experience entertainment, learn from each other, and even experience psychotherapy for improving ourselves. Augmented reality (AR) takes this to another level, placing VR experiences in real-world contexts.

Think of VR technologies as goggles or a helmet—often called a VR headset—that’s outfitted with earphones and small computer displays for each eye. The headset may be connected to a computer, while some may even use your smartphone as the display. Once you wear this VR headset, it’s as if you are immersed in a new environment: The VR headset rotates the world as your head moves.

 

So why is VR so special?

VR is an inexpensive way to experience new worlds. In just a few seconds, you can be “transported” to a completely different environment. This has obvious implications for those in areas with poor transportation options—such as remote airports, rural locations—but also violence-filled unsafe areas where walking outside can be dangerous.

VR can also be especially ideal for those with health issues. Think of those with mobility impairments, neurological movement disorders, or crippling social anxiety and agoraphobia, a type of fear where you do not want to be seen in public. Having a VR headset can help.

However, there are still some limitations with VR. Vertigo is a large issue. Up until now, delays and lags between the person’s head movements and what was displayed on-screen would induce vertigo—a type of dizzy sensation where the room feels like it’s spinning—because of the mismatch between sight and sensation. A variety of things caused this: underpowered graphics processors, terrible input sensors, or poorly-coded software.

So why is this a huge opportunity now? Well, in the past, VR was clunky, expensive, and required oversized computers. They didn’t have the same sophisticated sensors and inputs that we have today that captured our motions and movements. Our networks today are much faster and carry motion and graphics data much more quickly than in the past.

This has spawned a slew of new VR and AR consumer technologies, all of which have been released in the last two years. This means that the once-niche technologies have entered the mainstream.

However, there’s still a lot of room to experiment. There has not been a single winning platform for VR or AR yet. Microsoft, Facebook, Google, HTC, Vuzix, and other companies are putting lots of money into their experiments.

And one such area that’s ripe for experimentation: Talk.

 

What if you could talk to each other on Facebook—immersed in Facebook?

Over the past month, three California-based companies showed off new, exciting demos that show how important talk tech is for VR interactions. One of them, Oculus (@Oculus), is owned by social network giant Facebook (@Facebook).

Facebook Oculus VR 2016 social avatars 1Facebook Oculus VR 2016 social avatars 1Facebook Oculus VR 2016 social avatars play games

At Oculus Connect, Mark Zuckerberg demonstrated the Oculus Avatar feature with a live VR chat session with two other colleagues in VR, but also making a house call to a physician at the University of California, San Francisco (UCSF)—namely, his wife, Priscilla Chan. His colleagues appeared as 3D cartoon-like characters—or avatars—while his wife appeared as a video window.

What made this significant was the real-time facial expressions and hand gestures. While no facial expressions were tracked, the Oculus controller inputs allowed the users to communicate with raised eyebrows. Plus, mouth movements matched that of the speaker.

According to TechCrunch, Oculus will move much of their VR and AR technologies into traditional eye glasses, away from large cumbersome VR headsets. Ultimately, the company wants to use Facebook’s huge social network to create new social experiences for its users—and give them even more incentive to stay attached in the Facebook world.

This is important because using cartoon avatars avoids much of the uncanny valley found in 3D avatars. The uncanny valley is where a robot or avatar looks extremely human-like but has features missing—perhaps an inaccurate facial movement or hand gesture—that causes an uncomfortable feeling in the human viewer.

Plus, using cartoon avatars can be even better than real life. It could make for good entertainment.

 

Creating live VR talk shows—TV celebrities and news channels join the movement

What if talk shows allowed you to join celebrities face-to-face? Or even enter a celebrity’s world?

That’s precisely what Adam Savage (@DontTryThis), the former Discovery Channel co-host of Mythbusters, did in 2016’s Bay Area Science Festival, produced by the University of California, San Francisco (UCSF). There, he showed off some of his favorite movie props, including a replica Apollo A7L Spacesuit and Admiral Ackbar’s head.

This live VR talk show was spearheaded by FOO’s (@Foo_VR) Will Smith (@willsmith)—who broke ground in 2016 with the FOO VR talk show—and Norman Chan (@nchan) of the popular website, Tested (@TestedCom). The trio donned HTC Vive headsets—with large poles that captured and tracked their positions—and controllers that captured their hand positions.

Adam Savage, Will Smith, Norman Chan VR live talk show

Their avatars were blockier, more muscular versions of themselves, but showed off how they could interact with each other and 3D-scanned objects in their environment. This demonstrates how people can shape and remold traditional forms of entertainment—the television talk show—into something new.

Even NBC News has gone VR. A few weeks ago, NBC News held a mini talk show with Chuck Todd of Meet The Press. There, his Playmobil-like avatar discussed politics and held a Q&A session at a VR recreation of NBC News Democracy Plaza using California-based AltspaceVR.

The VR software company, AltspaceVR, assembled a complete communication platform that works across all the major computing platforms—Facebook’s Oculus Rift, HTC Vive, Samsung Gear VR, as well as 2D desktop computing platforms Windows and MacOS. AltspaceVR lets people see each other, message friends, and create private activities. Their activities are reminiscent of Second Life—a 3D avatar-based world—as AltspaceVR lets users play tabletop games, play miniature sports, watch videos, and explore new environments.

Chuck Todd at NBC News VR Democracy PlazaNBC News VR Democracy Plaza

But what does all of this new talk tech—fused with VR—have to do with how we can improve health? Imagine using VR to drop us into uncomfortable environments with uncomfortable objects that allow us to push and stretch our comfort zones to gain new experience. We could use VR to craft more interesting learning environments.

And in the hospital, what if we could experiment with a post-surgical representation of ourselves at home, to determine ahead of time what our needs are? Or, test drive different medical devices and wheelchairs before ordering them? Or even hold group psychotherapy sessions—thus, avoiding unpleasant face-to-face anxiety?

Although we’ve had a glimpse into VR applications for healthcare and at Health 2.0, in my next article, I’ll take a deeper look into the research behind virtual reality.

Steven Chan, MD, MBA (@StevenChanMD)
Dr. Chan is a Clinical Informatics fellow at UC San Francisco (UCSF)'s Division of Hospital Medicine, serving as editorial boardmember for the Journal of Medical Internet Research (JMIR) Mental Health, and develops cutting-edge research in the areas of digital mental health, with applications for cultural psychiatry and underserved minority health. Steve's ideas, thoughts, and research have been featured in JAMA, Healthcare, JMIR (Journal of Medical Internet Research), Wired, PBS, and NPR Ideastream. Steve serves as Vice Chair for the Workgroup on Mental Health & Psychiatric Apps at the American Psychiatric Association (APA), a part of the Committee on Mental Health Information Technology.

LEAVE A REPLY

Please enter your comment!
Please enter your name here

This site uses Akismet to reduce spam. Learn how your comment data is processed.