Virtual Reality Typing Lights Up Ideas and Keyboards
May 10, 2017 | Michigan Technological UniversityEstimated reading time: 3 minutes
What's better than a holographic keyboard? A real one, apparently. New research from computer scientists at Michigan Technological University delves into the different ways to type in a virtual reality (VR) space. They're presenting their work at ACM Conference on Human Factors in Computer Systems (CHI 2017), noting that while cranking out text is an integral part of our digital lives, it's a field of research that has a surprising lack of emphasis in VR development.
"Lots of people are buying head-mounted displays, but it's mostly for video games," says Scott Kuhl, an associate professor of computer science at Michigan Tech. "We're trying to figure out how we can use a head-mounted display for office work like writing and editing a document or sending a message to someone."
Though these mundane tasks aren't ideal yet in VR, by wearing a headset that looks an oversized sleeping mask, VR can make an enchanted forest walk feel real—or at least surreal—and can transport a viewer halfway around the world. The key is balancing what a person can see through the head-mounted display and what they can touch physically. For typing, people need both visual and tactile feedback.
Head-Mounted Display
James Walker, a lecturer in computer science, led the research as part of his dissertation, working with Kuhl. He says the challenge stems from the fact that people need to see what they're typing—a bit difficult with an over-eye headset on—so he developed a light-up virtual keyboard synced with a physical keyboard. This virtual keyboard lets a VR typist see in the head-mounted display what keys they typed on the physical keyboard.
Other VR typing systems rely on either mid-air virtual keyboards or overlaying of real-world video into the VR display. But both approaches require extra equipment such as tracking cameras that can be error-prone and intrusive. People's texting performance also declines with only virtual keyboards.
"Typing in midair is very fatiguing," Walker says. "Our solution is noteworthy because it enables people to continue using their physical peripherals, which gives the best performance, and it doesn't need any extra hardware or require superimposing a video feed into the virtual environment."
VelociTap: Autocorrect Algorithm
To assess the effectiveness of a physical keyboard, Walker conducted an experiment in which participants typed on a keyboard they couldn't see. In one part, participants wore a head-mounted display—in this case, an Oculus Rift. In the other part, participants used a desktop monitor with their view of the keyboard occluded by a cover. In each part, Walker tested participants’ performance with and without his virtual keyboard.
To start, a large number of people reported their typing prowess to be expert or at least proficient. Being human, participants made mistakes—lots of them—especially those without the virtual keyboard lighting up keys. That makes this experiment a perfect set-up not just for testing VR text hardware, but also to examine how autocorrect fits in.
"People underappreciate the redundancy in natural language," says Keith Vertanen, an assistant professor of computer science who assisted Walker and Kuhl with a language model to correct participants' typing. "Our recognition touch screen program, VelociTap, is extremely accurate as it's been trained on billions of words."
Still, Vertanen says he and the rest of the team were pleasantly surprised by how well the autocorrect algorithm shifted from its touch screen origins to predicting intended letters on a physical keyboard. In addition, the team observed that error rates declined as people continued typing with the VelociTap feedback and the model corrected about two-thirds of the text errors.
The team’s research has focused on the input of short, informal text messages. Next steps include supporting the input of numbers, punctuation and capitals, and to allow the editing and styling of text.
"The long-term goal is facilitating arbitrary input while wearing a head-mounted display," Walker says.
The team knows how to catalyze that integration: use what's at hand. Physical keyboards can help push VR into the text-filled world of work and play.
About Michigan Technological University
Michigan Technological University (www.mtu.edu) is a leading public research university developing new technologies and preparing students to create the future for a prosperous and sustainable world. Michigan Tech offers more than 120 undergraduate and graduate degree programs in engineering; forest resources; computing; technology; business; economics; natural, physical and environmental sciences; arts; humanities; and social sciences.
Suggested Items
Incap Estonia Teamed Up With Solaride to Fuel Youth Excitement for The World of Engineering
04/11/2024 | IncapIncap Estonia partnered with Solaride to ignite enthusiasm among young people for the world of engineering and sustainable technology. Together, they organised an inspiring day for students from grades 7 through 12 in Saaremaa. Held at Incap’s Kuressaare factory, this event brought together over 130 young minds eager to explore the innovative world of technology and learn more about advanced electronics manufacturing.
Accenture to Acquire CLIMB to Expand Technology Capabilities
04/08/2024 | BUSINESS WIREAccenture has agreed to acquire CLIMB, a technology services provider specializing in system integration, IT infrastructure management and operations, primarily in the Gunma Prefecture.
IPC APEX EXPO 2024: Kurt Palmer on the Formation of Schmoll America
04/01/2024 | Barry Matties, I-Connect007There’s exciting news for Schmoll Maschinen as Kurt Palmer leaves his role as president of Burkle North America to lead the new Schmoll America. In this interview, Kurt discusses his collaboration with Burkle North America, what it means for his new Schmoll teams, and the many machines and processes he’ll have on display at his busy booth at IPC APEX EXPO.
It’s Only Common Sense: When Your Company Starts Running Out of Popcorn
04/01/2024 | Dan Beaulieu -- Column: It's Only Common SenseCompanies, like people, get tired. I have seen companies just lose their focus, their “raison d’etre,” and just, well, fade away. More than once, I have sat in the office of a company owner, trying to console him while he asks me to “get him out of there.” Owners just get tired of the daily grind. They want off the treadmill of the constant juggling of people, strategies, customers, and, of course, cash flow.
Global Sourcing Spotlight: Don’t Be Afraid of Global Sourcing
04/03/2024 | Bob Duke -- Column: Global Sourcing SpotlightOn paper, it doesn’t seem like a great time for global sourcing. With several world conflicts, it may seem impossible to think that something like global sourcing and commercial trade between countries could begin to alleviate this tension between warring countries. But hey, I’ve always been a glass-half-full guy, so humor me. While trade between countries can indeed foster cooperation and contribute to peace in several ways, it's essential to recognize that it's not a panacea for all global issues. However, here are some ways in which trade can contribute to peace and global stability.