A flight simulator for the office: How VR is helping workers hone soft skills

As National Health Service waiting times continue to grow, medical staff in the UK are increasingly confronted by angry patients — and their families — affected by delays in accessing treatment.

“With all the stresses and struggles and strain on the NHS waiting times, patients are, understandably, quite frustrated. We are well aware of the heightened levels of confrontation that clinical staff have been dealing with post-COVID,” said Javidul Islam, physician associate at the Page Hall Medical Centre in Sheffield, England.

Such situations can be intimidating — scary even, especially for new staff. “When I did an A&E [Accident and Emergency] placement, some of the patients were very aggressive to staff and tried to attack them,” said Islam. “So that can be quite a sobering experience. You can definitely feel vulnerable. You are acutely aware of making sure you don’t say the wrong thing to aggravate a situation.”

Although medical professionals receive some communication training as students, this doesn’t compare with the firsthand experience of dealing with conflict. “You can read about it as much as you want, but nothing really equips you for dealing with a conversation like that,” Islam said.

That’s where new virtual reality (VR) training methods can help. Islam says training in an immersive VR environment has better prepared him for times when he may be forced to deal with confrontation for real. “This type of technology is a good bridge to being able to develop some techniques for yourself,” he said.

The VR training tool used by Islam, a learning module called Navigating Angry Conversations from VR training vendor Bodyswaps, was created in partnership with the UK’s Royal Society of Medicine (RSM), a charity that supports ongoing education for medical professionals.

The training, available to the medical society’s 20,000 members worldwide, puts the staff through a series of tasks in a 3D virtual environment. This includes interactions with avatars that replicate confrontational situations.

Bodyswaps

Upon completion, participants are given feedback on a range of outputs generated during the interaction, including their response dialogue and speech patterns, as well as eye- and gesture-tracking. Following some interaction scenarios, it’s possible to swap “bodies” and re-watch your own performance to learn from any mistakes.

By practicing in a virtual environment, medical professionals are able hone the skills needed to defuse a heated situation: empathy, patience, and effective communication.

“We’re not just aiming this at doctors — any health professional could be in this situation. Even a nurse who’s taking blood tests, for example, can still get that level of anger if somebody has waited three months for a test,” said Mary Bishop, director of learning at the RSM.

The training can be done via a browser, but it’s by donning a VR headset that doctors, nurses, and consultants get to practice in the most realistic way. “When I was in the consultation [scenario], I felt like when the patient was angry, I could feel it more. I was aware I was in the room,” said Islam, with the virtual environment and avatars heightening the sense of risk.

“You’re almost put in the place of being a fly on the wall, watching your own behavior,” said Bishop. “That’s an incredibly good way of learning, because you can reflect in a safe space: this is an area that needs deep reflection for good communication development.

“If you keep doing that, by the time you actually come under pressure from somebody who’s very upset, you’ve already met that situation maybe three or four times. You can refer to the breathing [exercises], and think ‘okay, don’t react straightaway, just give a moment,’ and then think about your choice of language.”

Bodyswaps

What are soft skills?

Not all workplace situations are as emotionally fraught and potentially dangerous as those experienced by frontline medical staff, but the need for so-called soft skills such as effective communication is felt across every sector.

Soft skills are non-technical skills that can be applied in a wide range of situations, often relating to problem solving and interpersonal interactions.

“We would define those as more like business skills — adaptability, communication, planning, the skills that you are using every day to interact with people — whereas hard and technical skills are more focused on a piece of technology or a tool,” said Travis Wickesberg, senior director analyst at Gartner’s HR Technology practice.  

These skills are critical for individuals to be successful in their job, he said, and are highly valued by employers seeking staff that are adaptable and effective both individually and in a team.

A recent analysis of 21 million job ads in the US, UK, Canada, and Australia by learning company Pearson showed that the five most in demand skills were so-called “human skills” — communication, customer service, leadership, attention to detail, and collaboration.

“When you look across job descriptions, soft skills outnumber hard skills,” said Wickesberg.

The market for soft skills training services is worth in the region of $20 billion a year, according to some estimates, and forms a part of the wider market for learning and development.

There are different approaches to developing these capabilities, with varying strengths and weaknesses. In-person training is an effective learning method but can be costly and time-consuming to implement. Online learning platforms, such as Skillsoft, LinkedIn, Udemy, and Udacity, provide another route to skill development, with self-service access to training materials — though support for employees can be limited, and staying motivated while completing modules on their own can be a challenge for some workers.

VR offers a new route to soft skills training

With advances in VR technology lowering headset costs, immersive training has emerged as another way to develop skills in the workplace, both for practical training (health and safety training, for example, or warehouse operations) and for these softer skills, such as presentations to colleagues, conflict resolution, and developing empathy and understanding.

Training materials are often tailored to a certain use case, such as public speaking, conflict resolution, or sales pitches and negotiations. Training for diversity, equity, and inclusion (DE&I) is another focus for vendors.

VR-based learning promises a greater level of immersion, with workers given the opportunity to practice skills at their leisure and build confidence in a virtual environment before taking on challenges in the real world. 

This is a key advantage over traditional soft skills learning platforms, said Christophe Mallet, CEO and co-founder of Bodyswaps. He describes the immersive training platform as “a flight simulator for soft skills.”

“There’s a gap between the classroom and performance in the real world, and that gap is experience,” he said. “Learning social influence or empathy or leadership on PowerPoint slides is about as useful as learning to swim on video: when you to get to the real world, you will drown.”

Bodyswaps

Training in simulated environments means employees can make mistakes in a safe environment before taking the next step. “You don’t have to fail fifteen negotiations to be decent,” said Mallet, “maybe you only have to fail five. You need the practice element to build your skills — that’s where VR is interesting.”

The ability to do access training on demand was a benefit noted by Islam. “If you’re weak in one area, you can tailor your own training in your own time — you’re not relying on anyone,” he said. “That’s where I think it’s useful.”

Another advantage is retention of information. “It’s very easy to deliver some learning, and then it gets forgotten because people don’t get the chance to apply it,” said Justin Maggs, leader of the Professional Development & Business Skills Learning Stream at Severn Trent, a UK-based water supply firm with around 7,000 employees.

Maggs provides VR training to staff for a range of purposes, including soft skills such as public speaking and conflict resolution. “Through practice, it enables people to build confidence, especially with things like presenting,” he said.

Severn Trent has several VR training initiatives in place, with around 30 Oculus VR headsets available. Last year, the company acquired licenses for VirtualSpeech, another immersive learning platform aimed at soft skills such as presentations.

“Public speaking is our most popular course,” said Sophie Thompson, CEO and founder of VirtualSpeech. The public speaking learning module available in VirtualSpeech’s library is around four hours long, teaching key theory and techniques online through video tutorials, case studies, quizzes, and interactive practice exercises, she said.

At certain points throughout the course, learners are prompted to practice using virtual reality. “They put on their VR headset and can be immersed into environments with different-sized audiences,” said Thompson. “If you want to practice a one-on-one conversation, you can. If you want to practice presenting at a team meeting, you can. Or if you want to practice speaking at a TEDx-style conference, you can also do that.”

After completing the training, users receive a score for certain aspects of performance, such as eye contact, speech pace, use of filler words, and more.

VirtualSpeech

Staff at Severn Trent have accessed the VirtualSpeech presentation training module to hone their skills speaking in front of an audience. “We find that people are lacking a little bit of self-confidence around presenting: they have some self-limiting beliefs,” said Maggs. “Giving them the opportunity to practice [in VR] helps build confidence and gives them the chance to self-reflect on their performance.”

Measuring the impact of soft skills training can be tricky, said Maggs, but employee self-assessments of the presentation training has shown long-term benefits around confidence and competence for staff.

“We’re seeing that confidence and competence are actually higher six months later than [immediately] post-course. So their self-assessment is saying that that learning has embedded, and they are able to use that learning a period of time later,” he said.

VR training tools

Bodyswaps and VirtualSpeech are two of the immersive learning vendors that provide corporate soft skills training, alongside others such as Strivr, immerse.io, and Mursion. For some vendors, such as Strivr, arguably the most prominent VR learning platform, soft skills are a part of a wider portfolio of learning tools aimed at workplace skills that also includes VR training for operational purposes such as accessing new machinery equipment, for instance.

Although the use of VR in the workplace more generally is in its infancy, employee training has arguably been the biggest success to date.

Walmart is perhaps the highest-profile example of VR training adopted on a large scale, with reports of over a million employees using VR headsets to learn. One part of the available training, developed with Strivr, relates to soft skills development, including de-escalation of conflict with angry customers.

This is the sort of training just isn’t as effective in a classroom, said Aneesh Kulkarni, Strivr’s CTO, and is where VR excels.

“The experiences are very real, you actually feel like you’re talking to a frustrated customer,” said Aneesh Kulkarni, Strivr’s CTO. “You’re able to observe what is good de-escalation, you are asked how you would respond to the situations, you can speak, and your response is recorded and then analyzed.”

Learners are then able to access analytics generated in the practice session to see how they performed, and then practice multiple times to improve their skills.

“You can look at the metrics to see your voice analysis, whether you paused or used a lot of filler words when you’re talking,” said Kulkarni. “That helps you in a real-life situation to be better prepared.”

The role of generative AI

A fast-developing area of VR training is the addition of generative AI to create more realistic interactions with avatars.  

VirtualSpeech is the first VR training vendor to incorporate generative AI into its platform, according to Thompson, the company’s CEO. The company launched a learning module in February that lets users interact with an avatar driven by OpenAI’s ChatGPT language model.

Just as you can enter into a lively debate with ChatGPT, generative AI enables more realistic interactions with avatars in the VirtualSpeech application, said Thompson, allowing users to engage in back-and-forth conversations before receiving personalized feedback on the interaction. This means that an employee can practice asking their boss for a pay raise, or a manager can practice giving a performance review or disciplinary meeting.

VirtualSpeech

The conversations require time for ChatGPT to process a response but provide greater flexibility than scripted interactions, according to Thompson.

“It enhanced our learning experiences so much, because now anybody can do any roleplay on demand, and it can be personalized to their industry,” he said. “Before ChatGPT, we had to upload generic questions [for users to answer], but now those questions can be very specific and on a huge range of topics.”

Conversation data generated in VirtualSpeech isn’t used to train OpenAI’s language models, said Thompson. Nevertheless, some customers, particularly those in regulated sectors such as finance, have so far preferred to turn off the generative AI features.

Bodyswaps has also added some AI functionality to its platform, though CEO Mallet is wary about how generative tools are implemented. “We are cautiously optimistic about AI,” he said. “We are optimistic because it’s very powerful, but we are also cautious because it’s very powerful.”

At a more basic level, AI can provide semantic analysis of a user’s speech when converted to text. This enables the software to detect the use of keywords related to a training task, for example.

With ChatGPT, it’s suddenly possible to have full, realistic conversations — the “holy grail,” as Mallet puts it — but also a prospect that brings some significant risks. “Can you do it safely? Can you do it without bias? If you ask ChatGPT to play a doctor, is it going to play a man? Is it going to play a white man? And it is protecting the safety of the learner?”

He pointed to the creation of the RSM’s conflict resolution training module as an example of where genAI in VR training could be risky. In this scenario, a scripted conversation is safest, he said.

To develop the script, Bodyswaps worked with Dame Lesley Fallowfield, a professor of psycho-oncology at Brighton & Sussex Medical School, University of Sussex and an expert in communication between doctors and patients. Fallowfield was involved “because the level of psychological safety that you have to safeguard is super, super important. If you gave that to an AI, what would happen?” said Mallet.

 “And so an interesting question now in our little world of soft skills training is where do you draw the line between human script and AI script, and how they interact with one another?” he added. “That line is constantly shifting.”

Kulkarni said interactions with generative AI are not currently available in Strivr but are on the company’s product roadmap. 

He also highlighted the potential of generative AI to create virtual environments more efficiently.  Today, a lot of the content generation is manual, he said. “You’re taking existing videos, pulling them into animation tools and editing them, or you’re creating new experiences and scenes, which takes time and is expensive.”

AI tools could help reduce the time to create 3D environments. “AI can be used to proceduralize content generation,” he said, which could enable the creation of realistic content “such as landscapes, objects, characters, environments, scenes.”

Will VR replace other types of learning?

The underlying technology is nascent, with VR software and hardware continuing to develop, but Gartner’s Wickesberg sees positive early signs of uptake of VR-based soft skills training tools.

“There’s growing interest — definitely exploration, experimentation,” he said. “Will it replace a traditional tool? Not right away. I think there’s value in online self-paced and classroom [learning]. But I think VR complements those tools very nicely, because it gives people that outlet to practice very safely, and to receive the feedback they need to make their behavior change.

“So I think there will be a progression: you experiment with VR and find it successful, then you find other scenarios you can experiment with, and it becomes complementary to your strategy.”

The majority of employees at Severn Trent had positive experiences with VR training, but it’s not for everybody, said Maggs.

“For some people there’s motion sickness. And for some people the reality is almost too much. They put the headset on and go, ‘oh, I wasn’t quite ready to stand in front of an audience of 80 people’ or whatever,” he said.

“We had a few people that find it a little bit difficult with the focus — the imagery is a bit blurry. If I did a rough estimate, maybe it’s 10% of people that can’t get on with it, though that’s very much ‘finger in the wind.’”

As for provision and management of devices, although the short battery life of VR headsets can be an issue at times, training facilitators at Severn Trent have generally encountered few problems, he said.

Most see VR as another tool in a blended approach to learning, with relative advantages and disadvantages compared to other training tools and methods.

For Page Hall Medical Centre’s Islam, VR training can be a “bridge” between a textbook and the real world, but it can’t replace real experience. “I wouldn’t say it would replace [other forms of training]; I think it would just enhance people’s learning,” he said.

“It will never replace in-person training, but there are many people who don’t get the opportunity for in-person training, so it will fill a market,” said Bishop at the Royal Society of Medicine.

Maggs said it’s important that learning and development programs show a real use for immersive, VR-based learning. “We’re really conscious of not using it as a gimmick,” he said, “so only using it where it serves a purpose and where it fits in with a blended learning solution.

“I think the danger is if you just go after it for the novelty factor, it will soon lose its value. It needs to be a very planned, conscious effort to use it.”

He sees VR playing a key role in soft skills training for staff at Severn Trent in the long term. “Soft skills are never going to go away,” he said. “They’re always going to be part of training across organizations. Finding new ways to develop those skills is part of our role.

“It’s quite exciting what virtuality reality could offer us going forward. It feels like it’s embryonic at the moment and people have just started stepping into space, so it’s interesting to see where it’s going to go. And ChatGPT and AI are going to open up new avenues.”