Overview
The objectives of this report are to summarize the work, results and findings of the extended reality analytics
suite, SANALEXA, developed by Simulanis, as part of which, simulation-oriented Augmented Reality (AR) and Virtual
Reality (VR) technology was used to train, assess, and evaluate and low-income workers performing dangerous tasks
in unsafe factory environments across a range of industries.
Conventional methods to train workers are not effective to meet the rigours and demands of modern-day factories
considering the high level of complexities involved in industrial operations. Moreover, training on such complex
and dangerous industrial operations presents a high risk to life, which conventional methods fail to address
effectively.
The aim of this project was to leverage AR and VR technologies and build software applications to train the
workers in a safe environment and develop a platform which embedded deeper and exhaustive analytics within the
AR and VR based training applications to comprehensively study, gauge, and track the learning, retention and
development of workers as they handle complex and dangerous industrial operations.
Through this project, we were successful in training batches of workers on complex and dangerous industrial
operations safely using AR and VR technology which was proven to be effective in achieving the learning outcomes
through an advanced 3-tiered cloud-based analytics suite layered on top of the AR and VR based training
applications.
The results proved that by receiving training through AR and VR applications, 80% of the batches of workers
achieved a reduction in errors committed over successive iterations of learning and training. A 56% reduction
in the violation of safety protocols was observed post training using AR and VR applications, along with a 52%
increment in cognitive skills instrumental in defining the hazard perception abilities of the workers. All
workers were able to learn the operations in a completely 100% safe manner. We obtained insightful and meaningful
data on the behaviour of workers and proved that intervention of AR and VR technologies for training positively
impacted their confidence levels and motivated them to perform the complex and dangerous industrial tasks in a
much-improved way.
IN THIS REPORT
Problems with Conventional Training Methods
The conventional methods used in manufacturing-based industries (such as pharmaceutical, automotive, oil and gas, automation, chemicals, paints, textiles, rubber, leather, power, utilities, and other allied industries to name a few) or in training institutes set up to train all types of workforces are either on-the-job or off-the-job.
Conventional Training
In an on-the-job mode, the following methods are often adopted:
Position/Job Rotation
Understudy Method
Coaching Method
In an off-the-job mode, the following methods are often adopted:
Vestibule Training
Lectures
Role Play
Limitations of conventional methods
The above methods (both on-the-job and off-the-job) have been employed and implemented across industries to varying levels of success and a wide array of limitations have been cited, such as:
-
Most of the companies and factories do not have formal off-the-job training programs. Moreover, most of the conventional training material is based on Standard Operating Procedures (SOPs) or operating manuals, and due to the underlying text-based nature of the same, training methods are not hands-on and practical, thereby lacking engagement.
-
Conventional off-the-job training programs lack visualization and are mostly theoretical in nature. This directly affects learning, retention, and recall of technical subject matter, adding to the ineffectiveness of such methods.
-
Most of the industrial tasks and operations are dangerous and hazardous in nature which pose a grave risk to life, hence making it difficult to impart conventional on-the-job training on those tasks and operations.
-
Conventional on-the-job training within the factory premises may require lengthy discussions and interactions between the trainer (or supervisor in most cases) and the workers which leads to long downtimes and high idle times for the manufacturing equipment and resources respectively. This trade-off situation presents a challenge to scale the entire activity due to the high operational cost-burden to the factory. Furthermore, during such training sessions, if the equipment gets subjected to damages, the incremental capital cost of repair adds to the burgeoning costs. To further factor-in, if the worker gets injured during training, the costs surmount leading to legal troubles, bad-press and negative social image for the factory, inhibiting their intentions to train the workers on-the-job.
-
Most of the conventional training methods fail to meet the rigorous demands from the industry and other strict regulations. Hence, a large segment of the worker population does not receive training in any form.
Consequences of ineffective training
As highlighted above, owing to the dangerous nature of industrial tasks and operations, in most
circumstances it becomes practically difficult to deliver training which benefits the workers
through conventional methods.
Subsequently, the lack of proper training becomes one of the predominant factors which contributes to
“risky” worker behaviour. Workers who are not trained properly as per the required standards lack
the cognitive ability to evaluate potential hazards and risks, which increases the probability of
occurrence of dangerous incidents and ultimately lead to an unsafe work environment.
Young, uneducated, poor, illiterate and less experienced workers in the industries usually do not read the
operating or safety instructions carefully and as a result, do not perform the related activities that
require special care and caution. The well-experienced workers, on the other hand, often become overly
self-confident and then do not strictly follow the safety instructions, relying on their inherent skills.
The monotonicity of the job causes a decrease in concentration when the same operations have to be
performed multiple times, thereby, increasing the possibilities of accidents
Lack of focus while training
Solution to Challenges: Extended Reality (XR)
The ineffectiveness of existing conventional (on-the-job and off-the-job) methods to train vulnerable workers, Augmented Reality (AR) and Virtual Reality (VR) technologies are novel and cutting-edge methods to address these challenges.
VR Training Flow
Augmented, Virtual and Mixed Reality fall under the same family of immersive technologies which can have a remarkable effect to alter our perceptions and enhance our visualization capabilities. While AR and MR adds overlays of information on our existing reality, VR on the other hand leaves us blind to our surroundings and transposes us to a different—simulated—world all together. While the ultimate objective of VR is true immersion, the value of it is a more intuitive way to interact with the digital world.
Need for AR VR Technologies for Training
The key issues in the manufacturing sector are that the technology landscape has evolved and employers
are looking for multi-skilled employees who can perform various tasks.
The acquisition of skill requires practice. Merely repeating a task alone, however, does not ensure
the acquisition of a skill. Skill acquisition is achieved when an observed behaviour has changed due to
experience or practice. This is known as learning and is not directly observable. The information
processing model, which incorporates this idea of experience, proposes that skills develop from the
interaction of four components central to information processing. These components include: processing
speed, the rate at which information is processed in our processing system; breadth of declarative
knowledge, the size of an individual's factual information store; breadth of procedural skill, the ability
to perform the actual skill; and processing capacity, synonymous with working memory. The processing
capacity is of importance to procedural memory because through the process of proceduralization an
individual stores procedural memory. This improves skill usage by linking environmental cues with
appropriate responses.
Since the majority of the employees in the manufacturing sector are blue collared workers, many of
them have a low literacy level which can make teaching complex tasks, concepts and skills a challenge
for organizations. In particular, training such workers on health, safety and regulatory compliance a
major challenge that organizations often face. The traditional classroom training programs and
Instructor-Led Training (ILT) can become lengthy, expensive and resource intensive. Furthermore,
many of these training programs lack the level of engagement necessary for effective training to take
place. Due to the workers operating under hazardous manufacturing conditions, in-effective training and
neglecting HSE compliance can lead to fatal and catastrophic accidents.
VR Technologies for Training
What makes AR VR Technologies "effective" for Training?
Virtual reality has unlimited possibilities. We’re looking at every opportunity we can right now. You can train people faster, and people learn faster in a virtual reality environment.
Laura Collings,
Training Manager at UPS
VR technologies have proven to be a game-changer to address the problems discussed in the above section. The section below summarizes the advantages of adopting AR-VR technologies for industrial training:
- Engagement: Engagement is a critical factor in learning and training outcomes, conventional training methods which lack engagement are often ones which are least effective. VR experiences which are both immersive and interactive, boost engagement levels which turn have a positive impact on the effectiveness of training.
- Cuts down access barriers: VR cuts down access barriers by having the capability of exposing individuals to experiences which they otherwise can’t access due to financial, geographical or resource constraints.
- Accelerated learning: Because VR is so immersive and compelling, learners absorb information
faster and retain what they have learned longer. Memory formation is linked to emotional response,
and VR is an emotionally charged experience.
- Visual imagery helps memory.
- Spatial encoding helps memory.
- Use of proprioception helps spatial understanding.
- Learn by doing: The immersive nature of Virtual Reality enables a practical or experiential learning to take place. Workers are encouraged to actually perform the task instead of plainly reading and memorizing them from a textual or verbal source. This has been widely known to be one of the most effective methods of improving worker understanding.
- Practice makes perfect: The replay ability factor thereby allows users to practice a training module multiple times and sharpen their skills at their convenience. It hence better prepares the user by increasing the workers confidence in operating hazardous equipment and reducing the risk of accidents – viz. harm to human life and damage to equipment
- Safely explore unsafe consequences: Because all training is virtual, workers can take risks they usually wouldn’t take; or pursue creative, high-risk solutions to problems. It allows workers to learn and practice their operating procedures in a completely safe virtual environment before they try their hands on the real operations.
- Previewing: VR technology allows workers to perform test runs of new initiatives to see how employees respond.
Research is showing that virtual reality can be a powerful tool for impacting people's perceptions and actions in the real world.
Stephanie Moffet,
Spokesperson, FEMA
Phase 1:
Selecting the key industrial task and operation
Post selection of the key geographies of focus based on manufacturing activity and nature of dangerous tasks, selecting appropriate industrial tasks and operations was critical in order to validate the overall success of the analytics suite we had envisaged.
We wanted to select operations which (were):
- Standard across all core industries viz. pharmaceutical, FMCG, chemicals, power, oil and gas, automotive, and automation to name a few.
- Had a high element of danger or risk accompanying them.
- Had a high proportion of vulnerable blue-collar workers operating them.
- Had a high degree of manual intervention where any error in the operating sequence could have led to catastrophic consequences.
- Required a moderate-to-high level of cognitive and motor skills for the retention of technical subject matter which could be applied to perform the complex operation.
Considering all the factors highlighted above, after brainstorming internally and externally with key stakeholders (viz. workers, factory-owners, industry experts, and senior technical subject matter experts), among a host of industrial tasks and operations, for the pilot project, we decided to base our work around a critical industrial procedure conducted in factories.
Commencing with on-site activities
The project activities on-site commenced with gathering the pre-requisite data needed to build the
corresponding VR application and the 3-tiered Analytics suite for the same.
The data gathering phase saw our team of engineers, designers and data scientists capture the process
activity in-person being applied to a machine. This required multiple visits to the site as the quantum
of data required to be collected was large and owing to its indispensable requirement for building the
VR application and the Analytics suite, it was vitally important to ensure it was collected correctly
and responsibly.
Our data science team captured the key data points of the procedure by closely studying the operation as
it was done routinely in real-life by the experienced and well-trained workers on-site and compared it
with the standard operating procedure put in place by the factory. This comparative study helped us to
understand the anatomy of the errors and deviations which emanated from worker behaviour, attitude towards
safety and knowledge of the risks and dangers involved in the activity being performed by the workers.
It was important to record these errors meticulously as they would form the basis to build the Analytics
ssuite layered within the corresponding VR application.
On site for data gathering
The Work in-house
Development Phase
Post the data gathering stages, the action shifted in-house where the engineering teams worked with
the design teams to create the VR environments using the inputs collected from the on-site visits.
The engineering, data science and design teams worked together to create the architecture of the VR
application which was handed over to the technology development team.
The bulk and ‘heart’ of the entire project was the technology powering the pilot project: coding to
build VR using popular game engines along with back-end and front-end dashboard development. As an
innovative technology intervention and value-add to the pilot, we integrated wearable devices with
the Analytics dashboard to comprehensively study the contribution of physiological aspects towards
worker behaviour.
Phase 2: Testing the VR in beta
Building VR environments replicating the real-world of plants and machinery in the factories required
testing from the workers to validate our hypothesis for memory recall, and an important step before
finalizing the development of the VR application.
The VR application was tested on-site with the workers and their feedback along with inputs from the
site stakeholders was taken to improve the final product.
Setting-up the experiment to test the Analytics engine
We employed a split A/B methodology to test the Analytics engine integrated with the VR application
by taking a sample population of a set of workers and subjected one batch to training in VR and the
other to training through conventional on-the-job and off-the-job training methods.
Workers subjected to training in VR were exposed to the virtual environment which mimicked and
replicated the real-world. The application (read: module) was immersive and simulated the entire
operation taking place on a virtual shop-floor.
To test the technical understanding of the workers, tests and assessments related to the
operation were interspersed within the VR module. Cognitive, psychometric, and behavioural tests
were an integral part of the overall testing and evaluation of retention and on-the-job application
skills of the workers, and a series of tests in VR were built to analyse worker performance on these
parameters.
Workers subjected to training through conventional methods of classroom training through presentations
and instructor-led on-the-job training were tested via paper-based assessments and questionnaires
taken in a timed manner under invigilation and supervision.
Conventional training v/s VR training
Phase 3: Deploying the final product
Final deployment by our team
Post the phase of testing and gathering valuable feedback and insights from the experiments conducted on-site, the final product was deployed within the factory and made accessible for the workers to get trained in VR. At this stage, we sought views from the entire stakeholder chain and based on their resoundingly positive feedback, we got immense confidence that we were on the right path to succeed in meeting the targets of the pilot.
Result & Finding
In summary, the overall results of the pilot successfully proved and validated the hypothesis that VR based training was more effective and better suited than conventional training methods to train vulnerable, low-income and blue-collar workers, as qualitatively:
-
The average score of the performance of workers who were trained on VR in assessments and tests conducted post training was 75% as compared to an average score of 52% achieved by those who were trained using conventional methods. The improvement in scores obtained by the workers who received training in VR can be directly attributed to their improvement (trend below) in performing the industrial operation whilst getting trained in VR as there was an 87.5% average reduction in the number of operational errors committed whilst performing the operation over the course of training in VR.
Results revealing effectiveness of VR based training over conventional training
A graph from the dashboard depicting reduction in error within VR
-
The violations of safety protocols, critical to health, committed by workers whilst training within VR were reduced by 56% over the course of training in VR.
-
The time taken by the workers to complete the industrial operation in VR was reduced by 17% over the course of training in VR.
A graph from the dashboard depicting reduction in time within VR
-
The results of training workers in VR and testing their cognitive skills further revealed a 37% increase in the retention abilities over the course in training in VR. Along the same lines, results revealed a 60% increase in visual awareness (visuo-spatial abilities) over the course of training in VR.
-
By capturing the data on the gaze and focus of workers (image below) as they were getting trained in VR and performing the industrial operation in VR, it was found that 90% of the workers attentively focused only on the areas where the tasks had to be performed.
-
In a remarkable set of observations on worker emotions as they received training in VR, it was seen that their negative emotions related to performing the operation in VR reduced (on average) from 55.9% before training (in VR) to 3.8% post training (in VR) and positive emotions improved from 61% before training (in VR) to 71% post training (in VR).
-
Through our Analytics dashboard, we obtained a comprehensive psychometric profile of the workers based on well-established psychological research-based (five-factor personality taxonomy by Costa & McCrae, 1950s) personality tests (viz. openness to experience, conscientiousness, extraversion, agreeableness, neuroticism and empathy).
-
Through our Analytics dashboard, we also obtained a comprehensive behavioural profile of the workers evaluating the motivation (intrinsic and extrinsic), directive behaviour causing the workers to initiate or repeat an action, acquired and inherited mental and emotional entity, as well as values and ethics or the principles in life that the worker would abide-by to live according to societal norms.
Impact Created
Delivering an impact to the vulnerable, low-income and blue-collar workers, by delivering a
better training experience addressing the needs and challenges faced by them was the primary
objective of this pilot.
We achieved the impact by conceiving a methodology of training leveraging VR technology and building
a comprehensive Analytics suite {the “output”) linked with the VR application to impart training on
a dangerous industrial operation.
The results and findings of the pilot were the outcomes which led us to determine the impact and
quantify effective change against relevant performance metrics and baseline, target and current
indicators, as elaborated in an account of outcomes below and summarized in the table below:
| Performance Metric | Indicator Baseline | Indicator Target (near-term) | Current Value (immediate) |
|---|---|---|---|
| Reduction in operational errors (in VR) | 100 | 10 (↓90%) | 12.5 (↓87.5%) |
| Reduction in safety protocol violations (in VR) | 100 | 75 (↓25%) | 44 (↓56%) |
| Reduction in operational errors (in VR) | 100 | 150 (↑50%) | 153 (↑53%) |
- Reducing the number of operational errors in VR:
The average performance of workers who received training in VR improved (from 52% to 75%) as opposed to those who got training through conventional methods, and this improvement was due to the reduction in errors (by 87.5%) committed by the workers whilst performing the industrial operations in VR.
Compared against the indicator targets of near-term (90% in 1-2 years), these immediate outcomes (87.5%) lead us to positively believe that within this time-frame, positive impact will be delivered by improving the technical proficiency and skills of the workers, thereby leading to better job growth and more compensation and benefits. - Reducing the violation of safety protocols in VR:
Results revealed that the violations in safety protocols, critical to health, were reduced by 56% amongst the workers trained in VR, further proving the effectiveness of VR as a training tool to safely deliver training to the workers, and tying-in well with the improvement of their technical proficiency and skills.
Compared against the indicator targets of near-term (25% in 1-2 years), these immediate outcomes (56%) lead us to positively believe that within this time-frame, the workers will be able to safeguard themselves from the various risks and hazards posed by dangerous industrial tasks and minimize the probability of endangering their lives. - Reducing the time to get trained:
The time taken by the workers to complete the industrial operation in VR was reduced by 17% over the course of training in VR, thereby proving that workers were more engaged within VR due to the high degree of interactivity and immersivity offered by the technology. This performance metric contributed positively towards improving the overall effectiveness of VR for training the workers.
Analysing the metric of overall effectiveness of VR for training the workers, compared against the indicator targets of near-term (50% in 1-2 years), these immediate outcomes (53%) lead us to positively believe that within this time-frame, more workers will be able to get trained effectively using VR in a short time-span, thereby resulting in improved skill levels, job growth and opportunities, more compensation and benefits, - Improving the cognitive abilities of workers:
Training workers in VR led to improvements in their cognitive skills in the form of a 37% increment in retention abilities and 60% increment in visual and spatial awareness, thereby proving that VR as a technology tool for training led to effective memorization of standard operating procedures and offered a better recall to the workers as they performed the industrial operations physically.
One of the primary reasons for improvement in cognitive skills is the likeliness and replication of real-world operations with high-quality, detailed and interactive virtual environments coupled with repeated exposure to the industrial operation in VR which gives the workers a completely immersive and next-to-real experience of practising the tasks multiple times and getting trained safely.
The impact of better cognitive abilities can lead to significant improvements in long-term memory and quicker reaction times to safety-related incidents reducing their possibilities to meet with accidents. - Improving the focus:
Data on gaze of workers captured as they performed the industrial operations in VR proved that 90% of the workers were found to be less distracted and more focused on critical tasks of the industrial operations whilst performing them in VR.
The impact of improved focus whilst performing dangerous industrial operations can make the workers more aware of the potential risks and hazardous situations around any industrial operation and make them more responsible towards their own safety, thereby reducing their probability of meeting with accidents whilst at work. - Positive emotions:
Data on emotions of workers captured as they performed the industrial operations in VR proved that their negative emotions related to performing the operation (on average) from 55.9% before training (in VR) to 3.8% post training (in VR) and positive emotions improved from 61% before training (in VR) to 71% post training (in VR)
The impact of improved focus whilst performing dangerous industrial operations can make the workers more aware of the potential risks and hazardous situations around any industrial operation and make them more responsible towards their own safety, thereby reducing their probability of meeting with accidents whilst at work.
The impact of increased positive emotions whilst training can make the workers more productive at work.
Conclusions and Next Steps
Simulanis envisions a future where any worker can have a desired level of skill
set, a growing level of confidence which will deliver a better future for the
blue-collared worker ensuring a stable lifestyle.
This project highlighted the grave need for a training mechanism for vulnerable
workers to ensure a safe and effective form of training leading to better job
growth, more compensation and benefits.
More than a training solution
We met several workers during the course of the pilot including those who were
working far away from their home towns, who were hired on a daily basis and
those who would constantly live in the fear of being without a job the very
next days (daily contractors).
Delivering training to workers was not a barrier for industries to achieve but
to assess the immersivity and the level of understanding of the technical
procedures among the workers who took the training was a clear barrier.
Currently, the training methodology adopted across majority of industries is
conventional form of training that is neither secure, just or sustainable. The
current training methodologies just create an environment of vulnerability,
restricted growth and poor technical proficiency among the blue collared workers.
This means a form of training has to be implemented that not only provides an
immersive form of training that captures the effectiveness of the training but
also implement a solution that assess the emotional and psychological state of
the workers. Thus, a training method that supports each worker as much as it helps
the organization and ensures the overall growth.
A training method for a vulnerable blue-collared worker and a secured future to come together.
The first and foremost thing on our radar is Extended Reality (XR) and making sure
our training tools are not limited to just VR as a medium but can also be taken
forward to other mediums like AR, MR, PCs, mobile and more. In the recent past,
we were able to deploy our analytics suite in AR for an automobile giant further
strengthening our belief that we can build something that is robust enough to be
deployed on different platforms and is truly cross platform as well.
In under-developed and developing nations, a common pattern has been the latest
generation skipping computers for smartphones as their first internet enabled
computing device. Serving this audience is key to scaling up our solution and
hence cross platform training modules are our biggest priority at the moment.
We plan to make further developments to our analytical suite by working towards
a solution which will be smart enough to provide support or guidance to the
worker based on his performance in the training module. Thus, we will aim to
include machine learning and artificial intelligence which will set the
difficulty levels of the training programs based on the performance on the
workers. In parallel, we will build an employee engagement app which will focus
primarily on the following:
- Create a database for all the workers.
- Provide a communication channel between the organization and the workers.
- Ensure continuous growth of a worker by communicating important updates and feedback from team leads.
Virtual Reality is unique in creating a sense of presence, like you’re right there with another person or in another place. If you can’t think of any way that your reality can’t be better, then you’re not thinking hard enough, it’s not about escaping reality, it’s about making it better
-Mark Zuckerberg
Key Definitions
- Virtual Reality:
Virtual reality is an interactive computer-generated experience taking place within a simulated environment. It incorporates mainly auditory and visual feedback, but may also allow other types of sensory feedback like haptic. This immersive environment can be similar to the real world or it can be fantastical. - Manufacturing Industry:
The branch of manufacture and trade based on the fabrication, processing, or preparation of products from raw materials and commodities. This includes all foods, chemicals, textiles, machines, and equipment. - Blue collared workers:
A blue-collar worker is a working-class person who performs manual labour. Blue-collar work may involve skilled or unskilled manufacturing, mining, sanitation, custodial work, textile manufacturing, commercial fishing, food processing, oil field work, waste disposal, and recycling, construction, mechanic, maintenance, warehousing, technical installation, and many other types of physical work. Blue-collar work often involves something being physically built or maintained. - SOP:
A standard operating procedures manual is a written document that lists the instructions, step-by-step, on how to complete a job task or how to handle a specific situation when it arises in the workplace. - Simulated environment:
A simulation-based learning environment is a setting for learning that includes a controlled, shielded and often simplified copy of a real-world process or system to be studied. - Analytics:
Analytics is the discovery, interpretation, and communication of meaningful patterns in data; and the process of applying those patterns towards effective decision making. - Eye tracking:
Eye tracking is a sensor technology that enables a device to know exactly where your eyes are focused. It determines your presence, attention, focus, drowsiness, consciousness or other mental states. - Visuospatial:
Visuospatial refers to cognitive processes necessary to "identify, integrate, and analyze space and visual form, details, structure and spatial relations" in more than one dimension. Visuospatial skills are needed for movement, depth and distance perception, and spatial navigation. - Instructor-led training:
Instructor-led training, or ILT, is the practice of training and learning material between an instructor and learners, either individuals or groups. - Immersion:
Immersion into virtual reality is a perception of being physically present in a non-physical world. The perception is created by surrounding the user of the VR system in images, sound or other stimuli that provide an engrossing total environment. - Hazardous energy:
Hazardous energy is any electrical, mechanical, hydraulic, pneumatic, chemical, nuclear, thermal, gravitational, or other energy that can harm personnel working on machines. - Pneumatic Energy:
Pneumatic energy is the energy stored within pressurized air. Like hydraulic energy, when under pressure, air can be used to move heavy objects and power equipment. Examples include spraying devices, power washers, or machinery. - Affect:
Affect is a concept used in psychology to describe the experience of feeling or emotion. - Guided module:
Guided module is a form of training where a user is provided audio (voice overs) and visual(pop-ups) guidance throughout the course of training. - Freestyle Module:
Freestyle module is the assessment form of training where a user is not provided any guidance and he has to finish the training the training on his own, also during the the course of this training his performance will be assessed. - Marker less AR:
Marker less AR is a term used to denote an Augmented Reality application that does not need any pre-knowledge of a user’s environment to overlay 3D content into a scene and hold it to a fixed point in space.