The Complete Guide to Job SimulationsPublished: 2022-12-08
Skilled workers are more important and in demand than ever before, placing much-needed pressure on organizations to validate candidates’ skills and predict job performance accurately to provide hiring confidence.
Despite this, most organizations’ hiring regimes still look something like this: hiring managers will start by sifting through a pile of resumes, invite the candidates who seem the most qualified on paper to an unstructured interview, and make a hiring decision based purely on assumed suitability.
Finding the right candidates is challenging. And to be honest, it’s no surprise to us. Why exactly? Traditional recruitment processes have been failing hiring managers for years, adding undue stress, length, and unreliability to an already tedious process. If this experience sounds all too familiar, it’s time to make the switch to a recruitment method that takes a pragmatic approach to hiring.
Enter: job simulations.
Want to learn more? This is the complete guide to job simulations — covering what they are, how they work, when to use them, the different types, and the benefits.
What is a job simulation?
A job simulation is a pre-employment assessment that mimics the real responsibilities and duties a candidate is expected to perform in the role. Job simulations take applicants through a realistic day in the life of the position to assess how their hard and soft skills align with the job’s requirements.
An online job simulation aims to achieve one of two things:
- Allow organizations and hiring teams to understand if the candidate has the skills required to succeed in the role.
- Give candidates a realistic preview of the job to assess whether or not it aligns with their expectations.
Job simulations provide much greater insight into a candidate’s capabilities than a cover letter or CV can offer. In a job market where resumes are unreliable, job simulations level the playing field for all candidates. By improving role and candidate transparency, job simulations can improve quality of hire, reduce hiring costs, and lower employee turnover and costs associated with that.
Why job simulations work and traditional methods don’t
To understand why job simulations work, it’s important to understand exactly how traditional recruitment methods are failing organizations.
A 2017 CareerBuilder survey found that 74% of organizations admit to hiring the wrong person for a role. Additionally, two in three employees acknowledge that they’ve accepted job offers that weren’t the right fit, resulting in 50% quitting within six months.
The impact of not finding the right candidate for a role is a costly error for organizations.
A CareerBuilder survey found that on average:
- A bad hire costs almost $15,000
- Losing a good employee costs $30,000
Yet, it’s not just the costs associated with bad hires or losing good employees. It’s also the cost and inconvenience of leaving positions unfilled for an extended period of time.
According to LinkedIn, it takes companies 41 days to hire, on average. And believe it or not, properly optimizing your recruitment process can shorten this by 60%.
So, what exactly does this say about traditional recruitment methods?
- They’re inaccurate
- They’re costly
- They’re time-consuming
So, how do job simulations solve the problem?
Simply put, they do what traditional methods can’t: they see candidates perform elements of the role before they get the job.
And for organizations and candidates alike, that insight is invaluable. It allows organizations to see whose hard and soft skills best align with the role to avoid a bad hire while also allowing candidates to preview the position to see if it meets their expectations, lowering attrition.
While traditional hiring methods rely on college degrees and experience to make assumptions about someone’s ability, a work simulation gets candidates to essentially do the job (or at least small parts of it) before choosing who to hire.
What are the different types of job simulations?
There are many approaches to running a job simulation. Some roles will benefit from taking candidates through role-playing simulations and situational judgment scenarios. Others will get to the crux of their candidates’ skills by taking them through a work sample or in-basket simulation. Simply put, organizations will benefit from different simulation assessments based on the nature of the role they’re hiring for and what soft and hard skills are fundamental to the role’s success. Let’s take a closer look at the different types of job simulations.
1. Role-playing simulation
Role-playing simulations are one of the most widely used job simulations. This type of simulation activity is designed to understand a candidate’s ability to navigate interpersonal relationships in the workplace. Generally, role-playing simulations help assess an applicant’s soft skills in relation to the role, including leadership, negotiation, conflict resolution, initiative, and more.
Examples: A role-playing simulation in practice could include:
- Group interviews — group interviews look to assess collaboration, teamwork, confidence, and communication skills in a group setting. Employers will invite multiple candidates to the same interview and ask them scenario-based questions individually and in teams.
- Presentation — this involves getting candidates to present on a specific topic, whether on the spot or with time to prepare. This particular simulation method may be used to assess more technical knowledge and hard skills, such as a salesperson being asked to present a sales pitch or a marketer being asked to conduct a presentation on the buying journey and marketing principles.
- Mock crisis management scenarios — this particular simulation exercise aims to assess a candidate’s response to high-stress situations. This might include putting an applicant for a senior management position in a mock crisis scenario where they have to announce staff layoffs to employees and the wider team. This type of job simulation looks to assess how candidates operate under pressure, as well as their empathy, communication, and conflict resolution.
2. Work sample simulation
A work sample assessment is one of the most predictive forms of job simulations. This method aims to place candidates in a real-world situation by giving them tasks that mimic the role’s responsibilities and the types of tasks they might encounter daily.
Unlike some other types of job simulations, a work sample simulation is designed to replicate the experience of the role, only testing candidates on skills that directly correlate with the position. For example, a web developer’s ability to perform their role is not dependent on presentation skills or verbal communication; it’s based on their proficiency in coding, operating systems, and more.
Example: Asking a graphic designer to design a concept for a product launch, asking a copywriter to write a blog introduction for a fictional topic, or a customer service specialist being asked to respond to a mock disgruntled customer email.
3. In-basket simulation
In-basket simulations are similar to work samples in that they generally take candidates through a realistic task that correlates with their daily responsibilities. In-basket simulations look at how candidates approach the task, including their step-by-step process, their demeanour and approach, and how they arrive at a certain outcome.
For example, this type of assessment aims to determine the following:
- Problem-solving and decision-making skills — i.e., how a candidate interprets and evaluates information, how a candidate approaches tasks, etc.
- Planning and evaluation skills — i.e., organizational skills, time management, etc.
- Management and information organizational skills — i.e., communication
- Self-management skills — i.e., ability to show initiative, self-motivation, time management, etc.
Example: an in-basket simulation is most commonly used to assess administrative skills like organization, delegation, time management, communication, and more. This could include asking the candidate to handle a customer call within a specific time frame.
For instance, consider the following real-life in-basket job simulation from the US Department of Justice:
4. Situational judgment simulation
Situational judgment simulations, also known as situational judgment tests (SJT), are used to assess a candidate’s situational judgment and decision-making. Using fictitious scenarios related to the role, candidates are placed in a work-like situation and asked to respond by selecting from the multiple-choice answer options. Generally, situational judgment simulations will ask candidates to choose their most likely and least likely responses, which allows hiring managers to measure their behavior and attitude toward each question.
Example: Situational judgment simulations can be used across all roles, from executive to entry-level positions. They’re also commonly used to assess college graduates, where real-world experience is lacking. Consider the following example from ResearchGate for evaluating teachers:
5. Take-home assignments
As the name suggests, a take-home assignment requires a candidate to complete an assessment at their own location within a specific deadline.
Take-home assignments are particularly beneficial for roles performed remotely or those requiring minimal supervision, as it allows hiring managers to assess their ability to perform tasks in their own space. This simulation will assess self-motivation, ability to work with tight deadlines, quality of work, and attention to detail.
Example: this simulation may benefit someone applying for a remote call center role or a company wanting to hire a contractor/freelancer.
What type of job simulation is the most effective?
The most effective type of job simulation is one that allows you to adequately assess candidates’ skills in the context of your role while giving applicants an equally valuable understanding of the job so they can choose to proceed or self-select out.
What works for one organization may be less effective for another. For example, how you assess a brick layer’s skills will differ from how you evaluate a graphic designer’s skills. As such, it’s essential to understand the hard and soft skills required to succeed in your role and the nature of your organization to ensure your chosen job simulation gives you greater insight into the most suitable candidate.
While all types of job simulations may be effective in different contexts, research suggests there are a couple that are significantly more valuable than others. Researchers at the University of Minnesota and Indiana University conducted a meta-analysis review of the “best selection procedures,” a 1998 study by Schmidt and Hunter, who researched 85 years of work to understand what hiring methods were most predictive of performance.
The original meta-analysis and 2022 review found that job knowledge and work sample tests are considered the most versatile and predictive of performance. It’s important to note that many other job simulation types aren’t featured in the meta-analysis, potentially suggesting minimal data on their effectiveness or they hold little value in the selection process.
At Vervoe, we believe the most effective job simulations are those that make it easy for hiring managers to spot their top performers, while giving candidates a realistic job preview. Our job simulation platform offers ready-made assessments or completely customizable options that can be personalized to your business.
Unlike many other job simulation platforms, Vervoe offers multiple question types to help organizations assess candidates’ suitability more thoroughly. This includes multiple choice, audio, video, presentation, spreadsheet, coding, typing tests, and more.
The cherry on top? Our job simulations are entirely graded and ranked by our intuitive machine-learning AI. This means you will quickly gain accurate insights into your top performers without human bias interfering with the process.
When to use a job simulation in your hiring process
Job simulations are most commonly used during the initial screening or final stage interview. Running candidates through a job simulation largely depends on your recruitment resources and the software available to support pre-employment testing. Let’s take a closer look.
1. In the job post
It’s not uncommon for organizations to put relatively strict requirements in job posts when seeking the right candidates. While this helps weed out unsuitable candidates, unfortunately, such requirements can also act as a barrier to potentially suitable people from underestimated groups (i.e., those without college degrees).
As such, placing a job simulation assessment, like the ones Vervoe offers, in your job post is an effective way to give every job seeker a chance to showcase their skills and to ensure no great candidates slip through the cracks.
However, this option is only effective with the right software. As Vervoe is an AI-powered job simulation platform, we can assess a large scale of candidates and automatically grade and rank their responses, so you can quickly determine your top performers. Unfortunately, with traditional recruitment methods, assessing candidates at scale manually is unrealistic, particularly if resources are lacking.
2. Initial candidate screening
Many organizations opt for a job simulation assessment at the initial screening stage as it helps remove any unsuitable candidates from the funnel almost immediately. In turn, this allows hiring managers to focus only on those whose skills align with the core requirements of the role.
According to a Harvard Business Review article, a UK supermarket chain saw great results from implementing a customized online situational judgment simulation to screen out the lowest performing 25% of candidates before progressing to reviewing CVs. Due to the better-suited candidates making it further along the hiring process, the number of candidates interviewed for each successful hire reduced from six to two. By implementing a situational judgment simulation, they saved 73,000 hours of managerial time.
The bottom line is, positioning your job simulation early ultimately allows for a more efficient and cost-effective hiring process as you’re working with an optimized talent pool.
3. During final stage interviewing
Role-playing and in-basket job simulations are commonly used further along the hiring process and often involve incorporating existing team members. As such, reserving these job simulations for shortlisted candidates is best to ensure minimal disruption.
The benefit of incorporating a job simulation later in the process is that there is generally less work involved in creating simulations for potentially unsuitable candidates and having to grade more applicants.
It’s important to note that it’s not uncommon for organizations to utilize several job simulation assessments throughout the hiring process. This could be an initial work sample (or skills assessment) to help optimize the talent pool, followed by a group interview, where shortlisted candidates will partake in a role-playing simulation.
How to setup and run a job simulation
1. Decide what hard and soft skills are fundamental to the success of your role
Job simulations have little to no value if you haven’t done the work to uncover the role’s most essential skills. This includes hard skills such as technical abilities like coding, copywriting, graphic design, design principles, etc., as well as soft skills such as personality traits like empathy, organization, time management, creativity, and so forth.
To create an effective job simulation, it needs to closely connect to the skills you want your ideal candidate to possess. As a general rule, we recommend selecting a few core skills, including a mixture of soft and hard skills. From here, you can build your scenario-based questions around assessing the chosen skills. Not only will this help you create a skills profile for each candidate, but it will also make it easier to grade each applicant objectively.
2. Create scenarios that are closely related to the role
The point of a job simulation is to gain an early preview into how a candidate might perform in your role before signing the dotted line. Therefore, the most effective way to determine their suitability is to run them through realistic scenarios they’re likely to encounter in the day-to-day job.
Consider what their daily tasks will look like, the dynamic and culture of your team, and the potential challenges they’re likely to face in the role. Ensure your questions aim to test the specific skills in the context of the role.
3. Place candidates in hypothetical situations
Once you’ve created work-related scenarios that tie in closely to the skills you’re testing, phrase the questions hypothetically by asking the candidates how they’d respond or react in the scenario.
Depending on the hiring process phase, this may look like written answers, short-form bullet points, or role-playing scenarios.
For example, if you run candidates through a job simulation during the screening process for a Director of Marketing Skills Assessment, your question may look something like this:
While trying to execute a new campaign, you find your team caught up in a disagreement with two other departments (sales and customer support). Both believe the other should be responsible for manually activating prospects in the system. In the meantime, your new campaign is generating leads that appear to be going nowhere. How do you respond to this situation, and what steps would you take to ensure your team doesn’t feel discouraged?
However, suppose you’re running candidates through a job simulation for a Sales Executive role in the final interview stage. In that case, your question may be a video or face-to-face role-playing scenario like this:
In a 60-second video, pitch the company to a prospect. Demonstrate your ability to focus on the company’s strengths, avoid mentioning its weaknesses, and wrap up with an enticing call to action.
4. Use software to help automate the process
Running job simulations might be a more effective way to ensure quality of hire, but it doesn’t come without challenges — the number one being the time it takes to create and coordinate.
To facilitate a job simulation manually, you need to delve into the role to understand the most critical skills, write scenario-based questions that relate to the role, coordinate the simulation with candidates, and objectively grade candidates to find your top performers. It’s certainly doable, but if you’re working with minimal recruitment resources, it may not be viable to do on your own.
Using a job simulation platform like Vervoe is a more effective way to add accuracy, quality, fairness, and efficiency to your hiring process. Using our ready-made or customizable skills assessments, Vervoe runs candidates through job simulations and uses intuitive machine learning AI to grade and rank applicants to avoid the manual and time-consuming process.
Example: a job simulation in practice
By now, you’ve learned there isn’t a one size fits all approach to job simulations. From the type of simulation to the scenario-based questions asked, it will depend on the nature and style of role you’re hiring for.
Firstly, let’s take a look at some simple ways to turn standard interview questions into hypothetical scenarios:
|Role||Standard interview questions||Scenario-based job simulation|
|Senior DevOps Specialist||Tell us about your experience as a DevOps Specialist?||A non-technical stakeholder asks you what is meant by Microservices application architecture and why we need it. Please record an audio of yourself explaining this to the stakeholder and outline 2-3 benefits.|
|Salesperson||Tell us about a time when you had to sell a product you knew little about?||Please give us a five minute sales pitch selling our product to us.|
|Marketing Analyst||Explain a time where two key stakeholders had opposing views on a campaign. What did you do?||You have run A/B testing for a campaign and there is no significant difference in the results. The CEO likes one campaign but the COO prefers the other. Run us through the conversation you’d have, and how you’d get to a decision.|
|Data Analyst||Tell us an experience you’ve had in the past with analyzing data. What was your process?||Please take the next 15 minutes to analyze the following data. Once complete, please talk us through your findings and observations.|
Now, let’s take a closer look at how you might incorporate multiple questions using different formats for the one job simulation. The following examples come from Vervoe’s job simulation platform:
|Role||Question one||Question two|
Skills tested: Excel, financial accounting, and problem solving
|Question Type: Text|
On 30 June 2021, an item of machinery had a written down value of $480,000. The machinery was purchased on 30 June 2020 at a cost of $600,000 at which time its estimated useful life was 5 years with no residual value. What is its written down value at 30 June 2023?
|Question Type: Spreadsheet|
In the attached excel, how would you decrease the balance of each account? Debit or Credit?
Skills tested: Attention to detail, communication, and initiative
|Question Type: Multiple Choice|
Refer to the following information: Acquisition Global’s account number is 11456789 – their status is active and authorisation code is TNAv8xUOr. For Landec Limited, their account number is 12178957, they are inactive and their authorization code is z2aRt8dlZ. Then there is Helios Incorporated who are active with an authorization code of Ws6R2tRRC. Which of the following is Acquisition Global’s account number?
|Question Type: Video|
It’s your first day at your new job. At the Monday morning meeting, you are asked to introduce yourself to the rest of the team, including the Director who is sitting in on the meeting. Please record and upload a video of yourself doing this.
|Customer Service Assistant |
Skills tested: Active listening, adaptability, and communication
|Question Type: Audio|
You are on the phone with a customer who is very irate and angry. What tactics could you use to calm the upset customer?
|Question Type: Text|
Have you ever had to say “no” to a customer request? How did you handle the situation in order to preserve the relationship?
|Cyber Security Specialist|
Skills tested: problem solving, risk management, cybersecurity
|Question Type: Video|
Tele2U’ expects that the web-platform could have over 1 million subscribers in the first 3 months alone. They are keen to ensure that authentication and access to the platform is as secure as possible, so that accounts cannot be compromised. They are also mindful that they will be capturing personal information that must be protected, as well as user passwords. Briefly explain two strategies that ‘Tele2U’ can implement to strengthen user authentication.
|Question Type: Text|
Tele2U’ have recently hired a Cyber Security professional to help provide guidance and recommendations to strengthen the overall security landscape. The Cyber Security professional maintains that the web-portal should undergo a vulnerability and/or penetration test prior to ‘go-live.’ The web developers believe that the test is unnecessary given the cost and the extensive testing that they have already completed. Describe the importance of penetration testing and why it should be considered by ‘Tele2U,’ both prior to go-live and periodically thereafter?
The benefits of job simulations
1. They add confidence to the hiring process
When you hire a candidate purely based on their resume and how well they interview, you cannot validate their skills or claims. And the truth is, this can be a risky move, given the likelihood of candidates embellishing the truth on their resumes.
According to a 2022 ResumeLab study, 36% of Americans admit to lying on their resumes. The study found that:
- 27% lie about experience
- 18% lie about their skills
- 17% lie about job duties
On the contrary, implementing job simulations as part of your hiring process allows you to see who can actually perform the job before hiring them. By placing candidates in scenarios they’re likely to encounter in the role, you’ll feel more confident that the chosen candidate is the most suitable for the role.
Likewise, opting for a job simulation platform like Vervoe adds an additional layer of reassurance. Using our in-depth skills assessments, our AI will take care of the heavy lifting, automatically grading and ranking candidates based on your preferences. Consider the following testimonials from a couple of our Vervoe customers:
2. They give candidates a realistic preview of the job
In today’s candidate short market, prioritizing employer brand and the candidate experience is fundamental to a company’s long-term success.
Job simulations are just as much for the candidate as for the employer. They allow applicants to understand the day-to-day duties and expectations of the role and gain greater insight into the company culture and team. With this knowledge, candidates can decide to proceed or self-select out if it doesn’t align with their values or expectations.
This also positively impacts a company’s attrition rate, as candidates are more likely to feel satisfied with the role and, thus, are less likely to leave due to job shock.
3. They improve diversity and inclusion
Many traditional recruitment methods still use college degrees as a proxy, relying on assumed suitability over verified suitability.
Why does this matter? Over 60% of Americans do not have a degree, yet 75% of roles posted in the past 10 years in the United States require a degree.
Let’s take, for example, the following statistics on college graduates by race or ethnicity in the United States in 2022:
- 59.1% of college graduates are white or caucasian
- 13.1% of college graduates are Hispanic or Latino
- 1% of college graduates are Black or African American
- 0.6% of college graduates are American Indian/Alaska Native.
Running candidates through a job simulation lifts the pressure for organizations to rely on college degrees to determine whether or not a candidate is a good match. While a degree may give hiring managers confidence in a candidate’s interest and exposure to a particular area or field of study, it isn’t an accurate indicator of job performance or a candidate’s suitability for a role. It’s merely based on assumption.
Using job simulations opens your organization to candidates who may have obtained their skills through non-traditional methods, such as on-the-job training or self-teaching. This gives employers access to a larger talent pool — including candidates from underrepresented communities — and adds a layer of confidence that the chosen candidate is the best possible person for the role.
But why does diversity matter in the workplace? In addition to improving inclusion, innovation, and productivity, a diverse workplace is also proven to benefit a company’s bottom line. According to McKinsey, organizations that fall into the top 25% for cultural and ethnic diversity were 36% more profitable than the 25% lowest-scoring companies.
4. They reduce bias
Traditional hiring practices — manually screening candidates’ CVs, interviewing, etc. — are riddled with unconscious biases.
Let’s take a closer look at a few key statistics that highlight the problem:
- Researchers from a 2011 Canadian study submitted 13,000 random resumes to open online job postings. The study concluded that resumes with English-sounding names were 39% more likely to receive a callback.
- Another similar study that sent out 5000 resumes concluded similar findings. Researchers discovered that applicants with “white” names received a callback for every ten resumes sent out. Yet, African-American-sounding names would receive a callback for every 15 resumes sent.
- A 2016 study tested a range of organizations using pro-diversity language in their job postings to assess whether their recruitment process was inclusive. Unfortunately, the study concluded that it did not result in reduced discrimination.
And guess what? It’s not just ethnic discrimination we’re witnessing in existing recruitment processes; it’s gender discrimination, too. Consider the following research findings to understand the reality of gender bias in the screening process:
- A Yale University study asked science faculty members to score and select a fictional job application of a prospective lab manager, which were randomly assigned feminine or masculine names. Although the applications were identical, the male and female science faculty members believed the male candidate was more competent than the female. The evaluators also suggested higher starting salaries for the male candidate.
- 238 US academic psychologists were asked to evaluate one of two identical resumes, one belonging to a female and the other to a male. Despite evaluators admitting to preferring the female candidate over the male, most put forth the male applicant for the role. Additionally, evaluators were four times more likely to question the female’s application, wanting further validation of her accomplishments.
So, how do job simulations help reduce bias?
While job simulations can help reduce bias by ensuring candidates are assessed on their proven capabilities rather than assumed suitability on a resume, the most powerful way to minimize human bias through job simulations is to use AI (like Vervoe’s).
Vervoe uses machine learning to screen candidates in, not out. It’s not uncommon for HR teams receive hundreds of resumes for a single open position. It’s nearly impossible to consider every candidate’s application carefully, which leads many recruiters to select candidates whose backgrounds seem most familiar or similar to theirs.
Vervoe’s AI brings greater efficiency to the process by matching candidates with the employer’s preferences. The algorithm allows HR teams to evaluate 10,000 candidates in the same time it takes for an individual recruiter to assess one.
5. They reduce the risk of a bad hire
It’s no secret that finding the right employee can be time-consuming and financially burdensome to an organization. But, believe it or not, hiring the wrong employee is far more costly.
Consider the following statistics to help illustrate the cost of a bad hire:
- The US Department of Labor suggests that a bad hire costs up to 30% of an employee’s income in the first year.
- According to a Robert Half study, 34% of CFOs admitted that bad hires cost them lost productivity too. The same study found that managers had to spend 17% of their time supporting underperforming employees — in a typical work week, that’s almost an entire day.
- 39% of CFOs agree that bad hires negatively impact team morale. And when team morale is low, so too are productivity and creativity, which can directly impact a company’s bottom-line success.
Job simulations reduce the risk of bad hires by placing candidates in situations that test their real-life hard and soft skills in the context of the role. Rather than assuming suitability or purely relying on a candidate’s resume, a job simulation will show you who is the most suitable performer to succeed in the role. They also show candidates what to expect in the role — minimizing the risk of job shock.
Filling a position is a costly, time-consuming, and often stressful process when using traditional hiring practices.
Unfortunately, relying on archaic methods to predict candidate performance can have a negative impact on a company’s success — doing little to improve diversity, being riddled with unconscious bias, and giving candidates and organizations alike little reassurance that they’re a sure fit.
The solution? Job simulations. This versatile recruitment method aims to give candidates a realistic job preview while showing organizations who their top performers are by validating their skills. In turn, job simulations help reduce the risk and costs associated with bad hires, and help to improve diversity and inclusion.
Yet, job simulations alone don’t have the power to solve all your recruitment problems. This type of hiring method is most effective when powered by machine-learning AI to help automatically grade and rank candidates efficiently and accurately.
Ready to hire with confidence? Book a demo now and let our experienced team run you through Vervoe’s full range of ready-made and tailored solutions.