Dec 13, 2022

Do stand-ins need to have the same skills as the appointed? 

Recently, we received several domestic and international inquiries about the value of using online simulations in the hiring selection process. We will briefly address the topic below.  

Briefly about the beginnings

In the ‘70s, simulation exercises for hiring were being used and examined. So using simulation techniques for selecting who to hire didn’t begin just a few years ago. In 1975 a thesis written at the University of South Florida analyzed the phenomena of organization in relation to applying the classic selection and evaluation practice. It’s worth looking at, if only to read a text in retro typewriter style. Although at that time candidates had also to be evaluated/selected according to other organizational paradigms, there was already familiarity with the more important questions related to the simulation of realistic workplace situations. As author W. Byham states in reference to a 1970 Harvard Business Review article: 

• Simulation exercises are designed to test specific skills relevant to the position, 
• The exercises are standardized, therefore measuring the candidates is fair and comparable, 
• During the simulation, the personal relationships between the stakeholders don’t play a meaningful role, 
• During testing, there isn’t any need to intervene in the functioning of real workplace collectives,
• During testing, it is possible to examine a skill deemed important in detail, 
• Experts using simulation exercises should be prepared to be observed and evaluated. 

So, we are writing about the ‘70s, still more than ten years before Lee Iacocca’s 1984 book “A Managerr’s Life” was published. And, during that period in the ‘70s, hardly anyone could have been prepared for the organizational paradigm shift that Ian Carlzon described in “Flatten In 1985” and which he began to talk about in his bestseller: “Flatten the Pyramid”. That was when online simulations began to offer something new. 

The benefits of modern digital tools

By 2020 online simulation practice has far more possibilities than can be had by “live” simulations. A serious management simulation is a development and analysis tool that allows for controlled and recorded experiments. At the pinnacle of online simulations are the interactive movies in which the user (or, as we often say – the “player”) is presented with complex challenges. In overcoming the “compulsion to conform”, their decisions about people and situations reflect their own real behavior patterns, strategies, and resources. 

Here are the advantages such a competency-based interactive film has in supporting the selection process: 

• Qualification of behavior – This is typically about more than ensuring the ability to measure performance. Recognizing the power of narrative and building on the alliance between psychology and digital game development (serious gaming) business actually profiles real behavior. Predictability (predictive analytics) shows quite well what we can expect from candidates in a given situation. With real-time measurement based on benchmarking, the results can be relativized, not only by industry, but also by leadership position (“leadership pipeline”).
• Realism – The organizational dilemmas processed in the simulation provide excellent opportunities to test the candidates’ learning practices. How do they approach new challenges or what do they do in unexpected situations? Moreover, these observations provide “unbiased” information, i.e. the frame stories (narrative contexts) ensure the candidates identify with the tasks, and this significantly reduces their pressure to comply. In other words, they spend far less energy figuring out what the test scorers expect from them. In this respect, simulations are significantly different from other skills assessment techniques, e.g. from questionnaires. 
• Actuality – A well-constructed simulation story has the capacity to display the present reality. It operates similarly to airplane simulators that in actuality accurately model the real environment. If the simulation deals with the challenges of the VUCA (Volatile, Uncertain, Complex, Ambiguous) world, it provides an excellent field for observing the candidate in a real work environment. The unpredictability and complexity of the corporate world is very different from the 2000s. Today, direct employees of a company are not the only ones who must be managed. A person  whose supervisor is not a line manager can have a prominent role in the achievement of goals: freelancers with their own will or powerful stakeholders in the organization’s external environment. Simulations help us directly test new skills and strategies for managing these complicated networks. 

Performance in the simulation is about achievement – not appointment. It’s the preliminary assessment of whether the candidate actually has the ability to perform. 

• Motivation – Proper management of the selection process is a critical service for clients and candidates alike: the impressions the applicants get about the organization advertising the job are not relevant. The gamified approach offers the experience of the flow of solving complex situations. This “engagement” releases a significant motivational force that effectively counteracts the stress of the selection process, regardless of the final result. 
• Value Consistency Testing – More complex simulations are usually not value-neutral. During feedback, the question always arises: on what basis do we say that the player did ‘this well’ and that they did ‘not so well’? The feedback about the leadership roles are typically value judgments that are good when aligned with society’s moral expectations for leadership. When creating FLIGBY, we integrated Professor Csíkszentmihály’s “good business” ethics with his involvement, tuning the feedback system in such a way that it linked positive signals with the sense of responsibility towards the organization’s wider environment. It’s worth being aware of this, since the feedback is mainly created through value-based filters. This is also an excellent opportunity to begin meaningful conversation about which management principles the candidate prefers.

Application Considerations

Let’s try on how a simulation works during the selection process. Based on our experience, it’s worth paying attention to the following: 

• Choosing the right target group – We must realize that the serious game is not for everyone. With driving simulations, it is an obvious value to focus on leadership positions. This is OK so far, but who do we call the ‘leaders’ of this day? As we mentioned above, significant changes are taking place and all jobs may be worth supporting with simulation selection as the total workforce requires cooperation, responsibility for performance, conflict management, and innovative thinking. These competencies are increasingly less the privileges of managerial work, and increasingly more the characteristics of project-based operations. 
• Preparation of instructors (trained trainers) – The complex management simulations, so-called “instructor-led” programs, that is, actions implemented with the’ assistance of trained trainers. The instructors’ tasks include preparation for the simulation, negotiation of goals and, of course, feedback on the simulation results. Game-based profiling is beginning to develop into an independent profession, with its own paradigm system and jargon. Interpretation of serious online reports must be learned, just as giving valuable and impactful feedback is not a simple or routine task.

“With the management simulation, we do not measure how strong certain skills and competencies are in absolute terms. We quantify the extent to which the player used them during the simulation. Of course, there is a strong correlation between the two: if we are not good at something, we usually don’t use it very much.”

• Ensuring the right conditions – Although the “bite-sized learning” trend increasingly forces serious game developers to avoid simulation situation exercises that require a significant investment of time, the solution of management simulations requires quality dedicated time. The most frequently asked question in this regard is: where does the candidate solve the simulation? Is it best under controlled conditions and under active consultant supervision, or rather from home (even providing several days to process the simulation story)? We argue in favor of the latter. According to our experience, the quality of the measurement is in fact helped by the freedom of time management, and by appropriate tuning. By the way, the personal conversation initiated on the basis of simulation experiences clearly dispels doubts about whether the candidate received unethical support to solve the exercise. For this, trained trainers must familiarize themselves with the dilemmas processed in the simulation story and be prepared for the occasional emotional reactions of the users. For certain: the simulation measurement doesn’t work like: “I sent you the link. Let’s do it.” Proper tuning and meaningful feedback must be ensured. In addition to ensuring the success of the project, this is also an important element of trainer ethics. 
• Evaluation of individual results – The available benchmarks help a lot in easily spotting unusual user patterns. The question is, of course, what is beyond standardization. Can the individual behavior patterns of the candidate be taken into account during the evaluation, or are the mean values ​​of the normal distribution more optimal? First of all, well-functioning simulations cannot be solved in only one way. Interactive films are typically based on decision tree models in which the set goal can be achieved, albeit with different consequences (e.g. winning a wine award). Secondly, the complex simulations are prepared to handle the quantification of the intensity of the skills used during decision situations and the progress in the different branches of the story separately from each other. This high-level profiling ensures “unbiased” measurement, i.e., that after a while the candidate surrenders to the greatness of the intellectual challenge (flow!) and is not really driven by meeting the perceived or real expectations of the selection process.
• Choosing the simulation – The serious gaming industry is a rapidly developing business, so the number of available solutions is constantly increasing. How to choose the right simulation? We have already gone through the issue thoroughly, in a detailed post we prepared. If one aspect has to be highlighted here, we must mention the degree of support for the “blended learning” approach. In other words, it’s worth choosing a simulation for which the developer also provides methodological support beyond the boundaries of the online world. The experiences gained in the virtual space live on in the real world, and provide excellent opportunities to expand the selection process and strengthen employer branding.

By 2020 (of course), online simulation practice has far more possibilities than can be had by “live” simulations. A serious management simulation is a development and analysis tool that allows for controlled and recorded experiments. At the pinnacle of online simulations are the interactive movies in which the user (or, as we often say – the “player”) is presented with complex challenges. In overcoming the “compulsion to conform”, their decisions about people and situations reflect their own real behavior patterns, strategies, and resources. 

Conclusion

Simulation management training has transformed since the 1970s, and online simulation practice possibilities have almost no limits. The ability to control and record the experiment has advantages over live training platforms, such as not allowing the participant to game-ify the questions because of their natural “compulsion to conform”. competency-based interactive films support the selection process through realism, qualification of behavior, and actuality.