Would you rather have that 1% of people who were not interested enough to take a 1 hour assessment?

Posted by  Steven Jarrett, Ph.D.

How would you like an assessment that can measure all of the aspects that are important to job performance for a variety of positions within your organization? And, how would you like it if that assessment only took 5 minutes to complete? If I heard this sales pitch, the next thing I would expect to hear is, “If you do not like this I also have some magic beans in the back and I will trade them for only one cow.”

In a world where information is at our fingertips, we can transfer money from a plane and connect with six friends from six continents in an instant, companies want faster ways to process and onboard employees. This is especially the case for entry-level employees where HR staffs fear that if things take too long they will lose out on quality candidates. In some cases this is very true, instant feedback is important, and scheduling for the next stage in the process should be done real-time, if possible. However, when it comes to trying to determine if an individual has the right knowledge, skills, and abilities to perform well in the role, saving a few extra minutes will hurt significantly in the long run. A wise man once told me, “You need to go slow, to go fast.” Using a longer test to make the correct hiring decision will certainly be a better solution than using a shorter test to potentially make a poor decision.

We are sometimes asked why we ask similar types of questions multiple times in an assessment; can’t we just shorten the assessment by eliminating questions that ask about similar things? Well, there is good reason for doing so, and it’s not because we’re trying to trick candidates into answering differently.  Instead, years of research in the field shows that measuring the same construct with several questions and methods improves the reliability of the measure and can improve validity. Thus, asking those questions several times in different ways ensures that the person is as conscientious or outgoing as they say they are.  This is critical from a measurement standpoint because the more confident we can be that the tool is accurately measuring the trait of interest, the more confident you can be that you are making the correct hiring decision. A good example of this is a measure of personality that began with 100 items to measure 5 traits and had reliability coefficients ranging between .8 - .9 (this is considered fairly high reliability, as reliability ranges on a scale between 0.0 and 1.0). Then, researchers thought that measure was too long, so they scientifically cut it down to 50 items and the reliability fell to between .6 -.7.  Then, that was still too long so they created a 10 item measure and saw the reliability plummet to around .2. So they went from roughly the reliability of a pregnancy test to the reliability of two 3-year olds telling you the plot of the movie they just watched. The lower the reliability, the less confident you can be that the results are reflecting the individuals standing on a given competency.

So, why do companies want shorter assessments? A common fear that we hear is that individuals will not complete the whole assessment and they will opt out of the process. Interestingly, our data shows that whether the assessment is 20 minutes or 2 hours, once the individual begins the assessment they are likely to finish it (roughly 99% of candidates finish). Thus, I guess the real question is, would you rather have sufficient measurement to know you are making an appropriate hiring decision or would you rather have that 1% of people who were not interested enough to take a 1 hour assessment?