The EU AI Act calls recruitment
a High Risk Activity!

If you do recruitment or have applied for a job sometime in the last 12 months, you may have seen a short text in the email you got back: ”AI is used as part of this recruitment process.” (or something similar)

Do you know why that text is there?
Do you know what it means?

You may think that the EU AI Act will provide some protection for candidates when the employer is using AI systems to select candidates and then also to decide who gets the job. To some extent it does – or tries to – but the actual effects and the risks are well worth discussing, because there is sooo much more behind this, and it’s not all pretty.

Stay with me, I will try to make this as clear and readable as possible. Here we go:

Risk levels

The AI Act defines 4 levels of risk for AI systems:

• Unacceptable risk
• High risk
• Transparency risk (somtimes called ’Limited risk’)
• Minimal or no risk

(https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai)

When I see that triangle, it kind of reminds me of this other one:

Is recruitment really mentioned as a level 3 thing?

Yep. It is.

”AI systems intended to be used for the recruitment or selection of natural persons, in particular to place targeted job advertisements, to analyse and filter job applications, and to evaluate candidates”

(https://artificialintelligenceact.eu/annex/3/)

Since recruitment is specifically mentioned as a high risk area, this means that some extra requirements are put in place. For high-risk AI use in general, there is this requirement:

…”, no action or decision is taken by the deployer on the basis of the identification resulting from the system unless that identification has been separately verified and confirmed by at least two natural persons with the necessary competence, training and authority.”

(https://artificialintelligenceact.eu/article/14/ ) (article 14(5))

Ok, so having things verified by two humans sounds reassuring, right?

…but here is what you should know

Because recruitment is never mentioned in 14.5, the statement that applies to recruitment is 14.1, which is this:

”High-risk AI systems shall be designed and developed in such a way […] that they can be effectively overseen by natural persons during the period in which they are in use.”

I would argue that ”effectively overseen by natural persons” is a vague statement. People have noted that vagueness:

• Tandfonline.com (a large British commercial academic publisher) has an article called ”Human oversight’ in the EU artificial intelligence act: what, when and by whom?”. 
  The original publication can also be found at Researchgate.net as a PDF.
  Author: Lena Enqvist, Department of Law, Umeå University )
  It’s a good read. Among other things, the article brings up this:
  
  ”AIA’s unclarities and gaps […]” and ”[…] the implications of vesting too much trust in providers to secure the oversight infrastructure of high-risk AI systems.”
  
  Thank you, Lena.
  
  
  
• Holistic AI comments on the EU AI Act and calls out the vague wording:
  ”Strikingly, there is no clear guidance about the standard of meaningful human oversight under EU policy.”
  
  (Holistic AI is a commercial UK-based AI governance and compliance company. Their mission is to empower organizations to adopt and scale AI with confidence by ensuring it is trustworthy and responsible.)

So what’s the problem?

What on earth does ”effectively overseen” mean? I see several risks in this.

Issue #1: ”Enabled to do” does not equal ”will do”

The EU AI Act does not explicitly what that means, but article 14.4 states what the oversight person must be enabled to do.
(Note: it says enabled to do, not do.)

My comment: That in itself leaves room for people to not do things, as long as they could have done them.

Issue #2: Will users be able to monitor and detect anomalies?

”(a) to properly understand the relevant capacities and limitations of the high-risk AI system and be able to duly monitor its operation, including in view of detecting and addressing anomalies, dysfunctions and unexpected performance”

My comment: Do we think people will be able to do this?

Issue #3: We are only humans and nobody is perfect all the time

”(b) to remain aware of the possible tendency of automatically relying or over-relying on the output produced by a high-risk AI system (automation bias) […];”

My comment: How many of the people making recruitment decisions can do this?

”(c) to correctly interpret the high-risk AI system’s output […];”

My comment: I’m not sure normal people will be able to do this continously during daily use of the AI system.

Issue #4: ”Just skip the system and think for yourself”

”(d) to decide, in any particular situation, not to use the high-risk AI system or to otherwise disregard, override or reverse the output […]”

My comment: I don’t think people will be able to – or even know how to. 
People will follow the path of least resistance, which is to believe the system.

Issue #5: This one is almost funny – except is isn’t

”(e) to intervene in the operation of the high-risk AI system or interrupt the system through a ’stop’ button or a similar procedure.”

My comment: Yeah, right.

Conclusion

It is dangerous to think that you are only using the AI system as a tool and that you yourself always make the real decisions. Even without AI, you are already affected by bias of all kinds.

With AI systems doing screening and selection, there may be even more of that but it may be hidden. And you are still biased before you even start the system.
It is crucial that we all know this.

It’s real people we are dealing with.

This article was written with support from (not by) an AI, and thoroughly fact-checked by a human (me 😉 ). All quotes can be found in the links provided.
…but of course I may still have made mistakes.

Take care, everyone!

Björn