• Workers who override AI decisions with better human judgment are often penalised, says new research
• AI systems are being widely adopted, but workers shouldn’t blindly follow their recommendations
• The findings are particularly critical for industries like healthcare, where decision-making is vital

Put yourselves in the shoes of a doctor. You’re speaking with a patient who has shortness of breath, chest pain, and discomfort – your assigned AI assistant decides that the symptoms you’ve entered into its system indicate the patient is likely suffering with indigestion, and suggests sending them home for rest and medication.

But in your experience, this seems a little more severe than indigestion. You use this experience to override the algorithm and make the call to send the patient to the hospital for emergency care. And it’s a good job you did – the patient was having a heart attack, and your knowledge and intuition saved their life – you’re a hero!

At least you thought you were.

Instead of being rewarded by your bosses for your heroic acts, you’re told off for overriding the very expensive AI system the hospital has invested in!

Though an extreme example, it’s very much a realistic one, according to new research by Frankfurt School of Finance and Management and ESMT Berlin.

In fact, in a new study, researchers from the German institutions found that workers are often penalised for disregarding AI recommendations-even when their own decisions led to better results.

The study, carried out Prof. Dr. Mirko Kremer, Professor of Supply Chain Management at the Frankfurt School, Francis De Véricourt, Academic Director of the Institute for Deep Tech Innovation at ESMT Berlin, and Hossein Nikpayam, Postdoc in Operations Management, highlights how an over-reliance on AI can undermine the value of human expertise in workplaces across various industries, often to the detriment of the organisation and those within it.

The Experiment: AI vs. Human Decision-Making

To understand this issue, the researchers set up an experiment where employees were given recommendations by an AI system. The workers could choose to follow the AI’s advice or make their own decision based on their knowledge and experience. Afterwards, managers evaluated the outcomes and adjusted the workers’ bonuses based on the success of their decisions.

The results were surprising: even when workers ignored the AI’s advice and made better decisions, their bonuses were reduced. This suggests that organisations are becoming overly dependent on AI recommendations, and human judgment is being undervalued-despite the fact that workers sometimes perform better without following the AI’s guidance.

The Pressure to Follow AI

The research highlights the pressure that employees feel to follow AI suggestions, regardless on their own instincts. This growing reliance on algorithms, partly driven by fear of financial penalties, means that workers are less likely to trust their own instincts and more likely to follow the AI’s advice, even when they know their own decisions might lead to better results.

Prof. Dr. Kremer, who led the study, warns that over-relying on AI can be risky. “While businesses are eager to use AI to improve decision-making and efficiency, it’s important to remember that these systems shouldn’t be followed blindly. Human expertise is still crucial, especially in areas like healthcare and aviation, where AI errors could have serious consequences,” he says.

Why Human Expertise Still Matters

Whilst AI may be great at processing data and making recommendations based on patterns, it lacks the qualities that are often necessary for good decision-making. Human judgment, intuition, and experience cannot be replaced by algorithms. The study highlights this by evidencing that when organisations fail to value human expertise, they risk making poor decisions that could have been avoided by trusting employees’ knowledge.

Rather than seeing AI as a replacement for human decision-making, the study suggests it should be used as a tool to support and enhance it. AI can be powerful when used correctly, but organisations must remember that employees bring invaluable insights and experience that machines cannot replicate. In short, human and artificial intelligence must be leveraged in partnership, affording organisations to benefit from both.

Transparency: A Key to Building Trust

Another key takeaways from the study is the importance of transparency in AI systems. For AI to be truly effective, managers need to understand how the system makes decisions in order to assess its recommendations critically. They must also be transparent in sharing these insights with their teams. Without this level of transparency, employees may feel confused or frustrated, not knowing why AI is making certain suggestions or how to challenge them.

The researchers also stress the importance of restoring trust in human judgment. Organisations should encourage employees to trust their expertise and feel confident in questioning AI when necessary. By creating a culture where employees can voice their concerns about AI recommendations, companies can foster better decision-making and improve overall workplace dynamics.

Creating a Healthy Work Environment

The study also recommends that organisations create a supportive environment where employees are encouraged to think critically. Training sessions, regular feedback, and open communication are key to helping workers understand the capabilities and limitations of AI. This will ensure that they can work effectively with AI while still valuing their own judgment.

Clear communication is essential for ensuring that workers feel comfortable discussing their experiences with AI systems. Organisations should set up channels for employees to share their thoughts and concerns, which could include feedback sessions or regular check-ins with managers. This approach not only helps build trust but also ensures that AI is used in a way that complements human expertise.

The Right Balance Between AI and Human Decision-Making

Ultimately, the study calls for a balanced approach to AI in decision-making. AI should not replace human decision-making but should be used as a tool to support it. By encouraging employees to trust their instincts and work collaboratively with AI, organisations can create a more effective decision-making process.

As AI continues to play a larger role in the workplace, companies must recognise the importance of human input and strive to strike the right balance between technology and human judgment. By doing so, businesses can create a healthier work environment, improve decision-making, and avoid the pitfalls of relying too heavily on AI.

By, Peter Remon

Interested in this topic? You might also like this…