AI is increasingly being used to make business decisions. Leaders must now adjust, working hard to understand AI-mediated risk.
Who would you trust more if it came to making a decision regarding your life, a human or a computer? Most people would turn to the computer due to a phenomenon known as automation bias. As crucial decisions are increasingly being made by autonomous systems, leaders bear the brunt should these go wrong. Thus, automated decision risk and its understanding are essential for anyone integrating AI into their decision-making process.
Business Leaders and AI
Much talk is given to how AI will impact workers and staff. Yet it also presents a huge challenge for those leading a company. With AI doing the bulk of the work, what is left to lead? These systems can create emails and streamline recruiting, yet the bulk of a leader's job, such as making tough decisions, fostering workplace accountability, and making the human side of a business work, has largely been untouched. Or they were until now.
AI is beginning to play a part in crucial decision-making processes, often known as 'black box' decisions. This is because the systems by which AI reaches a conclusion are often a mystery to those on the outside. People see only what goes in and what comes out, but not the inner workings. In business terms, imagine it from a hiring perspective. Leaders may see the candidates and then see which ones the AI has chosen. Between that, there is no information on why this decision was reached.
For business leaders who use AI to make a decision, when this goes wrong, it can be hard to justify. It leads to a case of "Because the AI said so" blame and little else. This can make a leader look lazy and incompetent, damaging their personal reputation. Oddly, around 6% of people in an organization believe no support or oversight is needed for AI according to a study by McKinsey.
Automation Bias
When a leader makes any decision, it comes with the potential for risk. This could be an undesirable outcome, which is more likely to occur if the decision has not been planned, thought through, or researched.
Compounding this is automation bias. This is the human tendency to give preference to automated artificial systems, even going so far as to ignore decisions made in other ways, even if they are correct. Prevalent in areas where systems are used to make a lot of decisions or have a say in them, it can impact everything from financial forecasting to medicine. A study by Deloitte found that computer vision targeting systems used on battlefields were incorrect 5% of the time, leading to 1% incorrect casualty rates. Errors like this move from ethical dilemmas to ones that have legal impacts.
Expertise in AI Governance
Matthew Bertram is an expert on AI governance, and became a practitioner in 2024. He believes that it is imperative to authorise before, as opposed to detect afterwards. This means that decision boundaries must be approved up front, not built from logs after something has gone wrong.
His second point is to validate the premise before building. This ensures a clear problem statement is centred, so that AI models don't fail at the question point. His three main points are as follows:
Authorize before. Detect after.
Validate the premise before you build.
Compounding outranks brilliance
AI Decision Risk
Because of this, there is a high level of AI decision risk when it comes to these systems. This refers to the process by which financial loss, or harm to a brand, can occur through decisions made by or influenced by AI. AI is logical and only makes decisions based on the information it is fed. Thus, it can blur the lines and cause more trouble than people may expect.
Examples of how bad decisions made using AI can go are on the increase. In the United Kingdom, the West Midlands police made a grave error, banning Maccabi Tel Aviv football fans from visiting for a game against West Ham. They later had to come clean, admitting that this was due to incorrect information that arose due to the use of Microsoft Co-Pilot. This led to the issuance of a "profound apology," though the damage to their reputation was already done.
Responsible AI Leadership
The big question is what responsible AI leadership actually looks like. This can differ for each business, depending on how integrated AI is in the decision-making process. Over time, AI should compound value, providing a balanced overview of the pros and cons of making a decision.
Throughout this, transparency should be paramount. Companies should not hide AI-mediated decisions, but instead discuss where and when it has been used. In areas where risk is extremely minimal, then fast tracks can be used, but they should never be fully autonomous and always checked by a human side of your business.
If you are confused about what this looks like, then speak to an expert. It is better to get this right now than try to get the genie back in the bottle later and suffer damage to your reputation and margins.
