A.I. and Risk

A.I. Is Changing What Is Possible. How Do We Decide What We Are Prepared to Do With It?

Artificial Intelligence is changing the way we search, communicate, create, analyse information and make decisions. What began for many of us as a tool for asking questions or generating text is rapidly becoming part of everyday life, work and business. A.I. can help us do things faster, discover possibilities we may not have considered, and give individuals and smaller businesses access to capabilities that once required far greater resources.

While some individuals and companies have embraced A.I. and are going full speed ahead, many local SMEs have yet to tap into its wider potential. At the time of writing, government initiatives continue to encourage businesses to adopt A.I. beyond using it simply as a text generator. The opportunity is there, but so is the question of how far, how fast and for what purpose we should use it?

With the possibilities of A.I. come new questions. Should I trust A.I.? Can A.I. make better decisions than humans? What could go wrong when my company starts using A.I.? Can employees put company information into ChatGPT? Who is responsible when A.I. makes a mistake? How much control should we give A.I.?

And increasingly, there is an even bigger question: what happens when A.I. moves from answering us to acting for us?

Isn't it ironic that the very thing we want A.I. to do for us could potentially become something we also need to manage? Perhaps it will. Perhaps it won't. That uncertainty itself is what makes this such a thrilling area of Risk Management, do you agree? Haha.

Where Does A.I. Fit Into Risk Management?

When we started updating this website, one question was whether A.I. should simply sit under Technology Risk. After all, A.I. is technology.

But A.I. now touches so many parts of our lives and organisations that we believe it deserves a major section of its own.

At the same time, A.I. does not make the established Pillars of Risk Management obsolete. In fact, those pillars help us make sense of many A.I. questions.

If an organisation relies on A.I., we may be looking at Technology Risk. If A.I. accesses information or systems, Cyber Risk becomes relevant. If the A.I. comes from an external provider, there may be Third-Party Risk. If its failure could interrupt an important operation, we may need to consider Operational Resilience and Business Continuity. Questions about responsibility and acceptable use can bring us into Governance & Compliance. And when we ask what could go wrong, how serious it could be and whether we are comfortable proceeding, we are applying Risk Assessment and Enterprise Risk Management.

So although A.I. may appear to present completely new risks (after all we got bombarded with this term A.I. maybe 5 years ago?), many familiar Risk Management principles can still help us ask the right questions.

The Bigger Question Is Not Simply “Is A.I. Risky?”

Almost everything we do involves some form of risk. I recently went to attend a WSQ Food Safety Certificate and I 'learnt' that germs are everywhere, so doesn't it meant that we are breathing germs as much as we are breathing oxygen?

Anyway let me go back. A more useful question is:

What are we using A.I. for, what are we allowing it to access or do, what could happen if it gets something wrong, and are we comfortable with that exposure?

The answer will not be the same for everyone.

Using A.I. to suggest ideas for an office celebration is very different from giving it confidential customer information. Asking A.I. to draft an email is different from allowing an A.I. system to influence a financial decision. Using A.I. to summarise information is different again from giving an A.I. agent access to systems and allowing it to take actions.

The technology may belong to the same broad family. The risk depends greatly on how we choose to use it.

A.I. Changes More Than Technology

This is also why our exploration of A.I. cannot stop at technical risk.

A.I. raises questions about trust. How do we know whether an answer is correct?

It raises questions about human judgement. If an A.I. recommendation sounds convincing, will we still challenge it?

It raises questions about responsibility. If someone acts on an A.I. recommendation and something goes wrong, where does accountability sit?

It raises questions about information. What are we giving A.I. access to, and what happens to that information?

And it raises questions about autonomy. How much authority are we comfortable handing to technology?

These are some of the reasons A.I. & Risk has become a major area of this updated website.

What We Will Explore in A.I. & Risk

Rather than trying to answer every A.I. question on one page, we have divided A.I. & Risk into seven areas, each looking at a different part of the subject.

A.I. Risk Management looks at how we identify, assess and manage the risks that arise from using A.I.

A.I. Governance considers responsibility, accountability and how organisations decide what A.I. should and should not be allowed to do.

A.I. Policy explores the practical rules organisations may need as employees increasingly use A.I. in everyday work.

A.I. & Data Privacy looks at one of the most immediate questions for individuals and businesses: what information are we giving A.I., and what should we be careful about sharing?

Human Oversight examines where human judgement remains important and when people should check, challenge or intervene in what A.I. is doing.

A.I. Hallucinations looks at the very practical problem of A.I. producing information that appears convincing but may be inaccurate or fabricated.

Agentic A.I. Risk takes us into the next stage: what happens when A.I. moves beyond generating answers and is given greater ability to plan, use tools and take actions?

Each of these areas will lead to more specific questions and articles as A.I. itself continues to develop.

We Are Learning Alongside a Rapidly Changing Technology

There is something particularly interesting about writing about A.I. risk now: the subject is changing while we are living through it.

Today's A.I. capabilities will not necessarily be tomorrow's. New applications will emerge. Some risks may become easier to control, while others may appear that we have not yet considered. Yesterday a post popped up on my Facebook that sent me into a depression of sorts because I was only learning the tip of the ice-berg in the whole piece of A.I. 

Because we all have limited time and energy (and no, we have not saved 2 days every week yet because in reality, most of us have not maximised the promise of A.I.), thus we do not need to predict everything A.I. will become before we can use it. Nor do we need perfect certainty before taking advantage of what it can offer. We can begin by understanding what we are trying to achieve, looking at the opportunities, considering what could affect the outcome, and deciding what safeguards make sense.

Then, as the technology changes, we reassess.

That is something Risk Management has always been good at.

Start With a Simple Question

Whether you are an individual experimenting with ChatGPT, an SME introducing A.I. into your business, a manager wondering what your employees are already using, or an organisation considering more autonomous A.I. systems, you can start in exactly the same place:

What am I asking A.I. to do?

Then ask:

What am I giving it access to? What am I relying on it for? What could happen if it gets it wrong? Who would be affected? And am I comfortable taking that risk?

You may discover that the answer leads to technology, cyber security, governance, privacy, third-party management, human oversight... or a combination of several areas.

That is why we created this section.

A.I. is changing what is possible. Risk Management gives us a structured way to decide what we are prepared to do with those possibilities. Oh oh, the passe word in Risk Management (i.e. 'structured way') appears here again. 

Questions We Will Be Exploring

These are some of the practical questions we will explore throughout A.I. & Risk:

• What Is A.I. Risk Management?

• What Could Go Wrong When My Company Uses A.I.?

• Do Small Businesses Need an A.I. Policy?

• Can Employees Put Company Information Into ChatGPT?

• What Is A.I. Governance?

• Who Is Responsible When A.I. Makes a Mistake?

• What Is Human Oversight of A.I.?

• How Do I Check Whether an A.I. Answer Is Correct?

• What Are A.I. Hallucinations?

• What Is Agentic A.I. and Why Is It Different?

• What Risks Do A.I. Agents Create?

• How Much Autonomy Should We Give an A.I. Agent?

• Can A.I. Make Business Decisions?

• How Do You Audit A.I.?

• How Do You Assess an A.I. Vendor?

• What Is A.I. Model Risk?

• What Is A.I. Data and Privacy Risk?

• What Is Prompt Injection?

• Should Companies Keep an A.I. Use Register?

• How Do I Create an A.I. Risk Register?

• What Should a Board Know About A.I. Risk?

• Can A.I. Replace Risk Managers?

• What Happens When A.I. Gets It Wrong?

Just putting these commonly asked questions here, so we get a sense of how much uncertainty we are still facing a couple of years from now.