As artificial intelligence continues to transform organisations, effective approaches to AI risk, governance and assurance are becoming increasingly important. The IAAA and AGRC are collaborating on a survey exploring perspectives on AI risk and governance. Your insights will help contribute to a broader understanding of the challenges and priorities. We invite you to take part and share your views. Survey available here.
You founded the International Association of Algorithmic Auditors (IAAA) after witnessing first-hand how fragmented and inconsistently defined AI auditing had become. What convinced you that the field needed a professional body, and what was missing from the existing AI governance landscape?
It wasn’t one single thing that convinced me. I looked at how other professions and techniques had emerged over time, and how they evolved into something with proper standards, oversight, and guarantees. The obvious gap in this space was that there was no space for the actual professionals doing the work. I waited for a while to see if someone else would build something like this, but nothing happened, so I built it myself.
When I started IAAA, there was essentially nothing in the AI governance landscape. This was back in 2023, when awareness of these issues was far lower than it is today. It was an obvious gap, and one I felt had to be filled.
As demand for AI assurance grows, the term ‘AI audit’ is being used increasingly loosely. What warning signs should organisations look for when selecting an AI auditor or evaluating the credibility of an audit?
The term ‘AI audit’ has always been used quite loosely, but the positive side is that there’s now a lot more practice out there: more people doing audits, evaluations, assessments, and assurance work. That gives us something to actually work with, since we can start grouping what exists and defining what qualifies as an audit, rather than working in the abstract. IAAA will soon release a document, ‘What Counts as an AI Audit,’ that aims to contribute to exactly this discussion.
The main warning sign is a lack of combined expertise: people who are only proficient in legal and compliance matters, or people who only do technical evaluations. You need both, and that combination is genuinely hard to find right now. Most of the industry still leans on compliance professionals who have no real understanding of how a machine learning system works, and it’s impossible to properly inspect something you don’t understand.
IAAA’s ‘What Makes an AI Audit’ framework identifies four minimum conditions for a credible audit, could you explain what these are and why they are so important?
At its core, a credible audit needs to include an acknowledgment of what has and has not been tested; the conditions of access to information; the conditions of the engagement itself; and accountability for what the auditor did versus what the company, system, or client provided.
It also needs to be clear about what was actually covered, whether this is a compliance audit, a model audit, or a production/impact audit. All of that needs to be disclosed for the exercise to be credible.
It doesn’t need to disclose how the client performed, at least not initially. What matters right now is that there’s a clear understanding of what was actually done.
AI audits can be carried out internally, by an external firm hired by the organisation or by a fully independent third party. How does the auditor’s relationship with the organisation affect the credibility of the audit?
Right now, the auditor’s relationship with the organisation doesn’t necessarily affect credibility directly: you can have zero credibility regardless of where you sit. Of course, a third-party auditor working from the outside, or a community trying to reverse-engineer a system, is more independent, but that doesn’t automatically make them more credible.
What’s actually missing is a clear definition of what these different layers of engagement mean, and how the presence or absence of independence translates into something that’s actually credible.
So, the real question is what credibility looks like for someone working inside the company, for a second-party auditor paid by the company, and for someone doing a fully independent third-party exercise. We could easily have cases where a third-party auditor does a genuinely bad audit that wrongly accuses a good system of problems it doesn’t actually have. Credibility and independence are two different things.
How should AI auditing fit within an organisation’s wider risk governance and three lines of defence structure? In particular, where should responsibility sit between developers, operational management, compliance, internal audit, and independent external assurance?
This is a genuinely complex question, and I think the field as a whole needs to evolve toward clearer product liability frameworks and legal structures that ensure every action can be traced back to a legal person who is accountable for what happened. Whether responsibility sits with developers, operational management, compliance, internal audit, or independent external assurance depends entirely on what each of them actually did.
In most other fields, independent external assurance actors, internal audit, and compliance don’t typically hold liability in the way developers and operational management do, which is exactly why companies need strong governance structures, so that when something goes wrong, it’s possible to establish liability clearly while also appropriately limiting it.
Right now, we’re relying on regulation to extend the same oversight mechanisms we have for other products to AI products and services. If a child’s stroller breaks, there’s clear product liability and a path to establish who’s responsible. We don’t have the equivalent for AI yet, so we’re essentially trying to fill a gap that needs the backing of regulation.
I think liability is a huge open issue, and if it isn’t addressed by regulation, it will eventually be addressed by the courts. The problem is that courts take much longer, and in the meantime, people will suffer real harm to their rights, their livelihoods, or their mental health before anyone is held to account.
Could you tell us more about the association’s Credentialed AI Auditor (CAIA) programme? How is it setting the standard in this field and ensuring that AI auditors have the right combination of technical knowledge, risk awareness, and practical auditing skills?
CAIA is unique because it’s the first credential that requires applicants to demonstrate genuinely socio-technical experience. Applicants need to show they understand legal, compliance, and regulatory concepts, but also technical issues: how machine learning systems work, how statistics are used to measure outcomes, and how benchmarks are established.
We also value, though don’t strictly require, experience with third-party audits and community engagement, because that gives people visibility into what actually happens once systems are in production.
We’ve tried to build a credentialing programme that acts as a lever, one that creates incentives for training providers to start building genuinely socio-technical profiles and helps people build bridges across disciplines. It’s both a credential and, in a sense, a guiding star for where the field needs to go.
We have a points-based system, which we’ll be making public shortly, that captures information across all these dimensions and recognises when someone has that mixed background, matching the credential to their actual expertise.
You don’t need to be an auditor to be part of IAAA. You can simply be a member without holding a credential. And even among credentialed members, there are two levels: Associate and Professional. So, people can climb the ladder as they professionalise in AI auditing.
Regulation and standards are evolving rapidly, what must happen next for AI auditing to mature into a trusted profession and what risks arise if commercial demand for audits grows faster than professional standards and auditor capacity?
A combination of things needs to happen. We need regulation. We need the private sector to recognise that unsafe systems are simply bad business. We need insurers to start demanding better governance and auditing practices from the companies they insure when those companies use AI. And we need users and clients to start demanding that their providers are more accountable and transparent about the systems they’re selling or providing.
A lot of this is already happening, even if it’s still messy and no one quite knows which direction to take yet. What I see are more and more of the levers I’ve been working toward for the past ten years finally starting to appear. It’s a genuinely good time for this work. We need to make sure that all the concern around AI safety, accountability, auditability, and evaluation translates into robust mechanisms, so that anyone entering this space claiming they can audit a system has actually done the work and can do it in a way that’s professional and accountable to the profession itself.
Most auditors get paid by the organisations they audit. So, the only way auditing becomes a genuinely healthy, independent profession is by ensuring auditors are bound by the rules of the profession, rather than by the contract with whoever is paying them. That’s what we’re building at IAAA.
Gemma Galdón-Clavell is a European expert in AI governance, algorithmic accountability and data ethics, and the founder and CEO of Eticas. Her work focuses on the assessment, auditing, and governance of high-risk AI and algorithmic systems, particularly those used in the public sector, law enforcement, surveillance, and other contexts where automated decision-making can have a direct impact on fundamental rights.
