AI Governance: the Role Only Leadership Can Play

A company installs an AI tool to speed up credit decisions and delegates its implementation to the technology department, with nobody on the board reviewing what the tool is actually deciding. Months later, a regulator asks who approved the criteria being used. Nobody on the board can answer, because nobody beyond the technical team ever looked at the matter. This scenario is no longer hypothetical. It is, increasingly, the real starting point of conversations between boards and European regulators.

What does the EU AI Act require of leadership?

The EU's Artificial Intelligence Regulation, in force since 2024 with obligations phasing in through 2027, classifies AI systems by risk level and requires, for high-risk systems, effective human oversight, clear documentation of how they function, and accountability for whoever uses them, not only whoever builds them. A company using AI in recruitment, credit or staff performance evaluation is, in all likelihood, in this high-risk category, and the law no longer allows this to be treated as a technical detail left to the IT department.

Who should be responsible for AI governance in a company?

Final responsibility has to sit with top leadership, even when technical implementation is delegated to specialists. A competent AI committee brings together different perspectives, legal, technical, operational, but the decision about where the organisation is willing to use AI, and where it is not, is a risk management decision, the same kind as decisions about financial exposure or brand reputation. Leaving this decision entirely to the technology department is abdicating a responsibility that the law, and plain common sense, clearly place elsewhere.

How do you integrate AI governance into ESG?

ESG brought dimensions to the decision table that the traditional financial model ignored, but it has a limitation that is rarely discussed: it asks, above all, whether the organisation is complying with rules already written down. It is a compliance instrument, useful but limited. AI governance sits naturally within the governance pillar of ESG, but it only produces real value once it goes beyond box ticking and starts asking whether the organisation has enough maturity to use these tools with judgement, not just with legal permission.

What mistake does a company make by treating this as legal compliance and nothing else?

It treats AI governance as a checklist exercise, settled once a year with a legal opinion, and forgets that the real risk shifts every month, as new tools enter the organisation through the informal initiative of individual teams. An organisation can be legally compliant and still have dozens of AI tools in use with no real oversight, because documented compliance and effective control are, quite often, different things.

What role does the Deep Capital Premium play in this governance?

The Deep Capital Premium, described in The Last Asset, proposes measuring an organisation's real maturity through indicators such as decision quality and resilience under crisis, going beyond the simple compliance that traditional ESG measures. An organisation with high Deep Capital tends to have better AI governance as a natural consequence, because the same maturity that allows it to decide well under pressure is what allows it to question a tool before adopting it blindly.

What signs show a company's AI governance is only for show?

One clear sign is a well written AI policy that nobody in the organisation actually knows or consults day to day. Another is the absence of any record of which AI tools are genuinely in use, a decision that tends to be scattered across different teams, each adopting whatever seems useful with no central coordination. A third, more subtle sign is a board that discusses AI only in terms of business opportunity, productivity gains, new products, and never in terms of the risk that same adoption is introducing into the organisation.

What does it cost an organisation to ignore this until it is too late?

The cost rarely shows up as a single fine, although the AI Act provides for substantial penalties for serious breaches. It shows up more often as reputational damage, when it emerges that an automated decision discriminated against a group of people, or as the cost of rebuilding, under pressure and against the clock, an entire process that should have been well designed from the start. Companies that treat AI governance as a priority from early on face this risk in an orderly way. Those that put it off end up dealing with it in crisis mode, with a regulator or a journalist already asking questions.

How do you start building this governance without waiting for an audit?

Start with a simple inventory: which AI tools are already in use, in which decisions, and who would take responsibility if something went wrong. Then set out in writing which types of decision can never be automated without human review, work that connects directly with what should not be delegated to artificial intelligence. A leadership diagnostic helps assess whether your top team has, today, the decision maturity needed to take this responsibility seriously, rather than discovering the gap only when a regulator, a journalist or a wronged customer demands it first.