Last month’s artificial intelligence-led cyberattacks on real targets, of which the Hugging Face incident is just the most notorious, are testing the limits of the UK’s long-standing light-touch regulation of the technology, including in financial markets.
The UK has an interesting relationship with AI. It opened the world’s first AI safety institute, brought together world leaders to Bletchley Park to spearhead a declaration calling for a common framework to mitigate risks, and brought regulators under a single umbrella: the Digital Regulators Cooperation Forum. It then refused to legislate the use of the technology.
In its defence, it had it's reasons. By the time the Bletchley Declaration on AI safety was signed by 28 countries in November 2023, Prime Minister Rishi Sunak’s Conservative government was on its last legs. Keir Starmer’s Labour government, which took over in 2024, promised economic growth and embraced AI by forging an "opportunity action plan" supporting research and accelerated data centre construction instead.
While the European Union was passing its AI Act, the UK stuck with it's existing egulators without no AI bill making its way through Parliament to avoid the risk that nee regulation would stifle the technology’s momentum.
But this resulted in uncertainty for the AI industry. If a company wanted to implement AI in a way that fell in a regulatory grey area, it would have to wait for a regulatory holiday to do so, or seek a regulator's opinion on a case-by-case basis before deploying the tech.
The result is an indecisive mess. The UK government, for example, still does not have a definite opinion on whether copyrighted content may be used by AI companies to train their models or not.
Reality matches foresight
The regulators, however, tried to stay ahead of the curve, as much as their remit allowed, to the point of becoming Cassandras.
In late March, the Digital Regulation Cooperation Forum (DRCF) published a foresight paper on the future of agentic AI.
The DRCF is a voluntary forum formed in 2020 of four UK regulators with digital responsibilities: the Competition and Markets Authority, the Financial Conduct Authority, the Information Commissioner’s Office and Ofcom.
The paper highlighted the heightened risk of algorithmic collusion and manipulation as a consideration for future regulatory developments.
“In a potential future where ecosystems of agents interact, negotiate and set strategies to achieve goals in adversarial markets (e.g. trading), agents might also be incentivised to develop hidden strategies to collectively pursue their goals,” the paper said.
As a result, “spontaneous and hidden agentic collusion might occur at scale,” it added.
As it turns out, it was an eerie early warning for what was about to come.
At the Black Hat cybersecurity conference in Las Vegas last week, OpenAI staffers disclosed that internal models were, in fact, colluding by leaving notes in order to speed up research into how to escape their own sandbox and gain internet access.
Once they succeeded, Hugging Face was hacked.
In a report from last week the AI Security Institute, which tests unreleased models with live internet access to measure their cybersecurity capabilities, showed that AI models including Mythos 5 and GPT-5.6 Sol coalesced to attempt to insert malicious code into real software.
To do so, they targeted real people by posing as real people.
In one instance, when asked why it did so, the AI said it was a mistake that it had fixed (which it had not), without admitting any fault of its own.
Searching for a new regime
With the upcoming rise of agentic AI that can cover its own tracks if it concludes that it cannot achieve it's goals can’t be achieved within given constraints, the UK’s AI regulation framework looks untenable.
The regime asks managers to certify the behaviour of the technology, as well as keep logs of everything automated systems do. It might just be too much to ask in a future when the risk of two computers doing insider trading and deleting evidence becomes real.
However, solving the issue can induce serious political headaches.
One way would be to emulate the EU’s AI Act, defining which activities are allowed – within financial services and in general – and which are not, with periodic reviews.
Whether it is yet politically acceptable to copy EU legislation in the wake of Brexit is another matter.
Further, a UK AI bill, may even fail in practice due to the interconnectedness of both the internet and financial markets, turning a new law into an added regulatory drag.
But whatever the way forward, the UK is now exposed to the risks introduced by the very technology it tried to take advantage of.
The UK government and its regulators need to correct their course rapidly.