Breaking news: U.S. banks are taking hits from AI agents

“We believe that the world should accept some bad things happening for the benefits of this technology and people having the agency”

— Sam Altman, October 3

Breaking news from Madison Mills at Axios:

This is disconcerting not just because hackers are stepping up their malicious use of agents, but also because they don’t need state-of-the-art “frontier” models to cause mayhem.

This should serve as a wakeup call for those who focus all their attention on regulating “frontier models”, often measured in terms of how much compute they use. Even if we were to stop the development of future frontier models, we would still face serious problems from the misuse of existing models.

Clearly, regulatory efforts should not be focused solely on the frontier, nor solely focused on concerns about what future models might do._Current_models are_already_causing serious problems across society, in domains ranging from education to politics (via deepfakes and wholesale disinformation) to banking. Ignoring all that entirely in favor of speculative futures would be a mistake.

§

At the beginning of 2025I warned in Politicothat we were about to face a perfect storm in cybersecurity, for four reasons.

First, generative AI, rising in popularity and declining in price, is a perfect tool for cyberattackers. Although it is unreliable and prone to hallucinations, it is terrific at making plausible sounding text (e.g., phishing attacks to trick people into revealing credentials) and deepfaked videos at virtually zero cost, allowing attackers to broaden their attacks. Already, a cybercrew bilked aHong Kong bank out of $25 million.

Second, large language models are notoriously susceptible to jailbreaking and things like “prompt-injection attacks,” for which no known solution exists.

Third, generative AI tools are increasingly being used to create code; in some cases those coders don’t fully understand the code written, and the autogenerated code has already been shown in some cases to introduce new security holes.

Finally, in the midst of all this, the new U.S. administration seems determined to deregulate as much as possible, slashing costs and even publicly shaming employees. Federal employees who do their jobs may be frightened, and many will be tempted to look elsewhere; enforcement and investigations will almost certainly decline in both quality and quantity, leaving the world quite vulnerable to ever more audacious attacks.

Well, here we are.

§

Here are three further predictions:

1. No matter how much chaos generative AI causes, the AI industry will not take real responsibility

2. The Trump administration will do little to hold them responsible.

3. Things will get worse.

   Subscribe now
   https://bender.layer3.press/articles/79c1d086-b964-4f86-bc7b-a851448636f6