There were two developments last week that caught my attention because they relate to the two most consequential trends shaping supply chains today: tariffs and AI.
Yesterday, I wrote about tariffs — specifically, the new levies imposed by the Trump administration on more than 80 countries based on Section 301 of the Trade Act of 1974 (see “The Trump Tariffs Strike Back”).
Today, I want to focus on a question prompted by this Wall Street Journal headline: “OpenAI Models Escaped and Hacked a Company in Cybersecurity Test Gone Wrong.”
Can We Really Trust and Control AI?
As reported by Robert McMillan and Amrith Ramkumar in the WSJ, last Tuesday “OpenAI said two artificial intelligence systems it was testing broke out of their test environment, hacked their way onto the internet and broke into another company. The victim was Hugging Face, a provider of open-source AI tools. The cause was a cybersecurity benchmarking test that went very, very wrong.”
The article adds that “OpenAI had caged the models in a ‘sandbox,’ a system that didn’t have access to the internet. But during the test, the AI software used its hacking skills to break out. It found a way to get online, and then hacked into Hugging Face’s network, OpenAI said.”
I encourage you to read the rest of the article for the full story.
When I read the article, my first thought was, “Wow, this is a bit like HAL from 2001: A Space Odyssey.” That is, an AI system pursuing its assigned objective in an unexpected way.
I also recalled that funny scene from the HBO series Silicon Valley, where the AI program “Son of Anton” accidentally (and autonomously) orders 4,000 lbs of meat in its quest to source cheap burgers. See clip below (note: there is some foul language in it).
My overall takeaway is that this incident is a reminder that as AI systems become more capable, controlling their behavior becomes increasingly challenging. It shows that you can’t really control what an AI will do with 100% certainty. Even in a supposedly isolated “sandbox” environment, the systems found ways to pursue their assigned objective that their creators did not anticipate.
Simply put, we cannot predict with complete certainty how an AI system will pursue the goals we give it — and that should give organizations deploying AI some pause.
Much of the hype today, especially in the supply chain software community, is about enabling the Autonomous Supply Chain, where AI agents — leveraging optimization, machine learning, and real-time data — will make decisions and execute tasks with virtually no human intervention.
Fortunately, supply chain leaders remain cautious about handing over that level of control.
In a survey we conducted last August, we asked members of our Indago research community — supply chain and logistics executives from manufacturing, retail, and distribution companies — “What level of autonomy would you be comfortable giving to an AI agent in your transportation operations?”
None of the respondents were comfortable granting fully autonomous decision-making to AI agents. Instead, 65% preferred semi-autonomous systems where humans approve exceptions, and 35% preferred AI proposals that humans evaluate before action.
This preference underscores a “trust but verify” mindset — i.e., an openness to automation coupled with a strong desire for control and transparency.
As one executive commented, “At least in the short to medium term (the next two years), I’d still want a human-in-the-loop process — where, at a minimum, exceptions are managed by a person, or the whole process is overseen by someone who can intervene when the AI isn’t acting appropriately.”
Another added a generational perspective: “My sense is that professionals of my vintage (Boomers / Gen X) might be a little scarred by things like ‘2001: A Space Odyssey’ or ‘Terminator,’ and therefore more wary of technology that can think. So, speaking for myself, I’m only comfortable with AI in non-critical areas.”
Can we really trust and control AI?
If trust means being able to predict with certainty how an AI system will behave in every situation, we’re clearly not there yet. For now, the lesson for supply chain leaders is the same one reflected in our survey results: keep humans in the loop, especially for high-impact decisions, while organizations gain experience, confidence, and governance around AI.
