Autonomous, intelligent agents offer some great opportunities for our digital lives*. There, look, I said it. They will book meetings for us, negotiate cheap holidays, order our children’s complete school outfit for the beginning of term, and let us know when it’s time to go to the nurse for our check-up. Our business lives, our personal lives, our family relationships – they’ll all be revolutionised by autonomous agents. Autonomous agents will learn our preferences, have access to our diaries, pay for items, be able to send messages to our friends.
This is all fantastic, and I’m very excited about it. The problem is that I’ve been excited about it for nearly 20 years, when I was involved in a project around autonomous agents in Java. It was very neat then, and it’s still very neat now***.
Of course, technology has moved on. Some of the underlying capabilities are much more advanced now than then. General availability of APIs, consistency of data formats, better Machine Learning (or Artificial Intelligence, if you must), less computationally expensive cryptography, and the rise of blockchains and distributed ledgers: they all bring the ability for us to build autonomous agents closer than ever before. We talked about disintermediation back in the day, and that looked plausible. We really can build scalable marketplaces now in ways which just weren’t as feasible two decades ago.
The problem, though, isn’t the technology. It was never the technology. We could have made the technology work 20 years ago, even if it wasn’t as fast, secure or wide-ranging as it could be today. It isn’t even vested interests from the large platform players, who arguably own much of this space at the moment – though these interests are much more consolidated than they were when I was first looking at this issue.
The problem is not the autonomy. The problem isn’t even particularly with the intelligence: you can program as much or as little in as you want, or as the technology allows. The problem is with the agency.
How much of my life do I want to hand over to what’s basically a ‘bot? Ignore***** the fact that these things will get hacked******, and assume we’re talking about normal, intended usage. What does “agency” mean? It means acting for someone: being their agent – think of what actors’ agents do, for example. When I engage a lawyer or a builder or an accountant to do something for me, or when an actor employs an agent for that matter, we’re very clear about what they’ll be doing. This is to protect both me and them from unintended consequences. There’s a huge legal corpus around defining, in different fields, exactly the scope of work to be carried out by a person or a company who is acting as an agent. There are contracts, and agreed restitutions – basically punishments – for when things go wrong. Say that an accountant buys 500 shares in a bank, and then I turn round and say that she never had the authority to do so: if we’ve set up the relationship correctly, it should be entirely clear whether or not she did, and whose responsibility it is to deal with any fall-out from that purchase.
Now think about that in terms of autonomous, intelligent agents. Write me that contract, and make it equivalent in software and the legal system. Tell me what happens when things go wrong with the software. Show me how to prove that I didn’t tell the agent to buy those shares. Explain to me where the restitution lies.
And these are arguably the simple problems. How to I rebuild the business reputation that I’ve built up over the past 15 years when my agent posts on Twitter a tweet about how I use a competitor’s products, when I’m just trialling them for interest? How does an agent know not to let my wife see the diary entry for my meeting with that divorce lawyer*******? What aspects of my browsing profile are appropriate for suggesting – or even buying – online products or services with my personal or business credit card*********? And there’s the classic “buying flowers for the mistress and having them sent to the wife” problem**********.
I don’t think we have an answer to these questions: not even close. You know that virtual admin assistant we’ve been promised in sci-fi movies for decades now: the one with the futuristic haircut who appears as a hologram outside our office? Holograms – nearly. Technology behind it – pretty much. Trust, reputation and agency? Nowhere near.
*I hate this word: “digital”. Well, not really, but it’s used far too much as a shorthand for “newest technology”**.
**”Digital businesses”. You mean, unlike all the analogue ones? Come on.
***this is one of those words that my kids hate me using. There are two types of word that come into this category: old words and new words. Either I’m showing how old I am, or I’m trying to be hip****, which is arguably worse. I can’t win.
****yeah, they don’t say hip. That’s one of the “old person words”.
*****for now, at least. Let’s not forget it.
******_everything_ gets hacked*******.
*******I could say “cracked”, but some of it won’t be malicious, and hacking might be positive.
********I’m not. This is an example.
*********this isn’t even about “dodgy” things I might have been browsing on home time. I may have been browsing for analyst services, with the intent to buy a subscription: how sure am I that the agent won’t decide to charge these to my personal credit card when it knows that I perform other “business-like” actions like pay for business-related books myself sometimes?
**********how many times do I have to tell you, darling…?
Alice, Eve and Bob - a security blog 
One thought on “Wow: autonomous agents!”