open source – Page 9 – Alice, Eve and Bob

Is homogeneity bad for security?

Can it really be good for security to have such a small number of systems out there?

For the last three years, I’ve attended the Linux Security Summit (though it’s not solely about Linux, actually), and that’s where I am for the first two days of this week – the next three days are taken up with the Open Source Summit. This year, both are being run both in North America and in Europe – and there was a version of the Open Source Summit in Asia, too. This is all good, of course: the more people, and the more diversity we have in the community, the stronger we’ll be.

The question of diversity came up at the Linux Security Summit today, but not in the way you might necessarily expect. As with most of the industry, this very technical conference (there’s a very strong Linux kernel developer bias) is very under-represented by women, ethnic minorities and people with disabilities. It’s a pity, and something we need to address, but when a question came up after someone’s talk, it wasn’t diversity of people’s background that was being questioned, but of the systems we deploy around the world.

The question was asked of a panel who were talking about open firmware and how making it open source will (hopefully) increase the security of the system. We’d already heard how most systems – laptops, servers, desktops and beyond – come with a range of different pieces of firmware from a variety of different vendors. And when we talk about a variety, this can easily hit over 100 different pieces of firmware per system. How are you supposed to trust a system with some many different pieces? And, as one of the panel members pointed out, many of the vendors are quite open about the fact that they don’t see themselves as security experts, and are actually asking the members of open source projects to design APIs, make recommendations about design, etc..

This self-knowledge is clearly a good thing, and the main focus of the panel’s efforts has been to try to define a small core of well-understood and better designed elements that can be deployed in a more trusted manner. The question that was asked from the audience was in response to this effort, and seemed to me to be a very fair one. It was (to paraphrase slightly): “Can it really be good for security to have such a small number of systems out there?” The argument – and it’s a good one in general – is that if you have a small number of designs which are deployed across the vast majority of installations, then there is a real danger that a small number of vulnerabilities can impact on a large percentage of that install base.

It’s a similar problem in the natural world: a population with a restricted genetic pool is at risk from a successful attacker: a virus or fungus, for instance, which can attack many individuals due to their similar genetic make-up.

In principle, I would love to see more diversity of design within computing, and particular security, but there are two issues with this:

management: there is a real cost to managing multiple different implementations and products, so organisations prefer to have a smaller number of designs, reducing the number of the tools to manage them, and the number of people required to be trained.
scarcity of resources: there is a scarcity of resources within IT security. There just aren’t enough security experts around to design good security into systems, to support them and then to respond to attacks as vulnerabilities are found and exploited.

To the first issue, I don’t see many easy answers, but to the second, there are three responses:

find ways to scale the impact of your resources: if you open source your code, then the number of expert resources available to work on it expands enormously. I wrote about this a couple of years ago in Disbelieving the many eyes hypothesis. If your code is proprietary, then the number of experts you can leverage is small: if it is open source, you have access to almost the entire worldwide pool of experts.
be able to respond quickly: if attacks on systems are found, and vulnerabilities identified, then the ability to move quickly to remedy them allows you to mitigate significantly the impact on the installation base.
design in defence in depth: rather than relying on one defence to an attack or type of attack, try to design your deployment in such a way that you have layers of defence. This means that you have some time to fix a problem that arises before catastrophic failure affects your deployment.

I’m hesitant to overplay the biological analogy, but the second and third of these seem quite similar to defences we see in nature. The equivalent to quick response is to have multiple generations in a short time, giving a species the opportunity to develop immunity to a particular attack, and defence in depth is a typical defence mechanism in nature – think of human’s ability to recognise bad meat by its smell, taste its “off-ness” and then vomit it up if swallowed. I’m not quite sure how this particular analogy would map to the world of IT security (though some of the practices you see in the industry can turn your stomach), but while we wait to have a bigger – and more diverse pool of security experts, let’s keep being open source, let’s keep responding quickly, and let’s make sure that we design for defence in depth.

Jargon – a force for good or ill?

Distinguish an electrical engineer from a humanities graduate: ask them how many syllables are in the word “coax”.

I was involved in an interesting discussion with colleagues recently about the joys or otherwise or jargon. It stemmed from a section I wrote in a recent article, How to talk to security people: a guide for the rest of us. In it, I suggested that jargon “has at least two uses:

as an exclusionary mechanism for groups to keep non-members in the dark;
as a short-hand to exchange information between ‘in-the-know’ people so that they don’t need to explain everything in exhaustive detail every time.”

Given the discussion that arose, I thought it was worth delving more deeply into this question. It’s more than an idle interest, as well, as I think that there are important lessons around our use of jargon that impact on how we interact with our colleagues and peers, and which deserve some careful thought. These lessons apply particularly to my chosen field of security.

Before we start, there should be some definition. It’s always nice to have two conflicting versions, so here we go:

Special words or expressions used by a profession or group that are difficult for others to understand. (Oxford dictionaries – https://en.oxforddictionaries.com/definition/jargon)
Without qualifier, denotes informal ‘slangy’ language peculiar to or predominantly found among hackers. (The Jargon File – http://catb.org/jargon/html/distinctions.html)

I should start by pointing out that the Jargon File (which was published in paper form in at least two versions as The Hacker’s Dictionary (ed. Steele) and the New Hacker’s Dictionary (ed. Raymond)) has a pretty special place in my heart. When I decided that I wanted to “take up” geekery properly[1][2], I read my copy of the the New Hacker’s Dictionary from cover to cover, several times, and when a new edition came out, I bought that and did the same. In fact, for many of the more technical readers of this blog, I suspect that a fair amount of your cultural background is expressed within the covers (paper or virtual) of the same, even if you’re not aware of it. If you’re interested in delving deeper and like the feel of paper in your hands, then I encourage you to purchase a copy, but be careful to get the right one: there are some expensive versions which seem just to be print-outs of the Jargon File, rather than properly typeset and edited versions[3].

But let’s get onto the meat of this article: is jargon a force for good or ill?

The case against jargon – ambiguity

You would think that jargon would serve to provide agreed terms within a particular discipline, and help ambiguity around contexts. It may be a surprise, then, that first problem that we often run into with jargon is name-space clashes. Consider the following. There’s an old joke about how to distinguish an electrical engineer from a humanities[3] graduate: ask them how many syllables are in the word “coax”. The point here, of course, is that they come from different disciplines. But there are lots of words – and particularly abbreviations – which have different meanings or expansions depending on context, and where disciplines and contexts may collide. What do these mean to you[5]?

scheduling (kernel level CPU allocation to processes OR placement of workloads by an orchestration component)
comms (I/O in a computer system OR marketing/analyst communications)
layer (OSI model OR IP suite layer OR other architectural abstraction layer such as host or workload)
SME (subject matter expert OR small/medium enterprise)
SMB (small/medium business OR small message block)
TLS (Transport Layer Security OR Times Literary Supplement)
IP (Internet Protocol OR Intellectual Property OR Intellectual Property as expressed as a silicon component block)
FFS (For Further Study OR …[6])

One of the interesting things is that quite a lot of my background is betrayed by the various options that present themselves to me: I wonder how many of the readers of this will have thought of the Times Literary Supplement, for example. I’m also more likely to think of SME as the term relating to organisations, because that’s the favoured form in Europe, whereas I believe that the US tends to SMB. I’m sure that your experiences will all be different – which rather makes my point for me.

That’s the first problem. In a context where jargon is often praised as a way of short-cutting lengthy explanations, it can actually be a significant ambiguating force.

The case against jargon – exclusion

Intentionally or not – and sometimes it is intentional – groups define themselves through use of specific terminology. Once this terminology becomes opaque to those outside the group, it becomes “jargon”, as per our first definition above. “Good” use of jargon generally allows those within the group to converse using shared context around concepts that do not need to be explained in detail every time they are used. An example would be a “smoke test” – a quick test to check that basic functionality is performing correctly (see the Jargon File’s definition for more). If everyone within the group understands what this means, then why go into more detail? But if you are joined at a stand-up meeting[7] by a member of marketing who wants to know whether a particular build is ready for release, and you say “well, no – it’s only been smoke-tested so far”, then it’s likely that you’ll need to explain.

Another enduring and complex piece of jargon is the use of “free” in relation to software. In fact, the term is so ambiguous that different terms have evolved to describe some variants – “open source”, “FOSS” – and even phrases such as “free as in speech, not as in beer”.

The problem is that there are occasions when jargon can be excluding from others, whether that usage is intended or not. There have been times for most of us, I’m sure, when we want to show that we’re part of group and so use terms that we know another person won’t understand. On other occasions, the term may be so ingrained in our practice that we use it without thinking, and the other person is unintentionally excluded. I would argue that we need to be careful to avoid both of these usages. Intentional exclusion is rarely helpful, but unintentional exclusion can be just as damaging – in ways more so, as it is typically unremarked and is therefore difficult to remedy.

The security world seems particularly prone to this behaviour, I feel. It may be that many security folks have a sound enough grounding in other spheres – especially those in which they’re working with others – that the overlap in the other direction is less noticeable. The other issue is that there’s a certain elitism with security folks which is both laudable – you want the best people, with the best training, to be leading the field – and lamentable – elitism tends to lead to exclusion of members of other groups.

A quick paragraph in praise of jargon

It’s not all bad, however. We need jargon, as I noted above, to enable us to discuss concepts, and the use of terms in normal language – like scheduling – as jargon leads to some interesting metaphors which guide us in our practice[8]. We absolutely need shared practice, and for that we need shared language – and some of that language is bound, over time, to become jargon. But consider a lexicon, or a FAQ, or other ways to allow your colleagues to participate: be inclusive, not exclusive.

1 – “properly” – really? Although I’m not sure “improperly” is any better.

2 – oh, remember that I actually studied English Literature and Theology at university, so this was a conscious decision to embrace a rather different culture.

3 – or “Liberal Arts”.

4 – the most recent “real” edition of which I’m aware is Raymond, Eric S., 1996, The New Hacker’s Dictionary, 3rd ed., MIT University Press, Cambridge, Mass.

5 – I’ve added the first options that spring to mind when I come across them – I’m aware there are almost certainly others.

6 – believe me, when I saw this abbreviation in a research paper for the first time, I was most confused, and actually had to look it up.

7 – oh, look: jargon…

8 – though metaphors can themselves be constraining as they tend to push us to thinking in a particular way, even if that way isn’t entirely applicable in this context.

The gift that keeps on giving: passwords

A Father’s Day special

There’s an old saying: “if you give a man a fish, he’ll eat for a day, but if you teach a man to fish, he’ll eat for a lifetime.” There are some cruel alternatives with endings like “he’ll buy a silly hat and sit outside in the rain”, but the general idea is that it’s better to teach someone something rather than just giving them something.

With Father’s Day coming up this Sunday in many parts of the world, I’d like to suggest the same for passwords. Many people’s password practices are terrible. There are three things that people really don’t get about passwords:

what they should look like
how they should be stored
how they should be communicated.

Let’s go through each of these in turn, and I’ll try to give brief tips that you can pass onto your father (or, indeed, mother, broader family, friends or colleagues) to help them with password safety.

What should passwords look like?

There’s a famous xkcd comic called password strength which aims to help you find a useful password. This is great advice if you only have a few passwords, but about twenty years ago I got above ten, and then started re-using passwords for certain levels of security. This was terrible at the time, and even worse now. Look at the the number of times a week we see news about information being lost when companies or organisations are hacked. If you share passwords between accounts, there’s a decent chance that your login details for one will be exposed, which means that all your other accounts that share that set are compromised.

I know some people who used to have permutations of passwords. Let’s say the base was “p4ssw0rd”: they would then add a suffix for the website or account, such as “p4ssw0rdNetflix”. This might be fine if we believed that all passwords are stored in hashed form, but, well, we know they’re not, so don’t do this, either. Extrapolating from one account to another is too easy.

What does a good password look like, then? Here’s one: “W9#!=_twXhRb” And another? This one is 16 characters long: “*Wdb_%|#N^X6CR_b” What are the chances of a human guessing these? Pretty slim. And a computer? Not much better, to be honest. They are randomly generated by software, and as long as I use a different one for each account, I’m pretty safe against password-guessing attacks.

“But,” you say, “how am I supposed to remember them? I’ve got dozens of accounts, and I can’t remember one of those, let alone fifty!”

How should you store passwords?

Well, you shouldn’t try to remember passwords, in the same way that you shouldn’t try to generate them. Oh, there will be a handful that you might remember – maybe low-importance ones like the wifi key to your home AP – but most of them you should trust to a password manager. These are nifty pieces of software that will generate and then remember hundreds of passwords for you. Some of them will even automatically fill website fields for you if you ask them to. The best ones are open source, which means that people have pored over their code (hopefully) to check they’re trustworthy, and that if you’re not entirely sure, then you can pore of their code, too. And make changes and improvements and generally improve the world bit by bit.

You will need to remember one password, though, and that’s the one to unlock the password manager. Make it really, really strong: it’s about the only one you mustn’t lose (though most websites will help you reset a password if you forget it, so it’s just a matter of going through each of the several hundred until they’re done…). Use the advice from the xkcd cartoon, or another strong password algorithm that’s easy to remember.

To make things more safe, store the (password protected) key store somewhere that is not easily accessed by other people – not a shared drive at work, for instance, but maybe on your phone or on some cloud-based storage that you can get to if you lose your phone. Always set the password manager to auto-lock itself after some time, in case you leave your computer logged on, or your phone gets stolen.

How to communicate passwords

Would you send a password via email? What about by SMS? Is post[2] better? Is it acceptable to reveal a password over the phone in a crowded train carriage[4]? Would you give your laptop password to a random person conducting a survey on a railway station for the prize of a chocolate bar?

In an ideal world, we would never share passwords, but there are times when we need to – and times when it’s worthwhile for material rewards[5]. There are some accounts which are shared – TV or film streaming accounts for the family – or that you’ve set up for somebody else, or which somebody urgently needs to access because you’re on holiday, for instance. So you may need to give out passwords from time to time. What’s the best mechanism? What’s the worst?

This may sound surprising, but I’d generally say that the worst (marginally) is post. What you’re trying to avoid happening is a Bad Person[tm] from marrying two pieces of information: the username and the password. If someone has access to your post, then there’s a good chance that they might be able to work out enough information about you that they can guess the account name. The others? Well, they’re OK as long as you’re not also sending the username via the same channel. That, in fact, is the key test: you should never provide the two pieces of information in such a way that a person with access to one channel can put them together. So, telling someone a password in a crowded train carriage may be rude in relation to all of the other people in the carriage[6], but it may be very secure in terms of account safety.

The reason I posed the question about the survey is that every few months a survey company in the UK asks people at mainline railway stations to tell them their password in exchange for a chocolate bar, and then write a headline about how awful it is that many people will give them their password. This is a stupid headline, for a stupid survey, for two reasons:

I’d happily lie and tell them a false password in order to get a free chocolate bar AND
even if I gave them the correct password, how are they going to marry that with my account details?

Conclusion

If you’re the sort of person reading there’s a fairly high chance that you’re the sort of person who’s asked to clear up the mess what family, friends or colleagues get their accounts compromised[7]. Here are four rules for password security:

don’t reuse passwords – use a different one for every single account
don’t think up your own passwords – get a password manager to generate them for you
use a password manager to store your passwords – if they’re strong enough in the first place, you won’t be able to remember them
never send usernames and passwords over the same channel – you want to avoid the situation where an attacker has access to both and can use them.

I’ll add a fifth one for luck: feel free to use underhand tactics to get chocolate bars from people performing poorly-designed surveys on railway stations.

1 – I thought about changing the order, as they do impact on each other, but it made my head hurt, so I stopped.

2 – note for younger readers: there used to be something called “snail mail”. It’s nearly dead[3].

3 – unless you forget to turn on “electronic statements” for your bank account. Then you’ll get loads of it.

4 – whatever the answer to this is from a security point of view, the correct answer is “no”, because a) you’re going to annoy me by shouting it repeatedly down the phone because reception is so bad on the train that the recipient can’t hear it and b) because reception is so bad on the train that the recipient can’t hear it (see b)).

5 – I like chocolate.

6 – I’m not a big fan of phone conversations in railway carriages, to be honest.

7 – Or you’ve been sent a link to this because you are one of those family, friends or colleagues, and the person who sent you the link is sick and tired of doing all of your IT dirty work for you.

I got phished this week: what did I do?

I was a foolish – but was saved by my forward planning.

The first thing I did was not panic. The second was to move quickly.

But what happened to get to this stage, you may ask, and how could I have been so stupid? I’ll tell you the story.

Every day, like most people, I suspect, I get lots of emails[1]. I have a variety of email accounts, and although I’m sure that I should be more disciplined, I tend to just manage them as they come in. First thing in the morning, though, I tend to sit down with a cup of tea and go through what’s come in an manage what I can then. Most work emails that require more than a glance and a deletion[2] will wait until later in the day, but I like to deal with any home-related ones before breakfast.

The particular email I’m talking about came in overnight, and I was sitting down with my cup of tea[3] when I noticed an email from a company with whom I have a subscription. The formatting was what I’d expect, and it looked fine. It was asking me to change my payment details.

“Danger!” is what you’ll be thinking, and quite rightly. However, I had some reasons for thinking that I might need to do this. I’ve recently changed credit cards, and I was aware that there was quite a high likelihood that I’d used the old credit card to subscribe. What’s more, I had a hazy recollection that I’d first subscribed to this service about this time of year, so it might well be due for renewal.

Here’s where I got even more unlucky: I told myself I’d come back to it because I didn’t have my wallet with me (not having got dressed yet). This meant that I’d given myself a mental task to deal with the issue later in the day, and I think that this gave it a legitimacy in my head which it wouldn’t have got if I’d looked at it in the first place. I also mentioned to my wife that I needed to do this: another step which in my head gave the task more legitimacy.

So I filed the mail as “Unread”, and went off to have a proper breakfast. When I was dressed, I sat down and went back to the email. I clicked on the link to update, and here’s where I did the really stupid thing: I didn’t check the URL. What I really should have done was actually enter the URL I would have expected directly into the browser, but I didn’t. I was in a rush, and I wanted to get it done.

I tried my account details, and nothing much happened. I tried them again. And then I looked at the URL in the browser bar. That’s not right…

This was the point when I didn’t panic, but moved quickly. I closed the page in my browser with the phishing site, and I opened a new one, into which I typed the correct URL. I logged in with my credentials, and went straight to the account page, where I changed my password to a new, strong, machine-generated password. I checked to see that the rest of the account details – including payment details – hadn’t been tampered with. And I was done.

There’s something else that I did right, and this is important: I used a different set of account details (username and password) for this site to any other site to which I’m subscribed. I use a password keeper (there are some good ones on the market, but I’d strongly advise going with an open source one: that way you or others can be pretty sure that your passwords aren’t leaking back to whoever wrote or compiled it), and I’m really disciplined about using strong passwords, and never reusing them at all.

So, I think I’m safe. Let’s go over what I did right:

I didn’t panic. I realised almost immediately what had happened, and took sensible steps.
I moved quickly. The bad folks only had my credentials for a minute or so, as I immediately logged into the real site and changed my password.
I checked my account. No details had been changed.
I used a strong, machine-generated password.
I hadn’t reused the same password over several sites.

A few other things worked well, though they weren’t down to me:

the real site sent me an email immediately to note that I’d changed my login details. This confirmed that it was done (and I checked the provenance of this email!).
the account details on the real site didn’t list my full credit card details, so although the bad folks could have misused my subscription, they wouldn’t have had access to my credit card.

Could things have gone worse? Absolutely. Do I feel a little foolish? Yes. But hopefully my lesson is learned, and being honest will allow others to know what to do in the same situation. And I’m really, really glad that I used a password keeper.

1 – some of them, particularly the work ones, are from people expecting me to do things. These are the worst type.

2 – quite a few, actually – I stay subscribed to quite a few lists just to see what’s going on.

3 – I think it was a Ceylon Orange Pekoe, but I can’t remember now.

What’s an attack surface?

“Reduce your attack surface,” they say. But what is it?

“Reduce your attack surface,” they[1] say. But what is it? The instruction to reduce your attack surface is one of the principles of IT security, so it must be a Good Thing[tm]. The problem is that it’s not always clear what an attack surface actually is.

I’m going to go for the broadest possible description I can think of, or nearly, because I’m pretty paranoid, and because I’m not convinced that the Wikipedia definition[2] is sufficient[3]. Although I’ll throw in a few examples of how to reduce attack surfaces, the purpose of this post is really to explain what one is, rather than to help protect you – but a good understanding really is required before you start with anything else, so hopefully this will be useful.

So, here’s my start at a definition:

The attack surface of a system is the sum of areas where attacks could be launched against it.

That feels a little bit circular – let’s define some terms. First of all, what’s an an “area” in this definition? Well, I’d say that any particular component of a system may have many points of possible vulnerability – and therefore attack. The sum of those points is an area – and the sum of the areas of the different components of a system gives us our system’s attack surface.

To understand better, we’re going to have to talk about systems – one of my favourite topics[4] – because I think it’s important to clarify a key difference between the attack surface of a component considered alone, and the area that a component adds when part of a system. They will not generally be the same.

Here’s an example: you’re deploying an Operating System. Let’s look at two options for deployment, and compare the attack surfaces. In both cases, I’m going to take a fairly restricted look at points of vulnerability, excluding, for instance, human factors, as I don’t want to get bogged down in the details.

Deployment one – bare metal

You install your Operating System onto a physical machine, and plug it into the network. What are some of the attack points?

your network connection
the physical hardware
services which are listening on the network connection
connections via USB – keyboard and mouse, for example.

There are more, but this should give us enough to do some comparisons. I’d generally think of the attack surface as being associated with the physical bounds of the hardware, with the addition of the network port and USB connections.

How can we reduce the attack surface? Well, we could unplug the network connection – though that might significantly reduce the efficacy of the system! – or we might take steps to reduce the number of services listening on the connection, to reduce the privilege level at which they run, or increase the authentication requirements for connecting to them. We could reduce our surface area by using a utility such as “usbguard” to restrict USB connections, and, if we’re worried about physical access to the machine, we could put it in a locked cabinet somewhere. These are all useful and appropriate ways to reduce our system’s attack surface.

Deployment two – a Virtual Machine

In this deployment scenario, we’re going to install the Operating System onto a Virtual Machine (VM), running on a physical host. What does my attack surface look like now? Well, that rather depends on how you define your system. You could, of course, look at the wider system – the VM and the physical host – but for the purposes of this discussion, I’m going to consider that the operation of the Operating System is what we’re interested in, rather than the broader system[6]. So, what does our attack surface look like this time? Here’s a quick list.

your network connection
the hypervisor
services which are listening on the network connection
connections via USB – keyboard and mouse, for example.

You’ll notice that “the physical hardware” is missing from this list, and that’s because it’s been replace with “the hypervisor”. This is a little simplistic, for a few reasons, including that the hypervisor is arguably implemented via a combination of software and hardware controls, but it’s certainly different from the entire physical hardware we were talking about before, and in fact, there’s not much you can do from the point of the Virtual Machine to secure it, other than recognise its restrictions, so we might want to remove it from our list at this level.

The other entries are also somewhat different from our first scenario, although you might not realise at first glance. First, it’s quite likely (though not certain) that your network connection may in fact be a virtual network connection provided by the hosting system, which means that some of the burden of defending it goes to the hosting system. The same goes for the connections via USB – the hypervisor generally provides “virtual hardware” (via something like qemu, for example), which can be attached – or removed – from virtual machines.

So, you still have the services which are listening on the network connection, but it’s definitely a different attack surface from the first deployment scenario.

Now, if you take the wider view, then there’s definitely an attack surface at the physical machine level as well, and that needs to be considered – but it’s quite likely that this will be under the control of somebody completely different (such as a Cloud Service Provider – CSP).

Another quick example

When I deploy a webserver (using, for instance, Apache), I’ll need to consider a variety of attack vectors, from authentication to denial of service to storage attacks: these are part of our attack surface. If I deploy it with a database (e.g. PostgreSQL or MySQL), the attack surface looks different, assuming that I care about the data in the database. Whereas I might previously have been concerned to ensure that an HTTP “PUT” command didn’t overwrite or scramble a file on my filesystem, a malformed command to my database server could delete or corrupt multiple tables. On the other hand, I might now be able to lock down some of the functions of my webserver that I no longer need to worry about filesystem attacks. The attack surface of my webserver is different when it’s combined in a system with other components[7].

Why do I want to reduce my attack surface?

Well, this is quite an easy one. By looking back at my earlier definition, you’ll see that the smaller a system’s attack surface, the fewer points of attack there are available to malicious actors. That’s got to be a piece of good news.

You will, of course, never be able to reduce your attack surface to zero (see There are no absolutes in security), but the more you reduce (and document, always document!), the better position you’ll be in. It’s always about raising the bar to make it more difficult for malicious actors to affect you.

1 – the mythical IT Security Community, that’s who.

2 – to give one example.

3 – it only talks about data, and only about software: that’s not broad enough for me.

4 – as long-standing[4] readers of this blog will know.

5 – and long-suffering.

6 – yes, I know we can’t ignore that, but we’ll come back to it, honest.

7 – there are considerations around the attack surface of the database as well, of course.

Defending our homes

Your router is your first point of contact with the Internet: how insecure is it?

I’ve always had a problem with the t-shirt that reads “There’s no place like 127.0.0.1”. I know you’re supposed to read it “home”, but to me, it says “There’s no place like localhost”, which just doesn’t have the same ring to it. And in this post, I want to talk about something broader: the entry-point to your home network, which for most people will be a cable or broadband router[1]. The UK and US governments just published advice that “Russia”[2] is attacking routers. This attack will be aimed mostly, I suspect, at organisations (see my previous post What’s a State Actor, and should I care?), rather than homes, but it’s a useful wake-up call for all of us.

What do routers do?

Routers are important: they provide the link between one network (in this case, our home network) and another one (in this case, the Internet, via our ISP’s network. In fact, for most of us, the box we think of as “the router”[3] is doing a lot more than that. The “routing” bit is what is sounds like: it helps computers on your network to find routes to send data to computers outside the network – and vice-versa, for when you’re getting data back. But most routers will actual be doing more than that. The other purpose that many will be performing is that of a modem. Most of us [4] connect to the Internet via a phoneline – whether cable or standard landline – though there is a growing trend for mobile Internet to the home. Where you’re connecting via a phone line, there’s a need to convert the signals that we use for the Internet to something else and then (at the other end) back again. For those of us old enough to remember the old “dial-up” days, that’s what the screechy box next to your computer used to do.

But routers often do more things as, well. Sometimes many more things, including traffic logging, being an WiFi access point, providing a VPN for external access to your internal network, child access, firewalling and all the rest.

Routers are complex things these days, and although state actors may not be trying to get into them, other people may.

Does this matter, you ask? Well, if other people can get into your system, they have easy access to attacking your laptops, phones, network drives and the rest. They can access and delete unprotected personal data. They can plausibly pretend to be you. They can use your network to host illegal data or launch attacks on others. Basically, all the bad things.

Luckily, routers tend to come set up by your ISP, with the implication being that you can leave them, and they’ll be nice and safe.

So we’re safe, then?

Unluckily, we’re really not.

The first problem is that the ISPs are working on a budget, and it’s in their best interests to provide cheap kit which just does the job. The quality of ISP-provided routers tends to be pretty terrible. It’s also high on the list of things to try to attack by malicious actors: if they know that a particular router model will be installed in a several million homes, there’s a great incentive to find an attack, as an attack on that model will be very valuable to them.

Measures to take

Here’s a quick list of steps you can take to try to improve the security of your first hop to the Internet. I’ve tried to order them in terms of ease – simplest first. Before you do any of these, however, save the configuration data so that you can bring it back if you need it.

Passwords – always, always, always change the admin password for your router. It’s probably going to be one that you rarely use, so you’ll want to record it somewhere. This is one of the few times where you might want to consider taping it to the router itself, as long as the router is in a secure place where only authorised people (you and your family[5]) have access.
Internal admin access only – unless you have very good reasons, and you know what you’re doing, don’t allow machines to administer the router unless they’re on your home network. There should be a setting on your router for this.
Wifi passwords – once you’ve done 2., you need to ensure that wifi passwords on your network – whether set on your router or elsewhere – are strong. It’s easy to set a “friendly” password so that it’s easy for visitors to connect to your network, but if it’s guessed by a malicious person who happens to be nearby, the first thing they’ll do will be to look for routers on the network, and as they’re on the internal network they’ll have access to it (hence why 1 is important).
Only turn on functions that you understand and need – as I noted above, modern routers have all sorts of cool options. Disregard them. Unless you really need them, and you actually understand what they do, and what the dangers of turning them on are, then leave them off. You’re just increasing your attack surface.
Buy your own router – replace your ISP-supplied router with a better one. Go to your local computer store and ask for suggestions. You can pay an awful lot, but you can conversely get something fairly cheap that does the job and is more robust, performant and easy to secure than the one you have at the moment. You may also want to buy a separate modem. Generally setting up your own modem or router is simple, and you can copy the settings from the ISP-supplied one and it will “just work”.
Firmware updates – I’d love to have this further up the list, but it’s not always easy. From time to time, firmware updates appear for your router. Most routers will check automatically, and may prompt you to update when you next log in. The problem is that failure to update correctly can cause catastrophic results[6], or lose configuration data that you’ll need to re-enter. But you really do need to consider doing this, and keeping a look-out of firmware updates which fix severe security issues.
Go open source – there are some great open source router projects out there which allow you to take an existing router and replace all of the firmware/software on it with an open source alternative. You can find a list of at least some of them on Wikipedia – https://en.wikipedia.org/wiki/List_of_router_firmware_projects, and a search on “router” on Opensource.com will open your eyes to a set of fascinating opportunities. This isn’t a step for the faint-hearted, as you’ll definitely void the warranty on your existing router, but if you want to have real control, open source is always the way to go.

Other issues…

I’d love to pretend that once you’ve improved the security of your router, that all’s well and good, but it’s not on your home network.. What about IoT devices in your home (Alexa, Nest, Ring doorbells, smart lightbulbs, etc.?) What about VPNs to other networks? Malicious hosts via Wifi, malicious apps on your childrens phones…?

No – you won’t be safe. But, as we’ve discussed before, although there is no “secure”, that doesn’t mean that we shouldn’t raise the bar and make it harder for the Bad Folks[tm].

1 – I’m simplifying – but read on, we’ll get there.

2 -“Russian State-Sponsored Cyber Actors”

3 – or, in my parents’ case, “the Internet box”, I suspect.

4 – this is one of these cases where I don’t want comments telling me how you have a direct 1 Terabit/s connection to your local backbone, thank you very much.

5 – maybe not the entire family.

6 – your router is now a brick, and you have no access to the Internet.

Do you know what’s lurking on your system?

Every utility, every library, every executable … increases your attack surface.

I had a job once which involved designing hardening procedures for systems that we were going to be use for security-related projects. It was fascinating. This was probably 15 years ago, and not only were guides a little thin on the ground for the Linux distribution I was using, but what we were doing was quite niche. At first, I think I’d assumed that I could just write a script to close down a few holes that originated from daemons[1] that had been left running for no reasons: httpd, sendmail, stuff like that. It did involve that, of course, but then I realised there was more to do, and started to dive down the rabbit hole.

So, after those daemons, I looked at users and groups. And then at file systems, networking, storage. I left the two scariest pieces to last, for different reasons. The first was the kernel. I ended up hand-crafting a kernel, removing anything that I thought it was unlikely we’d need, and then restarting several times when I discovered that the system wouldn’t boot because the things I thought I understood were more … esoteric than I’d realised. I’m not a kernel developer, and this was a salutary lesson in quite how skilled those folks are. At least, at the time I was doing it, there was less code, and fewer options, than there are today. On the other hand, I was having to hack back to a required state, and there are more cut-down kernels and systems to start with than there were back then.

The other piece that I left for last was just pruning the installed Operating System applications and associated utilities. Again, there are cut-down options that are easier to use now than then, but I also had some odd requirements – I believe that we needed Java, for instance, which has, or had …. well let’s say a lot of dependencies. Most modern Linux distributions[3] start off by installing lots of pieces so that you can get started quickly, without having to worry about trying to work out dependencies for every piece of external software you want to run.

This is understandable, but we need to realise when we do this that we’re making a usability/security trade-off[5]. Every utility, every library, every executable that you add to a system increases your attack surface, and increases the likelihood of vulnerabilities.

The problem isn’t just that you’re introducing vulnerabilities, but that once they’re there, they tend to stay there. Not just in code that you need, but, even worse, in code that you don’t need. It’s a rare, but praiseworthy hacker[6] who spends time going over old code removing dependencies that are no longer required. It’s a boring, complex task, and it’s usually just easier to leave the cruft[7] where it is and ship a slightly bigger package for the next release.

Sometimes, code is refactored and stripped: most frequently, security-related code. This is a very Good Thing[tm], but it turns out that it’s far from sufficient. The reason I’m writing this post is because of a recent story in The Register about the “beep” command. This command used the little speaker that was installed on most PC-compatible motherboards to make a little noise. It was a useful little utility back in the day, but is pretty irrelevant now that most motherboards don’t ship with the relevant hardware. The problem[8] is that installing and using the beep command on a system allows information to be leaked to users who lack the relevant permissions. This can be a very bad thing. There’s a good, brief overview here.

Now, “beep” isn’t installed by default on the distribution I’m using on the system on which I’m writing this post (Fedora 27), though it’s easily installable from one of the standard repositories that I have enabled. Something of a relief, though it’s not a likely attack vector for this machine anyway.

What, though, do I have installed on this system that is vulnerable, and which I’d never have thought to check? Forget all of the kernel parameters which I don’t need turned on, but which have been enabled by the distribution for ease of use across multiple systems. Forget the apps that I’ve installed and use everyday. Forget, even the apps that I installed once to try, and then neglected to remove. What about the apps that I didn’t even know were there, and which I never realised might have an impact on the security posture of my system? I don’t know, and have little way to find out.

This system doesn’t run business-critical operations. When I first got it and installed the Operating System, I decided to err towards usability, and to be ready to trash[9] it and start again if I had problems with compromise. But that’s not the case for millions of other systems out there. I urge you to consider what you’re running on a system, what’s actually installed on it, and what you really need. Patch what you need, remove what you don’t. It’s time for a Spring clean[10].

1- I so want to spell this word dæmons, but I think that might be taking my Middle English obsession too far[2].

2 – I mentioned that I studied Middle English, right?

3 – I’m most interested in (GNU) Linux here, as I’m a strong open source advocate and because it’s the Operating System that I know best[4].

4 – oh, and I should disclose that I work for Red Hat, of course.

5 – as with so many things in our business.

6 – the good type, or I’d have said “cracker”, probably.

7 – there’s even a word for it, see?

8 – beyond a second order problem that a suggested fix seems to have made the initial problem worse…

9 – physically, if needs be.

10 – In the Northern Hemisphere, at least.

Meltdown and Spectre: thinking about embargoes and disclosures

The technical details behind Meltdown and Spectre are complex and fascinating – and the possible impacts wide-ranging and scary. I’m not going to go into those here, though there are some excellent articles explaining them. I’d point readers in particular at the following URLs (which both resolve to the same page, so there’s no need to follow both):

http://meltdownattack.com/ and http://spectreattack.com/

I’d also recommend this article on the Red Hat blog, written by my colleague Jon Masters: What are Meltdown and Spectre? Here’s what you need to know. It includes a handy cut-out-and-keep video[1] explaining the basics. Jon has been intimately involved in the process of creating and releasing mitigations and fixes to Meltdown and Spectre, and is very well-placed to talk about it.

All that is interesting and important, but I want to talk about the mechanics and process behind how a vulnerability like this moves from discovery through to fixing. Although similar processes exist for proprietary software, I’m most interested in open source software, so that’s what I’m going to describe.

Step 1 – telling the right people

Let’s say that someone has discovered a vulnerability in a piece of open source software. There are a number of things they can do. The first is ignore it. This is no good, and isn’t interesting for our story, so we’ll ignore it in turn. Second is to keep it to themselves or try to make money out of it. Let’s assume that the discoverer is a good type of person[2], so isn’t going to do this[3]. A third is to announce it on the project mailing list. This might seem like a sensible thing to do, but it can actually be very damaging to the project and the community at large. The problem with this is that the discoverer has just let all the Bad Folks[tm] know about a vulnerability which hasn’t been fixed, and they can now exploit. Even if the discoverer submits a patch, it needs to be considered and tested, and it may be that the fix they’ve suggested isn’t the best approach anyway. For a serious security issue, the project maintainers are going to want to have a really good look at it in order to decide what the best fix – or mitigation – should be.

It’s for this reason that many open source projects have a security disclosure process. Typically, there’s a closed mailing list to which you can send suspected vulnerabilities, often something like “security@projectname.org“. Once someone has emailed that list, that’s where the process kicks in.

Step 2 – finding a fix

Now, before that email got to the list, somebody had to decide that they needed a list in the first place, so let’s assume that you, the project leader, has already put a process of some kind in place. There are a few key decisions to make, of which the most important are probably:

are you going to keep it secret[5]? It’s easy to default to “yes” here, for the reasons that I mentioned above, but the number of times you actually put in place restrictions on telling people about the vulnerability – usually referred to as an embargo, given its similarity to a standard news embargo – should be very, very small. This process is going to be painful and time-consuming: don’t overuse it. Oh, and your project’s meant to be about open source software, yes? Default to that.
who will you tell about the vulnerability? We’re assuming that you ended up answering “yes” to te previous bullet, and that you don’t want everybody to know, and also that this is a closed list. So, who’s the most important set of people? Well, most obvious is the set of project maintainers, and key programmers and testers – of which more below. These are the people who will get the fix completed, but there are two other constituencies you might want to consider: major distributors and consumers of the project. Why these people, and who are they? Well, distributors might be OEMs, Operating System Vendors or ISVs who bundle your project as part of their offering. These may be important because you’re going to want to ensure that people who will need to consume your patch can do so quickly, and via their preferred method of updating. What about major consumers? Well, if an organisation has a deployed estate of hundreds or thousands of instances of a project[6], then the impact of having to patch all of those at short notice – let alone the time required to test the patch – may be very significant, so they may want to know ahead of time, and they may be quite upset if they don’t. This privileging of major over rank-and-file users of your project is politically sensitive, and causes much hand-wringing in the community. And, of course, the more people you tell, the more likely it is that a leak will occur, and news of the vulnerability to get out before you’ve had a chance to fix it properly.
who should be on the fixing team? Just because you tell people doesn’t mean that they need to be on the team. By preference, you want people who are both security experts and also know your project software inside-out. Good luck with this. Many projects aren’t security projects, and will have no security experts attached – or a few at most. You may need to call people in to help you, and you may not even know which people to call in until you get a disclosure in the first place.
how long are you going to give yourself? Many discoverers of vulnerabilities want their discovery made public as soon as possible. This could be for a variety of reasons: they have an academic deadline; they don’t want somebody else to beat them to disclosure; they believe that it’s important to the community that the project is protected as soon as possible; they’re concerned that other people have found – and/or are exploiting – the vulnerability; they don’t trust the project to take the vulnerability seriously and are concerned that it will just be ignored. This last reason is sadly justifiable, as there are projects who don’t react quickly enough, or don’t take security vulnerabilities seriously, and there’s a sorry history of proprietary software vendors burying vulnerabilities and pretending they’ll just go away[7]. A standard period of time before disclosure is 2-3 weeks, but as projects get bigger and more complex, balancing that against issues such as pressure from those large-scale users to give them more time to test, holiday plans and all the rest becomes important. Having an agreed time and sticking to it can be vital, however, if you want to avoid deadline slip after deadline slip. There’s another interesting issue, as well, which is relevant to the Meltdown and Spectre vulnerabilities – you can’t just patch hardware. Where hardware is involved, the fixes may involve multiple patches to multiple projects, and not just standard software but microcode as well: this may significantly increase the time needed.
what incentives are you going to provide? Some large projects are in a position to offer bug bounties, but these are few and far between. Most disclosers want – and should expect to be granted – public credit when the fix is finally provided and announced to the public at large. This is not to say that disclosers should necessarily be involved in the wider process that we’re describing: this can, in fact, be counter-productive, as their priorities (around, for instance, timing) may be at odds with the rest of the team.

There’s another thing you might want to consider, which is “what are we going to do if this information leaks early?” I don’t have many good answers for this one, as it will depend on numerous factors such as how close are you to a fix, how major is the problem, and whether anybody in the press picks up on it. You should definitely consider it, though.

Step 3 – external disclosure

You’ve come up with a fix? Well done. Everyone’s happy? Very, very well done[8]. Now you need to tell people. But before you do that, you’re going to need to decide what to tell them. There are at least three types of information that you may well want to prepare:

technical documentation – by this, I mean project technical documentation. Code snippets, inline comments, test cases, wiki information, etc., so that when people who code on your project – or code against it – need to know what’s happened, they can look at this and understand.
documentation for techies – there will be people who aren’t part of your project, but who use it, or are interested in it, who will want to understand the problem you had, and how you fixed it. This will help them to understand the risk to them, or to similar software or systems. Being able to explain to these people is really important.
press, blogger and family documentation – this is a category that I’ve just come up with. Certain members of the press will be quite happy with “documentation for techies”, but, particularly if your project is widely used, many non-technical members of the press – or technical members of the press writing for non-technical audiences – are going to need something that they can consume and which will explain in easy-to-digest snippets a) what went wrong; b) why it matters to their readers; and c) how you fixed it. In fact, on the whole, they’re going to be less interested in c), and probably more interested in a hypothetical d) & e) (which are “whose fault was it and how much blame can we heap on them?” and “how much damage has this been shown to do, and can you point us at people who’ve suffered due to it?” – I’m not sure how helpful either of these is). Bloggers may also be useful in spreading the message, so considering them is good. And generally, I reckon that if I can explain a problem to my family, who aren’t particularly technical, then I can probably explain it to pretty much anybody I care about.

Of course, you’re now going to need to coordinate how you disseminate all of these types of information. Depending on how big your project is, your channels may range from your project code repository through your project wiki through marketing groups, PR and even government agencies[9].

Conclusion

There really is no “one size fits all” approach to security vulnerability disclosures, but it’s an important consideration for any open source software project, and one of those that can be forgotten as a project grows, until suddenly you’re facing a real use case. I hope this overview is interesting and useful, and I’d love to hear thoughts and stories about examples where security disclosure processes have worked – and when they haven’t.

1 – if you find a way to cut it out and keep it, do let me know.

2 – because only good people read this blog, right?

3 – some government security agencies have a policy of collecting – and not disclosing – security vulnerabilities for their own ends. I’m not going to get into the rights or wrongs of this approach.[4]

4 – well, not in this post, anyway.

5 – or try, at least.

6 – or millions or more – consider vulnerabilities in smartphone software, for instance, or cloud providers’ install bases.

7 – and even, shockingly, of using legal mechanisms to silence – or try to silence – the discloser.

8 – for the record, I don’t believe you: there’s never been a project – let alone an embargo – where everyone was happy. But we’ll pretend, shall we?

9 – I really don’t think I want to be involved in any of these types of disclosure, thank you.

Top 5 resolutions for security folks – 2018

Yesterday, I wrote some jokey resolutions for 2018 – today, as it’s a Tuesday, my regular day for posts, I decided to come up with some real ones.

1 – Embrace the open

I’m proud to have been using Linux[1] and other open source software for around twenty years now. Since joining Red Hat in 2016, and particularly since I started writing for Opensource.com, I’ve become more aware of other areas of open-ness out there, from open data to open organisations. There are still people out there who are convinced that open source is less secure than proprietary software. You’ll be unsurprised to discover that I disagree. I encourage everyone to explore how embracing the open can benefit them and their organisations.

2 – Talk about risk

I’m convinced that we talk too much about security for security’s sake, and not about risk, which is what most “normal people” think about. There’s education needed here as well: of us, and of others. If we don’t understand the organisations we’re part of, and how they work, we’re not going to be able to discuss risk sensibly. In the other direction, we need to be able to talk about security a bit, in order to explain how it will mitigate risk, so we need to learn how to do this in a way that informs our colleagues, rather than alienating them.

3 – Think about systems

I don’t believe that we[2] talk enough about systems. We spend a lot of our time thinking about functionality and features, or how “our bit” works, but not enough about how all the bits fit together. I don’t often link out to external sites or documents, but I’m going to make an exception for NIST special publication 800-160 “Systems Security Engineering: Considerations for a Multidisciplinary Approach in the Engineering of Trustworthy Secure Systems”, and I particularly encourage you to read Appendix E “Roles, responsibilities and skills: the characteristics and expectations of a systems security engineer”. I reckon this is an excellent description of the core skills and expertise required for anyone looking to make a career in IT security.

4 – Examine the point of conferences

I go to a fair number of conferences, both as an attendee and as a speaker – and also do my share of submission grading. I’ve written before about how annoyed I get (and I think others get) by product pitches at conferences. There are many reasons to attend the conferences, but I think it’s important for organisers, speakers and attendees to consider what’s most important to them. For myself, I’m going to try to ensure that what I speak about is what I think other people will be interested in, and not just what I’m focussed on. I’d also highlight the importance of the “hallway track”: having conversations with other attendees which aren’t necessarily directly related to the specific papers or talks. We should try to consider what conferences we need to attend, and which ones to allow to fall by the wayside.

5 – Read outside the IT security discipline

We all need downtime. One way to get that is to read – on an e-reader, online, on your phone, magazines, newspapers or good old-fashioned books. Rather than just pick up something directly related to work, choose something which is at least a bit off the beaten track. Whether it’s an article on a topic to do with your organisation’s business, a non-security part of IT[3], something on current affairs, or a book on a completely unrelated topic[4], taking the time to gain a different perspective on the world is always[5] worth it.

What have I missed?

I had lots of candidates for this list, and I’m sure that I’ve missed something out that you think should be in there. That’s what comments are for, so please share your thoughts.

1 GNU Linux.

2 the mythical IT community

3 – I know, it’s not going to be as sexy as security, but go with it. At least once.

4 – I’m currently going through a big espionage fiction phase. Which is neither here nor there, but hey.

5 – well, maybe almost always.

Who’s saying “hello”? – agency, intent and AI

Who is saying “hello world?”: you, or the computer?

I don’t yet have one of those Google or Amazon talking speaker thingies in my house or office. A large part of this is that I’m just not happy about the security side: I know that the respective companies swear that they’re only “listening” when you say the device’s trigger word, but even if that’s the case, I like to pretend[1] that I have at least some semblance of privacy in my life. Another reason, however, is that I’m not sure that I like what happens to people when they pretend that there’s a person listening to them, but it’s really just a machine.

It’s not just Alexa and the OK, Google persona, however. When I connect to an automated phone-answering service, I worry when I hear “I’ll direct your call” from a non-human. Who is “I”? “We’ll direct your call” is better – “we” could be the organisation with whom I’m interacting. But “I”? “I” is the pronoun that people use. When I hear “I”, I’m expecting sentience: if it’s a machine I’m expecting AI – preferably fully Turing-compliant.

There’s a more important point here, though. I’m entirely aware that there’s no sentience behind that “I”[2], but there’s an important issue about agency that we should unpack.

What, then, is “agency”? I’m talking about the ability of an entity to act on its or another’s behalf, and I touched on this this in a previous post, “Wow: autonomous agents!“. When somebody writes some code, what they’re doing is giving ability to the system that will run that code to do something – that’s the first part. But the agency doesn’t really occur, I’d say, until that code is run/instantiated/executed. At this point, I would argue, the software instance has agency.

But whose agency, exactly? For whom is this software acting?

Here are some answers. I honestly don’t think that any of them is right.

the person who owns the hardware (you own the Alexa hardware, right? You paid Amazon for it… Or what about running applications on the cloud?).
the person who started the software (you turned on the Alexa hardware, which started the software… And don’t forget software which is automatically executed in response to triggers or on a time schedule.)
the person who gave the software the instructions (what do you mean, “gave it the instructions”? Wrote its config file? Spoke to it? Set up initial settings? Typed in commands? And even if you gave it instructions, do you think that your OK Google hardware is implementing your wishes, or Google’s? For whom is it actually acting? And what side effects (like recording your search history and deciding what to suggest in your feed) are you happy to believe are “yours”?)
the person who installed the software (your phone comes with all sorts of software installed, but surely you are the one who imbues it with agency? If not, whom are you blaming: Google (for the Android apps) or Samsung (which actually put them on the phone)?)
the person who wrote the software (I think we’ve already dealt with this, but even then, is it a single person, or an organisation? What about open source software, which is typically written, compiled and documented by many different people? Ascribing “ownership” or “authorship” is a distinctly tricky (and intentionally tricky) issue when you discuss open source)

Another way to think of this problem is to ask: when you write and execute a program, who is saying “hello world?”: you, or the computer?

There are some really interesting questions that come out of this. Here are a couple that come to mind, which seem to me to be closely connected.

In the film Wargames[3], is the automatic dialling that Matthew Broderick’s character’s computer carries out an act with agency? Or is it when it connects to another machine? Or when it records the details of that machine? I don’t think anyone would argue that the computer is acting with agency once David Lightman actually gets it to complete a connection and interact with it, but what about before?
Google used to run automated programs against messages received as part of the Gmail service looking for information and phrases which it could use to serve ads. They were absolutely adamant that they, Google, weren’t doing the reading: it was just a computer program. I’m not sure how clear or safe a distinction that is.

Why does this all matter? Well, one of the more pressing reasons is because of self-driving cars. Whose fault is it when one goes wrong and injures or kills someone? What about autonomous defence systems?

And here’s the question that really interests – and vexes – me: is this different when the program which is executing can learn. I don’t even mean strong AI: just that it can change what it does based on the behaviour it “sees”, “hears” or otherwise senses. It feels to me that there’s a substantive difference between:

a) actions carried out at the explicit (asynchronous) request of a human operator, or according to sets of rules coded into a program

AND

b) actions carried out in response to rules that have been formed by the operation of the program itself. There is what I’d called synchronous intent within the program.

You can argue that b) has pretty much always been around, in basic forms, but it seems to me to be different when programs are being created with the expectation that humans will not necessarily be able to decode the rules, and where the intent of the human designers is to allow rulesets to be created in this way.

There is some discussion about at the moment as to how and/or whether rulesets generated by open source projects should be shared. I think the general feeling is that there’s no requirement for them to be – in the same way that material I write using an open source text editor shouldn’t automatically be considered open source – but open data is valuable, and finding ways to share it is a good idea, IMHO.

In Wargames, that is the key difference between the system as originally planned, and what it ends up doing: Joshua has synchronous intent.

I really don’t think this is all bad: we need these systems, and they’re going to improve our lives significantly. But I do feel that it’s important that you and I start thinking hard about what is acting for whom, and how.

Now, if you wouldn’t mind opening the Pod bay doors, HAL…[5]

1. and yes, I know it’s a pretense.

2. yet…

3. go on – re-watch it: you know you want to[4].

4. and if you’ve never watched it, then stop reading this article and go and watch it NOW.

5. I think you know the problem just as well as I do, Dave.