Mike Bursell

Confessions of an auditor

Moving to a postive view of security auditing.

So, right up front, I need to admit that I’ve been an auditor. A security auditor. Professionally. It’s time to step up and be proud, not ashamed. I am, however, dialled into a two-day workshop on compliance today and tomorrow, so this is front and centre of my thinking right at the moment. Hence this post.

Now, I know that not everybody has an entirely … positive view of auditors. And particularly security auditors[1]. They are, for many people, a necessary evil[2]. The intention of this post is to try to convince you that auditing[3] is, or can and should be, a Force for Good[tm].

The first thing to address is that most people have auditors in because they’re told that they have to, and not because they want to. This is often – but not always – associated with regulatory requirements. If you’re in the telco space, the government space or the financial services space, for instance, there are typically very clear requirements for you to adhere to particular governance frameworks. In order to be compliant with the regulations, you’re going to need to show that you meet the frameworks, and in order to do that, you’re going to need to be audited. For pretty much any sensible auditing regime, that’s going to require an external auditor who’s not employed by or otherwise accountable to the organisation being audited.

The other reason that audit may happen is when a C-level person (e.g. the CISO) decides that they don’t have a good enough idea of exactly what the security posture of their organisation – or specific parts of it – is like, so they put in place an auditing regime. If you’re lucky, they choose an industry-standard regime, and/or one which is actually relevant to your organisation and what it does. If you’re unlucky, … well, let’s not go there.

I think that both of the reasons above – for compliance and to better understand a security posture – are fairly good ones. But they’re just the first order reasons, and the second order reasons – or the outcomes – are where it gets interesting.

Sadly, one of key outcomes of auditing seems to be blame. It’s always nice to have somebody else to blame when things go wrong, and if you can point to an audited system which has been compromised, or a system which has failed audit, then you can start pointing fingers. I’d like to move away from this.

Some positive suggestions

What I’d like to see would be a change in attitude towards auditing. I’d like more people and organisations to see security auditing as a net benefit to their organisations, their people, their systems and their processes[4]. This requires some changes to thinking – changes which many organisations have, of course, already made, but which more could make. These are the second order reasons that we should be considering.

Stop tick-box[5] auditing. Too many audits – particularly security audits – seem to be solely about ticking boxes. “Does this product or system have this feature?” Yes or no. That may help you pass your audit, but it doesn’t give you a chance to go further, and think about really improving what’s going on. In order to do this, you’re going to need to find auditors who actually understand the systems they’re looking at, and are trained to some something beyond tick-box auditing. I was lucky enough to be encouraged to audit in this broader way, alongside a number of tick-boxes, and I know that the people I was auditing always preferred this approach, because they told me that they had to do the other type, too – and hated it.
Employ internal auditors. You may not be able to get approval for actual audit sign-off if you use internal auditors, so internal auditors may have to operate alongside – or before – your external auditors, but if you find and train people who can do this job internally, then you get to have people with deep knowledge (or deeper knowledge, at least, than the external auditors) of your systems, people and processes looking at what they all do, and how they can be improved.
Look to be proactive, not just reactive. Don’t just pick or develop products, applications and systems to meet audit, and don’t wait until the time of the audit to see whether they’re going to pass. Auditing should be about measuring and improving your security posture, so think about posture, and how you can improve it instead.
Use auditing for risk-based management. Last, but not least – far from least – think about auditing as part of your risk-based management planning. An audit shouldn’t be something you do, pass[6] and then put away in a drawer. You should be pushing the results back into your governance and security policy model, monitoring, validation and enforcement mechanisms. Auditing should be about building, rather than destroying – however often it feels like it.

1 – you may hear phrases like “a special circle of Hell is reserved for…”.

2 – in fact, many other people might say that they’re an unnecessary evil.

3 – if not auditors.

4 – I’ve been on holiday for a few days: I’ve maybe got a little over-optimistic while I’ve been away.

5 – British usage alert: for US readers, you’ll probably call this a “check-box”.

6 – You always pass every security audit first time, right?

The Other CIA: Confidentiality, Integrity and Availability

Any type of even vaguely useful system will hold, manipulate or use data.

I spend a lot of my time on this blog talking about systems, because unless you understand how your systems work as a set of components, you’re never going to be able to protect and manage them. Today, however, I want to talk about security of data – the data in the systems. Any type of even vaguely useful system will hold, manipulate or use data in some way or another[1], and as I’m interested[2] in security, I think it’s useful to talk about data and data security. I’ve touched on this question in previous articles, but one recent one, What’s a State Actor, and should I care? had a number of people asking me for more detail on some of the points I raised, and as one of them was the classic “C.I.A.” model around data security, I thought I’d start there.

The first point I should make is that the “CIA triad” is sometimes over-used. You really can’t reduce all of information security to confidentiality, integrity and availability – there are a number of other important principles to consider. These three arguably don’t even cover all the issues you’d want to consider around data security – what, for instance, about data correctness and consistency, for example? – but I’ve always found them to be a useful starting point, so as long as we don’t kid ourselves into believing that they’re all we need, they are useful to hold in mind. They are, to use a helpful phrase, “necessary but not sufficient”.

We should also bear in mind that for any particular system, you’re likely to have various types and sets of data, and these types and sets may have different requirements. For instance, a database may store not only key data about, for instance, museum exhibits, but will also store data about who can update the key data, and even metadata about that – this might[3] include information about a set of role-based access controls (RBAC), and the security requirements for this will be different to the security requirements for thee key data. So, when we’re considering the data security requirements of a system, don’t assume that they will be uniform across all data sets.

Confidentiality

Confidentiality is quite an easy one to explain. If you don’t want everybody to be able to see a set of data, then you wish it to be confidential with regards to at least some entities – whether they be people or systems entities, internal or external. There are a number of ways to implement confidentiality, the most obvious being encryption of data, but there are other approaches, of which the easiest is just denying access to data through physical, geographical or other authorisation controls.

When you might care that data is confidentiality-protected: health records, legal documents, credit card details, financial information, firewall rules, system administrator rights, passwords.

When you might not care that data is confidentiality-protected: sports records, examination results, open source code, museum exhibit information, published company financial results.

Integrity

Integrity, when used as a term in this context, is slightly different to its standard usage. The property we’re looking for is not the same integrity that we expect[4] from our politicians, but is that data has not been changed from what it should be. Data is often useless unless it can be changed – we want to update information about our museum exhibits from time to time, for instance – but who can change it, and when, are the sort of things we want to control. Equally important may be the type of changes that can be made to it: if I have a careful classification scheme for my Tudor music manuscripts, I don’t want somebody putting in binary data which means nothing to me or our visitors.

I struggled to think of any examples when you wouldn’t want to protect the integrity of your data from at least some entities, as if data can be changed willy-nilly[5], it seems be worthless. It did occur to me, however, that as long as you have integrity-protected records of what has been changed, you’re probably OK. That’s the model for some open source projects or collaborative writing endeavours, for example.

[Discursion – Open source projects don’t generally allow you to scribble directly onto the main “approved” store – whose integrity is actually very important. That’s why software projects – proprietary or open source – have for decades used source control systems or versioning systems. One of the success criteria for scaling an open source project is a consensus on integrity management.]

Availability

Availability is the easiest of the triad to ignore when you’re designing a system. When you create data, it’s generally useless unless the entities that need it can get to it when they need it. That doesn’t mean that all systems need to have 100% up-time, or even that particular data sets need to be available for 100% of the up-time of the system, but when you’re designing a system, you need to decide what’s going to be appropriate, and how to manage with degradation. Why degradation? Because one of the easiest ways to affect the availability of data is to slow down access to it – as described in another recent post What’s your availability? DoS attacks and more. If I’m using a mobile app to view information about museum exhibits in real-time, and it takes five minutes for me to access the description, then things aren’t any good. On the other hand, if there’s some degradation of the service, and I can only access the first paragraph of the description, with an apology for the inconvenience and a link to other information, that might be acceptable. From a different point of view, if I notice that somebody is attacking my museum system, but I can’t get into it with administrative access to lock it down or remove their access, that’s definitely bad.

As with integrity protection, it’s difficult to think of examples when availability protection isn’t important, but availability isn’t necessarily a binary condition: it may vary from time to time.

Conclusion

Although they’re not perfect descriptions of all the properties you need to consider when designing in data security, confidentiality, integrity and availability should at least cause you to start thinking about what your data is for, how it should be accessed, how it should be changed, and by whom.

1 – I just know that somebody’s going to come up with a counter-example.

2 – And therefore assume that you, the reader, are interested.

3 – as a nested example, which is quite nice, as we’re talking about metadata.

4 – And far too rarely get, it seems.

5 – Not a rude phrase, even if it sounds like it should be. Look it up if you don’t believe me.

What’s your availability? DoS attacks and more

In security we talk about intentional degradation of availability

A colleague of mine recently asked me about protection from DoS attacks[1] for a project with which he’s involved – Denial of Service attacks. The first thing that sprung to mind, of course, was DDoS: Distributed Denial of Service attacks, where hundreds or thousands[2] of hosts are used to send vast amounts of network traffic to – or maybe more accurately “at” – servers in the hopes of bringing the servers to their knees and stopping them providing the service for which they’re designed. These are the attacks that get into the news, and with good reason.

There are other types of DoS however, and the more I thought about it, the more I wondered whether he – and I – should be worrying about these other DoS attacks and also considering other related types of issue which could cause problems to systems. And because I realised it was an interesting topic, I decided to write about it[3].

I’m going to return to the classic “C.I.A.” model of computer security: Confidentiality, Integrity and Availability. The attacks we’re talking about here are those most often overlooked: attempts to degrade the availability of a service. There’s an overlap with the related discipline of resilience here, but I think that the key differentiator is that in security we’re generally talking about intentional degradation of availability, whereas resilience also covers (and maybe focuses on) unintentional degradation.

So, what types of availability attacks might we want to consider?

Denial of service attacks

I think it’s worth linking to Wikipedia’s pretty awesome entry “Denial of service attack” – not something I often do, but I thought it was excellent. Although they’re not mutually exclusive at all, here are some of the key types as I’d define them:

Distributed DoS – where you have lots of different hosts attacking at the same time, flooding the target with traffic. These days, this can be easily automated, and it’s possible to rent compromised machines to perform a coordinated attack.
Application layer – where the attack is aimed at the service, rather than at the host beneath. This may seem like an academic distinction, but it’s not: what it really means is that the attack is performed with knowledge of the application layer. So, for instance, if you’re attacking a web server, you might initiate lots of HTTP sessions, or if you were attacking a Kerberos server, you might request lots of authentication tickets. These types of attacks may be quite costly to perform, but they’re also difficult to protect against, as each attack looks like a “legal” interaction with the service, and unless you’re on the look-out in a way which is typically not automated at this level, they’re difficult to avoid.
Host level – this is a family of attacks which go for the host and/or associated Operating System, rather than the service itself. A classic attack would be the SYN flood, which misused the TCP protocol to use up resources on the host, thereby stopping any associated services from being able to respond. Host attacks may be somewhat simpler to defend against, as it’s easier to invest in logic to detect them at this level (or maybe “set of layers”, if we adopt the OSI model), and to correlate responses across different hosts. Firewalls and similar defences are also more likely to be able to be configured to help defend hosts which may be targeted.

Resource starvation

The term “resource starvation” most accurately refers[4] to situations where a process (or application) is denied sufficient CPU allocation to perform correctly. How could this occur? Well, it’s going to be rarer than in the DoS case, because in order to do it, you’re going to need some way to impact the underlying scheduling of the Operating System and/or virtualisation management (think hypervisor, typically). That would normally mean that you’d need pretty low-level access to the machine, but there is a family of attacks known as “noisy neighbour”[5] where workloads – VMs or containers, typically – use up so many resources that other workloads are starved.

However, partly because of this case, I’d argue that resource starvation can usefully be associated with other types of availability attacks which occur locally to the machine hosting the targeted service, which might be related to CPU, file descriptor, network or other resources.

Generally, noisy neighbour attacks can be fairly easily mitigated by controls in the Operating System or virtualisation manager, though, of course, compromised or malicious components at this layer are very difficult to manage.

Dependency blocking

I’m not sure what the best term for this type of attack is, but what I’m thinking of is attacks which impact a service by reducing or removing access to external services on which they depend – remote components, if you will. If, for instance, my web application requires access to a database, then an attack on that database – however performed – will impact my service. As almost any kind of service will have external dependencies these days[6], this is can be a very effective attack, as it allows knowledgeable attackers to target the weakest link in the “chain” of components that make up your service.

There are mitigations against some of these attacks – caching and later reconciliation/synching being one – but identifying and defending against these sorts of attacks depends largely on considering your service as a system, and realising the types of impact degradation of the different parts might have.

Conclusion – managed degradation

Which leads me to a final point, which is that when considering availability attacks, understanding and planning Service degradation: actually a good thing is going to be invaluable – and when you’ve done that, you’ll definitely going to need to test it, too (If it isn’t tested, it doesn’t work).

1 – yes, I checked the capitalisation – he wasn’t worried about DRDOS, MS-DOS or any of those lovely 80s era command line Operating Systems.

2 – or millions or more, these days.

3 – here, for the avoidance of doubt.

4 – I believe.

5 – you know my policy on spellings by now. I’m British, and we’ll keep it that way.

6 – unless you’re still using green-screen standalone machines to run your business, in which case either a) yikes or b) well done.

What’s a State Actor, and should I care?

The bad thing about State Actors is that they rarely adhere to an ethical code.

How do you know what security measures to put in place for your organisation and the systems that you run? As we’ve seen previously, There are no absolutes in security, so there’s no way that you’re ever going to make everything perfectly safe. But how much effort should you put in?

There are a number of parameters to take into account. One thing to remember is that there are always trade-offs with security. My favourite one is a three-way: security, cost and usability. You can, the saying goes, have two of the three, but not the third: choose security, cost or usability. If you want security and usability, it’s going to cost you. If you want a cheaper solution with usability, security will be reduced. And if you want security cheaply, it’s not going to be easily usable. Of course, cost stands in for time spent as well: you might be able to make things more secure and usable by spending more time on the solution, but that time is costly.

But how do you know what’s an appropriate level of security? Well, you need to think about what you’re protecting, the impact of it being:

exposed (confidentiality);
changed (integrity);
inaccessible (availability).

These are three classic types of control, often recorded as C.I.A.[1]. Who would be impacted? What mitigations might you put in place? What vulnerabilities might exist in the system, and how might they be exploited?

One of the big questions to ask alongside all of these is “who exactly might be wanting to attack my systems?” There’s a classic adage within security that “no system is secure against a sufficiently motivated and resourced attacker”. Luckily for us, there are actually very few attackers who fall into this category. Some examples of attackers might be:

insiders[3]
script-kiddies
competitors
trouble-makers
hacktivists[4]
… and more.

Most of these will either not be that motivated, or not particularly well-resourced. There are two types of attackers for whom that is not the case: academics and State Actors. The good thing about academics is that they have adhere to an ethical code, and shouldn’t be trying anything against your systems without your permission. The bad thing about State Actors is that they rarely adhere to an ethical code.

State Actors have the resources of a nation state behind them, and are therefore well-resourced. They are also generally considered to be well-motivated – if only because they have many people available to perform attack, and those people are well-protected from legal process.

One thing that State Actors may not be, however, is government departments or parts of the military. While some nations may choose to attack their competitors or enemies (or even, sometimes, partners) with “official” parts of the state apparatus, others may choose a “softer” approach, directing attacks in a more hands-off manner. This help may take a variety of forms, from encouragement, logistical support, tools, money or even staff. The intent, here, is to combine direction with plausible deniability: “it wasn’t us, it was this group of people/these individuals/this criminal gang working against you. We’ll certainly stop them from performing any further attacks[5].”

Why should this matter to you, a private organisation or public company? Why should a nation state have any interest in you if you’re not part of the military or government?

The answer is that there are many reasons why a State Actor may consider attacking you. These may include:

providing a launch point for further attacks
to compromise electoral processes
to gain Intellectual Property information
to gain competitive information for local companies
to gain competitive information for government-level trade talks
to compromise national infrastructure, e.g.
- financial
- power
- water
- transport
- telecoms
to compromise national supply chains
to gain customer information
as revenge for perceived slights against the nation by your company – or just by your government
and, I’m sure, many others.

All of these examples may be reasons to consider State Actors when you’re performing your attacker analysis, but I don’t want to alarm you. Most organisations won’t be a target, and for those that are, there are few measures that are likely to protect you from a true State Actor beyond measures that you should be taking anyway: frequent patching, following industry practice on encryption, etc.. Equally important is monitoring for possible compromise, which, again, you should be doing anyway. The good news is that if you might be on the list of possible State Actor targets, most countries provide good advice and support before and after the act for organisations which are based or operate within their jurisdiction.

1 – I’d like to think that they tried to find a set of initials for G.C.H.Q. or M.I.5., but I suspect that they didn’t[2].

2 – who’s “they” in this context? Don’t ask. Just don’t.

3 – always more common than we might think: malicious, bored, incompetent, bankrupt – the reasons for insider-related security issues are many and varied.

4 – one of those portmanteau words that I want to dislike, but find rather useful.

5 – yuh-huh, right.

Why I should have cared more about lifecycle

Every deployment is messy.

I’ve always been on the development and architecture side of the house, rather than on the operations side. In the old days, this distinction was a useful and acceptable one, and wasn’t too difficult to maintain. From time to time, I’d get involved with discussions with people who were actually running the software that I had written, but on the whole, they were a fairly remote bunch.

This changed as I got into more senior architectural roles, and particularly as I moved through some pre-sales roles which involved more conversations with users. These conversations started to throw up[1] an uncomfortable truth: not only were people running the software that I helped to design and write[3], but they didn’t just set it up the way we did in our clean test install rig, run it with well-behaved, well-structured data input by well-meaning, generally accurate users in a clean deployment environment, and then turn it off when they’re done with it.

This should all seem very obvious, and I had, of course, be on the receiving end of requests from support people who exposed that there were odd things that users did to my software, but that’s usually all it felt like: odd things.

The problem is that odd is normal. There is no perfect deployment, no clean installation, no well-structured data, and certainly very few generally accurate users. Every deployment is messy, and nobody just turns off the software when they’re done with it. If it’s become useful, it will be upgraded, patched, left to run with no maintenance, ignored or a combination of all of those. And at some point, it’s likely to become “legacy” software, and somebody’s going to need to work out how to transition to a new version or a completely different system. This all has major implications for security.

I was involved in an effort a few years ago to describe the functionality, lifecycle for a proposed new project. I was on the security team, which, for all the usual reasons[4] didn’t always interact very closely with some of the other groups. When the group working on error and failure modes came up with their state machine model and presented it at a meeting, we all looked on with interest. And then with horror. All the modes were “natural” failures: not one reflected what might happen if somebody intentionally caused a failure. “Ah,” they responded, when called on it by the first of the security to be able to form a coherent sentence, “those aren’t errors, those are attacks.” “But,” one of us blurted out, “don’t you need to recover from them?” “Well, yes,” they conceded, “but you can’t plan for that. It’ll need to be on a case-by-case basis.”

This is thinking that we need to stamp out. We need to design our systems so that, wherever possible, we consider not only what attacks might be brought to bear on them, but also how users – real users – can recover from them.

One way of doing this is to consider security as part of your resilience planning, and bake it into your thinking about lifecycle[5]. Failure happens for lots of reasons, and some of those will be because of bad people doing bad things. It’s likely, however, that as you analyse the sorts of conditions that these attacks can lead to, a number of them will be similar to “natural” errors. Maybe you could lose network connectivity to your database because of a loose cable, or maybe because somebody is performing a denial of service attack on it. In both these cases, you may well start off with similar mitigations, though the steps to fix it are likely to be very different. But considering all of these side by side means that you can help the people who are actually going to be operating those systems plan and be ready to manage their deployments.

So the lesson from today is the same as it so often is: make sure that your security folks are involved from the beginning of a project, in all parts of it. And an extra one: if you’re a security person, try to think not just about the attackers, but also about all those poor people who will be operating your software. They’ll thank you for it[6].

1 – not literally, thankfully[2].

2 – though there was that memorable trip to Singapore with food poisoning… I’ll stop there.

3 – a fact of which I actually was aware.

4 – some due entirely to our own navel-gazing, I’m pretty sure.

5 – exactly what we singularly failed to do in the project I’ve just described.

6 – though probably not in person. Or with an actual gift. But at least they’ll complain less, and that’s got to be worth something.

Talking in school

Learning by teaching

A few months ago, I was asked by a teacher at a local school to come in and talk to year 10 and year 11 students (aged 14-16 or so) about my job, what I do, my background, how I got into my job and to give any further thoughts and advice. Today I got the chance to go in and talk to them.

I very much enjoyed myself[1], and hopefully it was interesting for the pupils as well. I went over my past – from being “a bit of a geek at school” through to some of the stuff I need to know to do my job now – and also talked about different types of work within IT security. I was at pains to point out that you don’t need to be a great mathematician or even a great coder to get a career in IT security, and talked a lot about the importance of systems – which absolutely includes people.

What went down best – as is the case with pretty much any crowd – was stories. “War stories”, as they’re sometimes called, about what situations you’ve come across, how you dealt with them, how other people reacted, and the lessons you’ve learned from them, give an immediacy and relevance that just can’t be beaten. I was careful not to make them very technical – and one about a member of staff who had lost weight while on holiday and got stuck in a two-door man-trap (which included a weight sensor) went down particularly well[3].

The other thing that was useful – and which isn’t always going to work in a C-level meeting, for instance – was some exercises. Codes and ciphers are always interesting, so I started with a ROT13, then a Caesar cipher, then a simple key, then a basic alphabet substitution. We talked about letter frequency, repeated words, context and letter groupings, and the older group solved all of the puzzles, which was excellent.

There was time for some questions, too, which included:

“how much do you get paid?” Somewhat cheeky, this one, but I answered by giving them a salary range for a job which someone had contacted me about, but which I’d not followed up on – and gave no indications of the reasons for rejecting it
“do you need an IT or computing degree?” No, though it can be helpful.
“do you need a degree at all?” No, and though it can be difficult to get on without one, there are some very good apprentice schemes out there.

I went into the school to try to help others learn, but it was a very useful experience for me, too. Although all of the pupils there are taking a computing class by choice, not all of them were obviously engaged. But that didn’t mean that they weren’t paying attention: one of the pupils with the least “interested” body language was the fastest at answering some of the questions. Some of the pupils there had similar levels of understanding around IT security to some C-levels who aren’t in IT. Thinking about pace, about involving members of the audience who weren’t necessarily paying attention – all of these were really useful things for me to reflect on.

So – if you get the chance[4] – consider contacting a local school or college and seeing if they’d like someone to talk to them about what you do. Making it interesting, be ready to move on where topics aren’t getting the engagement you’d hope, and be ready for some questions. I can pretty much guarantee that you’ll learn something.

1 – one of my daughters, who attends the school, gave me very strict instructions about not talking to her, her friends or anyone she knew[2].

2 – (which I have every intention of ignoring, but sadly, I didn’t see her or any of her friends that I recognised. Maybe next time.)

3 – though possibly not with the senior manager who had to come out on a Sunday to rescue him and reset the system.

4 – and you’re willing to engage a tough audience.

There are no absolutes in security

There is no “secure”.

Let’s stop using the word “secure”. There is no “secure” in IT.

I know that sounds crazy, but it’s true.

Sometimes, when I speak to colleagues and customers, there will be non-technical or non-security people there, and they ask how to get a secure system. So I explain how I’d make a system secure. It goes a bit like this.

Remove any non-critical USB connections: in particular external or “thumb” drives.
Turn off all bluetooth.
Turn off all wifi.
Remove any network cables.
Remove any other USB connections, including mouse or keyboard.
Disconnect any monitors.
Disconnect any other cables that are connected to the system.
Yes, that includes the power cable.
Now take out any hard drives – SSD, HDD or other.
Destroy them. My preferred method is to gouge tracks in all spinning media, break the heads, bash all pieces with a hammer and then throw them into Mount Doom, but any other volcano[1] will do. Thermite lances are probably acceptable. You should do the same with all other components that you removed in earlier steps.
Destroy the motherboard, including all chips and RAM.
Tip all remaining pieces down a well.
Pour concrete down the well.[2]
You probably now have a secure which is about as secure as you’re going to get.

Yes, it’s a bit extreme, but the point is that all of the components there are possible threat vectors or information leakage channels.

Can we design and operate a system where we manage and mitigate the risks of threats and information leakage? Yes. That’s where we improve the security of a system. Is that a secure system? No, it’s not. What we’ve done is raise the bar, but we’ve not made it absolutely secure.

Part of the problem is that there’s just no way, these days[4], that any single person can be certain of the security of all parts of a system: they are just too many, and too complex. You may understand the application layer, but what about the virtualisation layer, for instance? I presented a simplified layer diagram in my post Isolationism a few months back, in which I listed the host as the bottom layer, but that was, of course, just asking for trouble. Along came Meltdown and Spectre, and now it’s clear (as if we didn’t know it already) that you should never ignore the fact that you can’t even trust the silicon you’re running on to do the thing you think it ought.

None of this, however, stops people and companies telling you that they’ll “secure your perimeter”, or provide you with “secure systems”. And it annoys me[5]. “We’ll help you secure your perimeter” isn’t too bad, but anything that suggests that you can have “secure systems” smacks to me of marketing – bad marketing.

So here you go: please stop using the word “secure” as an unqualified adjective or verb. We’re grown-ups, now, and we know it’s not real. So let’s not pretend.

Now – where was that well-cover? I need to deal with little Tommy.

1 – terrestrial/Middle Earth. I’m not sure about volcano temperatures on other planets or in the Undying Lands across the Western Sea.

2 – it should probably therefore be a disused well. Check there are no animals down there first[3]. In fact, before you throw anything down there.

3 – what’s that, Lassie? Little Tommy’s down the well? Well, I wonder whether little Tommy is waiting for us to throw the components down there so that he can do bad things. Bad Tommy.

4 – I’d like to think that maybe there was, once, in the distant past, but I’m probably kidding myself.

5 – you might be surprised at the number of things that annoy me[6].

6 – unless you’re my wife, in which case you probably won’t be[7].

7 – surprised. Or, in fact, reading this article.

3 tests for NOT moving to blockchain

How to tell when you can avoid the hype.

So, there’s this thing called “blockchain” which is quite popular…

You know that already, of course. I keep wondering if we’ve hit “peak hype” for blockchain and related technologies yet, but so far there’s no sign of it. As usual for this blog, when I’m talking about blockchain, I’m going to include DLTs – Distributed Ledger Technologies – which are, by some tight definitions of the term, not really blockchains at all. I’m particularly interested, from a professional point of view, in permissioned blockchains. You can read more about how that’s defined in my previous post Is blockchain a security topic? – the key point here is that I’m interested in business applications of blockchain beyond cryptocurrency[1].

And, if the hype is to be believed – and some of it probably should be[2] – then there is an almost infinite set of applications for blockchain. That’s probably correct, but that doesn’t mean that they’re all good applications for blockchain. Some, in fact, are likely to be very bad applications for blockchain.

The hype associated with blockchain, however, means that businesses are rushing to embrace this new technology[3] without really understanding what they’re doing. The drivers towards this move are arguably three-fold:

you can, if you try, make almost any application with multiple users which stores data into a blockchain-enable application;
there are lots of conferences and “gurus” telling people that if they don’t embrace blockchain now, they’ll go out of business within six months[4];
it’s not easy technology to understand fully, and lots of the proponents “on-the-ground” within organisations are techies.

I want to unpack that last statement before I get a hail of trolls flaming me[5]. I have nothing against techies – I’m one myself – but one of our characteristics tends to be enormous enthusiasm about new things (“shinies”) that we understand, but whose impact on the business we don’t always fully grok[6]. That’s not always a positive for business leaders.

The danger, then, is that the confluence of those three drivers may lead to businesses deciding to start moving to blockchain applications without fully understanding whether that’s a good idea. I wrote in another previous post (Blockchain: should we all play?) about some tests that you can apply to decide whether a process is a good fit for blockchain and when it’s not. They were useful, but the more I think about it, the more I’m convinced that we need some simple tests to tell us when we should definitely not move a process or an application to a blockchain. I present my three tests. If your answer any of these questions is “yes”, then you almost certainly don’t need a blockchain.

Test 1 – does it have a centralised controller or authority?

If the answer is “yes”, then you don’t need a blockchain.

If, for instance, you’re selling, I don’t know, futons, and you have a single ordering system, then you have single authority for deciding when to send out a futon. You almost certainly don’t need to make this a blockchain. If you are a purveyor of content that has to pass through a single editorial and publishing process, they you almost certainly don’t need to make this a blockchain.

The lesson is: blockchains really don’t make sense unless the tasks required in the process execution – and the trust associated with those tasks – is distributed between multiple entities.

Test 2 – could it work fine with a standard database?

If the answer to this question is “yes”, then you don’t need a blockchain.

This question and the previous one are somewhat intertwined, but don’t need to be. There are applications where you have distributed processes, but need to store information centrally, or centralised authorities but distributed data, where one may be yes, but the other “no”. But if this is question is a “yes”, then use a standard database.

Databases are good at what they do, they are cheaper in terms of design and operation than running a blockchain or distributed ledger, and we know how to make them work. Blockchains are about letting everybody[8] see and hold data, but the overheads can be high, and the implications costly.

Test 3 – is adoption going to be costly, or annoying, to some stakeholders?

If the answer to this question is “yes”, then you don’t need a blockchain.

I’ve heard assertions that blockchains always benefit all users. This is a patently false. If you are creating an application for a process, and changing the way that your stakeholders interact with you and it, you need to consider whether that change is in their best interests. It’s very easy to create and introduce an application, blockchain or not, which reduces business friction for the owner of the process, but increases it for other stakeholders.

If I make engine parts for the automotive industry, it may benefit me immensely to be able to track and manage the parts on a blockchain. I may be able to see at a glance who’s supplied what, when, and the quality of the steel used in the ball-bearings. On the other hand, if I’m a ball-bearing producer, and I have an established process which works for the forty companies to whom I sell ball-bearings, then adopting a new process for just one of them, with associated changes to my method of work, new systems and new storage and security requirements is unlikely to be in my best interests: it’s going to be both costly and annoying.

Conclusion

Tests are guidelines: they’re not fixed in stone. One of these tests looks like a technical test (the database one), but is really as much about business roles and responsibilities as the other two. All of them, hopefully, can be used as a counter-balance to the three drivers I mentioned.

1 – which, don’t get me wrong, is definitely interesting and a business application – it’s just not what I’m going to talk about in this post.

2 – the trick is knowing which bits. Let me know if you work out how, OK?

3 – it’s actually quite a large set of technologies, to be honest.

4 – which is patently untrue, unless the word “they” refers there to the conferences and gurus, in which case it’s probably correct.

5 – which may happen anyway due to my egregious mixing of metaphors.

6 – there’s a word to love. I’ve put it in to exhibit my techie credentials[7].

7 – and before you doubt them, yes, I’ve read the book, in both cut and uncut versions.

8 – within reason.

Moving to DevOps, what’s most important?

Technology, process or culture? (Clue: it’s not the first two)

You’ve been appointed the DevOps champion in your organisation: congratulations. So, what’s the most important issue that you need to address?

It’s the technology – tools and the toolchain – right? Everybody knows that unless you get the right tools for the job, you’re never going to make things work. You need integration with your existing stack – though whether you go with tight or loose integration will be an interesting question – a support plan (vendor, 3rd party or internal), and a bug-tracking system to go with your source code management system. And that’s just the start.

No! Don’t be ridiculous: it’s clearly the process that’s most important. If the team doesn’t agree on how stand-ups are run, who participates, the frequency and length of the meetings, and how many people are required for a quorum, then you’ll never be able institute a consistent, repeatable working pattern.

In fact, although both the technology and the process are important, there’s a third component which is equally important, but typically even harder to get right: culture. Yup, it’s that touch-feely thing that we techies tend to struggle with[1].

Culture

I was visiting a medium-sized government institution a few months ago (not in the UK, as it happens), and we arrived a little early to meet the CEO and CTO. We were ushered into the CEO’s office and waited for a while as the two of them finished participating in the daily stand-up. They apologised for being a minute or two late, but far from being offended, I was impressed. Here was an organisation where the culture of participation was clearly infused all the way up to the top.

Not that culture can be imposed from the top – nor can you rely on it percolating up from the bottom[3] – but these two C-level execs were not only modelling the behaviour they expected from the rest of their team, but also seemed, from the brief discussion we had about the process afterwards, to be truly invested in it. If you can get management to buy into the process – and to be seen to buy in – you are at least likely to have problems with other groups finding plausible excuses to keep their distance and get away with it.

So let’s say that management believes that you should give DevOps a go. Where do you start?

Developers, tick?[5]

Developers may well be your easiest target group. Developers are often keen to try new things, and to find ways to move things along faster, so they are often the group that can be expected to adopt new technologies and methodologies. DevOps has arguably been mainly driven by the development community. But you shouldn’t assume that all developers will be keen to embrace this change. For some, the way things have always been done – your Rick Parfitts of dev, if you will[7] – is fine. Finding ways to help them work efficiently in the new world is part of your job, not just theirs. If you have superstar developers who aren’t happy with change, you risk alienating them and losing them if you try to force them into your brave new world. What’s worse, if they dig their heels in, you risk the adoption of your DevSecOps vision being compromised when they explain to their managers that things aren’t going to change if it makes their lives more difficult and reduces their productivity.

Maybe you’re not going to be able to move all the systems and people to DevOps immediately. Maybe you’re going to need to choose which apps start with, and who will be your first DevOps champions. Maybe it’s time to move slowly.

Not maybe: definitely

No – I lied. You’re definitely going to need to move slowly. Trying to change everything at once is a recipe for disaster.

This goes for all elements of the change – which people to choose, which technologies to choose, which applications to choose, which user base to choose, which use cases to choose – bar one. For all of those elements, if you try to move everything in one go, you will fail. You’ll fail for a number of reasons. You’ll fail for reasons I can’t imagine, and, more importantly, for reasons you can’t imagine, but some of the reasons will include:

people – most people – don’t like change;
technologies don’t like change (you can’t just switch and expect everything to work still);
applications don’t like change (things worked before, or at least failed in known ways: you want to change everything in one go? Well, they’ll all fail in new and exciting[9] ways;
users don’t like change;
use cases don’t like change.

The one exception

You noticed that, above, I wrote “bar one”, when discussing which elements you shouldn’t choose to change all in one go? Well done.

What’s that exception? It’s the initial team. When you choose your initial application to change, and you’re thinking about choosing the team to make that change, select the members carefully, and select a complete set. This is important. If you choose just developers, just test folks, or just security folks, or just ops folks, or just management, then you won’t actually have proved anything at all. If you leave out one functional group from your list, you won’t actually have proved anything at all. Well, you might have proved to a small section of your community that it kind of works, but you’ll have missed out on a trick. And that trick is that if you choose keen people from across your functional groups, it’s much harder to fail.

Say that your first attempt goes brilliantly. How are you going to convince other people to replicate your success and adopt DevOps? Well, the company newsletter, of course. And that will convince how many people, exactly? Yes, that number[12]. If, on the other hand, you have team members from across the functional parts or the organisation, then when you succeed, they’ll tell their colleagues, and you’ll get more buy-in next time.

If, conversely, it fails, well, if you’ve chosen your team wisely, and they’re all enthusiastic, and know that “fail often, fail fast” is good, then they’ll be ready to go again.

So you need to choose enthusiasts from across your functional groups. They can work on the technologies and the process, and once that’s working, it’s the people who will create that cultural change. You can just sit back and enjoy. Until the next crisis, of course.

1 – OK, you’re right. It should be “with which we techies tend to struggle”[2]

2 – you thought I was going to qualify that bit about techies struggling with touchy-feely stuff, didn’t you? Read it again: I put “tend to”. That’s the best you’re getting.

3 – is percolating a bottom-up process? I don’t drink coffee[4], so I wouldn’t know.

4 – do people even use percolators to make coffee anymore? Feel free to let me know in the comments. I may pretend interest if you’re lucky.

5 – for US readers (and some other countries, maybe?), please substitute “tick” for “check” here[6].

6 – for US techie readers, feel free to perform “s/tick/check/;”.

7 – this is a Status Quo[8] reference for which I’m extremely sorry.

8 – for Millennial readers, please consult your favourite online reference engine or just roll your eyes and move on.

9 – for people who say, “but I love excitement”, trying being on call at 2am on a Sunday morning at end of quarter when your Chief Financial Officer calls you up to ask why all of last month’s sales figures have been corrupted with the letters “DEADBEEF”[10].

10 – for people not in the know, this is a string often used by techies as test data because a) it’s non-numerical; b) it’s numerical (in hexadecimal); c) it’s easy to search for in debug files and d) it’s funny[11].

11 – though see [9].

12 – it’s a low number, is all I’m saying.

Q: when is a backdoor not a backdoor?

An encryption backdoor isn’t the same as a house backdoor: the metaphor is faulty.

A: when you’re a politician.

I’m getting pretty bored of having to write about this, to be honest. I’ve blogged twice already on encryption backdoors:

But our politicians keep wanting us to come up with them, as the Register helpfully points out – thanks, both the UK Prime Minister and FBI Director.

I feel sorry for their advisers, because all of the technical folks I’ve ever spoken to within both the UK and US Establishments[1] absolutely understand that what’s being asked for by these senior people really isn’t plausible.

I really do understand the concern that the politicians have. They see a messaging channel which bad people may use to discuss bad things, and they want to stop those bad things. This is a good thing, and part of their job. The problem starts when they think “it’s like a phone: we have people who can tap phones”. Those who are more technologically savvy may even think, “it’s like email, and we can read email.” And in the old days[3], before end-to-end encryption, they weren’t far wrong.

The problem now is that many apps these days set up a confidential (encrypted) link between the two ends of the connection. And they do it in a way which means that nobody except the initiators of the two ends of the connection can read it. And they use strong encryption, which means that there’s no easy way for anyone[4] to break it.

This means that it’s difficult for anyone to read the messages. So what can be done about it, then? Well, if you’re a politician, the trend is to tell the providers of these popular apps to provide a backdoor to let you, the “good people” in.

Oh, dear.

I believe that the problem here isn’t really that politicians are stupid, because I honestly don’t think that they are[5]. The problem is with metaphor. Metaphors are dangerous, because humans need them to get a handle on an aspect of something which is unfamiliar, but once they’ve latched on to a particular metaphor, they assume that all the other aspects of the thing to which the metaphor refers are the same.

An encryption backdoor isn’t the same as a house backdoor: the metaphor is faulty[6].

The key[7] similarity is that in order to open up your house backdoor, you need a key. That key gives you entry to the house, and it also allows any other person you give that key access to it, as well. So far, so good.

Here’s where it gets bad, though. I’m going to simplify things a little here, but let’s make some points.

When you give a backdoor key to somebody, it’s not easily copyable if somebody happens to see it. In the electronic world, if you see the key once, you have it.
The cost of copying an electronic key is basically zero once you have it. If one person decides to share the key indiscriminately, then the entire Internet has it.
Access to a house Backdoor let’s you see what’s in the house at that particular moment. Access to an electronic backdoor lets you look at whatever the contents of the house were all the way up to the time the lock was changed, if you’ve taken copies (which is often easy).
And here’s the big one. When you create a backdoor, you’re creating a backdoor for every house, and not just one. Let’s say that I’m a house builder. I’m very, very prolific, and I build thousands of houses a week. And I put the same lock in the backdoor of every house that I build. Does that make sense? No, it doesn’t. But that’s what the politicians are asking for.

So, the metaphor breaks down. Any talk about “skeleton keys” is an attempt to reestablish the metaphor. Which is broken.

What’s the lesson here? We should explain to politicians that backdoors are a metaphor, and that the metaphor only goes so far. Explain that clever people – clever, good people – don’t believe that what they (the politicians) think should be done is actually possible, and the move on to work that can be done. Because they’re right: there are bad people out there, doing bad things, and we need to address that. But not this way.

1 – the capital “E” is probably important here. In the UK, at least, “establishment” can mean pub[2].

2 – and people in pubs, though they may start up clued up, tend to get less clever as the evening goes on, though they may think, for a while, that they’re becoming more clever. This is in my (very) limited experience, obviously.

3 – 10 years ago? Not very long ago, to be honest.

4 – well, who’s owning up, anyway.

5 – mostly.

6 – or Fawlty, for John Cleese fans.

7 – ooh, look what I did there.