Enarx news: WebAssembly + SGX

We can now run a WebAssembly workload in a TEE

Regular readers will know that I’m one of the co-founders for the Enarx project, and write about our progress fairly frequently. You’ll find some more information in these articles (newest first):

I won’t spend time going over what the project is about here, as you can read more in the articles above, and lots more over at our GitHub repository (https://enarx.io), but aim of the project is to allow you run sensitive workloads in Trusted Execution Environments (TEEs) from various silicon vendors. The runtime we’ll be providing is WebAssembly (using the WASI interface), and the TEEs that we’re targetting to start with are AMD’s SEV and Intel’s SGX – though you, the client running the workload, shouldn’t care which is being used at any particular point, as we abstract all of that away.

It’s a complicated architecture, with lots of moving parts and a fairly complex full-stack architecture. Things have been moving really fast recently, partly due to a number of new contributors to the project (which recently reached 150 stars on our main GitHub repository), and one of the important developments is the production of a new component architecture, included below.

I’ve written before about how important it is to have architectural diagrams for projects (and particularly open source projects), and so I’m really pleased that we have this updated – and much more detailed – version of the diagram. I won’t go into it in detail here, but the only trusted components (from the point of view of Enarx and a client using Enarx) are those in green: the Enarx client agent, the Attestation measurement database and the TEE instance containing the Application, Wasm, Shim, etc., which we call the Keep.

It’s only just over a couple of months since we announced our most recent milestone: running binaries within TEEs, including both SEV and SGX. This was a huge deal for us, but then, at the end of last week, one of the team, Daiki Ueno, managed to get a WebAssembly binary loaded using the Wasm/WASI layers and to run it. In other words, we can now run a WebAssembly workload in a TEE. Over the past 10 minutes, I’ve just managed to replicate that on my own machine (for my own interest, not that I doubted the results!). Circled, below, are the components which are involved in this feat.

I should make it clear that none of these components is complete or stable – they are all still very much in development, and, in fact, even the running test yielded around 40 lines for error messages! – but the fact that we’ve got a significant part of the stack working together is an enormous deal. The Keep loader creates an SGX TEE instance, loads the Shim and the Wasm/WASI components into the Keep and starts them running. The App Loader (one of the Wasm components) loads the Application, and it runs.

It’s not just an enormous deal due to the amount of work that it’s taken and the great teamwork that’s been evident throughout – though those are true as well – but also because it means we’re getting much closer to having something that people can try out for themselves. The steps to just replicating the test were lengthy and complex, and my next step is to see if I can make make some changes and create my own test, which may be quite a marathon, but I’d like to stress that my technical expertise lies closer to the application layer than the low-level pieces, which means that if I can make this work now (with a fair amount of guidance), then we shouldn’t be too far off being in a position where pretty much anyone with a decent knowledge of the various pieces can do the same.

We’re still quite a way off being able to provide a simple file with 5-10 steps to creating and running your own workload in a Keep, but that milestone suddenly feels like it’s in sight. In the meantime, there’s loads of design work, documentation, infrastructure automation, testing and good old development to be done: fancy joining us?

In praise of triage

It’s all too easy to prioritise based on the “golfing test”.

Not all bugs are created equal.

Some bugs need fixing now, some bugs can wait. Some bugs are in your implementation, some are in the underlying design. Some bugs will annoy a few customers, some will destroy your business.

Bugs come in all shapes and sizes, and one of the tasks of a product owner, product manager, chief architect – whoever makes the call about where to assign resources – is to decide which ones to address in which order: to prioritise them. The problem is deciding how to prioritise them. It’s all too easy to prioritise based on the “golfing test”: your CEO meets someone on golf course who mentions that his or her company loves your product, except for one tiny issue. The CEO comes back, and makes it clear that fixing this “major bug” is now your one and only task until it’s done, and your world is turned upside down. You have to fix the bug as quickly as possible, with no thought to the impact it has on the rest of the project, or the immense pile technical debt that’s just been accrued. You don’t want to live in this world. What, then, is the alternative?

The answer – though it’s only the beginning of the answer – is triage. Triage (from the French for “separating out”) comes from the world of battlefield medicine. When deciding which wounded soldiers to treat, rapid (hopefully objective) assessments are carried out, allowing a quick sorting of each soldier, typically into categories such as “not urgent: wait”, “urgent: treat immediately” and “not saveable: do not treat”. We can apply the same to software bugs in order to decide what to treat (fix) and with what priority. The important thing is not so much the categories – which will vary based on your context – but the assessment criteria, and how they are applied. Here are a list of just some of the possible criteria:

  • likely monetary impact per customer
  • number of customers impacted
  • reputational impact on your organisation
  • ease to fix
  • impact on system security
  • impact on system performance
  • impact on system stability
  • annoyance of CEO not to be listened to.

We do not, of course, only need to apply one of these: a number of them can be combined with a weighting system, though the more you add, the less clear your priorities will be, and the more likely it is that someone will “put a finger on the scales” – tweak the numbers to give the outcome they want. Another important point about the categories that you decide to apply is that they should be as measurable as you can make them, to allow as objective scoring as possible. I wrote a review of the book Building Evolutionary Architectures a while ago: the methodology adopted there, where you measure and test in order to meet specific criteria, is exactly the sort of approach you should be choosing when designing your triage system.

This is (ostensibly) a blog about security, and so you might expect me to say that “security always wins”, but that should absolutely not be the approach you take. Security might be the most important category for you (that is, carry the most weight), but you need to understand why that is the case – at this particular time – and what exactly you mean by “security”. The “security of the system” is not an objective measure: in order to mean anything, such a phrase needs to reference measurements that can be made (“resistance to physical tampering”, “resistance to brute force attacks”, “number or PhD students likely to be needed to reverse engineer our ‘secure’ protocol”[1]). More importantly, it may be that at this point in your organisation’s life, the damage done by lack of stability or decreased performance outweighs the impact of a security bug. If that’s the case, then your measurements should encapsulate that information and lead you to prioritise bugs with impact in these categories over security issues[3].

There’s one proviso that I feel I need to put in at this point, and it’s about the power of what, in Agile Methodology terms, is called the Product Owner. This is the person who represents the users of the product/project, and should have final say about the direction of development in terms of features, functionality and, most relevant in this discussion, bug-fixing. As noted above, this may be an architect, product manager or someone enjoying another title, but their role should be clear: they get to call the shots. There are times when this person goes against the evidence provided by the triage, and makes a decision to prioritise a particular bug over others despite the outcome of the measurements. This is typically very painful for the technical team[4], but, when it comes down to it, as the product owner, they get to decide. The technical team – after appropriate warnings and discussion[5] – must be ready to step aside and accept the decision. Such decisions (and related discussions) should be recorded, and the product owner must be ready to stand or fall based on the outcome, but that is their job. Triage is a guide, and there are occasions when there are measurements which cannot be easily made objectively, and which sit outside the expertise or scope of knowledge of the technical team. If this sort of decision keeps being made, and you think you know better, you may have a future in technical product management, where people with a view of both the technical and the business side of technology are much in demand. In the end, though, the product owner will need to justify their decision to management, and if they get it wrong, then they must be ready to take the blame (this is one reason why you should make sure that you’ve recorded the process taken to get to this decision – you don’t want to take the blame for a poor decision which you advised against).

So: go out an design a triage process, be ready to follow it, and be ready to defend it. Oh, and one last point: you might want to buy a set of golf clubs.

—–

1 – this last one is a joke: don’t design your own protocol, or if you do, make it open and have it peer-reviewed[2].

2 – and then throw it away and use an open source implementation of better, more thoroughly-reviewed one.

3 – much as it pains me to say it.

4 – I’ve been on both sides of these decisions: I know.

5 -often rather heated, in my experience.

8 tips on how to do open source (badly)

Generally, a “build successful” or “build failed” message should be sufficient.

A while ago, I published my wildly popular[1] article How not to make a cup of tea. Casting around for something to write this week, it occurred to me that I might write about something that I believe is almost as important as to world peace, the forward march of progress and brotherly/sisterly love: open source projects. There are so many guides out there around how to create an open source project that it’s become almost too easy to start a new, successful, community-supported one. I think it’s time to redress the balance, and give you some clues about how not to do it.

Throw it over the wall

You know how it is: you’re a large corporation, you’ve had a team of developers working on a project for several years now, and you’re very happy with it. It’s quite expensive to maintain and improve the code, but luckily, it’s occurred to you that other people might want to use it – or bits of it. And recently, some of your customers have complained that it’s difficult to get improvements and new features, and partners complain that your APIs are obscure, ill-defined and subject to undocumented change.

Then you hit on a brilliant idea: why not open source it and tell them their worries are over? All you need to do is take the existing code, create a GitHub or GitLab[2] project (preferably under a Game of Thrones-themed username that happens to belong to one of your developers), make a public repository, upload all of the code, and put out a press release announcing: a) the availability of your project; and b) what a great open source citizen you are.

People will be falling over themselves to contribute to your project, and you’ll suddenly have hundreds of developers basically working for you for free, providing new features and bug fixes!

Keep a tight grip on the project

There’s a danger, however, when you make your project open source, that other people will think that they have a right to make changes to it. The way it’s supposed to work is that your product manager comes up with a bunch of new features that need implementing, and posts them as issues on the repository. Your lucky new contributors then get to write code to satisfy the new features, you get to test them, and then, if they’re OK, you can accept them into the project! Free development! Sometimes, if you’re lucky, customers or partners (the only two parties of any importance in this process, apart from you) will raise issues on the project repository, which, when appropriately subjected to your standard waterfall development process and vetted by the appropriate product managers, can be accepted as “approved” issues and earmarked for inclusion into the project.

There’s a danger that, as you’ve made the project code open source, some people might see this as an excuse to write irrelevant features and fixes for bugs that none of your customers have noticed (and therefore, you can safely assume, don’t care about). Clearly, in this case, you should reject close any issues related to such features or fixes, and reject (or just ignore) any related patched.

Worse yet, you’ll sometimes find developers[2] complaining about how you run the project. They may “fork” the project, making their own version. If they do this, beware setting your legal department on them. There’s a possibility that as your project is open source, they might be able to argue that they have a right to create a new version. Much better is to set your PR department on them, rubbishing the new project, launching ad hominem[4] attacks on them and showing everybody that you hold the moral high ground.

Embrace diversity (in licensing)

There may be some in your organisation who say that an open source project doesn’t need a licence[5]. Do not listen to their siren song: they are wrong. What your open source project needs is lots of licences! Let your developers choose their favourite for each file they touch, or, even better, let the project manager choose. The OSI maintains a useful list, but consider this just a starting point: why not liberally sprinkle different licences through your project? Diversity, we keep hearing, is good, so why not apply it to your open source project code?

Avoid documentation

Some people suggest that documentation can be useful for open source projects, and they are right. What few of them seem to understand is that their expectations for the type of documentation are likely to be skewed by their previous experience. You, on the other hand, have a wealth of internal project documentation and external product marketing material that you accrued before deciding to make your project open source: this is great news. All you need to do is to create a “docs” folder and copy all of the PDF files there. Don’t forget to update them whenever you do a new product version!

Avoid tooling

All you need is code (and docs – see above). Your internal developers have carefully constructed and maintained build environments, and they should therefore have no problems building and testing any parts of the project. Much of this tooling, being internal, could be considered proprietary, and the details must therefore be kept confidential. Any truly useful contributors will be able to work out everything they need for themselves, and shouldn’t need any help, so providing any information about how actually to build the code in the repository is basically redundant: don’t bother.

An alternative, for more “expert” organisations, is to provide build environments which allow contributors to batch builds to see if they compile or not (whilst avoiding giving them access to the tooling themselves, obviously). While this can work, beware providing too much in the way of output for the developer/tester, as this might expose confidential information. Generally, a “build successful” or “build failed” message should be sufficient.

Avoid diagrams

Despite what some people think, diagrams are dangerous. They can give away too much information about your underlying assumptions for the project (and, therefore, the product you’re selling which is based on it), and serious developers should be able to divine all they need from the 1,500 source files that you’ve deposited in the repository anyway.

A few “marketiture” diagrams from earlier iterations of the project may be acceptable, but only if they are somewhat outdated and don’t provide any real insight into the existing structure of the code.

Keep quiet

Sometimes, contributors – or those hoping to become contributors, should you smile upon their requests – will ask questions. In the old days, these questions tended to be sent to email lists[6], where they could be safely ignored (unless they were from an important customer). More recently, there are other channels that developers expect to use to contact members of the project team, such as issues or chat.

It’s important that you remember that you have no responsibility or duty to these external contributors: they are supposed to be helping you. What’s more, your internal developers will be too busy writing code to answer the sort of uninformed queries that are likely to be raised (and as for so-called “vulnerability disclosures“, you can just fix those in your internal version of the product, or at least reassure your customers that you have). Given that most open source projects will come with an issue database, and possibly even a chat channel, what should you do?

The answer is simple: fall back to email. Insist that the only channel which is guaranteed attention[7] is email. Don’t make the mistake of failing to create an email address to which people can send queries: contributors are much more likely to forget that they’re expecting an answer to an email if they get generic auto-response (“Thanks for your email: a member of the team should get back to you shortly”) than if they receive a bounce message. Oh, and close any issues that people create without your permission for “failing to follow project process”[8].

Post huge commits

Nobody[9] wants to have to keep track of lots of tiny changes to code (or, worse, documentation – see above), or have contributors picking holes in it. There’s a useful way to avoid much of this, however, which is to train your developers only to post large commits to the open source project. You need to ensure that your internal developers understand that code should only be posted to the external repository when the project team (or, more specifically, the product team) deems it ready. Don’t be tempted to use the open source repository as your version control system: you should have perfectly good processes internally, and, with a bit of automation, you can set them up to copy batches of updates to the external repository on a regular basis[10].


1 – well, lots of you read it, so I’m assuming you like it.

2 – other public repositories may be available, but you won’t have heard of them, so why should you care?

3 – the canonical term for such people is “whingers”: they are invariably “experts”. According to them (and their 20 years of security experience, etc., etc.).

4 – or ad mulierem – please don’t be sexist in your attacks.

5 – or license, depending on your spelling choice.

6 – where they existed – a wise organisation could carefully avoid creating them.

7 – “attention” can include a “delete all” filter.

8 – you don’t actually need to define what the process is anywhere, obviously.

9 – in your product organisation, at least.

10 – not that “regular” does not equate to “frequent”. Aim for a cadence of once every month or two.

What’s a hash function?

It should be computationally implausible to work backwards from the output hash to the input.

Note – many thanks to a couple of colleagues who provided excellent suggestions for improvements to this text, which has been updated to reflect them.

Sometimes I like to write articles about basics in security, and this is one of those times. I’m currently a little over a third of the way through writing a book on trust in computing and the the cloud, and ended up creating a section about cryptographic hashes. I thought that it might be a useful (fairly non-technical) article for readers of this blog, so I’ve edited it a little bit and present it here for your delectation, dears readers.

There is a tool in the security practitioner’s repertoire that it is helpful for everyone to understand: cryptographic hash functions. A cryptographic hash function, such as SHA-256 or MD5 (now superseded for cryptographic uses as it’s considered “broken”) takes as input a set of binary data (typically as bytes) and gives as output which is hopefully unique for each set of possible inputs. The length of the output – “the hash” – for any particularly hash function is typically the same for any pattern of inputs (for SHA-256, it is 32 bytes, or 256 bits – the clue’s in the name). It should be computationally implausible (cryptographers hate the word “impossible”) to work backwards from the output hash to the input: this is why they are sometimes referred to as “one-way hash functions”. The phrase “hopefully unique” when describing the output is extremely important: if two inputs are discovered that yield the same output, the hash is said to have “collisions”. The reason that MD5 has become deprecated is that it is now trivially possible to find collisions with commercially-available hardware and software systems. Another important property is that even a tiny change in the message (e.g. changing a single bit) should generate a large change to the output (the “avalanche effect”).

What are hash functions used for, and why is the property of being lacking in collisions so important? The simplest answer to the first question is that hash functions are typically used to ensure that when someone hands you a piece of binary data (and all data in the world of computing can be described in binary format, whether it is text, an executable, a video, an image or complete database of data), it is what you expect. Comparing binary data directly is slow and arduous computationally, but hash functions are designed to be very quick. Given two files of several Megabytes or Gigabytes of data, you can produce hashes of them ahead of time, and defer the comparisons to when you need them[1].

Indeed, given the fact that it is easy to produce hashes of data, there is often no need to have both sets of data. Let us say that you want to run a file, but before you do, you want to check that it really is the file you think you have, and that no malicious actor has tampered with it. You can hash that file very quickly and easily, and as long as you have a copy of what the hash should look like, then you can be fairly certain that you have the file you wanted. This is where the “lack of collisions” (or at least “difficulty in computing collisions”) property of hash functions is important. If the malicious actor can craft a replacement file which shares the same hash as the real file, then the process is essentially useless.

In fact, there are more technical names for the various properties, and what I’ve described above mashes three of the important ones together. More accurately, they are:

  1. pre-image resistance – this says that if you have a hash, it should be difficult to find the message from which it was created, even if you know the hash function used;
  2. second pre-image resistance – this says that if you have a message, it should be difficult to find another message which, when hashed, generates the same hash;
  3. collision resistance – this says that it should be difficult to find any two messages which generate the same hash.

Collision resistance and second pre-image resistance sound like the same property, at first glances, but are subtly (but importantly) different. Pre-image resistance says that if you already have a message, finding another with a matching hash, whereas collision resistance should make it hard for you to find any two messages which will generate the same hash, and is a much harder property to fulfil in a hash function.

Let us go back to our scenario of a malicious actor trying to exchange a file (with a hash which we can check) with another one. Now, to use cryptographic hashes “in the wild” – out there in the real world beyond the perfectly secure, bug-free implementations populated by unicorns and overflowing with fat-free doughnuts – there are some important and difficult provisos that need to be met. More paranoid readers may already have spotted some of them, in particular:

  1. you need to have assurances that the copy of the hash you have has also not been subject to tampering;
  2. you need to have assurances that the entity performing the hash performs and reports it correctly;
  3. you need to have assurances that the entity comparing the two hashes reports the result of that comparison correctly.

Ensuring that you can meet such assurances it not necessarily an easy task, and is one of the reasons that Trusted Platform Modules (TPMs) are part of many computing systems: they act as a hardware root of trust with capabilities to provide such assurances. TPMs are a useful an important tool for real-world systems, and I plan to write an article on them (similar to What’s an HSM?) in the future.


1 – It’s also generally easier to sign hashes of data, rather than large sets of data themselves – this happens to be important as one of the most common uses of hashes is for cryptographic (“digital”) signatures.

Silencing my voice

Just links. I didn’t write these: others did.

Just links. I didn’t write these: others did.

(Image by Pexels from Pixabay).

Open source projects, embargoes and NDAs

Vendors may only disclose to project members under an NDA.

This article is a companion piece to one that I wrote soon after the advent of Meltdown and Spectre, Meltdown and Spectre: thinking about embargoes and disclosures, and another which I wrote more recently, Security disclosure or vulnerability management?. I urge you to read those first, as they provide background and context to this article which I don’t plan to reiterate in full. The focus of this article is to encourage open source projects to consider a specific aspect of the security disclosure/vulnerability management process: people.

A note about me and the background for this article. I’m employed by Red Hat – a commercial vendor of open source software – and I should make the standard disclosure that the views expressed in this article don’t necessarily reflect those of Red Hat (though they may!). The genesis of this article was a conversation I had as part of my membership of the Confidential Computing Consortium, of which Enarx (a project of which I’m a co-founder) is a member. During a conversation with other members of the Technical Advisory Board, we were discussing the importance of having a vulnerability management process (or whatever you wish to call it!), and a colleague from another organisation raised a really interesting point: what about NDAs?

What’s an NDA?

Let’s start with the question: “what’s an NDA”? “NDA” stands for Non-Disclosure Agreement, and it’s a legal document signed by one or more individuals or organisations agreeing not to disclose confidential material[1]. NDAs have a bad reputation in the wider world, as they’re sometimes used as “gagging orders” within legal settlements by powerful people or organisations in cases of, for instance, harassment. They are, however, not all bad! In fact, within the technology world, at least, they’re a very important tool for business.

Let’s say that company A is a software vendor, and creates software that works with hardware (a video card, let’s say) from company B. Company B wants to ensure that when its new product is released, there is software ready to make use of it, so that consumers will want to use it. In order to do that, Company A will need to know the technical specifications of the video card, but Company B wants to ensure that these aren’t leaked to the press – or competitors – before release. NDAs to the rescue! Company A can sign an NDA with Company B, agreeing not to disclose technical details about Company B’s upcoming products and plans to anyone outside Company A until the plans are public. If it’s a “two-way NDA”, and Company B signs it as well, then Company A can tell Company B about their plans for selling and marketing their new software, for instance, to help provide business partnership opportunities. If either of parties breaks the terms of the agreement, then it’s time for expensive legal proceedings (which nobody wants) and the likelihood that any trust from one to the other will break down, leading to repercussions later.

NDAs are common in this sort of context, and employees of the companies bound by NDAs are expected to abide by them, and also not to “carry over” confidential information to a new employer if they get a new job.

What about open source?

One of the things about open source is that it’s, well, “open”. So you’d think that NDAs would be a bad thing for open source. I’d argue, however, that they have actually served a very useful purpose in the development and rise of the open source ecosystem. The reason for this is that companies are often reluctant to have individuals sign NDAs with them, and individuals are often reluctant to sign NDAs. From the company’s point of view, NDAs are more difficult to enforce against individuals, and from the individual’s point of view, the legal encumbrance of signing an NDA – or multiple NDAs – may seem daunting.

Companies, however, are used to NDAs, and have legal departments to manage them. Back in the 1990s, I remember there being a long lag time between new hardware becoming available, and the open source support for it emerging. What did emerge was typically buggy, often reverse-engineered, and rarely took advantages of the newest features offered by the hardware. From around 2000, this has changed significantly: you can expect the latest hardware capabilities to emerge into, say, the Linux kernel, pretty much as soon as the supporting hardware is available. This is to some extent because there are now a large number of companies who employ engineers to work on open source. The hardware vendors have NDAs with these companies, which means that the engineers can find out ahead of time about new features, and have support ready for them when they launch.

There’s a balancing act here, of course, because sometimes providing code implementations and making them open can give away details of hardware features before the hardware vendors are ready to disclose them. This means that some of the engineering needs to take place “behind closed doors” and only be merged into projects once the hardware vendor is ready to announce their specifications to the world. This isn’t as open as one might like. On balance, though, I’d argue that this approach has provided a net benefit to the open source community, allowing hardware vendors to keep competitive information confidential, whilst allowing the open source community access to the latest and greatest hardware features at, or soon after, release.

NDAs and security

So far, we’ve not mentioned security: where does it come in? Well, say that you’re a hardware vendor, and you discover (or someone discloses to you) that there’s a vulnerability in one of your products. What do you do? Well, you don’t want to announce it to the world until a fix is ready (remember the discussion of embargoes in my previous article), but if there are open source projects which need to be involved in getting a fix out, then you need to be able to talk to them.

The key word in that last sentence is “them”. Most – one could say all – successful open source projects have members employed by a variety of different companies and organisations, as well as some who are not employed by any – or work on the project outside of any employer-related time or agreement. The hardware vendor is extremely keen to ensure that any embargo that is created around the work on this vulnerability is adhered to, and the standard way to enforce (or at least legally encourage) such adherence is through NDAs. This concern is understandable, and justified: there have been some high profile examples of information about vulnerabilities being leaked to the press before fixes were fully ready to be rolled out.

The chances, however, of the vendor having an NDA with all of the organisations and companies represented by engineers on a particular project is likely to be low, and there may also be individuals who are not associated with any organisation or company in the context of this project. Setting up NDAs can be slow and costly – and sometimes impossible – and the very fact of setting up an NDA might point to a vulnerability’s existence! Vendors, then, may make the choice only to disclose to open source project members who are governed by an NDA – whether through their employer or an NDA with them as an individual.

Process and NDAs

What does that mean to open source projects? Well, it means that if you’re working with specific hardware vendors – or even dependent on specific software vendors – then you may need to think very carefully about who is on your security/vulnerability response/management team (let’s refer to this as the “VMT” – “Vulnerability Management Team” – for simplicity). You may need to ensure that you have competent security engineers both with relevant expertise and who are also under an NDA (corporate or individual) with the hardware vendors with whom you work. Beyond that, you need to ensure that they are on your VMT, or can be tasked to your VMT if required.

This may be a tough ask, particularly for small projects. However, the world of enterprise software is currently going through a process of realising quite how important certain projects are to the underpinnings of their operation, and this is a time to consider whether core open source projects with security implications need more support from large organisations. Encouraging more involvement from organisations should not only help deal with this issue – that vendors may not talk to individuals on your project who are not under NDA with them. Of course, getting more contributors can have positive and negative effects in terms of governance and velocity, but you really do need to think about what would happen if your project were affected by a hardware (or upstream software) vulnerability, and were unable to react as you had nobody able to be involved in discussions.

The other thing you need to do is ensure that the process you have in place for reporting of vulnerabilities is flexible enough to manage this issue. There should be at least one person on the VMT to whom reports can be made who is under the relevant NDA(s): remember that for a vendor, even the act of getting in touch with a project may be enough to give away information about possible vulnerabilities. Although having anonymous VMT may seem attractive – so that you are not giving away too much information about your project’s governance – there can be times when a vendor (or individual) wishes to contact a named individual. If that individual is not part of the VMT, or has no way to trigger the process to deploy an appropriate member of the project to the VMT, then you are in trouble.

I hope that more open source projects will put VMTs and associated processes in place, as I wrote in my previous article. I believe that it’s also important that we consider the commercial and competitive needs of the ecosystem in which we operate, and that means being ready to react within the context of NDAs, embargoes and vulnerabilities. If you are part of an open source project, I urge you to discuss this issue – both internally and also with any vendors whose hardware or software you use.

Note: I would like to thank the members of the Technical Advisory Board of the CCC for prompting the writing of this article, and in particular the member who brought up the issue, which (to my shame) I had not considered in detail before. I hope this treatment of questions arising meets with their approval!


1 – Another standard disclosure: I am not a lawyer! This is my personal description, and should be used as the basis for any legal advice, etc., etc..

Security disclosure or vulnerability management?

Which do I need for an open source project?

This article is a companion piece to one I wrote soon after the advent of Meltdown and Spectre, Meltdown and Spectre: thinking about embargoes and disclosures, and I urge you to read that first, as it provides background and context to this article which I don’t plan to reiterate in full.

In that previous article, I mentioned that many open source projects have a security disclosure process, and most of the rest of the article was basically a list of decisions and steps that you might find in such a process. There’s another term that you might hear, however, which is a Vulnerability Management Process, or “VMP”. While a security disclosure process can be defined as a type of VMP, there are subtle differences to what these two processes might look like, and what they might mean, or be seen to mean, so I think it’s worth spending a little time examining possible differences before we continue.

I just did a “Google fight”[1] for “security disclosure” vs “vulnerability management”, and the former “won” by a ratio of around 5:3. I suspect that this is largely because security disclosures tend to sound exciting and are more “sexy” for headlines than are articles about managing things – even if those things are vulnerabilities.

I’m torn between the two, because both terms highlight important aspects – or reflect different viewpoints – of the same important basic idea: if a bad thing is discovered that involves your product or project, then you need to fix it. I’m going to come at this, as usual for me, from the point of view of open source, so let me give my personal feelings about each.

Security disclosure process

First off, I like the fact that the word “security” is front and centre here. Saying security focusses the mind in ways which “vulnerability” may not, and while a vulnerability may be the thing that we’re addressing, the impact that we’re trying to mitigate is on the security associated with the project or product. The second thing that I like about this phrase is the implication that disclosure is what we are aiming for. Now, this fits well with an open source mindset, but I wonder whether the accent is somewhat different for those who come from a more proprietary background. Where I read “a process to manage telling people about a security problem”, I suspect that others may read “a process to manage the fact that someone has told people about a security problem.” I urge everyone to move to the first point of view, for two reasons:

  1. I believe that security is best done in the open – though we need to find ways to protect people while fixes are being put in place and disseminated;
  2. if we don’t encourage people to come to us – project maintainers, product managers, architects, technical leads – first, in the belief that fixes will be managed as per point 1 above, and also that credit will be given where it’s due, then those who discover vulnerabilities will have little incentive to follow the processes we put in place. If this happens, they are more likely to disclose to the wider world before us, making providing and propagating fixes in a timely fashion much more difficult.

Vulnerability management process

This phrase shares the word “process” with the previous one, and, combined with the world “management”, conveys the importance of working through the issue at hand. It also implies to me, at least, that there is other work to be done around the vulnerability rather than just letting everybody know about it (“disclosure”). This may seem like a bad thing (see above), but on the other hand, acknowledging that vulnerabilities do need managing seems to be a worthwhile thing to signal. What worries me, however, is that managing can be seen to imply “sweeping under the carpet” – in other words, making the problem go away.

Tied with this is something about the word “vulnerability” in this context which is holds both negative and positive connotations. The negative is that one would say “it’s just another vulnerability”, underplaying the security aspect of what it represents. The positive is that sometimes, vulnerabilities are not all of the same severity – some aren’t that serious, compared to others – and it’s important to recognise this, and to have a process which allows you to address all severities of problem, part of which process is to rank and probably prioritise them – often known as “triaging”, a term borrowed from the medical world.

A third option?

There’s at least one alternative. Though it scores lower than either “vulnerability management” or “security disclosure” in a Google fight (which, as I mentioned in the footnotes, isn’t exactly a scientific measure), a “vulnerability disclosure” process is another option. Although it doesn’t capture the “security” aspect, it does at least imply disclosure, which I like.

Which do I need?

My main focus for this article – and my passion and background – is open source, so the next question is: “which do I need for an open source project?” The answer, to some degree, is “either” – or “any of them”, if you include the third option. Arguably, as noted above, a “security disclosure process” is a type of vulnerability management process anyway, but I think that in the open source world particularly, the implication of working towards disclosure – towards openness – is important. The open source community is very sensitive to words, and any suggestion of cover-up is unlikely to be welcomed, even if such an implication was entirely unintentional.

One point that I think it’s worth making is that I believe that pretty much any component or library that you are working on may have security implications down the road[2]. This means that there should be some process in place to deal with vulnerabilities or security issues. To be clear: many of these will be discovered by contributors to the project themselves, rather than external researchers or bug-hunters, but that doesn’t mean that such vulnerabilities are a) less important than externally found ones; OR b) less in need of a process for dealing with them.

A good place to look at what questions to start asking is my previous article, but I strongly recommend that every open source project should have some sort of process in place to deal with vulnerabilities or other issues which may have a security impact. I’ve also written a follow-up article with some options about how to deal with different types of disclosure, vulnerability and process, which you can find here: Open source projects, embargoes and NDAs.


1 – try it: Googlefight.com – it’s a fun (if unscientific) method to gauge the relative popularity of two words or terms on the web.

2 – who would ever have thought that font rendering software could lead to critical security issue?

More Rusty thoughts

I do feel that I’m now actually programming in Rust. And I like it.

I wrote an article a couple of weeks ago – 5 Rust reflections (from Java) – about learning Rust, and specifically, about moving to Rust from Java. I talked about five particular points:

  1. Rust feels familiar
  2. References make sense
  3. Ownership will make sense
  4. Cargo is helpful
  5. The compiler is amazing

I absolutely stand by all of these, but I’ve got a little more to say, because I now feel like a Rustacean[1], in that:

  • I don’t feel like programming in anything else ever again;
  • I’ve moved away from simple incantations.

What do I mean by these two statemetents? Well, the first is pretty simple: Rust feels like the place to be. It’s well-structured, it’s expressive, it helps you do the right thing[2], it’s got great documentation and tools, and there’s a fantastic community. And, of course, it’s all open source, which is something that I care about deeply.

And the second thing? Well, I decided that in order to learn Rust properly, I should take an existing project that I had originally written in Java and reimplement it in hopefully fairly idiomatic Rust. Sometime in the middle of last week, I started fixing mistakes – and making mistakes – around implementation, rather than around syntax. And I wasn’t just copying text from tutorials or making minor, seemingly random changes to my code based on the compiler output. In other words, I was getting things to compile, understanding why they compiled, and then just making programming mistakes[3].

This is a big step forward. When you start learning a language, it’s easy just to copy and paste text that you’ve seen elsewhere, or fiddle with unfamiliar constructs until they – sort of – work. Using code – or producing code -that you don’t really understand, but seems to work, is sometimes referred to as “using incantations” (from the idea that most magicians in fiction, film and gaming reciti collections of magic words which “just work” without really understanding what they’re doing or what the combination of words actually means). Some languages[4] are particularly prone to this sort of approach, but many – most? – people learning a new language will be prone to doing this when they start out, just because they want things to work.

And last night, I was up till 1am implementing a new feature – accepting command-line input – which I really couldn’t get my head round. I’d spent quite a lot of time on it (including looking for, and failing to find, some appropriate incantations), and then asked for some help on a rust-lang channel inhabited by some people I know. A number of people had made some suggestions about what had been going wrong, and one person in particular was enormously helpful in picking apart some of the suggestions so that I understood them better. He explained quite a lot, but finished with “I don’t know the return type of the hash function you’re calling – I think this is a good spot for you to figure this piece out on your own.”

This was just what I needed – and any learner of anything, including programming languages, needs. So when I had to go downstairs at midnight to let the dog out, I decided to stay down and see if I could work things out for myself. And I did. I took the suggestions that people had made, understood out what they were doing, tried to divine what they should be doing, worked out how they should be doing it, and then found the right way of making it happen.

I’ve still got lots to learn, and I’ll make lots of mistakes still, but I now feel that I’m in a place to find my way through those mistakes (with a little help along the way, probably – thanks to everyone who’s already pointed me in the right direction). But I do feel that I’m now actually programming in Rust. And I like it.


1 – this is what Rust programmers call themselves.

2 – it’s almost impossible to stop people doing the wrong thing entirely, but encouraging people do to the right thing is great. In fact, Rust goes further, and actually makes it difficult to do the wrong thing in many situations. You really have to try quite hard to do bad things in Rust.

3 – I found a particularly egregious off-by-one error in my code, for instance, which had nothing to do with Rust, and everything to do with my not paying enough attention to the program flow.

4 – *cough* Perl *cough*

Thunderspy – should I care?

Thunderspy is a nasty attack, but easily prevented.

There’s a new attack out there which is getting quite a lot of attention this week. It’s called Thunderspy, and it uses the Thunderbolt port which is on many modern laptops and other computers to suck data from your machine. I thought that it might be a good issue to cover this week, as although it’s a nasty attack, there are easy ways to defend yourself, some of which I’ve already covered in previous articles, as they’re generally good security practice to follow.

What is Thunderspy?

Thunderspy is an attack on your computer which allows an attacker with moderate resources to get at your data under certain circumstances. The attacker needs:

  • physical access to your machine – not for long (maybe five minutes), but they do need it. This type of attack is sometimes called an “evil maid” attack, as it can be carried out by hotel staff with access to your room;
  • the ability to take your computer apart (a bit) – all we’re talking here is a screwdriver;
  • a little bit of hardware – around $400 worth, according to one source;
  • access to some freely available software;
  • access to another computer at the same time.

There’s one more thing that the attacker needs, and that’s for you to leave your computer on, or in suspend mode. I’ve discussed different power modes before (in 3 laptop power mode options), and mentioned, as well, that leaving your machine in suspend mode is generally a bad idea (in 7 security tips for travelling with your laptop). It turns out I was right.

What’s the bad news?

Well, there’s quite a lot of bad news:

  • lots of machines have Thunderbolt ports (you can find pictures of both the port and connectors on Wikipedia’s Thunderbolt page, in case you’re not sure whether your machine is affected);
  • machines are vulnerable even if you have full disk encryption;
  • Windows machines are vulnerable;
  • Linux machines are vulnerable;
  • Macintosh machines are vulnerable;
  • most machines with a Thunderbolt port from 2011 onwards are vulnerable;
  • although protection is available on some newer machines (from around 2019)
    • the extent of its efficacy is unclear;
    • lots of manufacturers don’t implement it;
  • some protections that you can turn on break USB and other functionality;
  • one variant of the attack breaks Thunderbolt security permanently, meaning that the attacker won’t need to take your computer apart at all for subsequent attacks: they just need physical access to the port whilst your machine it turned on (or in suspend mode).

The worst thing to note is that full disk encryption does not help you if your computer is turned on or in suspend mode.

Note – I’ve been unable to find out whether any Chromebooks have Thunderbolt support. Please check your model’s specifications or datasheet to be certain.

What’s the good news?

The good news is short and sweet: if you turn your computer completely off, or ensure that it’s in Hibernate mode, then it’s not vulnerable. Thunderspy is a nasty attack, but it’s easily prevented.

What should I do?

  1. Turn your computer off when you leave it unattended, even for short amounts of time.

That was easy, wasn’t it? This is best practice anyway, and it turns out that hibernate mode is also OK. What the attacker is looking for is a powered-up, logged-on computer with Thunderbolt. If you can stop them finding a computer that meets those criteria, then you’re fine. Putting your computer into hibernate mode is also OK.

5 Rust reflections (from Java)

I’m a (budding) Rustacean.

It’s been a long time since I properly learned a new language – computer or human. Maybe 25 years. That language was Java, and although I’ve had to write little bits of C (very, very little) and Javascript in the meantime, the only two languages I’ve written much actual code in have been Perl and Java. As I’ve posted before, I’m co-founder of a project called Enarx (latest details here), which is written almost entirely in Rust. These days I call myself an “architect”, and it’s been quite a long time since I wrote any production code. In the lead up to Christmas last year (2019), I completed the first significant project I’ve written in quite a few years: an implementation of a set of algorithms around a patent application, in Java. It was a good opportunity to get my head back into code, and I was quite pleased with it. I wrote it with half a mind to compile it into WebAssembly as a candidate workload for Enarx, but actually compiling it turned out to be a bit of a struggle, and work got in the way, so completed a basic implementation, checked it into a private github repository and generally forgot about it.

My involvement with the Enarx project so far has been entirely design and architecture work – plus documentation, marketing, evangelism, community work and the rest, but no coding. I have suggested, on occasion – almost entirely in jest – that I commit some code to the project in Perl, and it’s become a bit of a running joke that this would be the extent of any code I submitted, as possibly my involvement with the project, as it would be immediately rejected. And then, about a week and a half ago, I decided to learn Rust. And then to rewrite (including, where necessarily refactoring) that Java project I wrote a few months ago. Here are some of my thoughts on Rust, from the point of a view of a Java developer with a strong Object-Oriented background.

1. Rust feels familiar

Although many of the tutorials and books you’ll find out there are written with C and C++in mind, there’s enough similarity with Java to make the general language feel familiar. The two tutorials I’ve been using the most are The Rust Programming Language online and Programming Rust in dead tree format, and the latter makes frequent references to similarities and differences to and from other languages, including not only C, C++ and Java, but also Python, Javascript and others. Things like control structures and types are similar enough to Java that they’re generally simple to understand, and although there are some major differences, you should be able to get your head round the basics of the language pretty simply. Beware, however: one of the biggest initial problems I’ve been having is that Rust sometimes feels too familiar, so I start trying to do things in the wrong way, have to back out, and try to work out a better way: a way which is more idiomatic to Rust. I have a long way to go on this!

2. References make sense

In Rust, you end up having to use references. Frankly, referencing and de-referencing variables was something that never made much sense to me when I looked at C or C++, but this time, it feels like I get it. If you’re used to passing Java variables by reference and value, and know when you need to take steps to do so differently in specific situations, then you’re ready to start understanding Rust references. The other thing you need to understand is why Rust needs you to use them: it’s because Rust is very, very careful about memory management, and you don’t have a Garbage Collector to clean up after you wherever you go (as in Java). You can’t just pass Strings (for instance) around willy-nilly: Rust is going to insist that you know the lifetime of a variable, and think about when it’s ready to be “dropped”. This means that you need to think hard about scope, and introduces a complex concept: ownership.

3. Ownership will make sense

Honestly, I’m not there yet. I’ve been learning and coding in Rust for under two weeks, and I’m beginning to get my head around ownership. For me (as, I suspect, for many newcomers), this is the big head-shift around moving to Rust from Java or most other languages: ownership. As I mentioned above, you need to understand when a variable is going to be used, and how long it will live. There’s more to it than that, however, and really getting this is something which feels a little foreign to me as a Java developer: you need to understand about the stack and the heap, a distinction which was sufficiently concealed from me by Java, but something which many C and C++ developers will probably understand much more easily. This isn’t the place to explain the concept (I’ve found the diagrams in Programming Rust particularly helpful), but in order to manage the lifetime of variables in memory, Rust is going to need to know what component owns each one. This gets complicated when you’re used to creating objects and instantiating them with variables from all over the place (as in Java), and requires some significant rethinking. Combining this with explicit marking of lifetimes is the biggest conceptual change that I’m having to perform right now.

4. Cargo is helpful

I honestly don’t use the latest and greatest Java tools properly. When I started to learn Java, it wasn’t even in 1.0, and by the time I finished writing production code on a regular basis, there wasn’t yet any need to pick up the very latest tooling, so it may be that Java is better at this than I remember, but the in-built tools for managing the various pieces of a Rust project, including dependencies, libraries, compilation and testing, are a revelation. The cargo binary just does the right thing, and it’s amazing to watch it do its job when it realises that you’ve made a change to your dependencies, for instance. It will perform automatic tests, optimise automatically, produce documentation – so many useful tasks, all within one package. Combine this with git repositories, and managing projects becomes saner and easier.

5. The compiler is amazing

Last, but very far from least, is the compiler. I love the Rust compiler: it really, really tries to help you. The members of the community[1] that makes and maintains clearly go out of their way to provide helpful guidance to correct you when you make mistakes – and I, for one, have been making many of them. Rather than the oracular pronouncements that may be familiar from other languages’ compilers, you’ll get colour-coded text with warnings and errors, and suggestions as to what you might actually be trying to do. You will even be given output such as For more information about this error, try rustc --explain E0308. When you do try this, you get (generally!) helpful explanation and code snippets. Sometimes, particularly when you’re still working your way into the language, it’s not always obvious what you’re doing wrong, but wading through the errors can help you get your head round the concepts in a way which feels very different to messages I’m used to getting from javac, for example.

Conclusion

I don’t expect ever to be writing lots of production Rust, nor ever truly to achieve guru status – in Rust or any other language, to be honest – but I really think that Rust has a lot to be said for it. Throughout my journey so far, I’ve been nodding my head and thinking “that’s a good way to do that”, or “ah, that makes so much more sense than the way I’m used to”. This isn’t an article about why Rust is such a good language – there are loads of those – nor about the best way to learn Rust – there are lots of those, too – but I can say that I’m enjoying it. It’s challenging, but one thing that the tutorials, books and other learning materials are all strong on is explaining the reasons for the choices that Rust makes, and that’s certainly been helpful to me, both in tackling my frustrations, but also in trying to internalise some of the differences between Java and Rust.

If I can get my head truly into Rust, I honestly don’t think I’m likely to write any Java ever again. I’m not sure I’ve got another 25 years of coding in me, but I think that I’m with Rust for the long haul now. I’m a (budding) Rustacean.


1 – Rust, of course, is completely open source, and the community support for it seems amazing.