enarx – Alice, Eve and Bob

Executive Director, Confidential Computing Consortium

I look forward to furthering the aims of the CCC

I’m very pleased to announce that I’ve just started a new role as part-time Executive Director for the Confidential Computing Consortium, which is a project of the The Linux Foundation. I have been involved from the very earliest days of the consortium, which was founded in 2019, and I’m delighted to be joining as an officer of the project as we move into the next phase of our growth. I look forward to working with existing and future members and helping to expand industry adoption of Confidential Computing.

For those of you who’ve been following what I’ve been up to over the years, this may not be a huge surprise, at least in terms of my involvement, which started right at the beginning of the CCC. In fact, Enarx, the open source project of which I was co-founder, was the very first project to be accepted into the CCC, and Red Hat, where I was Chief Security Architect (in the Office of the CTO) at the time, was one of the founding members. Since then, I’ve served on the Governing Board (twice, once as Red Hat’s representative as a Premier member, and once as an elected representative of the General members) acted as Treasurer, been Co-chair of the Attestation SIG and been extremely active in the Technical Advisory Council. I was instrumental in initiating the creation of the first analyst report into Confidential Computing and helped in the creation of the two technical and one general white paper published by the CCC. I’ve enjoyed working with the brilliant industry leaders who more than ably lead the CCC, many of whom I now count not only as valued colleagues but also as friends.

The position – Executive Director – however, is news. For a while, the CCC has been looking to extend its activities beyond what the current officers of the consortium can manage, given that they have full-time jobs outside the CCC. The consortium has grown to over 40 members now – 8 Premier, 35 General and 8 Associate – and with that comes both the opportunity to engage in a whole new set of activities, but also a responsibility to listen to the various voices of the membership and to ensure that the consortium’s activities are aligned with the expectations and ambitions of the members. Beyond that, as Confidential Computing becomes more pervasive, it’s time to ensure that (as far as possible), there’s a consistent, crisp and compelling set of messages going out to potential adopters of the technology, as well as academics and regulators.

I plan to be working on the issues above. I’ve only just started and there’s a lot to be doing – and the role is only part-time! – but I look forward to furthering the aims of the CCC:

The Confidential Computing Consortium is a community focused on projects securing data in use and accelerating the adoption of confidential computing through open collaboration.
The core mission of the CCC

Wish me luck, or, even better, get in touch and get involved yourself.

Enarx hits 750 stars

Yesterday, Enarx, the open source security project of which I’m co-founder and for which Profian is custodian, gained its 750th GitHub star. This is an outstanding achievement, and I’m very proud of everyone involved. Particular plaudits to Nathaniel McCallum, my co-founder for Enarx and Profian, Nick Vidal, the community manager for Enarx, everyone who’s been involved in committing code, design, tests and documentation for the project, and everyone who manages the running of the project and its infrastructure. We’ve been lucky enough to be joined by a number of stellar interns along the way, who have also contributed enormously to the project.

Enarx has also been supported by a number of organisations and companies, and it’s worth listing as many of them as I can think of:

Profian, the current custodian
Red Hat, under whose auspices the initial development began
the Confidential Computing Consortium, a Linux Foundation Project, which owns the project
Equinix, who have donated computing resources
PhoenixNAP, who have donated computing resources
Rocket.Chat, who have donated chat resources
Intel, who have worked with us along the way and donated various resources
AMD, who have worked with us along the way and donated various resources
Outreachy, with whom worked to get some of our fine interns

When it all comes down to it, however, it’s the community that makes the project. We strive to create a friendly, open community, and we want more and more people to get involved. To that end, we’ll soon be announcing some new ways to get involved with trying and using Enarx, in association with Profian. Keep an eye out, and keep visiting and giving us stars!

Enarx and Pi (and Wasm)

It’s not just Raspberry Pi, but also Macs.

A few weeks ago, I wrote a blog post entitled WebAssembly: the importance of language(s), in which I talked about how important it is for Enarx that WebAssembly supports multiple languages. We want to make it easy for as many people as possible to use Enarx. Today, we have a new release of Enarx – Elmina Castle – and with it comes something else very exciting: Raspberry Pi support. In fact, there’s loads more in this release – it’s not just Raspberry Pi, but also Macs – but I’d like to concentrate on what this means.

As of this release, you can run WebAssembly applications on your Raspberry Pi, using Enarx. Yes, that’s right: you can take your existing Raspberry Pi (as long as it’s running a 64bit kernel), and run Wasm apps with the Enarx framework.

While the Enarx framework provides the ability to deploy applications in Keeps (TEE[1] instances), one of the important features that it also brings is the ability to run applications outside these TEEs so that you can debug and test your apps. The ability to do this much more simply is what we’re announcing today.

3 reasons this is important

1. WebAssembly just got simpler

WebAssembly is very, very hot at the moment, and there’s a huge movement behind adoption of WASI, which is designed for server-based (that is, non-browser) applications which want to take advantage of all the benefits that Wasm brings – cross-architecture support, strong security model, performance and the rest.

As noted above, Enarx is about running apps within Keeps, protected within TEE instances, but access to the appropriate hardware to do this is difficult. We wanted to make it simple for people without direct access to the hardware to create and test their applications on whatever hardware they have, and lots of people have Raspberry Pis (or Macs).

Of course, some people may just want to use Enarx to run their Wasm applications, and while that’s not the main goal of the project, that’s just fine, of course!

2. Tapping the Pi dev community

The Raspberry Pi community is one of the most creative and vibrant communities out there. It’s very open source friendly, and Raspberry Pi hardware is designed to be cheap and accessible to as many people as possible. We’re very excited about allowing anyone with access to a Pi to start developing WebAssembly and deploying apps with Enarx.

The Raspberry Pi community also has a (deserved) reputation for coming up with new and unexpected uses for technology, and we’re really interested to see what new applications arise: please tell us.

3. Preparing for Arm9 Realms

Last, and far from least, is the fact that in 2021, Arm announced their CCA (Confidential Compute Architecture), coming out with the Arm9 architecture. This will allow the creation of TEEs called Realms, which we’re looking forward to supporting with Enarx. Running Enarx on existing Arm architecture (which is what powers Raspberry Pis) is an important step towards that goal. Extending Enarx Keeps beyond the x86 architecture (as embodied by the Intel SGX and AMD SEV architectures) has always been a goal of the project, and this provides a very important first step which will allow us to move much faster when chips with the appropriate capabilities start becoming available.

How do I try it on my Raspberry Pi?

First, you’ll need a Raspberry Pi running a 64bit kernel. Instructions for this are available over at the Raspberry Pi OS pages, and the good news is that the default installer can easily put this on all of the more recent hardware models.

Next, you’ll need to follow the instructions over at the Enarx installation guide. That will walk you through it, and if you have any problems, you can (and should!) report them, by chatting with the community over at our chat or by searching for/adding bug issues at our issue tracker.

We look forward to hearing how you’re doing. If you think this is cool (and we certainly do!), then please head to our main repository at https://github.com/enarx/enarx and give us a star.

1 – Trusted Execution Environments, such as Intel’s SGX and AMD’s SEV.

Image: Michael H. („Laserlicht“) / Wikimedia Commons

WebAssembly: the importance of language(s)

We provide a guide so that you can try each lanuage for yourself.

Over at Enarx, we’re preparing for another release. They’re coming every four weeks now, and we’re getting into a good rhythm. Thanks to all contributors, and also those working on streamlining the release process. It’s a complex project with lots of dependencies – some internal, and some external – and we’re still feeling our way about how best to manage it all. One thing that you will be starting to see in our documentation, and which we intend to formalise in coming releases, is support for particular languages. I don’t mean human languages (though translations of Enarx documentation into different languages, to support as diverse a community as we can, is definitely of interest), but programming languages.

Enarx is, at its heart, a way to deploy applications into different environments: specifically, Trusted Execution Environments (though we do support testing in kvm). The important word here is “execution”, because applications need a runtime in which to execute. Runtimes come in many different flavours: ELF (“Executable and Linking Format”, the main standard for Linux systems), JVM (“Java Virtual Machine”, for compiled Java classes) and PE (“Portable Executable”, used by Windows), to give but a few examples. Enarx uses WebAssembly, or, to be more exact, WASI, which you can think of as a “headless” version of WebAssembly: whereas WebAssembly was originally designed to run within browsers, WASI-compliant runtimes support server-type applications. The runtime which Enarx supports is called wasmtime, which is a Bytecode Alliance project, and written in Rust (like Enarx itself).

This is great, but (almost) nobody writes native WebAssembly code (there is actually a “human-readable” format supported by the standard, but I personally wouldn’t want to be writing in it directly!). One of the great things about WebAssembly is that it’s largely language-neutral: you can write in your favourite language and then compile your application to a “wasm” binary to be executed by the runtime (wasmtime, in our case). WebAssembly is attracting lots of attention within the computing community at the moment, and so people have created lots of different mechanisms to allow their favourite languages to create wasm binaries. There’s a curated list here, though it’s not always updated very frequently, and given the amount of interest in the space, it may be a little out of date when you visit the page. In the list, you’ll find common languages like C, C++, Rust, Golang, .Net, Python and Javascript, as well as less obvious ones like Haskell, COBOL and Scheme. Do have a look – you may be surprised to find support for your favourite “obscure” language is already started, or even quite mature.

This proliferation of languages with what we could call “compile target support” for WebAssembly is excellent news for Enarx, because it means that people writing in these languages may be able to write applications that we can run. I say may, because there’s a slight complication, which is that not all of these compile targets support WASI, which is the specific interface supported by wasmtime, and therefore by Enarx.

This is where the Enarx community has started to step in. They – we – have been creating a list of languages which do allow you to compile wasm binaries that execute under wasmtime, and therefore in Enarx. You’ll find a list over at our WebAssembly Guide and, at time of writing, it includes Rust, C++, C, Golang, Ruby, .NET, TypeScript, AssemblyScript, Grain, Zig and JavaScript[1]. You can definitely expect to see more coming in the near future. With this list, we don’t just say “you can run applications compiled from this language”, but provide a guide so that you can try each lanuage for yourself! Currently the structure of how the information is presented varies from language to language – we should probably try to regularise this – but in each case, there should be sufficient information for someone fairly familiar with the lanaguage to write a simple program and run it in Enarx.

As I noted above, not all languages with compile target support for WebAssembly will work yet, but we’re also doing “upstream” work in some cases to help particular languages get to a position where they will work by submitting patches to fix specific issues. This is an area where more involvement from the community (that means you!) can help: the more people contributing to this work, or noting how important it is to them, the quicker we’ll gain support for more languages.

And here’s where we hope to be: in upcoming releases, we want to be in a position where Enarx officially supports particular languages. What exactly that “support” entails is something we haven’t yet fully defined, but, at minimum, we hope to be able to say something like “applications written in this language using this set of capabilities/features are expected to work”, based on automated testing of “known good” code on a per-release basis. This will mean that users of Enarx will be able to have high confidence that an application working on one release will behave exactly the same on the next: a really important property for a project intended for commercial deployments.

How can you get involved? Well, the most obvious is to visit the page in our docs relating to your favourite language. Try it out, give us feedback or offer to improve the documentation if you think it needs it, or even go upstream and offer patches. If no such page exists, you could visit our chat channels and ask to see if anyone is working on support and/or create an issue requesting support, explaining why you think it’s important.

Finally, to encourage upstream developers to realise how important supporting “their” language is, you can provide a GitHub star by visiting https://enarx.dev or https://github.com/enarx/enarx. “Starring” the project is a way to register your interest, and to show the community that Enarx is something you’re interested in.

1 – Huge thanks to everyone involved in these efforts, with a special shout-out to Deepanshu Arora, who’s done lots of work in this area.

WebAssembly logo: By Carlos Baraza – Own work / https://github.com/carlosbaraza/web-assembly-logo, CC0, https://commons.wikimedia.org/w/index.php?curid=56494100

More Enarx milestones

It’s been a big month for Enarx.

It’s being a big month for Enarx. Last week, I announced that we’d released Enarx 0.3.0 (Chittorgarh Fort), with some big back-end changes, and some new functionality as well. This week, the news is that we’ve hit a couple of milestones around activity and involvement in the project.

1500 commits

The first milestone is 1500 commits to the core project repository. When you use a git-based system, each time you make a change to a file or set of files (including deleting old ones, creating new one and editing or removing sections), you create a new commit. Each commit has a rather long identifier, and its position in the project is also recorded, along with the name provided by the committer and any comments. Commit 1500 to the enarx was from Nathaniel McCallum, and entitled feat(wasmldr): add Platform API. He committed it on Saturday, 2022-03-19, and its commit number is 8ec77de0104c0f33e7dd735c245f3b4aa91bb4d2.

I should point out that this isn’t the 1500th commit to the Enarx project, but the 1500th commit to the enarx/enarx repository on GitHub. This is the core repository for the Enarx project, but there are quite a few others, some of which also have lots of commits. As an example, the enarx/enarx-shim-sgx repository ,which provides some SGX-specific capabilities within Enarx, had 968 commits at time of writing.

500 Github stars

The second milestone is 500 GitHub stars. Stars are measure of how popular a repository or project is, and you can think of them as the Github of a “like” on social media: people who are interested in it can easily click a button on the repository page to “star” it (they can “unstar” it, too, if they change their mind). We only tend to count stars on the main enarx/enarx repository, as that’s the core one for the Enarx project. The 500th star was given to the project by a GitHub user going by the username shebuel-oss, a self-described “Open Source contributor, Advocate, and Community builder”: we’re really pleased to have attracted their interest!

There’s a handy little website which allows you to track stars to a project called GitHub Star History where you can track the addition (or removal!) of stars, and compare other projects. You can check the status of Enarx whenever you’re reading by following this link, but for the purposes of this article, the important question is how did we get to 500? Here’s a graph:

You’ll see a nice steep line towards the end which corresponds to Nick Vidal’s influence as community manager, actively working to encourage more interest and involvement, and contributions to the Enarx project.

Why do these numbers matter?

Objectively, they don’t, if I’m honest: we could equally easily have chosen a nice power of two (like 512) for the number of stars, or the year that Michelangelo started work on the statue David (1501) for the number of commits. Most humans, however like round decimal numbers, and the fact that we hit 1500 and 500 commits and stars respectively within a couple of days of each provides a nice visual symmetry.

Subjectively, there’s the fact that we get to track the growth in interest (and the acceleration in growth) and contribution via these two measurements and their historical figures. The Enarx project is doing very well by these criteria, and that means that we’re beginning to get more visibility of the project. This is good for the project, it’s good for Profian (the company Nathaniel and I founded last year to take Enarx to market) and I believe that it’s good for Confidential Computing and open source more generally.

But don’t take my word for it: come and find out about the project and get involved.

Enarx 0.3.0 (Chittorgarh Fort)

Write some applications and run them in an Enarx Keep.

I usually post on a Tuesday, but this week I wanted to wait for a significant event: the release Enarx v0.3.0, codenamed “Chittorgarh Fort”. This happened after I’d gone to bed, so I don’t feel too bad about failing to post on time. I announced Enarx nearly three years ago, in the article Announcing Enarx on the 7th May 2019. and it’s admittedly taken us a long time to get to where we are now. That’s largely because we wanted to do it right, and building up a community, creating a start-up and hiring folks with the appropriate skills is difficult. The design has evolved over time, but the core principles and core architecture are the same as when we announced the project.

You can find more information about v0.3.0 at the release page, but I thought I’d give a few details here and also briefly add to what’s on the Enarx blog about the release.

What’s Enarx?

Enarx is a deployment framework for running applications within Trusted Execution Environments (TEEs). We provide a WebAssembly runtime and – this is new functionality that we’ve started adding in this release – attestation so that you can be sure that your application is protected within a TEE instance.

What’s new in v0.3.0?

A fair amount of the development for this release has been in functionality which won’t be visible to most users, including a major rewrite of the TEE/host interface component that we call sallyport. You will, however, notice that TLS support has been added to network connections from applications within the Keep. This is transparent to the application, so “Where does the certificate come from?” I hear you ask. The answer to that is from the attestation service that’s also part of this release. We’ll be talking more about that in further releases and articles, but key to the approach we’re taking is that interactions with the service (we call it the “Steward”) is pretty much transparent to users and applications.

How can I get involved?

What can you do to get involved? Well, visit the Enarx website, look at the code and docs over at our github repositories (please star the project!), get involved in the chat. The very best thing you can do, having looked around, is to write some applications and run them in an Enarx Keep. And then tell us about your experience. If it worked first time, then wow! We’re still very much in development, but we want to amass a list of applications that are known to work within Enarx, so tell us about it. If it doesn’t work, then please also tell us about it, and have a look at our issues page to see if you’re the first person to run across this problem. If you’re not, then please add your experiences to an existing issue, but if you are, then create a new one.

Enarx isn’t production ready, but it’s absolutely ready for initial investigations (as shown by our interns, who created a set of demos for v0.2.0, curated and aided by our community manager Nick Vidal).

Why Chittorgarh Fort?

It’s worth having a look at the Wikipedia entry for the fort: it’s really something! We decided, when we started creating official releases, that we wanted to go with the fortification theme that Enarx has adopted (that’s why you deploy applications to Enarx Keeps – a keep is the safest part of a castle). We started with Alamo, then went to Balmoral Castle, and then to Chittorgarh Fort (we’re trying to go with alphabetically sequential examples as far as we can!). I suggested Chittorgarh Fort to reflect the global nature of our community, which happens to include a number of contributors from India.

Who was involved?

I liked the fact that the Enarx blog post mentioned the names of some (most?) of those involved, so I thought I’d copy the list of github account names from there, with sincere thanks:

@MikeCamel @npmccallum @haraldh @connorkuehl @lkatalin @mbestavros @wgwoods @axelsimon @ueno @ziyi-yan @ambaxter @squidboylan @blazebissar @michiboo @matt-ross16 @jyotsna-penumaka @steveeJ @greyspectrum @rvolosatovs @lilienbm @CyberEpsilon @kubkon @nickvidal @uudiin @zeenix @sagiegurari @platten @greyspectrum @bstrie @jarkkojs @definitelynobody @Deepansharora27 @mayankkumar2 @moksh-pathak

Rahultalreja11 at English Wikipedia, CC BY-SA 3.0 https://creativecommons.org/licenses/by-sa/3.0, via Wikimedia Commons

Emotional about open source

Enarx is available to all, usable by all.

Around October 2019, Nathaniel McCallum and I founded the Enarx project. Well, we’d actually started it before then, but it’s around then that the main GitHub repo starts showing up, when I look at available info. In the middle of 2021, we secured funding a for a start-up (now named Profian), and since then we’ve established a team of engineers to work on the project, which is itself part of the Confidential Computing Consortium. Enarx is completely open source, and that’s really central to the project. We want (and need) the community to get involved, try it out, improve it, and use it. And, of course, if it’s not open source, you can’t trust it, and that’s really important for security.

The journey has been hard at times, and there were times when we nearly gave up on the funding, but neither Nathaniel nor I could see ourselves working on anything else – we really, truly believe that there’s something truly special going on, and we want to bring it to the world. I’m glad (and relieved) that we persevered. Why? Because last week, on Thursday, was the day that this came true for me. The occasion was OC3, a conference in Confidential Computing organised by Edgeless Systems. I was giving a talk on Understanding trust relationships for Confidential Computing, which I was looking forward to, but Nick Vidal, Community Manager for the Enarx project, also had a session earlier on. His session was entitled From zero to hero: making Confidential Computing accessible, and wasn’t really his at all: it was taken up almost entirely by interns in the project, with a brief introduction and summing up by Nick.In his introduction, Nick explained that he’d be showing several videos recorded by the interns of demos they had recorded. These demos took the Enarx project and ran applications that the (they interns) had created within Keeps, using the WebAssembly runtime provided within Enarx. The interns and their demos were:

TCP Echo Server (Moksh Pathak & Deepanshu Arora) – Mosksh and Deepanshu showed two demos: a ROT13 server which accepts connections, reads text from them and returns the input, ROT13ed; and a simple echo server.
Fibonacci number generator (Jennifer Chukwu) – a simple Fibonacci number generator running in a Keep
Machine learning with decision tree algorithm on Diabetes data set (Jennifer Kumar & Ajay Kumar) – implementation of Machine Learning, operating on a small dataset.
Zero Knowledge Proof using Bulletproof (Shraddha Inamdar) – implementation of a Zero Knowledge Proof with verification.

What is exciting about these demos is several-fold:

three of them have direct real-world equivalent use cases:
1. The ROT13 server, while simple, could be the basis for an encryption/decryptions service.
2. the Machine Learning service is directly relevant to organisations who wish to run ML workloads in the Cloud, but need assurances that the data is confidentiality and integrity protected.
3. the Zero Knowledge Proof demo provides an example of a primitive required for complex transaction services.
none of the creators of the demos knew anything about Confidential Computing until a few months ago.
none of the creators knew much – if anything – about WebAssembly before coming to the project.
none of the creators is a software engineering professional (yet!). They are all young people with an interest in the field, but little experience.

What this presentation showed me is that what we’re building with Enarx (though it’s not even finished at this point) is a framework that doesn’t require expertise to use. It’s accessible to beginners, who can easily write and deploy applications with obvious value. This is what made me emotional: Enarx is available to all, usable by all. Not just security experts. Not just Confidential Computing gurus. Everyone. We always wanted to build something that would simplify access to Confidential Computing, and that’s what we, the community, have brought to the world.

I’m really passionate about this, and I’d love to encourage you to become passionate about it, too. If you’d like to know more about Enarx, and hopefully even try it yourself, here are some ways to do just that;

visit our website, with documentation, examples and a guide to getting started
join our chat and then one of our stand-ups
view the code over at GitHub (and please star the project: it encourages more people to get involved!)
read the Enarx blog
watch the video of the demos.

I’d like to finish this post by thanking not only the interns who created the demos, but also Nick Vidal, for the incredible (and tireless!) work he’s put into helping the interns and into growing the community. And, of course, everyone involved in the project for their efforts in getting us to where we are (and the vision to continue to the next exciting stages: subscribe to this blog for upcoming details).

Enarx 0.2.0 – Balmoral Castle

Now it’s possible to write applications that you can talk to over the network

The big news this week from the Enarx project is our latest release: 0.2.0, which is codenamed “Balmoral Castle”, to continue with our castle/fortification theme.

The big change in Enarx 0.2.0 is the addition of support for networking. Until now, there wasn’t much you could really do in an Enarx Keep, honestly: you could run an application, but all it could to for input and output was read from stdin and write to stdout or stderr. While this was enough to prove that you could write and compile applications to WebAssembly and run them, any more complex interaction with the world outside the Keep was impossible.

So, why is this big news? Well, now it’s possible to write applications that you can talk to over the network. The canonical example which we’ve provided as part of the release is a simple “echo” server, which you start in a Keep and then listens on a port for incoming connections. You make a connection (for instance using the simple command-line utility ncat), and send it a line of text. The server accepts the connection, receives the text and sends it right back you. It can handle multiple connections and will send the text back to the right one (hopefully!).

This is new functionality with Enarx 0.2.0, and the ability to use networking mirrors an important change within WASI (the WebAssembly System Interface) specification, the runtime implemented within an Enarx Keep. Specifically, WASI snapshot preview 1, released in January (2022) now supports the the ACCEPT capability on sockets. The way that WASI works with managing permissions and capabilities is carefully designed, and we (the Profian folks working on Enarx) coordinated closely with the open source WASI/Wasm community to add this in a way which is consistent with the design philosophy of the project. Once the capability was added to the snapshot, there was one more step needed before Enarx could implement support, which was that it needed to appear in wasmtime, the WebAssembly runtime we use within Keeps to allow you to run your applications. This happened last week, in wasmtime release 0.34.0, and that allowed us to make this new release of Enarx.

This may not sound very exciting … but with this in place, you start to build proper applications and micro-services. What about an HTTP server? A ROT13 “encryption” service? A chatbot? An email server? A Wordle implementation[1]? And it’s not just text that you can send over a network connection, of course. What might write to process other types of data? A timestamp server? A logging service? With a network connection, you have the ability to write any of these. Micro-services are all about accepting connections, processing the data that’s come in, and then sending out the results. All of that is possible with this new release.

What we’d love you to do is to start writing applications (using networking) and running them in Enarx. Tell us what works – even better, tell us what doesn’t by creating an issue in our github repository . Please publish examples, join our chat channels, give us a github star, get involved.

What’s coming next? Well, keep an eye on the Enarx site, but be assured that I’ll announce major news here as well. You can expect work in attestation and deployment in the near future – watch this space…

1 – at time of writing, everyone’s talking about Wordle. For those of you coming from the future (say a couple of weeks from now), you can probably ignore this example.

[Image of Edward VII at Balmoral Castle from Wikimedia].

A new state of mind

I’m quite proud; though maybe slightly ashamed that I didn’t do it before.

Last year, I co-founded Profian with Nathaniel McCallum, a colleague from Red Hat. It’s a security start-up in the Confidential Computing Space, based on the open source Enarx project. There’s an update on that on the Profian blog with an article entitled Design to Roadmap to Product.

It’s an article on what we’ve been up to in the company, and a records the realisation that it’s time for me to step into yet another role as one of the founders: moving beyond the “let’s make sure that we have a team and that the basic day-to-day running of the company is working” to “OK, let’s really map out our product roadmap and how we present them to customers.”

A new state of mind

Which leads me to the main point of this short article. This is not an easy transition – it’s yet another new thing to learn, discover which bits I’m good at, improve the bits I’m not, get internal or external help to scale with, etc. – but it’s a vital part of being the CEO of a start-up.

It’s also something which I had, to be honest, been resisting. Most of us prefer to stick to stuff which we know – whether we’re good at it or not, sometimes! – rather than “embracing change”. Sometimes that’s OK, but in the position I’m in at the moment, it’s not. I have responsibility to the company and everyone involved in it to ensure that we can be successful. And that means doing something. So I’ve been listening to people say, “these are the things you need to do”, “here are the ways we can help you”, “this is what you should be looking for” and, while listening, just, well, putting it off, I suppose. Towards the end of last week, I ordered a book (The Founder Handbook) to try to get my head round it a bit more. There are loads of this type of book, but I did a little research, and this looked like it might be one of the better ones.

So, it arrived, and I started reading it. And, darn it, it made sense. It made me start seeing the world in a new way – a way which might not have been relevant to me (or the company) a few months ago, but really is, now. And I really need to embrace lots of the things the authors are discussing. I’m not saying that it’s a perfect book, or that no other book would have prompted this response, but at some point over the weekend, I thought: “right, it’s time to change and to move into this persona, thinking about these issues, being proactive and not putting it off anymore”.

I’m quite proud, to be honest; though maybe slightly ashamed that I didn’t do it before. I cemented the decision to jump into a new mindset by doing what I’ve done on a couple of occasions before (including when I decided to commit to writing my book): I told a few people what I was planning to do. This really works for me on several levels:

I’ve made a public commitment (even if it’s to a few people[1]), so it’s difficult to roll it back;
I’ve made a commitment to myself, so I can’t pretend that I haven’t and let myself drift back into the old mindset;
it sets expectations from other people as to what I’m going to do;
people are predisposed to being helpful when you struggle, or ask for help.

These are all big positives, and while telling people you’ve made a big decision may not work for everyone, it certainly helps for me. This is going to be only one of many changes I need to make if we’re to build a successful company out of Profian and Enarx, but acknowledging that it needed to be made – and that I was the one who was going to have to effect that change – is important to me, the company, our investors and our employees. Now all I need to do is make a success of it! Wish me luck (and keep an eye out for more…).

1 – a few more people now, I suppose, now that I’ve published this article!

How to hire an open source developer

Our view was that a pure “algorithm coding” exercise was pretty much useless for what we wanted.

We’ve recently been hiring developers to work on the Enarx project, a security project, written almost exclusively in Rust (with a bit of Assembly), dealing with Confidential Computing. By “we”, I mean Profian, the start-up for which I’m the CEO and co-founder. We’ve now found all the people we’re looking for initially on the team (with a couple due to start in the next few weeks), though we absolutely welcome contributors to Enarx, and, if things continue to go well, we’ll definitely want to hire some more folks in the future.

Hiring people is not easy, and we were hit with a set of interesting requirements which made the task even more difficult. I thought it would be useful and interesting for the community to share how we approached the problem.

What were we looking for?

I mentioned above some interesting requirements. Here’s what the main ones were:

systems programming – we mainly need people who are happy programming at the systems layer. This is pretty far down the stack, with lots of interactions directly with hardware or the OS. Where we are creating client-server pieces, for instance, we’re having to write quite a lot of the protocols, manage the crypto, etc., and the tools we’re using aren’t all very mature (see “Rust” below).
Rust – almost all of the project is written in Rust, and what isn’t is written in Assembly language (currently exclusively x86, though that may change as we add more platforms). Rust is new, cool and exciting, but it’s still quite young, and some areas don’t have all the support you might like, or aren’t as mature as you might hope – everything from cryptography through multi-threading libraries and compiler/build infrastructure.
distributed team – we’re building a team of folks where can find them: we have developers in Germany, Finland, the Netherlands, North Carolina (US), Massachusetts (US), Virginia (US) and Georgia (US), I’m in the UK, our community manager is in Brazil and we have interns in India and Nigeria. We knew from the beginning that we wouldn’t have everyone in one place, and this required people who we were happy would be able to communicate and collaborate with people via video, chat and (at worst) email.
security – Enarx is a security project, and although we weren’t specifically looking for security experts, we do need people who are able to think and work with security top of mind, and design and write code which is applicable and appropriate for the environment.
git – all of our code is stored in git (mainly GitHub, with a little bit of GitLab thrown in), and so much of our interaction around code revolves around git that anybody joining us would need to be very comfortable using it as a standard tool in their day-to-day work.
open source – open source isn’t just a licence, it’s a mindset, and, equally important, a way of collaborating. A great deal of open source software is created by people who aren’t geographically co-located, and who might not even see themselves as a team. We needed to be sure that the people we were hiring, while gelling as a close team within the company, will also be able to collaborate with people outside the organisation and be able to embrace Profian’s “open by default” culture not just for code, but for discussions, communications and documentation.

How did we find them?

As I’ve mentioned before, in Recruiting is hard. We ended up using a variety of means to find candidates, with varying levels of success:

LinkedIn job adverts
LinkedIn searches
Language-specific discussion boards and hiring boards (e.g. Reddit)
An external recruiter (shout out to Gerald at Interstem)
Word-of-mouth/personal recommendations

It’s difficult to judge between them in terms of quality, but without an external recruiter, we’d certainly have struggled with quantity (and we had some great candidates from that pathway, too).

How did we select them?

We needed to measure all of the candidates against all of the requirements noted above, but not all of them were equal. For instance, although we were keen to hire Rust programmers, we were pretty sure that someone with strong C/C++ skills at the systems level would be able to pick up Rust quickly enough to be useful. On the other hand, a good knowledge of using git was absolutely vital, as we couldn’t spend time working with new team members to bring them up-to-speed on our way of working. A strong open source background was, possibly surprisingly, not a requirement, but the mindset to work in that sort of model was, and anyone with a history of open source involvement is likely to have a good knowledge of git. The same goes for the ability to work in a distributed team: so much of open source is distributed that involvement in almost any open source community was a positive indicator. Security we decided was a “nice-to-have”.

How to proceed? We wanted to keep the process simple and quick – we don’t have a dedicated HR or People function, and we’re busy trying to get code written. What we ended up was this (with slight variations), which we tried to get complete within 1-2 weeks:

Initial CV/resume/github/gitlab/LinkedIn review – this to decide whether to interview
30-40 minute discussion with me as CEO, to find out if they might be a good cultural fit, to give them a chance to find out about us, and get an idea if they were as technically adept as they appeared from the first step
Deep dive technical discussion led by Nathaniel, usually with me there
Chat with other members of the team
Coding exercise
Quick decision (usually within 24 hours)

The coding exercise was key, but we decided against the usual approach. Our view was that a pure “algorithm coding” exercise of the type so beloved by many tech companies was pretty much useless for what we wanted. What we wanted to understand was whether candidates could quickly understand a piece of code, fix some problems and work with the team to do so. We created a github repository (in fact, we ended up using two – one for people a little higher up the stack) with some almost-working Rust code in it, some instructions to fix it, perform some git-related processes on it, and then improve it slightly, adding tests along the way. A very important part of the test was to get candidates to interact with the team via our chat room(s). We scheduled 15 minutes on a video call for set up and initial questions, 2 hours for the exercise (“open book” – as well as talking to the team, candidates were encouraged to use all resources available to them on the Internet), followed by a 30 minute wrap-up session where the team could ask questions and the candidate could also reflect on the task. This also allowed us to get an idea of how well the candidate was able to communicate with the team (combined with the chat interactions during the exercise). Afterwards, the candidate would drop off the call, and we’d generally make a decision within 5-10 minutes as to whether we wanted to hire them.

This generally worked very well. Some candidates struggled with the task, some didn’t communicate well, some failed to do well with the git interactions – these were the people we didn’t hire. It doesn’t mean they’re not good coders, or that they might not be a good fit for the project or the company later on, but they didn’t immediate meet the criteria we need now. Of the ones we hired, the levels of Rust experience and need for interaction with the team varied, but the level of git expertise and their reactions to our discussions afterwards was always sufficient for us to decide to take them.

Reflections

On the whole, I don’t think we’d change a huge amount about the selection process – though I’m pretty sure we could do better with the search process. The route through to the coding exercise allowed us to filter out quite a few candidates, and the coding exercise did a great job of helping us pick the right people. Hopefully everyone who’s come through the process will be a great fit and will produce great code (and tests and documentation and …) for the project. Time will tell!