There are three entities in an application that uses Conclave:
Clients send and receive encrypted messages to/from enclaves by interacting with the host over the network. Conclave doesn't mandate any particular network protocol for client<->host communication. It's up to you. However the content of the messages is defined, using the Mail API. Mail is described below.
Host programs load enclaves. From a security perspective they are fully untrusted and assumed to be malicious at all times. Hosts are relied on to provide the enclave with resources but beyond that work only with encrypted data. In some kinds of app their function is primarily proxying communications from clients to enclaves, but sometimes they also assist with application logic. Hosts use a standard JVM like HotSpot.
Enclaves are classes that are loaded into a dedicated sub-JVM with a protected memory space, running inside the same operating system process as the host JVM. Because they don't share heaps the host may exchange only byte buffers with the enclave. Direct method calls also don't work out of the box: that would require you to add some sort of RPC system on top. In this way it's similar to interacting with a server over a network, except the enclave is fully local.
In the above diagram orange shaded boxes are untrusted and could attack the enclave: the host and operating system (which includes the BIOS, drivers and peripherals). Blue shaded boxes are part of the trusted computing base - the set of components that must be correct and non-malicious for the system to work. That includes the enclave and of course the CPU. The client communicates with the enclave via the host. Both client and host interact with Intel servers to obtain relevant pieces of data proving the CPU is genuine and considered secure (see below for more information on this process). The enclave has a complex interaction with both operating system and host, in which the OS schedules the enclave onto the CPU and provides resources but is otherwise locked out of the enclave's operation. For its part, the enclave cannot interact with the OS directly and runs in what is effectively a "bare metal" embedded style environment. It cannot load DLLs/.so files or do system calls, so there's no way for it to do things like load files directly. It must ask the host to do it and use cryptography to prevent the malicious host from tampering with the data as it's loaded or saved.
Because the enclave runtime environment isn't the same as a normal HotSpot JVM, you will need to test your enclave code carefully and avoid advanced features that the embedded JVM doesn't support, such as Java Flight Recorder.
To use an enclave, clients need to know the network address where the host process for an enclave instance is
running so they can send it messages, and they also need to obtain an
EnclaveInstanceInfo object from the host.
This could be downloaded on demand from the host, or it could be published somewhere. This object encapsulates a
remote attestation. The client tests the
EnclaveInstanceInfo against a set of constraints, depending on
how flexible they want to be about software upgrades to the enclave. Constraints are represented by an
EnclaveConstraint object, which can be read from/written to
a small domain specific language suitable for embedding in config files, command line options and so on. A
constraint may specify that a specific set of code hashes is required i.e. every version is whitelisted and no
upgrade is possible until the constraint is adjusted. Or, it may specify a set of allowed signing keys, enabling
enclave authors to release new versions whenever they want. In that scenario the enclave creator is trusted,
but the entity hosting the enclave instance may not be.
When they're happy, they create encrypted messages using the key in the
EnclaveInstanceInfo. By sending and
receiving such messages to the host (and from there to the enclave), communication is established. See the Mail
section below for further discussion of this.
Whilst the high level setup has just those three entities, real deployments have more:
- Optionally, a cloud provider
- Optionally, an auditor
Intel. Intel's involvement in a deployed architecture is limited to providing the CPU hardware and running some servers. These servers provision the host with certificates guaranteeing authenticity of the chip (once, on initial setup) and provide the client with an assessment of the security of the host given a remote attestation (whenever the client requests it). If Intel's servers become unreachable by the clients that's not a problem, it just means the security of the host machine may degrade without the clients realising.
For instance if host's SGX software stack is out of date and has known vulnerabilities, or if the BIOS configuration is not correct, this will be reported to the client by Intel's servers as part of verifying and processing the data inside the remote attestation provided by the client.
Cloud provider. A cloud provider needs to support SGX for Conclave to be usable. They may operate their own provisioning servers that take over from Intel's.
Auditor. In the pure enclave-oriented model, the user is responsible for understanding what the enclave does before using it by reading the enclave's source code. If the user doesn't do this then enclaves have no point. In practice the user may wish to outsource this auditing to someone else.
Protocol sequence diagram¶
This is what a typical interaction looks like:
The first time SGX is used on a machine there are interactions with either the cloud provider or Intel to retrieve
machine certificates proving authenticity. The host then gets a remote attestation (
EnclaveInstanceInfo) to the
client somehow, the client verifies it and optionally asks Intel if the hardware setup of the machine is still
considered to be secure, or if there are known vulnerabilities (see renewability). This can be
repeated as often as the client wants, e.g. every day. Once this is done the client can send messages to the enclave
through the host.
Communicating with an enclave requires sending and receiving encrypted and authenticated messages. One possible approach is to embed a TLS stack into the enclave and use something like HTTPS, but this technique has some problems and limitations that are resolved via the Conclave Mail API. Mail makes communication between enclaves and clients easy, as well as solving common problems faced with doing application development.
Testing and debugging¶
Conclave provides full unit testing support for enclaves. Enclaves themselves can be compiled for one of three modes:
- Production/release: fully encrypted and protected memory.
- Debug: the same as production, but special instructions are provided that allow enclave memory to be read and modified. This mode provides no protection but is otherwise a faithful recreation of the standard environment.
- Simulation: SGX hardware isn't actually used at all. This is helpful during development when SGX capable hardware may not be available.
The modes must match between how the enclave was compiled and how it's loaded. This is handled for you automatically.
You can also load an enclave in "mock mode". This is not only suitable for unit testing but also for fast, iterative development. In mock mode the enclave class runs in the same JVM as the host, so message passing is just regular function calls. You can step through in a debugger and enjoy the regular Java development experience.
Inside the enclave
System.err are wired up to the host console, but there's no filesystem access.
Printing is disabled in release mode, so you can log to
stderr without worrying that it may leak
information outside the enclave when in production.
Future versions of the platform may automatically set up encrypted logging. For now, if you want release mode logging you can log to the in-memory filesystem and export logs from time to time to a trusted client or another enclave. Alternatively you may decide that due to the sensitive nature of what your enclave does, nobody should receive its logs when in release mode: this is a perfectly normal and reasonable choice!
You can get SGX capable virtual machines from cloud providers. Currently Microsoft Azure provides the easiest, up to
date source of SGX VMs. A Conclave host app includes the enclave bundled into the same JAR and the native libraries
required for working with the kernel driver and Intel infrastructure, so you can deploy a Conclave app by simply using
regular Java deployment procedures. For example, use the Gradle
assemble plugin to create a tarball/zip of your
app, or create a fat JAR and copy it to the server.
JVMs and performance¶
Enclave code can run on one of two different Java virtual machines. Both can execute bytecode. One is a small, specialised runtime called Avian. Avian is slow but can dynamically load bytecode, which some Java frameworks like to do. The other is GraalVM Native Image (SubstrateVM). The latter compiles your entire program to native code ahead of time, erasing any code that isn't used and optimising it as a whole. This can yield large speedups and memory usage improvements, at the cost of being unable to dynamically load new classes.
The differences between the two runtimes is summarised by the table below:
|Avian JVM||GraalVM Native Image|
|Dynamic class loading|
|Fast ahead of time code|
The speedups from using Native Image can be significant. However, as the enclave environment is small the performance will still be lower than with a regular HotSpot JVM. This table shows the performance difference and how they vary between a variety of benchmarks taken from the Computer Language Benchmarks Game.
The "empty" benchmark is measuring the overhead of entering and exiting the enclave, without doing any work. As entering/exiting triggers a variety of hardware mechanisms designed to block side channel attacks this is naturally expensive relative to the cost of a regular function call, however, once the enclave is doing real work this transition cost becomes less relevant.
Higher scores are better. As you can see, GraalVM based enclaves are around 4x-12x faster than with Avian, depending on the task. The performance hit overall of using an enclave is also highly dependent on what exactly the code is doing (primarily, memory access patterns).