[NOTES] Sandbox concept

[Giovanni]

Hi,

Just writing some notes about this idea: create an "unprivileged sandbox".

- Threads could be privileged or not privileged.
- The address space is by accessible only by privileged code.
- The application can create a "sandbox" using one or two MPU regions (for code and data).
- The sandbox code would start with an header containing some metadata (entry points and stack sizes).
- The application starts one or more threads to enter the sandbox and be trapped there using the metadata for entry points.
- The boxed threads cannot touch anything, just call SVC for services (API TBD), threads cannot call ChibiOS functions directly.
- Exceptions caused by sandboxed threads would just terminate the thread without affecting the rest of the system.
- The sandbox is static, no MPU reconfiguration at context switch (fast).
- Sandbox is pretty much like a Posix process but static and without overhead.
- It would be an optional module on top of both RT or NIL, it makes more sense for RT anyway.

Use cases:
- Run loadable code safely (sandbox in RAM, or in a dedicated flash that can be rewritten with external code).
- ISO26262 isolation concept (non-ASIL code in the sandbox in an ASIL system).

Possible enhancements:
- Multiple sandboxes isolated from each other, pretty much processes, would require MPU dynamic handling.

API ideas:
- exit, sleep for sure.
- Access to named global objects like mailboxes or message servers.
- Posix-like API? as an option?

To verify:
- How much isolation is really possible on Cortex-M.

Just writing notes right now, feedback is of course welcome.

Giovanni

[Giovanni]

Small update.

I got code boxing working, now threads can go in user mode and run within protected boxes with just an SVC access to the outside.

The SVC interface is very simple, it offers few standard APIs (to be finalized) and can be enhanced by the container host application to expose custom functionalities to clients in boxes.

Code is in the RT7 branch but probably I will migrate it to the current RT6 too, it requires nothing RT7-specific. There is still work to do: exception handling, testing and polishing mainly.

The thing proved challenging because the Cortex-M architecture does not implement separate stacks for privileged and unprivileged modes (as other ARM architectures do) so I had to implement a stack switching scheme, lucky it ended up quite efficient.

If everything works as expected it will be offered as ChibiOS/SB, a CM-specific, safety-oriented extension to RT and NIL. Something similar for Cortex-A and Cortex-R could be added later.

Giovanni

[Giovanni]

I added handling of exceptions now if the sandbox thread fails with an exception then it is terminated without affecting the rest of the system.

There is a nasty corner case:
1) The sandbox starts.
2) It moves the SP register outside the sandbox.
3) Causes an exception or waits for an interrupt to occur.

The exception/interrupt registers stacking is performed in non-privileged mode and this causes an exotic exception-on-stacking which is not very clear in the ARM documentation. I think I managed to address this, the code is a bit arcane but appears to work.

The sandbox should be fully isolated but there could be other ways that I have not considered, any suggestion would be welcome.

Next point is the host API, I just added few basic functions (time, intervals, sleep, exit) and need to think about a general mechanism for accessing services exposed by the host, perhaps through messages.

Giovanni

[mikeprotts]

I can think of a use case for a sandbox that could be terminated based on some form of watchdog. I can also see where I might like to start a sandbox for a single task, and where I'd like to have a long running sandbox repeating a task.

Would it make sense to use DMA or a reserved shared memory area? Perhaps an interrupt (or a pair of interrupts) could be used to communicate between the sandbox and a controlling non-sandboxed thread? The way I am thinking is to have logic like:

Controller:
Add data to shared area
Signal into sandbox with message
[do something else or wait]

Sandbox:
Wait until signalled
Read message
Process data
Update data
Signal controller with message
Back to wait or exit

Controller:
Read message
Process data

Does that make sense?
Mike

[Giovanni]

Hi,

The watchdog is an excellent idea, I think it should be added or a sandbox could monopolize the CPU (maliciously or because an error) and starve lower priority threads.

About communication mechanism you described, I agree, there should be something like that in two flavours:

1) The host is the initiator and the sandbox answers to incoming messages (the sandbox IS the service).
2) The sandbox is the initiation and the host answers to incoming messages (the host exposes services TO the sandbox).

Messages would be shared buffers inside the sandbox.

Giovanni

[mikeprotts]

For the watchdog, it's specifically missing or false interrupts that I can see are likely problems, e.g. in electrically noisy environments. Protecting against malicious causes would be an added bonus.

Keeping shared buffers within the sandbox is ideal, with something like a message queue or circular buffer. There may be a case for the simplest use with a single input and output message, to have minimal overheads.

Mike

[Giovanni]

More news.

I ported sandbox support on trunk, now the sandbox supports two modes:

1) Static Regions. Regions are statically assigned to sandbox(es). This means that there is a limited number of possible sandboxes and that each sandbox can access the areas of other sandboxes. The kernel does not touch MPU directly.

2) Dynamic Regions. Each thread has 1...4 regions saved with the thread context. This means that all sandboxes use the same MPU regions and cannot interfere each other. This causes some context-switch overhead and uses more RAM of course.

Why 1..4 regions? there are several scenarios:

1 Region: A RAM-based sandbox, code and data are in the same region.
2 Regions: Separated flash and RAM regions, it is what the demo does (or one RAM region and 1 extra region).
3 Regions: Separated flash and RAM plus one extra region (or one RAM region and 2 extra regions).
4 Regions: Same but one more extra regions.

Extra regions can be used, for example, for making peripherals visible to certain sandboxes, or making accessible certain buffer areas.

Note, the demo debug configuration contains an absolute path in the box of the "Statup" tab, replace it with your own path. For some reason relative paths do not work for me.

Giovanni

[Giovanni]

Added API for messages and events, I am thinking to add support for I/O streams too, that would enable use of chprintf() and similar.

Giovanni

[Giovanni]

I am going for a subset of Posix API too, basically the open/close/read/write/lseek group of functions as a start. The idea is to have a fully working C library inside the sandbox: file I/O, allocator, strings, formatting etc. The basic implementation would be to have stdin, stdout and stderr directed on an host-side ChibiOS stream (serial port or other kinds).

Of course this is in addition to the RTOS-like API already implemented.

Giovanni

[mikeprotts]

I think stdin, stdout and stderr would be an important link between the sandbox and the main system. The stderr route might benefit from being treated as a higher priority, perhaps options to have errors treated as either fatal (perhaps chSysHalt) or recoverable, where the sandbox might be terminated, but the main system handle the reported error.

Mike