Two months ago, I wrote about the idea/concept of a no-backend backend. The concept works well, but there is room for improvements:
- PouchDB/CouchDB does not fit naturally within the platform, – it is to imperative.
- CouchDB does not have super performance. It is ok, but it is an order of magnitude slower than LevelDB. And I only need a key-value store + clojure logic on top.
- I do not have the need for p2p yet, and the libraries are not mature enough.
- Authentication is not worked through.
So now I am revisiting the MuBackend design, – looking at authentication, communication and sync’able storage.
The database and communication channels are partitioned into realms. If you have the secret key to a realm, then you can write to the database, and receive communication. Any communication channel and data entry belongs in one realm. The realm identifier is the cryptographic hash of the secret key.
A message which is sent into a realm goes to one arbitrary client who have joined the realm. Anybody can send messages into a realm, but clients can only join realms for which they have the secret key. A client can join any number of realms. A message consist of:
- destination realm – the realm to which the message is sent
- destination mailbox – a handle for the client to sort incoming messages
- reply realm – optional, used for replying
- reply mailbox – optional, used for replying
- actual content.
If a realm is empty, and the message has a reply address, then an empty message is sent back.
The prerequisite for communication is a bidirectional message channel between client and MuBackend. This can be implemented over http, websockets, tcp, and other protocols. I intended to implement it on top of long-polling http first, to make it go though the cloudflare proxy, that the production server is living behind.
That it is message oriented, avoids unnecessary round-trips. The reply address allows easy rpc implementation, and notifications about when we clearly know delivery failed.
Having one random (or sender hashed or …) recipient for a messages in a realm allows for easy load-balancing computations etc.
A http-request will be transformed into a message into a given mailbox/realm, and the response message will be the http-response. Any client can handle http-requests, as long as they have the keys to that realm. Of course the keys would only be given to trusted clients.
The core API is the ability to send and receiver messages, ie.:
- Communication ; a message is (dst-realm, dst-mbox, reply-realm, reply-mbox, content)
handler(f(msg))– all results are received as messages given to the handler function.
- Storage ;
get(realm, key) -> (version, value)
subscribe(realm, key) -> (version) stream
set(secret, realm, key, version, value) -> version
traverse(secret, realm, min-version) -> (key, version) stream
- Realm-hash for http-server given as environment variable to server
Functions marked with * are those that need authentication. The realm id must be the SHA-224 hash of the secret.