Space to Think

I implemented a hash that takes previous data and current column and the previous hash to produce a hash of the entire database.

This allows us to synchronize with the minimum of data transmissions when I write the synchronizer part which shall rehash all its own data, then retrieve the hash of a binary search of the sorted data.

I spent a few hours working on the remainder of this problem and got documnent retrieval working after finishing save document.

This lets you save and retrieve JSON documents

The document data inserted is also queryable by SQL.

Joins against documents are not yet supported but I plan to implement this.

Most computers use locks to synchronize, which are very slow. This design is far faster and allows locks to be elided for performance.

I use unscynchronized data structures and achieve 100 million requests a second raw communication performance for a cost of 10 nanoseconds per batch of messages.

I managed to store a JSON document and I used the SQL inserter to insert the document. In theory the object is queryable by SQL

{ "items": [("name": "item1"), ("name": "item2")], "subobject": {"subobject_key": "value"} }

The idea is that each thread has a ringbuffer, when a socket is accepted, a message is put on the ringbuffer and the thread begins listening to that user.

All the other threads are notified of all the users so they can also echo data to them.

This is efficient as a thread can serve thousands-millions of connections and there are multiple threads.

One problem, which is a generic problem for multithreading is servicing one of multiple kinds of events from one thread. I rely on the fact I am polling a ringbuffer and epoll.