... | ... | @@ -113,3 +113,24 @@ Well, first these two. |
|
|
* Collect the data on the latency
|
|
|
* Read the Hulu paper
|
|
|
* Worker is a thing
|
|
|
|
|
|
# May 1
|
|
|
## Project Ideas
|
|
|
|
|
|
### Key-value store performance comparison (on YCSB, maybe YCSB+T)
|
|
|
In addition to Redis, which we already pretty much have, we could do experiments with a couple more platforms to see how their performance differs (especially if they have different trends). In particular:
|
|
|
|
|
|
- [Hyperdex](http://hyperdex.org/): has data types, supports ACID transactions, could be a good candidate for modifying or building on top of, built-in clustering capability
|
|
|
- [Riak](http://basho.com/riak/): eventual consistency, has data types, built-in clustering capability
|
|
|
- ??? (find other NoSQL stores, even better if they support transactions or data types)
|
|
|
|
|
|
The aim in evaluating these different key-value stores would be to figure out which one would be the best candidate for building on top of to try out ideas like phase reconciliation or transaction boosting, etc. We've been playing with Redis, but it may be that it's not the easiest option as far as extending it (for instance, it doesn't have distributed transactions already, whereas some others do).
|
|
|
|
|
|
### Distributed Transactions for Redis
|
|
|
We could aim to evaluate Redis on YCSB+T, which, unless we can find an existing implementation, would mean we have to implement some form of distributed transactions on top of Redis Cluster. The nice thing is we have the YCSB+T validation step to help us figure out if we've done it correctly or not. There's also an [old blog post](http://engineering.imvu.com/2012/02/09/distributed-redis-transactions/) on implementing distributed redis transactions, which has code on Github, which we might be able to use to help with our implementation.
|
|
|
|
|
|
Once we have distributed transactions implemented and evaluated, we would be all set to start implementing some of the transactions optimizations, such as transaction boosting which I've been having lots of success with in my toy key-value store.
|
|
|
|
|
|
### Explore replicated data records
|
|
|
|
|
|
[Adaptive strength geo-replication strategy](http://dl.acm.org/authorize?N96599): in this paper they replicate specific records based on how often they are accessed, and the replicas are kept in sync with each other asynchronously. This is in the context of *eventual consistency*, and the data types are called *convergent replicated data types (CRDTs)*. We could take their implementation ([Antidote](https://github.com/syncfree/antidote), in the "partial_replication" branch I think) and adapt it to do something like phase reconciliation to help keep the various replicas more in sync. |