why use etcd?Can I use redis to implement configuration management/service discovery etc.?

Question

I learned etcd for a few hours, but a question suddenly came into me. I found that redis is fully capable of covering functions which etcd owns.Like key/value CRUD && watch, and redis is very simple to use. why people choose etcd instead of redis? why? I googled a few posts, but no post told me the reason.

Thanks!

Tifanytiff

Possibly answered here – Indecipherable

Answer 1

Redis stores data in memory, which makes it very high performance but not very durable. If the redis server dies, it's easy to lose data. Etcd stores data in files on disc, and performs fsync across multiple nodes before resolving to guarantee consistency, which makes it very durable but not very performant.

That's a good trade-off for kubernetes, which is using etcd for cluster state and configuration, not user data. It would not be a good trade-off for something like user session data which you might be using redis for in your app because you need extremely fast response times and can tolerate a bit of data loss or inconsistency.

Outsmart

thanks!Sorry for taking so long to thank you. – Tifanytiff

Is this answer valid given that redis can be configured to log every write to disk, and replay the log at restart (redis.io/docs/management/persistence)? This log mode addresses the durability issue, but not the consistency. It seems to me the original question remains unanswered... why do people use etcd over redis? I'm faced with the same topic, and I'm unfamiliar with either solution. – Civvies

Answer 2

A major difference which is affecting my choice of one vs the other is:

• etcd keeps the data index in RAM and the data store on disk
• redis keeps both data index and data store in RAM

Theoretically, this means etcd ought to be a good fit for large data / small memory scenarios, where redis would require large RAM.

In practice, etcd's current behaviour is that it allocates some memory per transaction when data is accessed. Under heavy load, the memory footprint of the etcd server balloons unboundedly (appears limited by the rate of read requests), and the Go runtime eventually OOM's, killing the server.

In contrast, the redis design requires a virtual address space sized in relation to the dataset, or to the partition of the dataset stored locally.

Memory footprint examples

Eg, with redis, a 8GB dataset partition with an index size of 0.5GB requires 8.5GB of virtual address space (ie, could be handled with 1GB of RAM and 7.5GB of swap), but not less, and the requirement has an upper bound.

The same 8GB dataset, with etcd, would require only 0.5GB of virtual address space, but not less (ie, could be handled with 500MB of RAM and no swap), in theory. In practice, under high load, etcd's memory use is unbounded.

Other considerations

There are other considerations like data consistency, or supported languages, that have to be evaluated separately.

In my case, the language the server is written in is a factor, as I have in-house C expertise, but no Go expertise. This means I can maintain/diagnose/customize redis (written in C) in-house if needed, but cannot do the same with etc (written in Go), I'd have to use it as released by the maintainers.

My conclusion

Unfortunately, the memory behaviour of etcd, whereby it needs to allocate memory to access the indexed data, negates the memory advantages it might have theoretically, and the risk of crash by OOM due to high load make it unsuitable in applications that might experience unexpected usage spikes. Github bug 14362, Github bug 14352, other OOM reports

Furthermore, the ability to customize the server in-house (ie, available C vs Go expertise) is a business consideration that weighs in redis's favour, in my case.

Civvies