Elasticsearch in RAM data store

Question

Can it be possible to configure Elasticsearch in such a way, that it will keep all data in RAM (like Redis in-memory cache)? What I need is additional speed while querying data. If anybody has experience doing that or something similar, it will help a lot.

Harkness

This question is too broad, please provide information about your ES version, index size and documents count and use cases - search, aggregations etc. There are a lot of in heap structures and OS level memory mapped caches. – Cimmerian

Looking on your tags: full-text-search, data-mining; you could have different issue, more related to document mapping types and indexing. Could you explain why it slow, what are the document structure and show examples of you queries. – Mossberg

@pkhlop, i will edit my question and add additional information as soon as I finish the investigation on elasticsearch. the document structure i have is amino acids sequences in FASTA format and it definitely may not fit for indexing with elasticsearch. thanks – Harkness

This is not at all a broad question. How to make ES run with indices all in JVM heap which is in the physical memory. – Terracotta

Answer 1

1. Elasticsearch indexes are just files and they effectively cached in RAM by system. Usually if you have enough RAM Elasticsearch should work as fast as possible, especially for GET queries.

2. You can use /dev/shm/ shared memory folder as data folder

3. You can put perindex settings

   PUT /my_index { "settings": { "index.store.type": "mmapfs" } }

Mossberg

seems like this is the right way! somehow missed it while reading tons of stuff. – Harkness