Update max_map_count for ElasticSearch docker container Mac host

Question

I'm using this container to start elasticsearch in docker. In accordance with the manual I have to update max_map_count to start the container

sudo sysctl -w vm.max_map_count=262144

but.. I can update it in my host (container) AFTER I start it while I'm unable to start it. Am I doing something wrong?

ERROR: bootstrap checks failed max virtual memory areas vm.max_map_count [65530] likely too low, increase to at least [262144]

If I try to do it on my host machine (which is Mac) I get the following error.

sysctl: unknown oid 'vm.max_map_count'

Docker engine installs the Lunix VM where all containers are running. So the command to increase the limit should be executed for the Linux host, not for the Mac.

How can I access Linux VM via terminal installed by the Docker engine?

Clifford

You increased the vm max map count as a root user right ? – Thalweg

yes, updated the command. But was executed for the host (Mac OS), not for the container OS (Linux) and it seems that it's required to be done for guest OS (container OS where the ES is setup). – Clifford

Answer 1

On Docker Toolbox

If you are in docker toolbox try the docker client from terminal and then make the configs:

docker-machine ssh
sudo sysctl -w vm.max_map_count=262144
exit

On Docker For Mac:

The vm_max_map_count setting must be set within the xhyve virtual machine:

$ screen ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/tty

If it asks you for a username and passwordLog in with root and no password.

If it just has a blank screen, press RETURN.

Then configure the sysctl setting as you would for Linux:

sysctl -w vm.max_map_count=262144

Exit by Control-A Control-\.

Se the docs here.

Persistence

In some cases, this change does not persist across restarts of the VM. So, while screen'd into, edit the file /etc/sysctl.d/00-alpine.conf and add the parameter vm.max_map_count=262144 to the end of file.

Krick

you saved my day – Gunlock

just for completion: Be aware that this change is on the host OS and NOT inside the container. Containers share the same kernel as the host OS and for that reason it's working. On OSX the "docker host OS" is a VM which is wrapped by the docker-machine cli. – Vitriolize

you ask docker automatically sets some environment variables when creating a Linux container. docker run -e "MAX_MAP_COUNT=262144" .... – Colorblind

@Octavio Question: I can't seem to edit that file, it keeps erroring out with File is read-only. I think it's something with xhyve being a read-only filesystem but my Docker/xhyve-fu is not nearly strong enough. How did you edit that file? – Taskwork

When I hit screen ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/tty I go not such file or directory. – Lamprey

@Lamprey run find ~/Library/Containers/com.docker.docker/Data/ -name 'tty'. Mine was at ~/Library/Containers/com.docker.docker/Data/vms/0/tty on High Sierra 10.13.6 – Tomaso

As @Taskwork mentioned the file system is read only now: github.com/docker/for-mac/issues/2389#issuecomment-356547597 – Corroboration

Is there another way of persisting this now? – Legend

Now you can change the virtual memory from the docker desktop GUI. Preferences -> Resources -> Swap -> Apply and Restart. – Hypanthium

This works for podman, too: podman machine ssh; sudo sysctl -w vm.max_map_count=262144; exit – Glimp

Answer 2

On Latest Docker For Mac (Version 18.06.0-ce-mac70):

$ screen ~/Library/Containers/com.docker.docker/Data/vms/0/tty

On the blank screen, press RETURN.

Then configure the sysctl setting as you would for Linux:

sysctl -w vm.max_map_count=262144

Exit by Control-A Control-\

Ido

How can this be persisted across a reboot? – Legend

I cannot screen. I get Cannot exec '/Users/arielpontes/Library/Containers/com.docker.docker/Data/vms/0/tty': Permission denied. sudo doesn't work either. The file exists but apparently cannot be executed no matter what. – Jarrodjarrow

@Jarrodjarrow were you able to resolve this issue because I am facing the same challenge right now – Obscurantism

@Obscurantism in Docker Desktop, go to Preferences, after, Resources and change the Swap value – Strike

@Obscurantism ultimately the only thing that worked for me was adding bootstrap.system_call_filter=false in the elasticsearch environment in my docker-compose file. – Jarrodjarrow

Answer 3

To make it work with Mac Sonoma 14.0 + Docker Desktop 25.0, follow these steps:

• Enable Use Rosetta for x86/amd64 emulation on Apple Silicon in Settings -> General
• Disable Access experimental features in Settings -> Features in development -> Experimental Features
• Run the following commands (see https://github.com/docker/for-mac/issues/4822)

docker run -it --privileged --pid=host debian nsenter -t 1 -m -u -n -i sh

/ # sysctl -w vm.max_map_count=262144

vm.max_map_count = 262144

Justicz

only one working solution MAC Ventura 13.6 Docker Desktop 4.25.0 – Concessionaire

Sonoma 14.1 + Docker 24.0.6 worked as expected. Thank you – Ical

Answer 4

For those using Docker Desktop on windows 10 you have to execute:

wsl -d docker-desktop in the command line before sysctl -w vm.max_map_count=262144

Donny

Answer 5

For Colima the same process (like Octavio Soto answered) helped me

colima ssh
sudo sysctl -w vm.max_map_count=262144
exit

Faustofaustus

Answer 6

For mac users you might have a problem connecting to docker VM, therefore you should run this command to enter the shell of the Docker VM:

docker run -it --privileged --pid=host debian nsenter -t 1 -m -u -n -i sh

GitHub issue description: https://github.com/docker/for-mac/issues/4822

Prosperus

Answer 7

I fixed it on my Mac (Sonoma 14.0) without dealing with vm.max_map_count by downgrading Docker Desktop to v.23 (https://docs.docker.com/desktop/release-notes/#docker-community-edition-17120-ce-mac49-2018-01-19)

Daveta

While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes. - From Review – Spaniard

I confirm it does not work on Mac Sonoma + Docker Desktop 4.25.0 (126437). @Daveta can you provide details on how to fix it in this scenario? – Justicz

Answer 8

Podman Desktop for Mac OS

For those of us having the same problem with Podman Desktop, we have to update the settings of the virtual machine that Podman runs for running containers on Macs.

Steps...

Login into the Podman VM

From a terminal on your mac:

$ podman machine ssh

Edit Default /etc/sysctl.conf File

Using sudo edit the /etc/sysctl.conf and add the vm.max_map_count = 262144 property.

$ sudo echo "vm.max_map_count = 262144" | sudo tee /etc/sysctl.conf

After exiting the podman virtual machine you should be able to run the ElasticSearch container:

$ podman run --name es01 --net elastic -p 9200:9200 -it docker.elastic.co/elasticsearch/elasticsearch:8.8.0

I would then recommend stopping Podman Desktop and rebooting your mac. Then restarting Podman Desktop and trying to run the ElasticSearch container again. It should work.

Hertahertberg

Answer 9

Colima Permanent Fix for Docker/TestContainers on Mac.

Follow the following steps to make this work permanently for test containers with Colima on Mac. 1-time setup.

1. Assuming Colima is started, run colima ssh to get inside the VM.
2. Run sudo su - to become the root user.
3. Run sudo echo "vm.max_map_count = 262144" >> /etc/sysctl.conf to update the value.
4. Exit Colima and restart it to load the value permanently.

Pivoting

Answer 10

Folder has been moved and this is the new location -

$screen ~/Library/Containers/com.docker.docker/Data/vms/0/tty

and then

sysctl -w vm.max_map_count=262144

Viewer

Please don't repeat existing answers. – Bejarano

Answer 11

For those who use docker desktop on mac, you can easily increase the memory by the following steps:

1. click on docker desktop -> preferences...
2. navigate to 'Resources'
3. change the memory to whatever you need
4. click on 'Apply & Restard'

Larimore

This does not resolve the max vm issue. – Justicz

Answer 12

If you have installed Docker from docker's Mac installer then you will have Docker desktop installed(also includes includes Docker Engine, Docker CLI client, Docker Compose, Notary, Kubernetes, and Credential Helper.)

Here is what Docker desktop looks like in 2021 where you can change memory/swap or any other resources.

Step 1 - click on docker's preferences as shown below.

Step 2 - Click on resources tab, here you can tweak the resources and finally click on "Apply and Restart" button.

Please ignore configuration what I made. You can set based on your requirement.

Brownie

From that configuration screen you cannot change the vm.max_map_count – Braddock