Docker Desktop WSL ext4.vhdx too large [closed]

Question

I have WSL installed as well as Docker Desktop.

I tried to clean up docker as much as I could by running

docker system prune -a

docker volume rm $(docker volume ls -q -f dangling=true)

Then I verified with

❯ docker system df
TYPE            TOTAL     ACTIVE    SIZE      RECLAIMABLE
Images          0         0         0B        0B
Containers      0         0         0B        0B
Local Volumes   0         0         0B        0B
Build Cache     0         0         0B        0B

However, I see that I still have nearly 12G used by WSL.

I assume this file is docker related. Why is it so large despite me removing all containers and volumes?

How do I shrink it (or can I delete it?)

Fetiparous

Setting up compression to the ext4.vhdx file reduced the disk size from 104GB to 74GB for me. – Tussle

Try this, It cleaned up 220 GB. https://mcmap.net/q/144456/-docker-image-taking-up-space-after-deletion – Scriber

Answer 1

(Update for December 2022)

The windows utility diskpart can now be used to shrink Virtual Hard Disk (vhdx) files provided you freed up the space inside it by deleting any unnecessary files. I found the info in this guide.

I am putting the gist of the instructions below for reference but the guide above is more complete.

First make sure all WSL instances are shut down by opening an administrator command window, and typing:

>> wsl --shutdown 

Verify everything is stopped by:

>> wsl.exe --list --verbose

Then start diskpart:

>> diskpart

and inside diskpart type:

DISKPART> select vdisk file="<path to vhdx file>"

For example:

DISKPART> select vdisk file="C:\Users\user\AppData\Local\Packages\CanonicalGroupLimited.Ubuntu22.04LTS_12rqwer1sdgsda\LocalState\ext4.vhdx"

it should respond by saying DiskPart successfully selected the virtual disk file.

Then to shrink

DISKPART> compact vdisk

After this the vhdx file should shrink in usage. In my case it went from 40GB to 4GB. You can type exit to quit diskpart.

Merylmes

I think a shrinking by factor 10 is rather unlikely, for me I was able to reduce 55GB to 48GB. – Ludwick

Thanks alot bro! Mine from 107.9GB to 45.9GB – Albano

Thanks! 141.8GB to 137GB – Pyrogen

Nice! From 37GB to 17GB. I think the amount of disk space that you save depend on the number of images that you had. For example I had alot of images that were taking alot of space, but after deleting the images in docker desktop I couldn't reclaim that space because .vhdx can't shrink down automatically. But after this, the size of dynamically allocated .vhdx reduces. – Leatherman

Strangely, this did not free up any space. The file size is currently 7GB. Any other solutions? – Beneficent

If it did not free up any space, is ir likely that there are actual files taking up that space ? You can use a tool like ncdu to examine the usage of disk space in linux. Using ncdu -x / will give you a full breakdown of usage across the whole drive without crossing into another mounted filesystem. – Merylmes

Awesome! 503GB to 151GB. For context I downloaded lots of LLMs but then moved them to another drive, so running this command shrunk the ext4.vhdx down to what was actually being used by WSL (confirmed with df -h). – Vantage

This works great (128GB -> 30GB)! Remember to first remove unused or dangling images, unused volumes and clear the build cache. The "Disk usage" extension can be useful for that. – Sagacity

Thanks! Docker on win11 was responsible for constantly enlarging WSL vdisk despite my attempts to keep volumes cleaned up, 692 GB -> 39 GB, here's a gif of WinDirStat: i.imgur.com/j86wcsN.gif – Plop

It's a super cool instruction. I manage to clean up 320GB -> 98GB___________I did remove some big files in wsl using rm -rf command directly. But seems the disk is not cleaned up after I remove the file.__________I read some article to say in linux we don't need to care about disk defragment. Maybe it's not the case in Windows WSL. – Tynishatynwald

Mine went from 57GB to 3GB so yes, it can have a pretty hardcore shrinking factor. Also yes I did remove all my containers and images and volumes first...! – Ruckman

If you get the error saying diskpart process cannot access the file, you might need to run diskpart from the elevated command prompt. – Eris

Worked for me. Now down to 4.5GB. Thanks! – Facer

Got me down from 53.4GB to 43.8GB – Farmyard

Answer 2

If you're willing to wipe all of your Docker data, open the Docker Desktop client, click the bug icon in the title bar, and then click Clean / Purge data.

---

Source: Cleaning Up Docker Disk Space In WSL2

Synecology

Also need to check the relevant checkbox e.g "WSL 2 . This worked for me and brought the disk space used by ext4.vhdx down from 15GB to 1GB. – Lacking

It saved me over 100GB! Thanks – Ferdelance

Although it saved me more than 60GB. But I don't know why after following this answer my docker desktop has completely stopped. – Lindon

which deleted all the containers – Froze

Answer 3

September 2023 update: A new pre-release of WSL (2.0.0) is reported to provide a new "sparse" mode for disk images which will automatically shrink the image when files are removed.

While I have not tested this personally yet, from the DevBlog announcement, you can convert an existing disk image to sparse with the following command from PowerShell:

wsl --manage <distro> --set-sparse true

You can also add the following to your .wslconfig (located in your Windows profile directory, not inside WSL) to have any newly created distro image be sparse:

[experimental]
sparseVhd=true

---

Older answer

Still useful for "released" and earlier versions of WSL

Short answer:

• diskpart or the Optimize-VHD commandlet can be used to shrink the virtual disk. Optimize-VHD is typically preferred, but is only available under Windows Pro or higher. See below for details.

More Details:

For reference, there's an open Github issue on this topic.

WSL2 virtual disks are "dynamic" .vhdx's, which means they:

• Are allocated to a maximum size
• Are initialized with just a few kilobytes of structural data
• Grow dynamically as data is added, up to their maximum allocated size
• And here's the kicker -- They do not automatically shrink when data is removed.

However, it's certainly possible to manually optimize them.

Step 0: For others that are reading this, follow the OP's lead by first cleaning up the Docker data using the normal Docker commands.

Next, in all cases:

• Stop Docker Desktop
• wsl --shutdown from PowerShell or CMD

Since you've already removed all data (containers, images, volumes), you could just remove the images entirely:

• Uninstall Docker Desktop

• From PowerShell:

  wsl -l -v
  # If they still exist, remove via ...
  wsl --unregister docker-desktop
  wsl --unregister docker-desktop-data
  

  Important: Note that these are destructive operations, so make sure you really don't need any data from Docker Desktop.

• Reinstall Docker Desktop

For those that have data that they need to retain, but still need to reclaim some space:

• Copy the ext.vhdx as a backup. Note that there have been reports of corruption when using either of the techniques below.

Then, as noted in the comments in that Github issue:

• On Windows Pro, you can enable the Hyper-V feature and then run the Optimize-VHD cmdlet in PowerShell per the original Github issue:

  Optimize-VHD -Path .\ext4.vhdx -Mode full
  

• If using Windows Home, you'll need to use use diskpart per this comment.

• Restart Docker Desktop and confirm that it is functional and that all expected data is intact before removing the backed-up ext4.vhdx.

Dacy

Thank you, this helps. Was able to reduce to 5.5GB. The Windows Home applied to me as Hyper V is not supported on Windows Home, so the alternate worked. Thank you so much. Although 5GB still seems like quite a bit. Any way to increase it more? – Fetiparous

@KSplitX That's higher than I would expect as well. I don't think I have anything critical in mine at the moment. Let me see if I can clean it up, try optimizing mine, see what the size comes out to. – Dacy

it worked for me but after a quick restart. – Mint

it worked, to complement: this article helped me a lot about export/import/resizing/compact, and also, move the location: levelup.gitconnected.com/… – Emmi

You can download pre-releases here github.com/microsoft/WSL/releases – Phares

@Phares If needed (e.g., you can't install using App Store), but for most users wsl --update --pre-release will be easier ;-) – Dacy

Make sure to purge any images and system images and do a compact before switching to sparse, you can't compact a sparse image. I had to convert back to non-sparse after purging, compact, then re-sparse which saved another 20+gb (the amount that was purged) – Equuleus

Answer 4

(Update for October 2023)

• stop all docker services and WSL:

wsl --shutdown

• open PowerShell with admin rights
• run next command for VHDX compression:

Optimize-VHD -Path "${Env:LocalAppData}\Docker\wsl\data\ext4.vhdx" -Mode Full

This command uses the Optimize-VHD tool to compress the VHDX file. After completing this command, you can start Docker and WSL 2 again.

• PROFIT

Duwe

I did the fix you proposed and the problem reappeared the next day. (It uses 100% of my disk space vitual and physical). Any other idea? – Jeffryjeffy

This helped me with shrinking the file from 167GB to 110GB, but I can not get it lower. – Huth

"Optimize-VHD : The term 'Optimize-VHD' is not recognized as the name of a cmdlet, function, script file, or operable program" Does this need to be installed somewhere first? – Transgression

Answer 5

I want to mention wslcompact, this tools also list of WSL distro (also docker-desktop and docker-desktop-data) and estimates the time needed to compact the virtual disk size. It works by creating a copy of virtual disk in temporary directory and then replace original virtual disk. Personally, I always set user temporary dir in another drive so I dont worry about disk space needed.

PS> wslcompact
 WSL compact, v5.0 2023.02.02 (Groundhog edition)
 (C) 2023 Oscar Lopez
 wslcompact -h for help. For more information visit: https://github.com/okibcn/wslcompact

 Distro's name:  Ubuntu
 Image file:     C:\Users\Oki\WSL\Ubuntu\ext4.vhdx
 Current size:   12864 MB
 Estimated size: 7700 ± 188 MB
 The estimated process time using an SSD is about 2 minutes.

 Distro's name:  Kali
 Image file:     C:\Users\Oki\WSL\Kali\ext4.vhdx
 Current size:   1579 MB
 Estimated size: 723 ± 18 MB
 The estimated process time using an SSD is about 1 minutes.

 Distro's name:  Arch
 Image file:     C:\Users\Oki\WSL\Arch\ext4.vhdx
 Current size:   1075 MB
 Estimated size: 860 ± 21 MB
 The estimated process time using an SSD is about 1 minutes.

Imperial

Under the hood the project just iterates over vhdx files and perform: wsl --export $wsl_distro - | wsl --import $tmp_distro. No fancy low level EXT4 fs tricks or TRIM hardware command... – Agni

@Agni I believe yes. Thanks. – Imperial

Answer 6

From within a Linux VM (be it the Docker-provided or any other you installed yourself), you can run sudo fstrim -a (or sudo fstrim -av if you want details) and let the hypervisor deallocate the empty blocks of the virtual disk. This is managed by the hypervisor itself and works on KVM and Hyper-V (and, most likely, any self-respecting hypervisor).

Ova