Swift: Quickly get the size of a folder

Question

here on SO are a lot of topics about getting the size of a folder programatically in Swift. All accepted answers suggest to use the FileManager and then enumerate over all files to and sum the size of all files.

However, I was asking myself if there is a faster way to get the size of a folder? Enumerating over all files can take a very long time and when I use the Finder and right-click a folder and select "Information", the size is displayed instantly.

Is the size a "secret" property of a folder that is only available to the OS? Does the operating system index all folders/files in the background?

When I do the same thing in Windows, the subfolders and files are also enumerated.

Why do I ask this question? Because I am developing an app that is processing files of a directory and I would like to show some sort of progress bar. But If I have to enumerate all files twice (once to get the total count and once to actually process them), is not very good.

Regards, Sascha

Warford

I don't think that HFS+ (the current file system on macOS) has such a property. And the Finder often displays "Calculating Size" in the Get Info dialog, but apparently caches the information. – Unveiling

This is a known problem on HFS+, and one of the reasons for creating APFS. See: developer.apple.com/videos/play/wwdc2016/701 @18:00 – Enfranchise

Seems that this feature in APFS was cancelled or is not yet ready: 1, 2. – Barehanded

Answer 1

In the world of Unix, the command to get the file sizes is "du -sh" which I guess stands for disk usage. The -s is the sum and h means give it to me in gigabytes effectively.

This command has been around for 30+ years; you can find its source of it here. If you want lightning-fast then I suggest you look into this.

It is just over one thousand lines of code; shouldn't be too hard to figure out what is going on.

https://github.com/coreutils/coreutils/blob/master/src/du.c

Sterner

great tip! running on Mac this would be the way to go (note that you can't inside an iPhone) – Absentminded

Answer 2

However, I was asking myself if there is a faster way to get the size of a folder?

The answer may depend on the file system in use. Like most operating systems, macOS supports a variety of different file systems, and the best way to determine the size of a directory may not be the same for all file systems. Accordingly, I'll answer the remaining questions with Apple's file system, APFS, in mind.

Enumerating over all files can take a very long time and when I use the Finder and right-click a folder and select "Information", the size is displayed instantly.

Just because the information seems to be displayed instantly doesn't mean that Finder isn't calculating the sum of the file sizes in the directory. Indeed, if you try the Get Info command on a directory known to have a great many files, such as /System, you'll see that Finder takes considerable time to determine the size of the folder:

Is the size a "secret" property of a folder that is only available to the OS? Does the operating system index all folders/files in the background?

It certainly doesn't seem to. Again, try looking at a directory that contains thousands of files in a variety of subdirectories.

Also, try using the du command to determine the disk usage of a large directory, e.g. du -d0 somedirectory; it responds almost immediately for directories with dozens of files, but can take many seconds for large directories.

When I do the same thing in Windows, the subfolders and files are also enumerated.

Consider that if the file system were to constantly keep track of the sizes of all directories, writing to any file could change the size of the file, as well as every directory in the file's path. Since asking for the size of a directory happens much, much less often than writing to a file (or moving or deleting a file), the file system would be doing a lot of work for very little benefit.

Consider also that there's more than one way to ask about directory size. You might want to know how much space is used by the files in a directory, but exclude subdirectories. You might want to know about logical file size (the number of bytes in the file) or physical size (the number of blocks used to store those bytes). You might want to include symbolic links in the calculation, or not. Being prepared to answer just one version of the question probably isn't all that helpful, and being prepared to answer every version of the question would require a lot of work.

With all that in mind, it's not surprising that both macOS and Windows calculate directory sizes when asked by recursively enumerating all the files and subdirectories in the directory.

If you want an absolutely authoritative answer for the file system(s) that you're working with, look at the published information for that system. For example, here's the data for Apple File System (ApFS)(PDF). After a quick look, I don't see fields for the cumulative sizes of directories in the relevant structures.

---

Note that if you select an entire volume in MacOS and use the Get Info command, you do see the amount of space used almost instantly. I believe this is because the file system has to keep track of the volume's total size and free space because in order to be able to provide free blocks as they're needed. Determining the amount of space used is therefore very simple.

Hyperactive

Try to look at "System Settings -> Storage -> Developer". 2-3 second and it will displayin gme 65GB. Now Click on "i" button near "Developer". Lot of folders displayed with some size. Try to calculate with code size of the largest folder and it will take more than 20 seconds at least. So there is exist some way to fast calculation of the size. Not theoretically, but in practice. And I need to get sizes exactly of folders in list "System Settings -> Storage -> Developer" :) – Juliettejulina

as example I have location "/Users/uks/Library/Developer/Xcode/DerivedData" which is "XCode project build files" from "System Settings -> Storage -> Developer". Size calculation from "Get info" takes close to 15-20 seconds. But size calculation of the same folder from "System Settings -> Storage -> Developer" takes < 2 seconds. – Juliettejulina

@Juliettejulina it seems really likely that the System Settings page is doing some caching and/or prefetching of this information. I just opened that settings page for the first time and it took fully 2-3 minutes for every section to be loaded and display a size (with Applications and Mail taking particularly long). I don't think there's any trickery here. – Widner

On an M1 MacBook Pro with a fast SSD – Widner

After closing and reopening System Settings, the information is populated very quickly – Widner

@Widner that's the point! It tool 2-3 mins to calculate information for thousands(!!!) of gitabites(!!!) When I trying to calculate 25-30GB of derived data folder more than in a minute(!) So even with some sort of caches this is really fast calculation in this Settings view even on first open. (Even if caches are empty.) – Juliettejulina

Answer 3

Code taken from : gist of Nikolai Ruhe

//
//  Copyright (c) 2016, 2018 Nikolai Ruhe. All rights reserved.
//

import Foundation


public extension FileManager {

    /// Calculate the allocated size of a directory and all its contents on the volume.
    ///
    /// As there's no simple way to get this information from the file system the method
    /// has to crawl the entire hierarchy, accumulating the overall sum on the way.
    /// The resulting value is roughly equivalent with the amount of bytes
    /// that would become available on the volume if the directory would be deleted.
    ///
    /// - note: There are a couple of oddities that are not taken into account (like symbolic links, meta data of
    /// directories, hard links, ...).

    public func allocatedSizeOfDirectory(at directoryURL: URL) throws -> UInt64 {

        // The error handler simply stores the error and stops traversal
        var enumeratorError: Error? = nil
        func errorHandler(_: URL, error: Error) -> Bool {
            enumeratorError = error
            return false
        }

        // We have to enumerate all directory contents, including subdirectories.
        let enumerator = self.enumerator(at: directoryURL,
                                         includingPropertiesForKeys: Array(allocatedSizeResourceKeys),
                                         options: [],
                                         errorHandler: errorHandler)!

        // We'll sum up content size here:
        var accumulatedSize: UInt64 = 0

        // Perform the traversal.
        for item in enumerator {

            // Bail out on errors from the errorHandler.
            if enumeratorError != nil { break }

            // Add up individual file sizes.
            let contentItemURL = item as! URL
            accumulatedSize += try contentItemURL.regularFileAllocatedSize()
        }

        // Rethrow errors from errorHandler.
        if let error = enumeratorError { throw error }

        return accumulatedSize
    }
}


fileprivate let allocatedSizeResourceKeys: Set<URLResourceKey> = [
    .isRegularFileKey,
    .fileAllocatedSizeKey,
    .totalFileAllocatedSizeKey,
]


fileprivate extension URL {

    func regularFileAllocatedSize() throws -> UInt64 {
        let resourceValues = try self.resourceValues(forKeys: allocatedSizeResourceKeys)

        // We only look at regular files.
        guard resourceValues.isRegularFile ?? false else {
            return 0
        }

        // To get the file's size we first try the most comprehensive value in terms of what
        // the file may use on disk. This includes metadata, compression (on file system
        // level) and block size.

        // In case totalFileAllocatedSize is unavailable we use the fallback value (excluding
        // meta data and compression) This value should always be available.

        return UInt64(resourceValues.totalFileAllocatedSize ?? resourceValues.fileAllocatedSize ?? 0)
    }
}

Important thing: as this code can do huge/slow work.... better to rewrite it to return Channel. This will give you ability to take updates due to long work process of the function.

I did this but it's using lot of other internal code so I did'nt post this rewritten solution here :) Just giving you idea how to do the same thing better :)

Juliettejulina

Answer 4

FWIW to just get the "size of a directory" we just do this ...

Getting the size of one file is trivial, it's

file.resourceValues(forKeys: [.fileSizeKey]).fileSize

To deep iterate a directory, you just use the iterator but there's a lot of options. (Full discussion.)

Hence it's just

extension URL {
    ///Total mb of a directory.
    func overallMB() -> Int {
        var bb = 0
        let RK: [URLResourceKey] = [.creationDateKey, .isDirectoryKey]
        guard let eor = FileManager.default.enumerator(at: self, includingPropertiesForKeys: RK, options: [.skipsHiddenFiles]) else { return 0 }
        for case let fileURL as URL in eor {
            if let rv = try? fileURL.resourceValues(forKeys: Set(RK)), let isDirProperty = rv.isDirectory, isDirProperty == false, let b = try? fileURL.resourceValues(forKeys: [.fileSizeKey]).fileSize {
                bb += b
            }
        }
        return bb / (1024 * 1024)
    }
}

---

Note, if the app is strictly Mac (not iOS) you could just run "du -sh" as @user3069232 points out!

Absentminded