How to get free and total size of each StorageVolume?
Asked Answered
E

3

31

Background

Google (sadly) plans to ruin storage permission so that apps won't be able to access the file system using the standard File API (and file-paths). Many are against it as it changes the way apps can access the storage and in many ways it's a restricted and limited API.

As a result, we will need to use SAF (storage access framework) entirely on some future Android version (on Android Q we can, at least temporarily, use a flag to use the normal storage permission), if we wish to deal with various storage volumes and reach all files there.

So, for example, suppose you want to make a file manager and show all the storage volumes of the device, and show for each of them how many total and free bytes there are. Such a thing seems very legitimate, but as I can't find a way to do such a thing.

The problem

Starting from API 24 (here), we finally have the ability to list all of the storage volumes, as such:

    val storageManager = getSystemService(Context.STORAGE_SERVICE) as StorageManager
    val storageVolumes = storageManager.storageVolumes

Thing is, there is no function for each of the items on this list to get its size and free space.

However, somehow, Google's "Files by Google" app manages to get this information without any kind of permission being granted :

enter image description here

And this was tested on Galaxy Note 8 with Android 8. Not even the latest version of Android.

So this means there should be a way to get this information without any permission, even on Android 8.

What I've found

There is something similar to getting free-space, but I'm not sure if it's indeed that. It seems as such, though. Here's the code for it:

    val storageManager = getSystemService(Context.STORAGE_SERVICE) as StorageManager
    val storageVolumes = storageManager.storageVolumes
    AsyncTask.execute {
        for (storageVolume in storageVolumes) {
            val uuid: UUID = storageVolume.uuid?.let { UUID.fromString(it) } ?: StorageManager.UUID_DEFAULT
            val allocatableBytes = storageManager.getAllocatableBytes(uuid)
            Log.d("AppLog", "allocatableBytes:${android.text.format.Formatter.formatShortFileSize(this,allocatableBytes)}")
        }
    }

However, I can't find something similar for getting the total space of each of the StorageVolume instances. Assuming I'm correct on this, I've requested it here.

You can find more of what I've found in the answer I wrote to this question, but currently it's all a mix of workarounds and things that aren't workarounds but work in some cases.

The questions

  1. Is getAllocatableBytes indeed the way to get the free space?
  2. How can I get the free and real total space (in some cases I got lower values for some reason) of each StorageVolume, without requesting any permission, just like on Google's app?
Efficacy answered 19/6, 2019 at 8:47 Comment(15)
I have played around with this a little more and am getting good agreement with "Files by Google" or so it seems on my end. Take a look at this project. If it is promising, it might be worth pursuing.Dungeon
@Dungeon What do you mean "good agreement " ? Did you talk to them about this?Efficacy
I mean that the figures for the demo app and "Files by Google" match in my tests.Dungeon
The output numbers? I see. But it seems you use reflection, like on some of my workarounds. Do you think they use reflection too? Also, I see that you've stopped to even use StorageVolume, which is not what I asked in the question. You got the numbers for paths. The solution you provide is similar to what I have found, including the numbers. Now what you will have to do (as I did) is to map between the paths and the StorageVolume...Efficacy
All true. I do think that reflection can be eliminated. It seems that too many useful methods and variables have been "hidden" such as those that would link a StorageVolume to a path. (StorageVolume has a field for the path but, alas, it is not available.) I have no idea if they are using reflection or not.Dungeon
I have updated the demo app on GitHub. The update beefs up the data structures and isolates reflection to an extension function. This extension function does what the API should do. IMHO. This type of functionality would be a good thing to ask for.Dungeon
As I read in Restrictions on non-SDK interfaces, it is possible to Request a new public API if you find the current API lacking. You may not get anything undone, but you may stop further damage to the part of the API that is causing you trouble. It may be worth an official request.Dungeon
@Dungeon What you updated is just to make it nice-looking. About requests, already did it, in various ways to cover all possible APIs that could be related and missing : 130637990, 133047170 , 132434082 , 132825438 ,134379739 ,134370342 ,134367310 , 137084500. Please consider starring them and optionally write somethingEfficacy
More than that. It now uses StorageVolume and simplifies things.Dungeon
Are you asking for answers to more than your two questions at the end of your post? It seems to me that the second question has been addressed and the first can be addressed depending upon what kind of interpretation of "free space" you are looking for - just looking for some clarification.Dungeon
I just want a more official way to do it. Do you think Google uses reflection for this?Efficacy
Ok. I doubt that they use reflection. I think that there are a couple of issues: First, SD cards are not being treated the same as primary storage. The SD card implementation is problematic or just too restrictive. Secondly, there is some variance on how to determine a volume's size. I can post something at a later time that will better explain these issues - at least how I see them.Dungeon
I know that there are issues about using the API I've found for SD-card. For some reason it works only for primary storage. But somehow they got it (the free and total space) for SD-card too. That's why I don't get how it's done.Efficacy
What about NDK and using getInternalPath() and then use the Unix C functionality? This seems to be the easiest and most superior solution.Alphonsa
@Lother If you have a nicer solution that covers all storage-volumes, please write it as an answer.Efficacy
D
15

Is getAllocatableBytes indeed the way to get the free space?

Android 8.0 Features and APIs states that getAllocatableBytes(UUID):

Finally, when you need to allocate disk space for large files, consider using the new allocateBytes(FileDescriptor, long) API, which will automatically clear cached files belonging to other apps (as needed) to meet your request. When deciding if the device has enough disk space to hold your new data, call getAllocatableBytes(UUID) instead of using getUsableSpace(), since the former will consider any cached data that the system is willing to clear on your behalf.

So, getAllocatableBytes() reports how many bytes could be free for a new file by clearing cache for other apps but may not be currently free. This does not seem to be the right call for a general-purpose file utility.

In any case, getAllocatableBytes(UUID) doesn't seem to work for any volume other than the primary volume due to the inability to get acceptable UUIDs from StorageManager for storage volumes other than the primary volume. See Invalid UUID of storage gained from Android StorageManager? and Bug report #62982912. (Mentioned here for completeness; I realize that you already know about these.) The bug report is now over two years old with no resolution or hint at a work-around, so no love there.

If you want the type of free space reported by "Files by Google" or other file managers, then you will want to approach free space in a different way as explained below.

How can I get the free and real total space (in some cases I got lower values for some reason) of each StorageVolume, without requesting any permission, just like on Google's app?

Here is a procedure to get free and total space for available volumes:

Identify external directories: Use getExternalFilesDirs(null) to discover available external locations. What is returned is a File[]. These are directories that our app is permitted to use.

extDirs = {File2@9489
0 = {File@9509} "/storage/emulated/0/Android/data/com.example.storagevolumes/files"
1 = {File@9510} "/storage/14E4-120B/Android/data/com.example.storagevolumes/files"

(N.B. According to the documentation, this call returns what are considered to be stable devices such as SD cards. This does not return attached USB drives.)

Identify storage volumes: For each directory returned above, use StorageManager#getStorageVolume(File) to identify the storage volume that contains the directory. We don't need to identify the top-level directory to get the storage volume, just a file from the storage volume, so these directories will do.

Calculate total and used space: Determine the space on the storage volumes. The primary volume is treated differently from an SD card.

For the primary volume: Using StorageStatsManager#getTotalBytes(UUID get the nominal total bytes of storage on the primary device using StorageManager#UUID_DEFAULT . The value returned treats a kilobyte as 1,000 bytes (rather than 1,024) and a gigabyte as 1,000,000,000 bytes instead of 230. On my SamSung Galaxy S7 the value reported is 32,000,000,000 bytes. On my Pixel 3 emulator running API 29 with 16 MB of storage, the value reported is 16,000,000,000.

Here is the trick: If you want the numbers reported by "Files by Google", use 103 for a kilobyte, 106 for a megabyte and 109 for a gigabyte. For other file managers 210, 220 and 230 is what works. (This is demonstrated below.) See this for more information on these units.

To get free bytes, use StorageStatsManager#getFreeBytes(uuid). Used bytes is the difference between total bytes and free bytes.

For non-primary volumes: Space calculations for non-primary volumes is straightforward: For total space used File#getTotalSpace and File#getFreeSpace for the free space.

Here are a couple of screens shots that display volume stats. The first image shows the output of the StorageVolumeStats app (included below the images) and "Files by Google." The toggle button at the top of the top section switches the app between using 1,000 and 1,024 for kilobytes. As you can see, the figures agree. (This is a screen shot from a device running Oreo. I was unable to get the beta version of "Files by Google" loaded onto an Android Q emulator.)

enter image description here

The following image shows the StorageVolumeStats app at the top and output from "EZ File Explorer" on the bottom. Here 1,024 is used for kilobytes and the two apps agree on the total and free space available except for rounding.

enter image description here

MainActivity.kt

This small app is just the main activity. The manifest is generic, compileSdkVersion and targetSdkVersion are set to 29. minSdkVersion is 26.

class MainActivity : AppCompatActivity() {
    private lateinit var mStorageManager: StorageManager
    private val mStorageVolumesByExtDir = mutableListOf<VolumeStats>()
    private lateinit var mVolumeStats: TextView
    private lateinit var mUnitsToggle: ToggleButton
    private var mKbToggleValue = true
    private var kbToUse = KB
    private var mbToUse = MB
    private var gbToUse = GB

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        if (savedInstanceState != null) {
            mKbToggleValue = savedInstanceState.getBoolean("KbToggleValue", true)
            selectKbValue()
        }
        setContentView(statsLayout())

        mStorageManager = getSystemService(Context.STORAGE_SERVICE) as StorageManager

        getVolumeStats()
        showVolumeStats()
    }

    override fun onSaveInstanceState(outState: Bundle) {
        super.onSaveInstanceState(outState)
        outState.putBoolean("KbToggleValue", mKbToggleValue)
    }

    private fun getVolumeStats() {
        // We will get our volumes from the external files directory list. There will be one
        // entry per external volume.
        val extDirs = getExternalFilesDirs(null)

        mStorageVolumesByExtDir.clear()
        extDirs.forEach { file ->
            val storageVolume: StorageVolume? = mStorageManager.getStorageVolume(file)
            if (storageVolume == null) {
                Log.d(TAG, "Could not determinate StorageVolume for ${file.path}")
            } else {
                val totalSpace: Long
                val usedSpace: Long
                if (storageVolume.isPrimary) {
                    // Special processing for primary volume. "Total" should equal size advertised
                    // on retail packaging and we get that from StorageStatsManager. Total space
                    // from File will be lower than we want to show.
                    val uuid = StorageManager.UUID_DEFAULT
                    val storageStatsManager =
                        getSystemService(Context.STORAGE_STATS_SERVICE) as StorageStatsManager
                    // Total space is reported in round numbers. For example, storage on a
                    // SamSung Galaxy S7 with 32GB is reported here as 32_000_000_000. If
                    // true GB is needed, then this number needs to be adjusted. The constant
                    // "KB" also need to be changed to reflect KiB (1024).
//                    totalSpace = storageStatsManager.getTotalBytes(uuid)
                    totalSpace = (storageStatsManager.getTotalBytes(uuid) / 1_000_000_000) * gbToUse
                    usedSpace = totalSpace - storageStatsManager.getFreeBytes(uuid)
                } else {
                    // StorageStatsManager doesn't work for volumes other than the primary volume
                    // since the "UUID" available for non-primary volumes is not acceptable to
                    // StorageStatsManager. We must revert to File for non-primary volumes. These
                    // figures are the same as returned by statvfs().
                    totalSpace = file.totalSpace
                    usedSpace = totalSpace - file.freeSpace
                }
                mStorageVolumesByExtDir.add(
                    VolumeStats(storageVolume, totalSpace, usedSpace)
                )
            }
        }
    }

    private fun showVolumeStats() {
        val sb = StringBuilder()
        mStorageVolumesByExtDir.forEach { volumeStats ->
            val (usedToShift, usedSizeUnits) = getShiftUnits(volumeStats.mUsedSpace)
            val usedSpace = (100f * volumeStats.mUsedSpace / usedToShift).roundToLong() / 100f
            val (totalToShift, totalSizeUnits) = getShiftUnits(volumeStats.mTotalSpace)
            val totalSpace = (100f * volumeStats.mTotalSpace / totalToShift).roundToLong() / 100f
            val uuidToDisplay: String?
            val volumeDescription =
                if (volumeStats.mStorageVolume.isPrimary) {
                    uuidToDisplay = ""
                    PRIMARY_STORAGE_LABEL
                } else {
                    uuidToDisplay = " (${volumeStats.mStorageVolume.uuid})"
                    volumeStats.mStorageVolume.getDescription(this)
                }
            sb
                .appendln("$volumeDescription$uuidToDisplay")
                .appendln(" Used space: ${usedSpace.nice()} $usedSizeUnits")
                .appendln("Total space: ${totalSpace.nice()} $totalSizeUnits")
                .appendln("----------------")
        }
        mVolumeStats.text = sb.toString()
    }

    private fun getShiftUnits(x: Long): Pair<Long, String> {
        val usedSpaceUnits: String
        val shift =
            when {
                x < kbToUse -> {
                    usedSpaceUnits = "Bytes"; 1L
                }
                x < mbToUse -> {
                    usedSpaceUnits = "KB"; kbToUse
                }
                x < gbToUse -> {
                    usedSpaceUnits = "MB"; mbToUse
                }
                else -> {
                    usedSpaceUnits = "GB"; gbToUse
                }
            }
        return Pair(shift, usedSpaceUnits)
    }

    @SuppressLint("SetTextI18n")
    private fun statsLayout(): SwipeRefreshLayout {
        val swipeToRefresh = SwipeRefreshLayout(this)
        swipeToRefresh.setOnRefreshListener {
            getVolumeStats()
            showVolumeStats()
            swipeToRefresh.isRefreshing = false
        }

        val scrollView = ScrollView(this)
        swipeToRefresh.addView(scrollView)
        val linearLayout = LinearLayout(this)
        linearLayout.orientation = LinearLayout.VERTICAL
        scrollView.addView(
            linearLayout, ViewGroup.LayoutParams.MATCH_PARENT,
            ViewGroup.LayoutParams.WRAP_CONTENT
        )

        val instructions = TextView(this)
        instructions.text = "Swipe down to refresh."
        linearLayout.addView(
            instructions, ViewGroup.LayoutParams.WRAP_CONTENT,
            ViewGroup.LayoutParams.WRAP_CONTENT
        )
        (instructions.layoutParams as LinearLayout.LayoutParams).gravity = Gravity.CENTER

        mUnitsToggle = ToggleButton(this)
        mUnitsToggle.textOn = "KB = 1,000"
        mUnitsToggle.textOff = "KB = 1,024"
        mUnitsToggle.isChecked = mKbToggleValue
        linearLayout.addView(
            mUnitsToggle, ViewGroup.LayoutParams.WRAP_CONTENT,
            ViewGroup.LayoutParams.WRAP_CONTENT
        )
        mUnitsToggle.setOnClickListener { v ->
            val toggleButton = v as ToggleButton
            mKbToggleValue = toggleButton.isChecked
            selectKbValue()
            getVolumeStats()
            showVolumeStats()
        }

        mVolumeStats = TextView(this)
        mVolumeStats.typeface = Typeface.MONOSPACE
        val padding =
            16 * (resources.displayMetrics.densityDpi.toFloat() / DisplayMetrics.DENSITY_DEFAULT).toInt()
        mVolumeStats.setPadding(padding, padding, padding, padding)

        val lp = LinearLayout.LayoutParams(ViewGroup.LayoutParams.MATCH_PARENT, 0)
        lp.weight = 1f
        linearLayout.addView(mVolumeStats, lp)

        return swipeToRefresh
    }

    private fun selectKbValue() {
        if (mKbToggleValue) {
            kbToUse = KB
            mbToUse = MB
            gbToUse = GB
        } else {
            kbToUse = KiB
            mbToUse = MiB
            gbToUse = GiB
        }
    }

    companion object {
        fun Float.nice(fieldLength: Int = 6): String =
            String.format(Locale.US, "%$fieldLength.2f", this)

        // StorageVolume should have an accessible "getPath()" method that will do
        // the following so we don't have to resort to reflection.
        @Suppress("unused")
        fun StorageVolume.getStorageVolumePath(): String {
            return try {
                javaClass
                    .getMethod("getPath")
                    .invoke(this) as String
            } catch (e: Exception) {
                e.printStackTrace()
                ""
            }
        }

        // See https://en.wikipedia.org/wiki/Kibibyte for description
        // of these units.

        // These values seems to work for "Files by Google"...
        const val KB = 1_000L
        const val MB = KB * KB
        const val GB = KB * KB * KB

        // ... and these values seems to work for other file manager apps.
        const val KiB = 1_024L
        const val MiB = KiB * KiB
        const val GiB = KiB * KiB * KiB

        const val PRIMARY_STORAGE_LABEL = "Internal Storage"

        const val TAG = "MainActivity"
    }

    data class VolumeStats(
        val mStorageVolume: StorageVolume,
        var mTotalSpace: Long = 0,
        var mUsedSpace: Long = 0
    )
}

Addendum

Let's get more comfortable with using getExternalFilesDirs():

We call Context#getExternalFilesDirs() in the code. Within this method a call is made to Environment#buildExternalStorageAppFilesDirs() which calls Environment#getExternalDirs() to obtain the volume list from StorageManager. This storage list is used to create the paths we see returned from Context#getExternalFilesDirs() by appending some static path segments to the path identified by each storage volume.

We would really want access to Environment#getExternalDirs() so we can immediately determine space utilization, but we are restricted. Since the call we make depends upon a file list that is generated from the volume list, we can be comfortable that all volumes are covered by out code and we can get the space utilization information we need.

Dungeon answered 20/7, 2019 at 16:44 Comment(12)
So what you did is to write a lot about what I wrote and what I wrote in code. Even the bug report was made by me... Do you really think that Google just uses what I've found? That they use the getExternalFilesDirs (I used getExternalCacheDirs instead and as a fallback in case i can't get the path right away using reflection, but it's the same idea) to get the stats?Efficacy
I know you wrote the bug report but I didn't think it necessary for me to make that connection here. You asked two questions - I answered them and got good results (IMHO.) There is no reflection or any trickery or any reliance upon deprecated methods. If you want to know what Google does, you will have to ask them. I think that you will need to abandon the idea of displaying path names to the user. It doesn't look like "Files by Google" does but I can't get the latest beta to run, so maybe it does. If you want the path name, you will need to make some assumptions about the implementation.Dungeon
I didn't talk about the paths. I just said that you did it as I did - checking which path belongs to which StorageVolume, and then get the stats from there. The reflection of my solution is just a way to use it easier in case it succeeds. I know you did a good job, but do you think that what we found is what Google does? Is this the most official way to get the StorageVolume stats (total&free space) ?Efficacy
The "official" (and reasonable) way would be to request volumes and get space stats immediately from the volumes or through some identifier supplied by the volume such as the UUID. As we see, this doesn't work for non-primary volumes so, in my opinion, paths must be identified first then space determined. Whether this is done through reflection, making assumptions about the directory structure or asking for file dirs is up to the implementation. Maybe the sources for Q will shed some light when released.Dungeon
I see. But it's not just on Q. The app works on Android 8 too.Efficacy
@androiddeveloper I don't understand your last comment. When you say that the app works on Android 8, too, which app are you referring to? What is the significance of also working on 8?Dungeon
The "files by Google" app, which I've shown a screenshot of and wrote that it works even on Galaxy Note 8 with Android 8.Efficacy
Ok - the "Files by Google" app, but what is the significance of running on Android 8? Why shouldn't it? That is what I don't understand.Dungeon
You wrote "Maybe the sources for Q will shed some light when released." . Whatever they did on the app, it works even before Q.Efficacy
Anyway, since I don't see any other way that looks official to me, I will grant the bounty to you now, especially because you've made such a nice answer (though I personally don't need it, as I've come with a very similar solution already). Just please update this answer in case you find anything better, that looks more official, especially on Q, ok?Efficacy
@androiddeveloper I'll be on the lookout for a better solution.Dungeon
Let us continue this discussion in chat.Dungeon
D
7

The following uses fstatvfs(FileDescriptor) to retrieve stats without resorting to reflection or traditional file system methods.

To check the output of the program to make sure it is producing reasonable result for total, used and available space I ran the "df" command on an Android Emulator running API 29.

Output of "df" command in adb shell reporting 1K blocks:

"/data" corresponds to the "primary" UUID used when by StorageVolume#isPrimary is true.

"/storage/1D03-2E0E" corresponds to the "1D03-2E0E" UUID reported by StorageVolume#uuid.

generic_x86:/ $ df
Filesystem              1K-blocks    Used Available Use% Mounted on
/dev/root                 2203316 2140872     46060  98% /
tmpfs                     1020140     592   1019548   1% /dev
tmpfs                     1020140       0   1020140   0% /mnt
tmpfs                     1020140       0   1020140   0% /apex
/dev/block/vde1            132168   75936     53412  59% /vendor

/dev/block/vdc             793488  647652    129452  84% /data

/dev/block/loop0              232      36       192  16% /apex/com.android.apex.cts.shim@1
/data/media                793488  647652    129452  84% /storage/emulated

/mnt/media_rw/1D03-2E0E    522228      90    522138   1% /storage/1D03-2E0E

Reported by the app using fstatvfs (in 1K blocks):

For /tree/primary:/document/primary: Total=793,488 used space=647,652 available=129,452

For /tree/1D03-2E0E:/document/1D03-2E0E: Total=522,228 used space=90 available=522,138

The totals match.

fstatvfs is described here.

Detail on what fstatvfs returns can be found here.

The following little app displays used, free and total bytes for volumes that are accessible.

enter image description here

MainActivity.kt

class MainActivity : AppCompatActivity() {
    private lateinit var mStorageManager: StorageManager
    private val mVolumeStats = HashMap<Uri, StructStatVfs>()
    private val mStorageVolumePathsWeHaveAccessTo = HashSet<String>()
    private lateinit var mStorageVolumes: List<StorageVolume>
    private var mHaveAccessToPrimary = false

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)

        mStorageManager = getSystemService(Context.STORAGE_SERVICE) as StorageManager
        mStorageVolumes = mStorageManager.storageVolumes

        requestAccessButton.setOnClickListener {
            val primaryVolume = mStorageManager.primaryStorageVolume
            val intent = primaryVolume.createOpenDocumentTreeIntent()
            startActivityForResult(intent, 1)
        }

        releaseAccessButton.setOnClickListener {
            val takeFlags =
                Intent.FLAG_GRANT_READ_URI_PERMISSION or Intent.FLAG_GRANT_WRITE_URI_PERMISSION
            val uri = buildVolumeUriFromUuid(PRIMARY_UUID)

            contentResolver.releasePersistableUriPermission(uri, takeFlags)
            val toast = Toast.makeText(
                this,
                "Primary volume permission released was released.",
                Toast.LENGTH_SHORT
            )
            toast.setGravity(Gravity.BOTTOM, 0, releaseAccessButton.height)
            toast.show()
            getVolumeStats()
            showVolumeStats()
        }
        getVolumeStats()
        showVolumeStats()

    }

    private fun getVolumeStats() {
        val persistedUriPermissions = contentResolver.persistedUriPermissions
        mStorageVolumePathsWeHaveAccessTo.clear()
        persistedUriPermissions.forEach {
            mStorageVolumePathsWeHaveAccessTo.add(it.uri.toString())
        }
        mVolumeStats.clear()
        mHaveAccessToPrimary = false
        for (storageVolume in mStorageVolumes) {
            val uuid = if (storageVolume.isPrimary) {
                // Primary storage doesn't get a UUID here.
                PRIMARY_UUID
            } else {
                storageVolume.uuid
            }

            val volumeUri = uuid?.let { buildVolumeUriFromUuid(it) }

            when {
                uuid == null ->
                    Log.d(TAG, "UUID is null for ${storageVolume.getDescription(this)}!")
                mStorageVolumePathsWeHaveAccessTo.contains(volumeUri.toString()) -> {
                    Log.d(TAG, "Have access to $uuid")
                    if (uuid == PRIMARY_UUID) {
                        mHaveAccessToPrimary = true
                    }
                    val uri = buildVolumeUriFromUuid(uuid)
                    val docTreeUri = DocumentsContract.buildDocumentUriUsingTree(
                        uri,
                        DocumentsContract.getTreeDocumentId(uri)
                    )
                    mVolumeStats[docTreeUri] = getFileStats(docTreeUri)
                }
                else -> Log.d(TAG, "Don't have access to $uuid")
            }
        }
    }

    private fun showVolumeStats() {
        val sb = StringBuilder()
        if (mVolumeStats.size == 0) {
            sb.appendln("Nothing to see here...")
        } else {
            sb.appendln("All figures are in 1K blocks.")
            sb.appendln()
        }
        mVolumeStats.forEach {
            val lastSeg = it.key.lastPathSegment
            sb.appendln("Volume: $lastSeg")
            val stats = it.value
            val blockSize = stats.f_bsize
            val totalSpace = stats.f_blocks * blockSize / 1024L
            val freeSpace = stats.f_bfree * blockSize / 1024L
            val usedSpace = totalSpace - freeSpace
            sb.appendln(" Used space: ${usedSpace.nice()}")
            sb.appendln(" Free space: ${freeSpace.nice()}")
            sb.appendln("Total space: ${totalSpace.nice()}")
            sb.appendln("----------------")
        }
        volumeStats.text = sb.toString()
        if (mHaveAccessToPrimary) {
            releaseAccessButton.visibility = View.VISIBLE
            requestAccessButton.visibility = View.GONE
        } else {
            releaseAccessButton.visibility = View.GONE
            requestAccessButton.visibility = View.VISIBLE
        }
    }

    private fun buildVolumeUriFromUuid(uuid: String): Uri {
        return DocumentsContract.buildTreeDocumentUri(
            EXTERNAL_STORAGE_AUTHORITY,
            "$uuid:"
        )
    }

    private fun getFileStats(docTreeUri: Uri): StructStatVfs {
        val pfd = contentResolver.openFileDescriptor(docTreeUri, "r")!!
        return fstatvfs(pfd.fileDescriptor)
    }

    override fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent?) {
        super.onActivityResult(requestCode, resultCode, data)
        Log.d(TAG, "resultCode:$resultCode")
        val uri = data?.data ?: return
        val takeFlags =
            Intent.FLAG_GRANT_READ_URI_PERMISSION or Intent.FLAG_GRANT_WRITE_URI_PERMISSION
        contentResolver.takePersistableUriPermission(uri, takeFlags)
        Log.d(TAG, "granted uri: ${uri.path}")
        getVolumeStats()
        showVolumeStats()
    }

    companion object {
        fun Long.nice(fieldLength: Int = 12): String = String.format(Locale.US, "%,${fieldLength}d", this)

        const val EXTERNAL_STORAGE_AUTHORITY = "com.android.externalstorage.documents"
        const val PRIMARY_UUID = "primary"
        const val TAG = "AppLog"
    }
}

activity_main.xml

<LinearLayout 
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        android:orientation="vertical"
        tools:context=".MainActivity">

    <TextView
            android:id="@+id/volumeStats"
            android:layout_width="match_parent"
            android:layout_height="0dp"
            android:layout_marginBottom="16dp"
            android:layout_weight="1"
            android:fontFamily="monospace"
            android:padding="16dp" />

    <Button
            android:id="@+id/requestAccessButton"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:layout_gravity="center_horizontal"
            android:layout_marginBottom="16dp"
            android:visibility="gone"
            android:text="Request Access to Primary" />

    <Button
            android:id="@+id/releaseAccessButton"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:layout_gravity="center_horizontal"
            android:layout_marginBottom="16dp"
            android:text="Release Access to Primary" />
</LinearLayout>   
Dungeon answered 27/6, 2019 at 19:37 Comment(21)
How do you get the total and free space from fstatvfs ? Can you please update it?Efficacy
BTW, I've shown 2 solutions. One uses reflection, and the other doesn't. Don't you think the other solution should work fine?Efficacy
@androiddeveloper I may be mistaken, but with the changes in "Q" and beyond, isn't it better to stay away from traditional file paths, etc? The solution I posted uses only what the new framework offers.Dungeon
You have a point, but I think that it's still a something with a real path after all... OK here's an offer: make your code with nice functions instead of all in setOnClickListener , put a more readable text for stats, that will show total and free space , and have a fallback to what I did if it seems yours failed, and I will grant the bounty if indeed that's the best answer here. I don't think there are decent APIs for this SAF... :(Efficacy
For some reason this doesn't seem to work when I don't have access to the storage volumes , which misses the point of showing them with the space they have. If you run my code, it works without the need to grant any access to the storage volumes. In addition, just like in my case, for some reason it shows a total space that is lower than what's built in (~54GB instead of 64GB). I wonder if it's because that's a partition thing, as the system has its own partition to save for itself.Efficacy
Do you know of a way to get the same information you have now, without the need to request access to the storageVolume? Like what I did? Here, I've updated my answer to be even shorter. Also, do you know perhaps what's with the total space being different than what's really on the device?Efficacy
The demo stays within the confines of the SAF. In the future, will all traditional file-level access (directory paths, etc.) outside of an app's sandbox be shut down or not? In the meantime, I would go with the SAF whenever possible and fall back to directory paths when SAF cannot accommodate the request. As for space, my experience is that some of it always trickles out through low-level formatting, boot partitions, hidden partitions, VM heaps and the like. The demo agrees with what the "df" command reports when run in a shell.Dungeon
Since you don't always have access to the StorageVolumes , I would prefer to have something than nothing, even if it works for now. However, I understand your point. I will keep this question open. Maybe someone else will find a nicer solution/workaround. I hope Google will provide a better, official way to do it.Efficacy
@androiddeveloper We'll see what happens. The storage changes to Q are so fundamental, that I think they will be stamping out fires for the foreseeable future. Is "Q" for "questionable"? :(Dungeon
I really hate the recent restrictions: clipboard, storage, and even system-alert-window are planned to be ruined. And what did we get in Q? Just dark mode? Anyway, back to here. Do you think you can find a better workaround, that will work even without any permission or access?Efficacy
@androiddeveloper The storage functions either need a path (discouraged by the adoption of SAF but works for the time-being) or a file descriptor (not made available through SAF except with permission.) Unless there is some method that I am not aware of (always a possibility), I think this is the best we can do. As Q proceeds, the documentation improves, bugs are fixed, etc., something may come to light.Dungeon
OK I'm going to leave this question open till a better solution, then. I don't like any of those solutions. Here, get +1 for the comment.Efficacy
About the official API, there is StorageStatsManager , but for some reason I can't use it in most cases, where I can't create the UUID. This includes SD-card for example, but the primary storage stats can be achieved, and it even provides the real total space (64GB in my case). Your solution seems to work, but as I've found, "Files by Google" app shows the information of all StorageVolume instances without any permission to them. Can you please check it out?Efficacy
The UUIDs that I am getting from the emulator don't pass muster: they are too short. I don't know why this is. Is there an issue with the emulator and a real Q device (which I don't have) will be more forthcoming with genuine UUIDs, or, is the problem that revealing a real UUID would permit the unique identification of a device that, I think, Google is loathe to do. (Isn't that why the real UUID for the primary device is not revealed?) Evidently, "Files by Google" has a secret sauce, but I don't know what it is. @androiddeveloper , I'll post here if I gain any insightDungeon
I updated my answer and question about this. The UUID issue was also tested on a real device (Galaxy Note with Android 8) , and indeed it crashed when reaching SD-card, but worked fine for the primary storage. I don't think it's an issue on Q. I'm probably missing some API here that I didn't look at. One that will probably remove the need for weird workarounds.Efficacy
@androiddeveloper I wonder if the UUID the app sees on the Galaxy Note with 8 is a real UUID (128 bits) or the shorter version (32 bits). If the shorter version then, for a test, you could hard-code the real UUID for the SD card into the program and try _StorageStatsManager _ with the real UUID.Dungeon
How could I get the real UUID ?Efficacy
@androiddeveloper On second thought, I don't think my suggestion will be helpful. But, if you want to try it, on Windows, bring up PowerShell or the standard CLI and type in "mountvol". It should kick out the UUIDs of all mounted devices. (Make sure the SD card is available to Windows.) If not on Windows then you'll need to search for a method, but this is the idea.Dungeon
This technique takes a very long time on Android 12. Any idea why? Or any kind of workaround? #70018879Gladiate
@GavinWright I have looked at this in so long, I really couldn't say. I think that Android has taken file access, a solved problem, to a new level.Dungeon
FYI, I tried this technique, using openFileDescriptor on a content: URI, then fstatvfs() on the file descriptor. It was successful (on Android 11) for files, but not for directories. When I used a URI (from OpenDocumentTree) that points to a directory, openFileDescriptor() threw an IllegalArgumentException, and the stack trace showed readExceptionWithFileNotFoundExceptionFromParcel along the way. Apparently it doesn't like opening directories as files. Tried openAssetFileDescriptor() as well, but the result was the same. The workaround seems to be, target a known file.Bumbling
E
5

Found a workaround, by using what I wrote here , and mapping each StorageVolume with a real file as I wrote here. Sadly this might not work in the future, as it uses a lot of "tricks" :

        for (storageVolume in storageVolumes) {
            val volumePath = FileUtilEx.getVolumePath(storageVolume)
            if (volumePath == null) {
                Log.d("AppLog", "storageVolume \"${storageVolume.getDescription(this)}\" - failed to get volumePath")
            } else {
                val statFs = StatFs(volumePath)
                val availableSizeInBytes = statFs.availableBytes
                val totalBytes = statFs.totalBytes
                val formattedResult = "availableSizeInBytes:${android.text.format.Formatter.formatShortFileSize(this, availableSizeInBytes)} totalBytes:${android.text.format.Formatter.formatShortFileSize(this, totalBytes)}"
                Log.d("AppLog", "storageVolume \"${storageVolume.getDescription(this)}\" - volumePath:$volumePath - $formattedResult")
            }
        }

Seems to work on both emulator (that has primary storage and SD-card) and real device (Pixel 2), both on Android Q beta 4.

A bit better solution which wouldn't use reflection, could be to put a unique file in each of the paths we get on ContextCompat.getExternalCacheDirs, and then try to find them via each of the StorageVolume instances. It is tricky though because you don't know when to start the search, so you will need to check various paths till you reach the destination. Not only that, but as I wrote here, I don't think there is an official way to get the Uri or DocumentFile or File or file-path of each StorageVolume.

Anyway, weird thing is that the total space is lower than the real one. Probably as it's a partition of what's the maximum that's really available to the user.

I wonder how come various apps (such as file manager apps, like Total Commander) get the real total device storage.


EDIT: OK got another workaround, which is probably more reliable, based on the storageManager.getStorageVolume(File) function.

So here is the merging of the 2 workarounds:

fun getStorageVolumePath(context: Context, storageVolumeToGetItsPath: StorageVolume): String? {
    //first, try to use reflection
    if (Build.VERSION.SDK_INT < Build.VERSION_CODES.LOLLIPOP)
        return null
    try {
        val storageVolumeClazz = StorageVolume::class.java
        val getPathMethod = storageVolumeClazz.getMethod("getPath")
        val result = getPathMethod.invoke(storageVolumeToGetItsPath) as String?
         if (!result.isNullOrBlank())
            return result
    } catch (e: Exception) {
        e.printStackTrace()
    }
    //failed to use reflection, so try mapping with app's folders
    val storageVolumeUuidStr = storageVolumeToGetItsPath.uuid
    val externalCacheDirs = ContextCompat.getExternalCacheDirs(context)
    val storageManager = context.getSystemService(Context.STORAGE_SERVICE) as StorageManager
    for (externalCacheDir in externalCacheDirs) {
        val storageVolume = storageManager.getStorageVolume(externalCacheDir) ?: continue
        val uuidStr = storageVolume.uuid
        if (uuidStr == storageVolumeUuidStr) {
            //found storageVolume<->File match
            var resultFile = externalCacheDir
            while (true) {
                val parentFile = resultFile.parentFile ?: return resultFile.absolutePath
                val parentFileStorageVolume = storageManager.getStorageVolume(parentFile)
                        ?: return resultFile.absolutePath
                if (parentFileStorageVolume.uuid != uuidStr)
                    return resultFile.absolutePath
                resultFile = parentFile
            }
        }
    }
    return null
}

And to show the available and total space, we use StatFs as before:

for (storageVolume in storageVolumes) {
    val storageVolumePath = getStorageVolumePath(this@MainActivity, storageVolume) ?: continue
    val statFs = StatFs(storageVolumePath)
    val availableSizeInBytes = statFs.availableBytes
    val totalBytes = statFs.totalBytes
    val formattedResult = "availableSizeInBytes:${android.text.format.Formatter.formatShortFileSize(this, availableSizeInBytes)} totalBytes:${android.text.format.Formatter.formatShortFileSize(this, totalBytes)}"
    Log.d("AppLog", "storageVolume \"${storageVolume.getDescription(this)}\" - storageVolumePath:$storageVolumePath - $formattedResult")
}

EDIT: shorter version, without using the real file-path of the storageVolume:

fun getStatFsForStorageVolume(context: Context, storageVolumeToGetItsPath: StorageVolume): StatFs? {
    //first, try to use reflection
    if (Build.VERSION.SDK_INT < Build.VERSION_CODES.N)
        return null
    try {
        val storageVolumeClazz = StorageVolume::class.java
        val getPathMethod = storageVolumeClazz.getMethod("getPath")
        val resultPath = getPathMethod.invoke(storageVolumeToGetItsPath) as String?
        if (!resultPath.isNullOrBlank())
            return StatFs(resultPath)
    } catch (e: Exception) {
        e.printStackTrace()
    }
    //failed to use reflection, so try mapping with app's folders
    val storageVolumeUuidStr = storageVolumeToGetItsPath.uuid
    val externalCacheDirs = ContextCompat.getExternalCacheDirs(context)
    val storageManager = context.getSystemService(Context.STORAGE_SERVICE) as StorageManager
    for (externalCacheDir in externalCacheDirs) {
        val storageVolume = storageManager.getStorageVolume(externalCacheDir) ?: continue
        val uuidStr = storageVolume.uuid
        if (uuidStr == storageVolumeUuidStr) {
            //found storageVolume<->File match
            return StatFs(externalCacheDir.absolutePath)
        }
    }
    return null
}

Usage:

        for (storageVolume in storageVolumes) {
            val statFs = getStatFsForStorageVolume(this@MainActivity, storageVolume)
                    ?: continue
            val availableSizeInBytes = statFs.availableBytes
            val totalBytes = statFs.totalBytes
            val formattedResult = "availableSizeInBytes:${android.text.format.Formatter.formatShortFileSize(this, availableSizeInBytes)} totalBytes:${android.text.format.Formatter.formatShortFileSize(this, totalBytes)}"
            Log.d("AppLog", "storageVolume \"${storageVolume.getDescription(this)}\" - $formattedResult")
        }

Note that this solution doesn't require any kind of permission.

--

EDIT: I actually found out that I tried to do it in the past, but for some reason it crashed for me on the SD-card StoraveVolume on the emulator:

        val storageStatsManager = getSystemService(Context.STORAGE_STATS_SERVICE) as StorageStatsManager
        for (storageVolume in storageVolumes) {
            val uuidStr = storageVolume.uuid
            val uuid = if (uuidStr == null) StorageManager.UUID_DEFAULT else UUID.fromString(uuidStr)
            val availableSizeInBytes = storageStatsManager.getFreeBytes(uuid)
            val totalBytes = storageStatsManager.getTotalBytes(uuid)
            val formattedResult = "availableSizeInBytes:${android.text.format.Formatter.formatShortFileSize(this, availableSizeInBytes)} totalBytes:${android.text.format.Formatter.formatShortFileSize(this, totalBytes)}"
            Log.d("AppLog", "storageVolume \"${storageVolume.getDescription(this)}\" - $formattedResult")
        }

The good news is that for the primary storageVolume, you get the real total space of it.

On a real device it also crashes for the SD-card, but not for the primary one.


So here's the latest solution for this, gathering the above:

        for (storageVolume in storageVolumes) {
            val availableSizeInBytes: Long
            val totalBytes: Long
            if (storageVolume.isPrimary) {
                val storageStatsManager = getSystemService(Context.STORAGE_STATS_SERVICE) as StorageStatsManager
                val uuidStr = storageVolume.uuid
                val uuid = if (uuidStr == null) StorageManager.UUID_DEFAULT else UUID.fromString(uuidStr)
                availableSizeInBytes = storageStatsManager.getFreeBytes(uuid)
                totalBytes = storageStatsManager.getTotalBytes(uuid)
            } else {
                val statFs = getStatFsForStorageVolume(this@MainActivity, storageVolume)
                        ?: continue
                availableSizeInBytes = statFs.availableBytes
                totalBytes = statFs.totalBytes
            }
            val formattedResult = "availableSizeInBytes:${android.text.format.Formatter.formatShortFileSize(this, availableSizeInBytes)} totalBytes:${android.text.format.Formatter.formatShortFileSize(this, totalBytes)}"
            Log.d("AppLog", "storageVolume \"${storageVolume.getDescription(this)}\" - $formattedResult")
        }

Updated answer for Android R:

        fun getStorageVolumesAccessState(context: Context) {
            val storageManager = context.getSystemService(Context.STORAGE_SERVICE) as StorageManager
            val storageVolumes = storageManager.storageVolumes
            val storageStatsManager = context.getSystemService(Context.STORAGE_STATS_SERVICE) as StorageStatsManager
            for (storageVolume in storageVolumes) {
                var freeSpace: Long = 0L
                var totalSpace: Long = 0L
                val path = getPath(context, storageVolume)
                if (storageVolume.isPrimary) {
                    totalSpace = storageStatsManager.getTotalBytes(StorageManager.UUID_DEFAULT)
                    freeSpace = storageStatsManager.getFreeBytes(StorageManager.UUID_DEFAULT)
                } else if (path != null) {
                    val file = File(path)
                    freeSpace = file.freeSpace
                    totalSpace = file.totalSpace
                }
                val usedSpace = totalSpace - freeSpace
                val freeSpaceStr = Formatter.formatFileSize(context, freeSpace)
                val totalSpaceStr = Formatter.formatFileSize(context, totalSpace)
                val usedSpaceStr = Formatter.formatFileSize(context, usedSpace)
                Log.d("AppLog", "${storageVolume.getDescription(context)} - path:$path total:$totalSpaceStr used:$usedSpaceStr free:$freeSpaceStr")
            }
        }

        fun getPath(context: Context, storageVolume: StorageVolume): String? {
            if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.R)
                storageVolume.directory?.absolutePath?.let { return it }
            try {
                return storageVolume.javaClass.getMethod("getPath").invoke(storageVolume) as String
            } catch (e: Exception) {
            }
            try {
                return (storageVolume.javaClass.getMethod("getPathFile").invoke(storageVolume) as File).absolutePath
            } catch (e: Exception) {
            }
            val extDirs = context.getExternalFilesDirs(null)
            for (extDir in extDirs) {
                val storageManager = context.getSystemService(Context.STORAGE_SERVICE) as StorageManager
                val fileStorageVolume: StorageVolume = storageManager.getStorageVolume(extDir)
                        ?: continue
                if (fileStorageVolume == storageVolume) {
                    var file = extDir
                    while (true) {
                        val parent = file.parentFile ?: return file.absolutePath
                        val parentStorageVolume = storageManager.getStorageVolume(parent)
                                ?: return file.absolutePath
                        if (parentStorageVolume != storageVolume)
                            return file.absolutePath
                        file = parent
                    }
                }
            }
            try {
                val parcel = Parcel.obtain()
                storageVolume.writeToParcel(parcel, 0)
                parcel.setDataPosition(0)
                parcel.readString()
                return parcel.readString()
            } catch (e: Exception) {
            }
            return null
        }
Efficacy answered 21/6, 2019 at 8:15 Comment(12)
I've tested the last version you posted for Android R and for external USB drives the total and available size is always 0 on Android 11 on a Pixel 2 device. Have you by any chance found a workaround for those?Daudet
@Daudet External USB drive? No idea. Any way to test it without it? Especially on emulator? The emulator shows the size for both its storage volumes... Would connecting another smartphone be considered as external USB drive? can one smartphone access the storage of the other?Efficacy
For me it was easy to test because I had a USB pen lying around and some of my phones came with a USB-A to USB-C converter (Pixel phones). Unfortunately I don't know how to mount these on an emulator sorry. You don't happen to have those, do you? :)Daudet
@Daudet I have another smartphone and a USB-C to USB-C cable. Would that be the same? Can one device see the storage of the other this way? I wonder if it's even possible (and makes sense) to check the free storage this way.Efficacy
@Daudet I've tested being connected via USB to another smartphone. I think it's similar to what you describe. After setting on the other smartphone that I allow reaching its files, only the built-in Files app allowed me reaching there. It wasn't mentioned using the code I've made. However, I think searching for USBOTG will solve the issue for you (and sadly not for the case I test, of connected smartphones). Here are some possible links that might be useful: https://mcmap.net/q/131399/-play-video-using-connected-usb-via-otg-cable-in-android/878126 github.com/Batlin/OTGViewer . Please let me know what you find.Efficacy
Thanks. I tried looking there but the most I could find was the ability to see the total size of external storage. I'll keep looking to see if I can find available storage size as well. Thanks for your help!Daudet
@Daudet I think it might not be possible. For some reason I saw that I can access (with some informational-size-number) only on the built-in "Files" app. On other file manager apps I've failed to reach the paths of the other device (except for when using SAW) . So because I don't think it's possible requested here: issuetracker.google.com/issues/185527171 . Please consider starring.Efficacy
Ah yes, I just tried to connect to another phone via USB and indeed it seems different than simply connecting a USB OTG drive. The file browser I use (Solid Explorer) can not see the files or storage space available on the other phone while it CAN on USB OTG. Problem is that I can't even get the space on OTG myself... 😅Daudet
@Daudet Please create a new request there, then. I could star it.Efficacy
The problem is that I know that it's possible since some file browsers are able to access USB OTG full and available size. I just can't find out how they did it. So I can't really create a request since it's obviously something that already exists... Right?Daudet
@Daudet Maybe they use some weird workaround, then. Try to find it, and if you don't, you can ask here a new question , and if that doesn't work, you could assume it doesn't exist as a decent API and request it. If you are confident it doesn't exist, you can do those all at the same time.Efficacy
Awesome! Thanks for all the help!Daudet

© 2022 - 2024 — McMap. All rights reserved.