How to use the new SD card access API presented for Android 5.0 (Lollipop)?
Asked Answered
B

3

125

Background

On Android 4.4 (KitKat), Google has made access to the SD card quite restricted.

As of Android Lollipop (5.0), developers can use a new API that asks the user to confirm to allow access to specific folders, as written on the this Google-Groups post .

The problem

The post directs you to visit two websites:

This looks like an inner example (perhaps to be shown on the API demos later), but it's quite hard to understand what's going on.

This is the official documentation of the new API, but it doesn't tell enough details about how to use it.

Here's what it tells you:

If you really do need full access to an entire subtree of documents, start by launching ACTION_OPEN_DOCUMENT_TREE to let the user pick a directory. Then pass the resulting getData() into fromTreeUri(Context, Uri) to start working with the user selected tree.

As you navigate the tree of DocumentFile instances, you can always use getUri() to obtain the Uri representing the underlying document for that object, for use with openInputStream(Uri), etc.

To simplify your code on devices running KITKAT or earlier, you can use fromFile(File) which emulates the behavior of a DocumentsProvider.

The questions

I have a few questions about the new API:

  1. How do you really use it?
  2. According to the post, the OS will remember that the app was given a permission to access the files/folders. How do you check if you can access the files/folders? Is there a function that returns me the list of files/folders that I can access?
  3. How do you handle this problem on Kitkat? Is it a part of the support library?
  4. Is there a settings screen on the OS that shows which apps have access to which files/folders?
  5. What happens if an app is installed for multiple users on the same device?
  6. Is there any other documentation/tutorial about this new API?
  7. Can the permissions be revoked? If so, is there an intent that's being sent to the app?
  8. Would asking for the permission work recursively on a selected folder?
  9. Would using the permission also allow to give the user a chance of multiple selection by user's choice? Or does the app need to specifically tell the intent which files/folders to allow?
  10. Is there a way on the emulator to try the new API ? I mean, it has SD-card partition, but it works as the primary external storage, so all access to it is already given (using a simple permission).
  11. What happens when the user replaces the SD card with another one?
Blakeslee answered 4/11, 2014 at 20:54 Comment(7)
FWIW, AndroidPolice had a little article about this today.Selfimmolation
@Selfimmolation Thank you. but they are showing about what I've read already. It's good news though.Blakeslee
Each time a new OS comes out, they make our life more complicate...Dozier
@Funkystein true. Wish they did it on Kitkat. Now there are 3 types of behaviors to handle.Blakeslee
@Funkystein I think they allow both the exact and inexact ways to use it: developer.android.com/reference/android/app/…, long, android.app.PendingIntent)Blakeslee
Yes, but then there's one more check on the API level. If < 19 use the good old method else use the setInexact method... :(Dozier
@Funkystein I don't know. I used it a long time ago. It's not that bad to do this check, I think. You have to remember they are humans too, so they can make mistakes and change their minds from time to time...Blakeslee
S
151

Lots of good questions, let's dig in. :)

How do you use it?

Here's a great tutorial for interacting with the Storage Access Framework in KitKat:

https://developer.android.com/guide/topics/providers/document-provider.html#client

Interacting with the new APIs in Lollipop is very similar. To prompt the user to pick a directory tree, you can launch an intent like this:

    Intent intent = new Intent(Intent.ACTION_OPEN_DOCUMENT_TREE);
    startActivityForResult(intent, 42);

Then in your onActivityResult(), you can pass the user-picked Uri to the new DocumentFile helper class. Here's a quick example that lists the files in the picked directory, and then creates a new file:

public void onActivityResult(int requestCode, int resultCode, Intent resultData) {
    if (resultCode == RESULT_OK) {
        Uri treeUri = resultData.getData();
        DocumentFile pickedDir = DocumentFile.fromTreeUri(this, treeUri);

        // List all existing files inside picked directory
        for (DocumentFile file : pickedDir.listFiles()) {
            Log.d(TAG, "Found file " + file.getName() + " with size " + file.length());
        }

        // Create a new file and write into it
        DocumentFile newFile = pickedDir.createFile("text/plain", "My Novel");
        OutputStream out = getContentResolver().openOutputStream(newFile.getUri());
        out.write("A long time ago...".getBytes());
        out.close();
    }
}

The Uri returned by DocumentFile.getUri() is flexible enough to use with may different platform APIs. For example, you could share it using Intent.setData() with Intent.FLAG_GRANT_READ_URI_PERMISSION.

If you want to access that Uri from native code, you can call ContentResolver.openFileDescriptor() and then use ParcelFileDescriptor.getFd() or detachFd() to obtain a traditional POSIX file descriptor integer.

How do you check if you can access the files/folders?

By default, the Uris returned through Storage Access Frameworks intents are not persisted across reboots. The platform "offers" the ability to persist the permission, but you still need to "take" the permission if you want it. In our example above, you'd call:

    getContentResolver().takePersistableUriPermission(treeUri,
            Intent.FLAG_GRANT_READ_URI_PERMISSION |
            Intent.FLAG_GRANT_WRITE_URI_PERMISSION);

You can always figure out what persisted grants your app has access to through the ContentResolver.getPersistedUriPermissions() API. If you no longer need access to a persisted Uri, you can release it with ContentResolver.releasePersistableUriPermission().

Is this available on KitKat?

No, we can't retroactively add new functionality to older versions of the platform.

Can I see what apps have access to files/folders?

There's currently no UI that shows this, but you can find the details in the "Granted Uri Permissions" section of adb shell dumpsys activity providers output.

What happens if an app is installed for multiple users on the same device?

Uri permission grants are isolated on a per-user basis, just like all other multi-user platform functionality. That is, the same app running under two different users has no overlaping or shared Uri permission grants.

Can the permissions be revoked?

The backing DocumentProvider can revoke permission at any time, such as when a cloud-based document is deleted. The most common way to discover these revoked permissions is when they disappear from ContentResolver.getPersistedUriPermissions() mentioned above.

Permissions are also revoked whenever app data is cleared for either app involved in the grant.

Would asking for the permission work recursively on a selected folder?

Yep, the ACTION_OPEN_DOCUMENT_TREE intent gives you recursive access to both existing and newly created files and directories.

Does this allow multiple selection?

Yep, multiple selection has been supported since KitKat, and you can allow it by setting EXTRA_ALLOW_MULTIPLE when starting your ACTION_OPEN_DOCUMENT intent. You can use Intent.setType() or EXTRA_MIME_TYPES to narrow the types of files that can be picked:

http://developer.android.com/reference/android/content/Intent.html#ACTION_OPEN_DOCUMENT

Is there a way on the emulator to try the new API?

Yep, the primary shared storage device should appear in the picker, even on the emulator. If your app only uses the Storage Access Framework for accessing shared storage, you no longer need the READ/WRITE_EXTERNAL_STORAGE permissions at all and can remove them or use the android:maxSdkVersion feature to only request them on older platform versions.

What happens when the user replaces the SD-card with another one?

When physical media is involved, the UUID (such as FAT serial number) of the underlying media is always burned into the returned Uri. The system uses this to connect you to the media that the user originally selected, even if the user swaps the media around between multiple slots.

If the user swaps in a second card, you'll need to prompt to gain access to the new card. Since the system remembers grants on a per-UUID basis, you'll continue to have previously-granted access to the original card if the user reinserts it later.

http://en.wikipedia.org/wiki/Volume_serial_number

Sludge answered 5/11, 2014 at 19:53 Comment(21)
Thank you for the answers. Will you please consider adding some sort of control to the OS itself, to manage which apps have access to which folders? Also, how can I convert the Uris to File objects? via "getPath", right ? I ask this since sometimes a Uri object actually needs extra query (like the media store). Then I can delete, create, modify anything in it. You wrote that it's recursive. Does it also allow to rename the root file/folder that I got permission to handle? What happens when it gets deleted? About revoking, I also meant as an end user : can the end user revoke the permission?Blakeslee
Added more details to example code above to use DocumentFile.getUri() to open and write some data.Sludge
Thank you. Wait, so I can't use the normal "File" class? And what's with the "createFile" function? how would I be able to create a file with an extension that I choose?Blakeslee
The java.lang.File object only works with files on disk, but the SAF is an abstraction layer which is much more flexible. For example, the Vault example linked above encrypts files before they're actually written to disk; something that isn't possible with the traditional File object.Sludge
I see. so the decision was that instead of adding more to the known API (of File), to use a different one. OK. Can you please answer the other questions (written in the first comment) ? BTW, thank you for answering all of this.Blakeslee
@jeffSharkey you said that for Kit-Kat "No, we can't retroactively add new functionality to older versions of the platform." but google documentation its mentioned that " To simplify your code on devices running KITKAT or earlier, you can use fromFile(File) which emulates the behavior of a DocumentsProvider ". Can you please tell me how to fix this issue for devices running on Kit-Kat.Paschall
When I use the command on a Nvidia SHIELD TABLET with Android Lollipop, the dialog just shows the internal sd-card, but no way to see the external sd card (which is the one we want to get access). Help please!Bouillon
@JeffSharkey Any way to provide OPEN_DOCUMENT_TREE with a starting location hint? Users aren't always the best at navigating to what the application needs access to.Tagmemics
Is there a way, how to change Last Modified Timestamp (setLastModified() method in File class)? It is really fundamental for applications like archivers.Megillah
Lets say you have a stored folder document Uri and you want to list files in in later at some point after reboot of device. DocumentFile.fromTreeUri always list root files, no matter the Uri you give it (even child Uri), so how do you create a DocumentFile you can call listfiles on, where the DocumentFile is not respresenting either a the root or a SingleDocument.Shrift
@JeffSharkey How can this URI be used in MediaMuxer, since it accepts a string as the output file path? MediaMuxer(java.lang.String, intRiyal
This should be marked as the answer, very helpful and a great API on Google's part. Wish this came along before Lollipop.Dielle
Supposed I have a Uri (with _ID) from MediaStore.Audio.Media.EXTERNAL_CONTENT_URI received by querying database. Given that I have write access (with SAF properly) to a directory tree which the Uri's file resides, how to modify the file?Rrhagia
@JeffSharkey Would you be so kind to answer this question https://mcmap.net/q/88420/-sd-card-access-api-android-for-lollipop/753575 ?Delcine
@JeffSharkey Are you sure the granted permission works per user? I granted Uri permission with 1st account. Then I removed that account and sign in with 2nd account. Uri permission is still granted!Omniscient
@Tagmemics usability can be slightly improved by forcing the Advanced view: https://mcmap.net/q/88421/-extra-to-enable-the-show-hide-sd-card-with-storage-access-framework-saf In addition, you can add EXTRA_LOCAL_ONLY flagHuntingdonshire
I followed the steps, but the images are not showing on ListView. This ListView used to work before Kitkat. How do you display the images that are stored on the SD Card to the ListView? Thanks a lotNeuromuscular
@JeffSharkey how can we tackle this on devices running KitKat?Tyus
I have successfully readed the content of internal storage using this new document but unfortunately the stream returned by resolver.openOutputStream does not have random access (mark () and reset () position). Have I done somrthing wrong?Kierakieran
Why this painful user experience Android? For the developers and even for the end user. Why Why WHY??Inaudible
I have implemented my own file/folder selection dialog with advanced functionality (e.g. file/folder search based on regex or file name extension etc.). It seems that I'll have to throw it away after this new introduction. Otherwise I'll have to show user my file dialog first to select file/folder and then the second dialog to get the permission...Detection
C
48

In my Android project in Github, linked below, you can find working code that allows to write on extSdCard in Android 5. It assumes that the user gives access to the whole SD Card and then lets you write everywhere on this card. (If you want to have access only to single files, things get easier.)

Main Code snipplets

Triggering the Storage Access Framework:

@TargetApi(Build.VERSION_CODES.LOLLIPOP)
private void triggerStorageAccessFramework() {
    Intent intent = new Intent(Intent.ACTION_OPEN_DOCUMENT_TREE);
    startActivityForResult(intent, REQUEST_CODE_STORAGE_ACCESS);
}

Handling the response from the Storage Access Framework:

@TargetApi(Build.VERSION_CODES.LOLLIPOP)
@Override
public final void onActivityResult(final int requestCode, final int resultCode, final Intent resultData) {
    if (requestCode == SettingsFragment.REQUEST_CODE_STORAGE_ACCESS) {
        Uri treeUri = null;
        if (resultCode == Activity.RESULT_OK) {
            // Get Uri from Storage Access Framework.
            treeUri = resultData.getData();

            // Persist URI in shared preference so that you can use it later.
            // Use your own framework here instead of PreferenceUtil.
            PreferenceUtil.setSharedPreferenceUri(R.string.key_internal_uri_extsdcard, treeUri);

            // Persist access permissions.
            final int takeFlags = resultData.getFlags()
                & (Intent.FLAG_GRANT_READ_URI_PERMISSION | Intent.FLAG_GRANT_WRITE_URI_PERMISSION);
            getActivity().getContentResolver().takePersistableUriPermission(treeUri, takeFlags);
        }
    }
}

Getting an outputStream for a file via the Storage Access Framework (making use of the stored URL, assuming that this is the URL of the root folder of the external SD card)

DocumentFile targetDocument = getDocumentFile(file, false);
OutputStream outStream = Application.getAppContext().
    getContentResolver().openOutputStream(targetDocument.getUri());

This uses the following helper methods:

public static DocumentFile getDocumentFile(final File file, final boolean isDirectory) {
    String baseFolder = getExtSdCardFolder(file);

    if (baseFolder == null) {
        return null;
    }

    String relativePath = null;
    try {
        String fullPath = file.getCanonicalPath();
        relativePath = fullPath.substring(baseFolder.length() + 1);
    }
    catch (IOException e) {
        return null;
    }

    Uri treeUri = PreferenceUtil.getSharedPreferenceUri(R.string.key_internal_uri_extsdcard);

    if (treeUri == null) {
        return null;
    }

    // start with root of SD card and then parse through document tree.
    DocumentFile document = DocumentFile.fromTreeUri(Application.getAppContext(), treeUri);

    String[] parts = relativePath.split("\\/");
    for (int i = 0; i < parts.length; i++) {
        DocumentFile nextDocument = document.findFile(parts[i]);

        if (nextDocument == null) {
            if ((i < parts.length - 1) || isDirectory) {
                nextDocument = document.createDirectory(parts[i]);
            }
            else {
                nextDocument = document.createFile("image", parts[i]);
            }
        }
        document = nextDocument;
    }

    return document;
}

public static String getExtSdCardFolder(final File file) {
    String[] extSdPaths = getExtSdCardPaths();
    try {
        for (int i = 0; i < extSdPaths.length; i++) {
            if (file.getCanonicalPath().startsWith(extSdPaths[i])) {
                return extSdPaths[i];
            }
        }
    }
    catch (IOException e) {
        return null;
    }
    return null;
}

/**
 * Get a list of external SD card paths. (Kitkat or higher.)
 *
 * @return A list of external SD card paths.
 */
@TargetApi(Build.VERSION_CODES.KITKAT)
private static String[] getExtSdCardPaths() {
    List<String> paths = new ArrayList<>();
    for (File file : Application.getAppContext().getExternalFilesDirs("external")) {
        if (file != null && !file.equals(Application.getAppContext().getExternalFilesDir("external"))) {
            int index = file.getAbsolutePath().lastIndexOf("/Android/data");
            if (index < 0) {
                Log.w(Application.TAG, "Unexpected external file dir: " + file.getAbsolutePath());
            }
            else {
                String path = file.getAbsolutePath().substring(0, index);
                try {
                    path = new File(path).getCanonicalPath();
                }
                catch (IOException e) {
                    // Keep non-canonical path.
                }
                paths.add(path);
            }
        }
    }
    return paths.toArray(new String[paths.size()]);
}

 /**
 * Retrieve the application context.
 *
 * @return The (statically stored) application context
 */
public static Context getAppContext() {
    return Application.mApplication.getApplicationContext();
}

Reference to the full code

https://github.com/jeisfeld/Augendiagnose/blob/master/AugendiagnoseIdea/augendiagnoseLib/src/main/java/de/jeisfeld/augendiagnoselib/fragments/SettingsFragment.java#L521

and

https://github.com/jeisfeld/Augendiagnose/blob/master/AugendiagnoseIdea/augendiagnoseLib/src/main/java/de/jeisfeld/augendiagnoselib/util/imagefile/FileUtil.java

Chrysler answered 28/5, 2015 at 18:20 Comment(26)
Seems like a very complicated project. Would you be able to make a minimized version? Something that shows only what's related to the question?Blakeslee
I added the main code parts related to the question to the post.Importance
Can you please put it in a minimized project, that only handles SD-card ? Also, do you know how can I check if all external storages that are available are also accessible, so that I won't request their permission for nothing?Blakeslee
Excellent, exactly what I was looking for! Perfect work, thank you!Bouillon
Wish I could upvote this more. I was halfway to this solution and found the Document navigation so awful that I thought I was doing it wrong. Good to have some confirmation on this. Thanks Google...for nothing.Bathsheba
Guys, I’ve asked essentiality same question about SD-Card access from C++ code and it was closed as off topic! #32852593 Basically the question is: I have a C++ library to work with large binary files. To work properly, it does need RW access to a folder. Am I right, it’s not possible to grant access to a folder via SAF, so C++ code can use fopen() for files in the folder?Cursor
This means that we need to use DocumentFile instead of File for file creation/modification? If so, then we need to modify our apps everywhere File was used with a conditional statement according to the API?Angellaangelle
Yes, for writing on external SD you cannot use the normal File approach any more. On the other hand, for older Android versions and for primary SD, File is still the most efficient approach. Therefore you should use a custom utility class for file access.Importance
@JörgEisfeld: I have an app that uses File 254 times. Can you imagine fixing that?? Android is becoming a nightmare for devs with its total lack of backward compatibility. I still didn't find any place where they explain why Google took all this stupid decisions regarding the external storage. Some claim "security", but of course is nonsense since any app can mess up the internal storage. My guess is to try to force us to use their cloud services. Thankfully, rooting solves the problems... at least for Android < 6 ....Angellaangelle
Please one question, Does the normal file method still work for non removable external storage?Crore
@Ankit: Before Android 6, it works without any problem. With Android 6, it works only if you ensure to have permission WRITE_EXTERNAL_STORAGE using the new Permissions framework.Importance
@JörgEisfeld Thank you very much, Correct me if I am wrong just putting the permission in the manifest will not do,right ? Android is becoming a nightmare ,DamnCrore
@Ankit: As far as I see, in Android 6 it is not sufficient any more to put the permission in the manifest (at least it was not working when trying it with AVD). You have to verify in runtime if you have the permission, and if you don't have, you have to request the permission. See the documentationImportance
@JörgEisfeld thanks, I just read it, about normal and dangerous permissions. Another question if you don't mind. The solution you gave above, will it work on android 4.4 ?since it also supports storage access framework ?Crore
@Ankit: Kitkat (Android 4.4) by concept does not support having write access to complete folder structures on external SD card. You would have to request access for each single file. My solution tries to abuse a bug in Kitkat that (at least on some devices) circumvents this restriction.Importance
This thing doesn't work on my Lollipop device. I got java.io.FileNotFoundException: Failed to open for writing: java.io.FileNotFoundException: Read-only file system when doing at android.content.ContentResolver.openOutputStream(ContentResolver.java:662 Even though I got theReturnedExistingDocumentFile.canWrite() true and getContentResolver().getPersistedUriPermissions().get(0).isWritePermission() is true as well.Rrhagia
OK. Magically, after I restarted my phone, it works :)Rrhagia
@JörgEisfeld why do you use a KitKat workaround throughout your code? Shouldn't SAF work for KitKat as well?Tyus
@dwbrito: In Kitkat, SAF works only on individual file level. For accessing folder structures, you need some dirty workaround.Importance
@JörgEisfeld what do you mean by accessing folder structures? Moving a complete directory, is that it? Or navigating in depth to fetch a folder?Tyus
@dwbrito: when using SAF with KitKat, you cannot get permission on folder level. You have to request write permission for each individual file. Permissions on whole folder structures are only offered with Lolliop.Importance
@JörgEisfeld I get it now. That really sucks. Can you link me to any documentation ? Couldn't find much info about it.Tyus
Great answer, very to-the-point! After getting this to work in my app (which can now write to JPEGs both on device memory and removable/SD memory) I thought I'd point out one thing which clarified things for me. Jörgs solution is to prompt the user for the root of the SD card, and use that URI as a starting point to traverse/search the SAF document tree for the requested file. This resolves a DocumentFile corresponding to the target file, which can then be used to write/delete/etc that file. My app first tries normal File IO, and use this SAF method as a fallback on exceptions (if SD file)Quibbling
However, I do wish Android had managed to make this a bit simpler... What I understand, though, and a lot of people are complaining about this complexity, is that it's just a complex solution to a complex problem. Android runs on many different hardware devices, and adding support for removable media has been difficult from the start. The Storage Access Framework was released with KitKat, and at least makes it possible to support File IO also on removable media.Quibbling
How can I use ftruncate, which needs write access, but there is none "fdtruncate" version which accepts file descriptor?Parentage
Why didn't they SIMPLY showed another permission popup to users instead, like other permissions? Let me help with the popup message.. "This app wants to read/write on the external storage (sdcard1) of this device. Do you allow? Yes/No"Inaudible
T
3

SimpleStorage helps you by simplifying Storage Access Framework across API levels. It works with scoped storage as well. For example:

val fileFromExternalStorage = DocumentFileCompat.fromSimplePath(context, basePath = "Downloads/MyMovie.mp4")

val fileFromSdCard = DocumentFileCompat.fromSimplePath(context, storageId = "9016-4EF8", basePath = "Downloads/MyMovie.mp4")

Granting SD card's URI permissions, picking files & folders are simpler with this library:

class MainActivity : AppCompatActivity() {

    private lateinit var storageHelper: SimpleStorageHelper

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)

        storageHelper = SimpleStorageHelper(this, savedInstanceState)
        storageHelper.onFolderSelected = { requestCode, folder ->
            // do stuff
        }
        storageHelper.onFileSelected = { requestCode, file ->
            // do stuff
        }

        btnRequestStorageAccess.setOnClickListener { storageHelper.requestStorageAccess() }
        btnOpenFolderPicker.setOnClickListener { storageHelper.openFolderPicker() }
        btnOpenFilePicker.setOnClickListener { storageHelper.openFilePicker() }
    }

    override fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent?) {
        super.onActivityResult(requestCode, resultCode, data)
        storageHelper.storage.onActivityResult(requestCode, resultCode, data)
    }

    override fun onSaveInstanceState(outState: Bundle) {
        storageHelper.onSaveInstanceState(outState)
        super.onSaveInstanceState(outState)
    }

    override fun onRestoreInstanceState(savedInstanceState: Bundle) {
        super.onRestoreInstanceState(savedInstanceState)
        storageHelper.onRestoreInstanceState(savedInstanceState)
    }
}
Talus answered 11/5, 2018 at 20:33 Comment(2)
Where is getExternalFile() called?Detection
@Anggrayudi H Does it helps to laod raw files , i am it has a feature to convert documentFile.toRawFile() but on using that raw file i always get java.io.FileNotFoundException: open failed: EACCES (Permission denied)Audly

© 2022 - 2024 — McMap. All rights reserved.