How does AccessibilityService draw on top of other apps?

Question

Background

SAW (system alert window) permission can be used to draw content on top of other apps.

I was told a very long time ago that accessibility service can do this too, but I never found any tutorial, sample, documentation and even an app that does it... until recently:

https://play.google.com/store/apps/details?id=com.github.ericytsang.screenfilter.app.android

In fact, this app seems to be able to draw everywhere, as opposed to SAW permission. It draws even on top of the settings app and system dialogs, while SAW permission isn't allowed as such.

The problem

Sadly, as accessibility is quite a unique and rarely thing to use, just as I wrote, I couldn't find how such a thing works with drawing on top of other apps.

What I've found

The only thing I know is that this app somehow does it, and this is what it shows when it asks to grant it:

But that's not enough. I know that for some old POC I've made of using accessibility service, it showed the same, and checking it out, I can't see what triggers it. Pretty sure it's the minimal thing the users will see for any kind of accessibility service, so this won't help.

The questions

1. How does AccessibilityService draw on top of other apps?

2. Does it work the same as SAW permission ? Can you, for example, handle touch events on what it draws?

3. What are the restrictions of using it, if there are any ?

Eyecup

What does this have to do with maven dependencies? – Eskimo

@Eskimo Why do you think it has anything to do with maven depenendencies? Why the downvote? I didn't mention maven anywhere here. – Eyecup

Your title is How come maven dependencies can't be used? Also I didnt downvote – Eskimo

@Eskimo Oh sorry for that. Fixed now. The previous post's title I made (here: #66984261 ) somehow got into here and I didn't notice. Please don't downvote anymore :( – Eyecup

So the codelab for Accessibility Service: codelabs.developers.google.com/codelabs/… and where the text calls out WindowManager.LayoutParams.TYPE_ACCESSIBILITY_OVERLAY wasn't found in your search? – Kayser

@MorrisonChang Thank you. Updated the answer that mentioned it. Next time please create a new answer instead, so that I could grant the bounty. – Eyecup

I felt the question was a bit unfocused but still valuable. Also see: How much research effort is expected of Stack Overflow users? The codelab is referenced at the bottom of the Android Developer - Create your own accessibility service page – Kayser

@MorrisonChang Sorry. Sometimes I fail to find the information. :( – Eyecup

Answer 1

Using TYPE_ACCESSIBILITY_OVERLAY as type in the WindowManager.LayoutParams when adding the view from within the accessibility service seems to do the trick. I did a quick test and the overlay window was shown even in the settings menu. The overlay window also received touch events. This worked also without the SYSTEM_ALERT_WINDOW permission in the manifest and also without setting the "Display over other apps" permission interactively by the user. I did my testing using target SDK 29.

Sorry, I cannot answer to your third question about what specific restrictions apply.

---

EDIT: By looking at the old tutorial of Google here, here's a short sample:

GlobalActionBarService.java

public class GlobalActionBarService extends AccessibilityService {
    FrameLayout mLayout;

    @Override
    protected void onServiceConnected() {
        // Create an overlay and display the action bar
        WindowManager wm = (WindowManager) getSystemService(WINDOW_SERVICE);
        mLayout = new FrameLayout(this);
        WindowManager.LayoutParams lp = new WindowManager.LayoutParams();
        lp.type = WindowManager.LayoutParams.TYPE_ACCESSIBILITY_OVERLAY;
        lp.format = PixelFormat.TRANSLUCENT;
        lp.flags |= WindowManager.LayoutParams.FLAG_NOT_FOCUSABLE;
        lp.width = WindowManager.LayoutParams.WRAP_CONTENT;
        lp.height = WindowManager.LayoutParams.WRAP_CONTENT;
        lp.gravity = Gravity.TOP;
        LayoutInflater inflater = LayoutInflater.from(this);
        inflater.inflate(R.layout.action_bar, mLayout);
        wm.addView(mLayout, lp);
    }

    @Override
    public void onAccessibilityEvent(AccessibilityEvent event) {
    }

    @Override
    public void onInterrupt() {
    }
}

manifest

<manifest xmlns:android="http://schemas.android.com/apk/res/android" xmlns:tools="http://schemas.android.com/tools"
    package="com.lb.myapplication">

    <application
        android:allowBackup="true" android:icon="@mipmap/ic_launcher" android:label="@string/app_name"
        android:roundIcon="@mipmap/ic_launcher_round" android:supportsRtl="true"
        android:theme="@style/Theme.MyApplication" tools:ignore="AllowBackup">
        <activity
            android:name=".MainActivity" android:exported="true">
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />

                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>

        <service
            android:name=".GlobalActionBarService" android:exported="false"
            android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE">
            <intent-filter>
                <action android:name="android.accessibilityservice.AccessibilityService" />
            </intent-filter>
            <meta-data
                android:name="android.accessibilityservice" android:resource="@xml/global_action_bar_service" />
        </service>
    </application>

</manifest>

global_action_bar_service.xml

<accessibility-service xmlns:android="http://schemas.android.com/apk/res/android"
    android:accessibilityFeedbackType="feedbackGeneric" android:accessibilityFlags="flagDefault"
    android:canPerformGestures="true" android:canRetrieveWindowContent="true" />

action_bar.xml

<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android" android:layout_width="match_parent"
    android:layout_height="wrap_content" android:orientation="horizontal">

    <Button
        android:id="@+id/power" android:layout_width="wrap_content" android:layout_height="wrap_content"
        android:text="@string/power" />

    <Button
        android:id="@+id/volume_up" android:layout_width="wrap_content" android:layout_height="wrap_content"
        android:text="@string/volume" />

    <Button
        android:id="@+id/scroll" android:layout_width="wrap_content" android:layout_height="wrap_content"
        android:text="@string/scroll" />

    <Button
        android:id="@+id/swipe" android:layout_width="wrap_content" android:layout_height="wrap_content"
        android:text="@string/swipe" />
</LinearLayout>

Foreandafter

How do you use it? Can you please share some minimal code for it? Maybe even Github sample? Or is it exactly the same as using SAW? – Eyecup

OK I've found it and edited your answer to include some sample. – Eyecup

If you can, please explain even in what I've edited, the answers to the questions and what is the relevant part there. – Eyecup

@androiddeveloper Nice find! In the meantime I uploaded a small example to github. It isn't really that much different from the example you found. The crucial part is to add the view with type TYPE_ACCESSIBILITY_OVERLAY. If your questions are really about how this all works internally then I cannot answer them. – Foreandafter

I'm wondering which part of the "accessibility-service" tag is enough for the SAW-like ability. BTW your link doesn't work. – Eyecup

@androiddeveloper Sorry, I didn't realize the github repository was still private, should work now. I'm confused about the intention of your questions. I thought you were looking for a way to display content above all other apps including the system settings UI. The only way I know about to achieve this is in the context of an accessibility service as described above. Regarding the minimal set of code you would need for your specific scenario I suggest experimenting with the examples. E.g. you can strip the main activity and "accessibilityservice" meta data and still get the overlay working. – Foreandafter

I can't import it now. It seems like it's missing a lot of important files for Android Studio to be opened. – Eyecup

@androiddeveloper Indeed the code on github doesn't include any project configuration files etc. It does however include the complete src/main tree. You can take the code as an example and transfer it to your own project as needed. If you find issues with the code I suggest to discuss them on github. – Foreandafter

@androiddeveloper Hi, i copied your code it works fine but i can't remove WindowManager view, i try wm.removeView(mLayout); it doesn't work, there are still buttons on the screen. Can you help me? Thank you so much. – Schick

Answer 2

The answer is: You must use WindowManager which puts views to a window in the service to draw it on top of other apps.

    val typeApplication =
        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O)
            WindowManager.LayoutParams.TYPE_APPLICATION_OVERLAY
        else
            WindowManager.LayoutParams.TYPE_PHONE

    val layoutParams = WindowManager.LayoutParams(
        windowWidth,
        ViewGroup.LayoutParams.WRAP_CONTENT,
        typeApplication,
        WindowManager.LayoutParams.FLAG_NOT_FOCUSABLE,
        PixelFormat.TRANSLUCENT
    )
    // inflater view with above layoutParams variable

And the application granted the Overlay permission and make sure accessibility is enabled in the device setting (Setting-> Accessibility -> Enable your app). Or use this intent to go to it

Intent(Settings.ACTION_ACCESSIBILITY_SETTINGS)

. You can check the Overlay permission by the following code

  // check
  Settings.canDrawOverlays(applicationContext)
  // Use this intent to enable permision
  Intent(Settings.ACTION_MANAGE_OVERLAY_PERMISSION,
         Uri.parse("package:$packageName"))

Does it work the same as SAW permission? Can you, for example, handle touch events on what it draws? I've never used SAW before so I'm not sure about this question.

Hope can help you!

Osset

Other people here wrote I should use TYPE_ACCESSIBILITY_OVERLAY instead though. I don't see it being mentioned in your code. Have you tested what you wrote? – Eyecup

Sure, the code are using in my project. You can see it here play.google.com/store/apps/details?id=com.tailt.new_base – Osset

Seems like a cool app, but it has 3 issues: 1. It does have SAW permission, at least declared as such. 2. I can't make it pass granting the accessibility permission. It gets stuck on it, always asking for it. I think it's a serious bug. 3. Still prefer source code. – Eyecup