How to determine correct device orientation in Android N multi-window mode?

Question

From Multi-Window documentation:

Disabled features in multi-window mode

Certain features are disabled or ignored when a device is in multi-window mode, because they don’t make sense for an activity which may be sharing the device screen with other activities or apps. Such features include:

• Some System UI customization options are disabled; for example, apps cannot hide the status bar if they are not running in full-screen mode.
• The system ignores changes to the android:screenOrientation attribute.

I get that for most apps it doesn't make sense to distinct between portrait and landscape modes, however I am working on SDK which contains camera view which user can put on any activity they wish - including activity that supports multi-window mode. The problem is that camera view contains SurfaceView/TextureView which displays the camera preview and in order to display preview correctly in all activity orientations, knowledge about correct activity orientation is required so that camera preview can be correctly rotated.

The problem is that my code which calculates correct activity orientation by examining current configuration orientation (portrait or landscape) and current screen rotation. The problem is that in multi-window mode current configuration orientation does not reflect the real activity orientation. This then results with camera preview being rotated by 90 degrees because Android reports different configuration than orientation.

My current workaround is to check for requested activity orientation and use that as a basis, but there are two problems with that:

1. the requested activity orientation does not have to reflect actual activity orientation (i.e. request may still not be fulfilled)
2. the requested activity orientation can be 'behind', 'sensor', 'user', etc. which does not reveal any information about current activity orientation.
3. According to documentation, screen orientation is actually ignored in multi-window mode, so 1. and 2. just won't work

Is there any way to robustly calculate correct activity orientation even in multi-window configuration?

Here is my code that I currently use (see comments for problematic parts):

protected int calculateHostScreenOrientation() {
    int hostScreenOrientation = ActivityInfo.SCREEN_ORIENTATION_UNSPECIFIED;
    WindowManager wm = (WindowManager) getContext().getSystemService(Context.WINDOW_SERVICE);
    int rotation = getDisplayOrientation(wm);

    boolean activityInPortrait;
    if ( !isInMultiWindowMode() ) {
        activityInPortrait = (mConfigurationOrientation == Configuration.ORIENTATION_PORTRAIT);
    } else {
        // in multi-window mode configuration orientation can be landscape even if activity is actually in portrait and vice versa
        // Try determining from requested orientation (not entirely correct, because the requested orientation does not have to
        // be the same as actual orientation (when they differ, this means that OS will soon rotate activity into requested orientation)
        // Also not correct because, according to https://developer.android.com/guide/topics/ui/multi-window.html#running this orientation
        // is actually ignored.
        int requestedOrientation = getHostActivity().getRequestedOrientation();
        if ( requestedOrientation == ActivityInfo.SCREEN_ORIENTATION_PORTRAIT ||
                requestedOrientation == ActivityInfo.SCREEN_ORIENTATION_REVERSE_PORTRAIT ||
                requestedOrientation == ActivityInfo.SCREEN_ORIENTATION_SENSOR_PORTRAIT ||
                requestedOrientation == ActivityInfo.SCREEN_ORIENTATION_USER_PORTRAIT ) {
            activityInPortrait = true;
        } else if ( requestedOrientation == ActivityInfo.SCREEN_ORIENTATION_LANDSCAPE ||
                requestedOrientation == ActivityInfo.SCREEN_ORIENTATION_REVERSE_LANDSCAPE ||
                requestedOrientation == ActivityInfo.SCREEN_ORIENTATION_SENSOR_LANDSCAPE ||
                requestedOrientation == ActivityInfo.SCREEN_ORIENTATION_USER_LANDSCAPE ) {
            activityInPortrait = false;
        } else {
            // what to do when requested orientation is 'behind', 'sensor', 'user', etc. ?!?
            activityInPortrait = true; // just guess
        }
    }

    if ( activityInPortrait ) {
        Log.d(this, "Activity is in portrait");
        if (rotation == Surface.ROTATION_0) {
            Log.d(this, "Screen orientation is 0");
            hostScreenOrientation = ActivityInfo.SCREEN_ORIENTATION_PORTRAIT;
        } else if (rotation == Surface.ROTATION_180) {
            Log.d(this, "Screen orientation is 180");
            hostScreenOrientation = ActivityInfo.SCREEN_ORIENTATION_REVERSE_PORTRAIT;
        } else if (rotation == Surface.ROTATION_270) {
            Log.d(this, "Screen orientation is 270");
            // natural display rotation is landscape (tablet)
            hostScreenOrientation = ActivityInfo.SCREEN_ORIENTATION_PORTRAIT;
        } else {
            Log.d(this, "Screen orientation is 90");
            // natural display rotation is landscape (tablet)
            hostScreenOrientation = ActivityInfo.SCREEN_ORIENTATION_REVERSE_PORTRAIT;
        }
    } else {
        Log.d(this, "Activity is in landscape");
        if (rotation == Surface.ROTATION_90) {
            Log.d(this, "Screen orientation is 90");
            hostScreenOrientation = ActivityInfo.SCREEN_ORIENTATION_LANDSCAPE;
        } else if (rotation == Surface.ROTATION_270) {
            Log.d(this, "Screen orientation is 270");
            hostScreenOrientation = ActivityInfo.SCREEN_ORIENTATION_REVERSE_LANDSCAPE;
        } else if (rotation == Surface.ROTATION_0) {
            Log.d(this, "Screen orientation is 0");
            // natural display rotation is landscape (tablet)
            hostScreenOrientation = ActivityInfo.SCREEN_ORIENTATION_LANDSCAPE;
        } else {
            Log.d(this, "Screen orientation is 180");
            // natural display rotation is landscape (tablet)
            hostScreenOrientation = ActivityInfo.SCREEN_ORIENTATION_REVERSE_LANDSCAPE;
        }
    }
    return hostScreenOrientation;
}

private int getDisplayOrientation(WindowManager wm) {
    if (DeviceManager.getSdkVersion() < 8) {
        return wm.getDefaultDisplay().getOrientation();
    }

    return wm.getDefaultDisplay().getRotation();
}

private boolean isInMultiWindowMode() {
    return Build.VERSION.SDK_INT >= 24 && getHostActivity().isInMultiWindowMode();
}

protected Activity getHostActivity() {
    Context context = getContext();
    while (context instanceof ContextWrapper) {
        if (context instanceof Activity) {
            return (Activity) context;
        }
        context = ((ContextWrapper) context).getBaseContext();
    }
    return null;
}

EDIT: I've reported this also to Android issue tracker.

Darmit

Note that isInMultiWindowMode() suffers from a race condition, adding to the problem. – Divulge

Just for sake of completeness, I am adding here issue report in our SDK which also contains screenshots and some discussion on topic. – Darmit

Could you just look at screen dimensions? getMetrics(): "If requested from non-Activity context metrics will report the size of the entire display based on current rotation and with subtracted system decoration areas.”. Then compare width and height. – Slater

@natario, I'll try that - but I am not sure if I can conclude that if width is larger than height that activity is in definitely landscape. I'll first have to check what this method returns in multi-window mode. Thanks anyway! – Darmit

@Darmit not the activity, the phone. The activity lives in its window and you know it can now be landscape or portrait depending on the drag position. By looking at the phone dimensions, though, you should be able to tell whether it’s in landscape or portrait at the moment, and this does not depend on the drag position. If it helps in your case. – Slater

@natario, your solution works. Thank you very much. Please provide the solution as the answer (not as comment), so I will accept it and you will be able to collect reputation bounty from CommonsWare. – Darmit

Answer 1

I don’t know if this should be considered a solution or just a workaround.

As you say, your problems come with Android N and its multi-window mode. When the app is in multi window, your Activity is not tied to the full display dimensions. This redefines the concept of Activity orientation. Quoting Ian Lake:

Turns out: “portrait” really just means the height is greater than the width and “landscape” means the width is greater than the height. So it certainly makes sense, with that definition in mind, that your app could transition from one to the other while being resized.

So there is no link anymore between Activity orientation changing and device physically being rotated. (I think the only reasonable use of Activity orientation changes now is to update your resources.)

Since you are interested in device dimensions, just get its DisplayMetrics. Quoting docs,

If requested from non-Activity context metrics will report the size of the entire display based on current rotation and with subtracted system decoration areas.

So the solution is:

final Context app = context.getApplicationContext();
WindowManager manager = (WindowManager) app.getSystemService(Context.WINDOW_SERVICE);
Display display = manager.getDefaultDisplay();
DisplayMetrics metrics = new DisplayMetrics();
display.getMetrics(metrics);
int width = metrics.widthPixels;
int height = metrics.heightPixels;
boolean portrait = height >= width;

Width and height values will be swapped (more or less) when the device is tilted.

If this works, I would personally run it every time, deleting the isInMultiWindowMode() branch, because

• it’s not expensive
• our assumptions stand also in the non-multi-window mode
• it will presumably work well with any other future kinds of modes
• you avoid the race condition of isInMultiWindowMode() described by CommonsWare

Slater

Thanks! As you said, this works every time - both in multi-windows and normal mode, so there is no more need for depending on activity configuration at all. Btw. I call it a solution, as it solves my problem. – Darmit

That is totally wrong for tablets that have a default orientation of landscape (i.e. Samsung Tab SM-P600). You simply cannot rely on widthPixels and heightPixels giving the accurate value according to current orientation. – Leadsman

@Igor that width and height should be device-agnostic, that is, the default orientation should not matter. I didn't, but I'm sure DoDo did test this. – Slater

@Slater Look at your code: boolean portrait = height >= width; That matters. – Leadsman

@Darmit might answer to you. I think the Metrics width and height do not depend on the default orientation. – Slater

@IgorGanapolsky, this solution works correctly on tablets too. I've tested on Sony Xperia Z tablet, Nexus 10 and Tesco/Pegatron Hudl. The trick is in obtaining application context, not the current context - when window manager is obtained via it, default display metrics return widht/height of the entire screen in current orientation, regardless of the current activity real size. – Darmit

@Darmit Some Android devices have NATURAL system orientation of landscape, NOT portrait. See this: android-developers.googleblog.com/2010/09/…. Read the quote: recently, a few devices have shipped that run Android on screens that are naturally landscape in their orientation. Therefore, the width/height on these devices will be reported in reverse of your solution above. – Leadsman

@Igor here height means "screen dimension better aligned with gravity" and width is its counterpart. I suspect your question is not exactly the OPs one. Read the docs, getMetrics reports these values based on current rotation, and that takes into account the screen natural orientation (or its semantics are deeply flawed). – Slater

@Slater OP's question is crystal clear: How to determine correct activity orientation. Did I misunderstand something? – Leadsman

@Igor yes, the title is wrong. OP was asking how to get the device orientation, regardless of the activity one (since they do not necessarily agree now). – Slater

@IgorGanapolsky, the accepted answer's code works correctly also on devices with natural system orientation of landscape, such as Nexus 10 and Sony Xperia Z tablet (I've tested that). The width/height on these devices are reported correctly. Being landscape just means that reported width is larger than reported height, no matter of what is natural. Do you have an example on which device this solution does not work? – Darmit

@Darmit It doesn't report as expected on Samsung Tab 10.1 SMP600. When holding that device in landscape (user perceives physical horizontal width > vertical height), the code snippet above would calculate boolean portrait = true – Leadsman

@IgorGanapolsky, what are the values of width and height on your device. What is the actual orientation of the screen at the moment of observing the values? Generally, if height is larger than width, device is in portrait orientation, no matter what its natural orientation is. – Darmit

Answer 2

I thought you could utilise the accelerometer to detect where's "down" - and thus the orientation of the phone. The Engineer Guy explains that that's the way the phone itself does it.

I searched here on SO for a way to do that and found this answer. Basically you need to check which of the 3 accelerometers detect the most significant component of the gravitational pull, which you know is roughly 9.8m/s² near the ground of the earth. Here's the code snippet from it:

private boolean isLandscape;

mSensorManager = (SensorManager)getSystemService(SENSOR_SERVICE);
mSensorManager.registerListener(mSensorListener,     mSensorManager.getDefaultSensor(
                                      Sensor.TYPE_ACCELEROMETER),1000000);
private final SensorEventListener mSensorListener = new SensorEventListener() { 
    @Override 
    public void onSensorChanged(SensorEvent mSensorEvent) {   
        float X_Axis = mSensorEvent.values[0]; 
        float Y_Axis = mSensorEvent.values[1]; 

        if((X_Axis <= 6 && X_Axis >= -6) && Y_Axis > 5){
        isLandscape = false; 
        }
        else if(X_Axis >= 6 || X_Axis <= -6){
        isLandscape = true;
        }

    }

    public void onAccuracyChanged(Sensor sensor, int accuracy) {
    }
};  

Be careful as this code might not work in every situation, as you need to take into account scenarios like being on a stopping/accelerating train or moving the phone fast in a game - here's the orders of magnitude page on wiki to get you started. It looks like you're safe with the values Khalil put in his code (in his answer), but I would take extra caution and research into what values might be generated in the different scenarios.

It's not a flawless idea, but I think as long as the API is built the way it is - whithout allowing you to get their calculated orientation - I think it's a beneficial workaround.

Frame

Nice idea, but for me it is not a problem to calculate what is "down" (I get that indirectly with getDisplayOrientation) - the problem is to detect "what transformation has activity made to my view so I can counter it". The problem is that when activity is rotated, it also rotates all its views, including the view that displays camera preview. Therefore, I need to know exact rotation Android has made to counter it by local rotation inside the camera preview view itself. – Darmit

To better display the problem - imagine that sensor information from your code correctly concludes that device is in landscape because user actually holds it in landscape. However, the user has disabled screen rotation in settings and multi-window activities are actually in portrait. To display camera preview as in landscape is then wrong, as it would create incorrect rotation of camera preview. That is precisely the problem introduced with multi-window feature. – Darmit

Yeah I see what you mean. You're right, my solution will not help. – Frame

This code makes absolutely no sense: if((X_Axis <= 6 && X_Axis >= -6) – Leadsman

Why shouldn't it make sense? You can write it in another form like this: if((Math.abs(X_Axis)<=6). – Frame