AutomationElement.FindAll() performance issue

Question

We have been working with the MS UIA framework and noticed what appears to be a significant slowdown when finding collections of objects in Windows 10 / .NET 4.6.

When testing AutomationElement.FindAll() on a Windows 10 / .NET 4.6 box our times on average are roughly 3 to 5 times longer to find collections of elements than when finding the exact same elements on a Windows 8.1 / .NET 4.5.1 box. My test is against a WPF DataGrid with virtualization on (using Recycling) and getting all of the cells inside each row of the DataGrid.

On our Win10 box each FindAll call to get the cells in each row takes roughly 30 - 50ms or even longer. On the Win8.1 box it takes around 5 - 10ms. I am unable to figure out why, but I do not think the issue is limited to the DataGrid since there is nothing fancy about our FindAll() call.

        //get grid
    AutomationElement gridElement = AutomationElement.RootElement.FindFirst(TreeScope.Descendants,
           new PropertyCondition(AutomationElement.AutomationIdProperty, "dataGridAutomationId"));

    //get all visible rows
    AutomationElementCollection dataItems = gridElement.FindAll(TreeScope.Descendants,
                       new PropertyCondition(AutomationElement.ControlTypeProperty, ControlType.DataItem));

 foreach (AutomationElement dataItem in dataItems)
 {
    if (!string.IsNullOrEmpty(dataItem.Current.Name))
    {
        //call under test
        AutomationElementCollection cells = dataItem.FindAll(TreeScope.Children,
             new PropertyCondition(AutomationElement.ClassNameProperty, "DataGridCell"));
    }
 }

The use of AutomationElement.RootElement is for the test only. The 'dataItem.FindAll()' call is what I am testing.

Win 8.1 and Win10 machine specs:

• Xeon W3670 3.20GHz cpu
• 12gb ram
• 64bit OS

We have tried using the unmanaged MS uia API via com wrapping and saw no noticeable performance improvements on Win10.

Any advice would be much appreciated.

Raver

have you compared the structure of the datagrid between win8.1 and win10 already?(e.g. with inspect.exe) i wonder if win10 might use a different structure for it? not sure if this is even possible – Potter

According to inspect the datagrid structure is identical. – Raver

Answer 1

This appears to have been fixed in the latest round of Windows 10 updates (kb/3093266). According to an MS support representative I spoke with:

"UIA was frequently calling the NtQuerySystemInformation, the performance of calling that API frequently is not satisfactory. They made changes to that particular code path and do not call that API anymore and that improved the overall performance."

Unfortunately that was all the information they had so I am unable to determine exactly what about that call was causing the issue.

After updating and testing, performance is the same across both machines.

Raver

Answer 2

I'd suggest trying a different method than FindAll(), such as using TreeWalker. UIAutomation implementations tend to change (at least that's what I felt like) between different OS versions (they're compiled to UIAutomationCore.dll if you want to check the versions). Note that you can check the managed code of the latest .NET framework. Example link for TreeWalker: http://referencesource.microsoft.com/#UIAutomationClient/System/Windows/Automation/TreeWalker.cs

For example to check the immediate children with TreeWalker, you can use:

var walker = TreeWalker.RawViewWalker;
var current = walker.GetFirstChild(/* a parent automation element here*/);
while (current != null)
{
  // use current (an automationelement) here, than go to next:
  current = walker.GetNextSibling(current);
}

Hinojosa

Thanks for the suggestion. I've tried this approach as well and get the same problem with the Win10 box being vastly slower than the Win8.1 box (by about the same amount as when using FindAll()) – Raver

Answer 3

This might not be related to your particular situation, but sometimes there can be a difference in performance characteristics between the managed .NET UIA API and the native Windows UIA API. So if practical, you might want to consider seeing if you get better perf results with the UI that you're interacting with, by using the Windows UIA API.

As a test, I just created an app that presented a DataGrid with 25 rows in it and 10 cells in each row. I then wrote the UIA client code below to access the name of each cell, as exposed through UIA. (Some notes on how I have my C# code call the native Windows UIA API are at http://blogs.msdn.com/b/winuiautomation/archive/2015/09/30/so-how-will-you-help-people-work-with-text-part-2-the-uia-client.aspx.)

I think the really interesting thing about the test code, is that once I have the element which is the parent of the DataItems, I can access all the data I need with a single cross-process call. Given that cross-process calls are slow, I want to make as few of them as possible.

Thanks,

Guy

IUIAutomationElement rootElement = uiAutomation.GetRootElement();

// The first few steps below find a DataGridRowsPresenter for the 
// DataGrid we're interested in.
IUIAutomationElement dataGridRowsPresenter = null;

// We'll be setting up various UIA conditions and cache requests below.
int propertyIdControlType = 30003; // UIA_ControlTypePropertyId
int propertyIdName = 30005; // UIA_NamePropertyId
int propertyIdAutomationId = 30011; // UIA_AutomationIdPropertyId
int propertyIdClassName = 30012; // UIA_ClassNamePropertyId
int controlTypeIdDataItem = 50029; // UIA_DataItemControlTypeId

// Look for the test app presenting the DataGrid. For this test, assume there's
// only one such UIA element that'll be found, and the current language doesn't
// effect any of the searches below.
string testAppName = "Window1";

IUIAutomationCondition conditionTestAppName =
    uiAutomation.CreatePropertyCondition(
        propertyIdName, testAppName);

IUIAutomationElement testAppElement =
    rootElement.FindFirst(
        TreeScope.TreeScope_Children,
        conditionTestAppName);

// Did we find the test app?
if (testAppElement != null)
{
    // Next find the DataGrid. By looking at the UI with the Inspect SDK tool first,
    // we can know exactly how the UIA hierarchy and properties are being exposed.
    string dataGridAutomationId = "DataGrid_Standard";

    IUIAutomationCondition conditionDataGridClassName =
        uiAutomation.CreatePropertyCondition(
            propertyIdAutomationId, dataGridAutomationId);

    IUIAutomationElement dataGridElement =
        testAppElement.FindFirst(
            TreeScope.TreeScope_Children,
            conditionDataGridClassName);

    // Did we find the DataGrid?
    if (dataGridElement != null)
    {
        // We could simply look for all DataItems that are descendents of the DataGrid.
        // But we know exactly where the DataItems are, so get the element that's the 
        // parent of the DataItems. This means we can then get that element's children,
        // and not ask UIA to search the whole descendent tree.
        string dataGridRowsPresenterAutomationId = "PART_RowsPresenter";

        IUIAutomationCondition conditionDataGridRowsPresenter =
            uiAutomation.CreatePropertyCondition(
                propertyIdAutomationId, dataGridRowsPresenterAutomationId);

        dataGridRowsPresenter =
            dataGridElement.FindFirst(
                TreeScope.TreeScope_Children,
                conditionDataGridRowsPresenter);
    }
}

// Ok, did we find the element that's the parent of the DataItems?
if (dataGridRowsPresenter != null)
{
    // Making cross-proc calls is slow, so try to reduce the number of cross-proc calls we 
    // make. In this test, we can find all the data we need in a single cross-proc call below.

    // Create a condition to find elements whose control type is DataItem.
    IUIAutomationCondition conditionRowsControlType =
        uiAutomation.CreatePropertyCondition(
            propertyIdControlType, controlTypeIdDataItem);

    // Now say that all elements returned from the search should have their Names and
    // ClassNames cached with them. This means that when we access the Name and ClassName
    // properties later, we won't be making any cross-proc call at that time.
    IUIAutomationCacheRequest cacheRequestDataItemName = uiAutomation.CreateCacheRequest();
    cacheRequestDataItemName.AddProperty(propertyIdName);
    cacheRequestDataItemName.AddProperty(propertyIdClassName);

    // Say that we also want data from the children of the elements found to be cached 
    // beneath the call to find the DataItem elements. This means we can access the Names 
    // and ClassNames of all the DataItems' children, without making more cross-proc calls.
    cacheRequestDataItemName.TreeScope =
        TreeScope.TreeScope_Element | TreeScope.TreeScope_Children;

    // For this test, say that we don't need a live reference to the DataItems after we've 
    // done the search. This is ok here, because the cached data is all we need. It means
    // that we can't later get current data (ie not cached) from the DataItems returned.
    cacheRequestDataItemName.AutomationElementMode =
        AutomationElementMode.AutomationElementMode_None;

    // Now get all the data we need, in a single cross-proc call.
    IUIAutomationElementArray dataItems = dataGridRowsPresenter.FindAllBuildCache(
        TreeScope.TreeScope_Children,
        conditionRowsControlType,
        cacheRequestDataItemName);

    if (dataItems != null)
    {
        // For each DataItem found...
        for (int idxDataItem = 0; idxDataItem < dataItems.Length; idxDataItem++)
        {
            IUIAutomationElement dataItem = dataItems.GetElement(idxDataItem);

            // This test is only interested in DataItems with a Name.
            string dataItemName = dataItem.CachedName;
            if (!string.IsNullOrEmpty(dataItemName))
            {
                // Get all the direct children of the DataItem, that were cached 
                // during the search.
                IUIAutomationElementArray elementArrayChildren = 
                    dataItem.GetCachedChildren();
                if (elementArrayChildren != null)
                {
                    int cChildren = elementArrayChildren.Length;

                    // For each child of the DataItem...
                    for (int idxChild = 0; idxChild < cChildren; ++idxChild)
                    {
                        IUIAutomationElement elementChild =
                            elementArrayChildren.GetElement(idxChild);
                        if (elementChild != null)
                        {
                            // This test is only interested in the cells.
                            if (elementChild.CachedClassName == "DataGridCell")
                            {
                                string cellName = elementChild.CachedName;

                                // Do something useful with the cell name now...
                            }
                        }
                    }
                }
            }
        }
    }
}

Preheat

We have tried using the unmanaged api via com wrapping and saw no difference in the performance. I'm not sure this answers my question as it doesn't explain why there is a difference in performance between Win8.1 and Win10. – Raver

As an aside, is there a reason MS is pushing the native uia api instead of the managed one? – Raver

Answer 4

TreeScope other than immediate children should be avoided in general. Not only it can kill performance, but it may simply never end (depending on what lies beneath...).

I suggest you refine your search using other discriminants (ControlType, Name, etc.), or use it only if you're 100% sure the subtree is really small or bounded.

BTW, that's often the reason why most automatic recorder tools fall short with UI automation, they're not clever enough to determine good AutomationProperty criteria that only humans can see, in the context of what they want to automate.

Sunk

that doesn't explain the differnce in performance between win8.1 and win10, does it? – Potter

Are the automation trees you use exactly the same for both systems? – Sunk

According to Inspect the trees are the same for the datagrid on both systems. – Raver