WPF "Lazy" VisualBrush
Asked Answered
W

2

7

I'm trying to implement somesting like an "Lazy" VisualBrush right now. Does anybody have an idea how to to that? Meaning: Something that behaves like an VisualBrush but does not update on every change in the Visual but at max once a second (or whatever).

I better should give some background why I'm doing this and what I alreay tried I guess :)

Problem: My job right now is to improve the performance of an rather big WPF application. I tracked down the main performance issue (at the UI level anyway) to some visual brushes used in the application. The application consists of an "Desktop" area with some rather complex UserControls and an Navigation area containing a scaled down version of the Desktop. The navigation area is using visual brushes to get the job done. Everything is fine as long as the Desktop items are more or less static. But if the elements are changing frequently (because they contain an animation for example) the VisualBrushes go wild. They will update along with the framerate of the animations. Lowering the framerate helps of course, but I'm looking for an more general solution to this problem. While the "source" control only renders the small area affected by the animation the visual brush container is rendered completly causing the application performance to go to hell. I already tried to use BitmapCacheBrush instead. Doesn't help unfortunately. The animation is inside the control. So the brush have to be refreshed anyway.

Possible solution: I created a Control behaving more or less like an VisualBrush. It takes some visual (as the VisualBrush) but is using a DiapatcherTimer and RenderTargetBitmap to do the job. Right now I'm subscribing to the LayoutUpdated event of the control and whenever it changes it will be scheduled for "rendering" (using RenderTargetBitmap). The actual rendering then is triggered by the DispatcherTimer. This way the control will repaint itself at maximum in the frequency of the DispatcherTimer.

Here is the code:

public sealed class VisualCopy : Border
{
    #region private fields

    private const int mc_mMaxRenderRate = 500;
    private static DispatcherTimer ms_mTimer;
    private static readonly Queue<VisualCopy> ms_renderingQueue = new Queue<VisualCopy>();
    private static readonly object ms_mQueueLock = new object();

    private VisualBrush m_brush;
    private DrawingVisual m_visual;
    private Rect m_rect;
    private bool m_isDirty;
    private readonly Image m_content = new Image();
    #endregion

    #region constructor
    public VisualCopy()
    {
        m_content.Stretch = Stretch.Fill;
        Child = m_content;
    }
    #endregion

    #region dependency properties

    public FrameworkElement Visual
    {
        get { return (FrameworkElement)GetValue(VisualProperty); }
        set { SetValue(VisualProperty, value); }
    }

    // Using a DependencyProperty as the backing store for Visual.  This enables animation, styling, binding, etc...
    public static readonly DependencyProperty VisualProperty =
        DependencyProperty.Register("Visual", typeof(FrameworkElement), typeof(VisualCopy), new UIPropertyMetadata(null, OnVisualChanged));

    #endregion

    #region callbacks

    private static void OnVisualChanged(DependencyObject obj, DependencyPropertyChangedEventArgs args)
    {
        var copy = obj as VisualCopy;
        if (copy != null)
        {
            var oldElement = args.OldValue as FrameworkElement;
            var newelement = args.NewValue as FrameworkElement;
            if (oldElement != null)
            {
                copy.UnhookVisual(oldElement);
            }
            if (newelement != null)
            {
                copy.HookupVisual(newelement);
            }
        }
    }

    private void OnVisualLayoutUpdated(object sender, EventArgs e)
    {
        if (!m_isDirty)
        {
            m_isDirty = true;
            EnqueuInPipeline(this);
        }
    }

    private void OnVisualSizeChanged(object sender, SizeChangedEventArgs e)
    {
        DeleteBuffer();
        PrepareBuffer();
    }

    private static void OnTimer(object sender, EventArgs e)
    {
        lock (ms_mQueueLock)
        {
            try
            {
                if (ms_renderingQueue.Count > 0)
                {
                    var toRender = ms_renderingQueue.Dequeue();
                    toRender.UpdateBuffer();
                    toRender.m_isDirty = false;
                }
                else
                {
                    DestroyTimer();
                }
            }
            catch (Exception ex)
            {
            }
        }
    }
    #endregion

    #region private methods
    private void HookupVisual(FrameworkElement visual)
    {
        visual.LayoutUpdated += OnVisualLayoutUpdated;
        visual.SizeChanged += OnVisualSizeChanged;
        PrepareBuffer();
    }

    private void UnhookVisual(FrameworkElement visual)
    {
        visual.LayoutUpdated -= OnVisualLayoutUpdated;
        visual.SizeChanged -= OnVisualSizeChanged;
        DeleteBuffer();
    }


    private static void EnqueuInPipeline(VisualCopy toRender)
    {
        lock (ms_mQueueLock)
        {
            ms_renderingQueue.Enqueue(toRender);
            if (ms_mTimer == null)
            {
                CreateTimer();
            }
        }
    }

    private static void CreateTimer()
    {
        if (ms_mTimer != null)
        {
            DestroyTimer();
        }
        ms_mTimer = new DispatcherTimer { Interval = TimeSpan.FromMilliseconds(mc_mMaxRenderRate) };
        ms_mTimer.Tick += OnTimer;
        ms_mTimer.Start();
    }

    private static void DestroyTimer()
    {
        if (ms_mTimer != null)
        {
            ms_mTimer.Tick -= OnTimer;
            ms_mTimer.Stop();
            ms_mTimer = null;
        }
    }

    private RenderTargetBitmap m_targetBitmap;
    private void PrepareBuffer()
    {
        if (Visual.ActualWidth > 0 && Visual.ActualHeight > 0)
        {
            const double topLeft = 0;
            const double topRight = 0;
            var width = (int)Visual.ActualWidth;
            var height = (int)Visual.ActualHeight;
            m_brush = new VisualBrush(Visual);
            m_visual = new DrawingVisual();
            m_rect = new Rect(topLeft, topRight, width, height);
            m_targetBitmap = new RenderTargetBitmap((int)m_rect.Width, (int)m_rect.Height, 96, 96, PixelFormats.Pbgra32);
            m_content.Source = m_targetBitmap;
        }
    }

    private void DeleteBuffer()
    {
        if (m_brush != null)
        {
            m_brush.Visual = null;
        }
        m_brush = null;
        m_visual = null;
        m_targetBitmap = null;
    }

    private void UpdateBuffer()
    {
        if (m_brush != null)
        {
            var dc = m_visual.RenderOpen();
            dc.DrawRectangle(m_brush, null, m_rect);
            dc.Close();
            m_targetBitmap.Render(m_visual);
        }
    }

    #endregion
}

This works pretty good so far. Only problem is the trigger. When I use LayoutUpdated then the Rendering is triggered constantly even if the Visual itself is not changed at all (propably because of animations in other parts of the application or whatever). LayoutUpdated is just fired way to often. As a matter of fact I could just skip the trigger and just update the control using the timer without any trigger. It doesn't matter. I also tried to override OnRender in the Visual and raise an custom event to trigger the update. Doesn't work either because OnRender is not called when something deep inside the VisualTree changes. This is my best shot right now. It's working much better then the original VisualBrush solution already (from the performance point of view at least). But I'm, still looking for an even better solution.

Does anyone have an idea how to a) trigger the update only when nessasarry or b) get the job done with an altogether differen approach?

Thanks!!!

Whity answered 23/4, 2011 at 13:19 Comment(13)
Could you explain why you need the VisualBrush at all? If it is degrading performance it might be worth looking for a way to get rid of (instances of) the VisualBrush. I am afraid that with your current solution you are slowly evolving into an even more expensive Brush because tracking possible changes might be more costly than a standard VisualBrush. (costly in development/maintanance and in the end runtime too)Truncation
@Erno he describes the use of that VisualBrush in detail in his question. The problem he states is not so uncommon. Think of some "Overview Map" that displays a map larger than a display with a rectangle to choose the viewport that is currently shown. It is a good question and his solution is a good one, now it just needs some tweaking and this is a nice reference for the future. Think of the possibilities where you could just have a VisualBrush/VisualCopy Element where through a property you could just turn on/off immediate/async updating.Triolein
Absolutley correct! Unfortunately I can not get rid of it. It is an integral part of the application. The only other option would be to create an addional view (or control) instance in the overview area. But I#m pretty sure that would be overkill.Whity
Acually the solution would be pretty simple. Just take the VisualBrush implementation and add some code to decouple the trigger from the actual rendering. Unfortunately the way ViualBruh is implemented does not allow me to do that. If you look at the implementation here all the relevant stuff is internal. The Key to do it would be the MediaContext class. Since this is internal there is no way to hook into the rendering :(Whity
@Markus It seems that you know what I'm talking about. Did you have a simmilar problem? How did you solve it? Actually my "solution" is working good enough for now, but I't drives me crazy that I can not get .Net to do what I REALLY want. That should be doable, right? @Erno: I should read up to the end befor posting :). You are right. My solution is much more expensive then the actual VisualBrush. But the application performance is still much better when your Visual is updating rapidly. Thing about an animiation. That animation will cause the Brush to be updated 60 times a second by default.Whity
@Harri you have some issues in your code, but the core problem is: 1st: layoutupdated is not the correct event (you figured that alread), because it gets raised every time some layout in the visual tree is updated (so UpdateBuffer triggers another LayoutUpdated). That alone would not be so bad but there is 2nd: LayoutUpdated doesn't get raised when there is an animation in the visual. So eventually the problem is this one.Triolein
What if you made a custom control for the small map that used a VisualBrush inside it to fill a BitmapCache'd image AND then only attached and detached the visual from the visualbrush with respect to your timer?Clifton
@J Trana what he's got so far already does a refresh based on a timer (didn't you read the question?) But that also means this is getting refreshed even if the visual had no render update.Triolein
@Markus Hutter It sounds like he's using a RenderTargetBitmap which is exceedingly slow. I'm trying to recommend an option where he still uses the VisualBrush to take the snapshots but continually attaches and detaches the visual. I'm wondering about the overhead of VisualBrush attachment vs. RenderTargetBitmap. That's where I'm hoping he'll get the savings.Clifton
@J Trana the usual VisualBrush he was taking before was slow, as it was updating at 60fps. So he coded his solution with the RenderTargetBitmap, which is not slow, but updates repeatedly even if an update wouldn't be needed. That's why his final question is: " [how to] trigger the update only when nessasarry [?]"Triolein
Well, I've got something that is related and you might find of use, but definitely a hack. I started writing a chunk of code that would walk a portion of the visual tree on CompositionTarget_Rendering and check for any updates and then pull the raw MilCore rendering structs out using reflection to get at the internals. The check for updates looks at the raw flags of the Visuals marking their current state. I'll post that here - maybe you'll find it useful, maybe not.Clifton
@Markus Hutter: I realized that LayoutUpdated is not the perfect event to use in this case. But the thing is: I can not find a better one. That is my point :). If there would be another event I could use the problem would be solved.Whity
@J Trana. Good point. I will try to attach/detach the visual instead of using the RenderTargetBitmap. But: I don't think it will work. My expectation when detaching the VisualBrush would be an empty brush. Not sure about that ... I'll try it. Anyhow ... even if that works it won't solve the core problem. I'm not hunting for some CPU cycles. What I want is a clean way of refreshing my visual when needed. But still a good option to optimize the solution when it is working. I'll give it a shoot!Whity
C
4

I've monitored the visual status of controls using the internals of WPF via reflection. So the code I've written hooks into the CompositionTarget.Rendering event, walks the tree, and looks for any changes in the subtree. I was writing it to intercept data being pushed to MilCore and then use it for my own purposes, so take this code as a hack and nothing more. If it helps you, great. I was using this on .NET 4.

First, the code to walk the tree read the status flags:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Windows;
using System.Windows.Media;
using System.Reflection;

namespace MilSnatch.Utils
{
    public static class VisualTreeHelperPlus
    {
        public static IEnumerable<DependencyObject> WalkTree(DependencyObject root)
        {
            yield return root;
            int count = VisualTreeHelper.GetChildrenCount(root);
            for (int i = 0; i < count; i++)
            {
                foreach (var descendant in WalkTree(VisualTreeHelper.GetChild(root, i)))
                    yield return descendant;
            }
        }

        public static CoreFlags ReadFlags(UIElement element)
        {
            var fieldInfo = typeof(UIElement).GetField("_flags", BindingFlags.Instance | BindingFlags.NonPublic);
            return (CoreFlags)fieldInfo.GetValue(element);
        }

        public static bool FlagsIndicateUpdate(UIElement element)
        {
            return (ReadFlags(element) &
                (
                    CoreFlags.ArrangeDirty |
                    CoreFlags.MeasureDirty |
                    CoreFlags.RenderingInvalidated
                )) != CoreFlags.None;
        }
    }

    [Flags]
    public enum CoreFlags : uint
    {
        AreTransformsClean = 0x800000,
        ArrangeDirty = 8,
        ArrangeInProgress = 0x20,
        ClipToBoundsCache = 2,
        ExistsEventHandlersStore = 0x2000000,
        HasAutomationPeer = 0x100000,
        IsCollapsed = 0x200,
        IsKeyboardFocusWithinCache = 0x400,
        IsKeyboardFocusWithinChanged = 0x800,
        IsMouseCaptureWithinCache = 0x4000,
        IsMouseCaptureWithinChanged = 0x8000,
        IsMouseOverCache = 0x1000,
        IsMouseOverChanged = 0x2000,
        IsOpacitySuppressed = 0x1000000,
        IsStylusCaptureWithinCache = 0x40000,
        IsStylusCaptureWithinChanged = 0x80000,
        IsStylusOverCache = 0x10000,
        IsStylusOverChanged = 0x20000,
        IsVisibleCache = 0x400000,
        MeasureDirty = 4,
        MeasureDuringArrange = 0x100,
        MeasureInProgress = 0x10,
        NeverArranged = 0x80,
        NeverMeasured = 0x40,
        None = 0,
        RenderingInvalidated = 0x200000,
        SnapsToDevicePixelsCache = 1,
        TouchEnterCache = 0x80000000,
        TouchesCapturedWithinCache = 0x10000000,
        TouchesCapturedWithinChanged = 0x20000000,
        TouchesOverCache = 0x4000000,
        TouchesOverChanged = 0x8000000,
        TouchLeaveCache = 0x40000000
    }

}

Next, supporting code for Rendering event:

//don't worry about RenderDataWrapper. Just use some sort of WeakReference wrapper for each UIElement
    void CompositionTarget_Rendering(object sender, EventArgs e)
{
    //Thread.Sleep(250);
    Dictionary<int, RenderDataWrapper> newCache = new Dictionary<int, RenderDataWrapper>();
    foreach (var rawItem in VisualTreeHelperPlus.WalkTree(m_Root))
    {
        var item = rawItem as FrameworkElement;
        if (item == null)
        {
            Console.WriteLine("Encountered non-FrameworkElement: " + rawItem.GetType());
            continue;
        }
        int hash = item.GetHashCode();
        RenderDataWrapper cacheEntry;
        if (!m_Cache.TryGetValue(hash, out cacheEntry))
        {
            cacheEntry = new RenderDataWrapper();
            cacheEntry.SetControl(item);
            newCache.Add(hash, cacheEntry);
        }
        else
        {
            m_Cache.Remove(hash);
            newCache.Add(hash, cacheEntry);
        }
            //check the visual for updates - something like the following...
            if(VisualTreeHelperPlus.FlagsIndicateUpdate(item as UIElement))
            {
                //flag for new snapshot.
            }
        }
    m_Cache = newCache;
}

Anyways, in this way I monitored the visual tree for updates, and I think you can monitor them using something similar if you'd like. This is far from best practices, but sometimes pragmatic code has to be. Beware.

Clifton answered 25/4, 2011 at 4:59 Comment(3)
Definitely, this is just some example code. I needed all the debugging I could get for pulling raw structs out of milcore pushing buffers. :DClifton
Tip: use Debug.WriteLine next time.Truncation
Pretty nice. But if I use Reflection to access .Net internals in production code I propably get killed :). Also, since the controls I have to monitor are pretty complex, I'm not sure about the performance implications of walking the VisualTree on each Rendering event (which is called veeeery often if I remember corectly). But definetly a +1 for cleverness!Whity
C
1

I think your solution is pretty good already. Instead of a timer you could try to do it with a Dispatcher callback with a ApplicationIdle priority, this would effectively make the updates lazy since it will only occur when the application isn't busy. Also, as you have already stated you might try to use the BitmapCacheBrush instead of the VisualBrush to draw your overview image and see if this makes any difference.

Regarding your question on WHEN to redraw the brush:

Basically you want to know when things changed in a way that would mark your existing thumbnail image as dirty.

I think you could either attack this problem in the backend/model and have a dirty flag there or try to get it from the front end.

Backend obviously depends on your application so I can't comment.

In the front end the LayoutUpdated event seems the right thing to do but as you say it could fire more often than necessary.

Here is a shot in the dark - I don't know how LayoutUpdated works internally so it might have the same problem as LayoutUpdated: You could override ArrangeOverride in the control you want to observe. Whenever ArrangeOverride is called you fire your own layout updated event using a dispatcher so that it is fired after the layout pass finishes. (maybe even wait for a couple of milliseconds longer and don't queue more events if a new ArrangeOverride should be called in the meanwhile). Since a layout pass will always call Measure and then Arrange and travel up the tree this should cover any changes anywhere inside the control.

Communicate answered 5/5, 2011 at 5:8 Comment(4)
Good idea! I just tried it. But it doesn't make much difference since the application is not doing much in during normal operation. And the Visual update is done only every 500ms anyway. And still: what I really would like to know is how I can determine WHEN to redraw the brush. The main problem is thar I do not have an event to property update the visual.Whity
added some more suggestions although I think your timer or the ApplicationIdle event is probably good enough for this kind of feature.Communicate
I'd love to be proved wrong, but I don't believe ArrangeOverride is called on a Control if one of it's sub-sub controls has a layout/render pass. Would have been nice if that worked.Triolein
Just tried using ArrangeOverride. It works sometimes but sadly not reliable. It seems that it depends on what changes in the control and how the control is layouted. Can't do it in the "backend" eiter. If I have (for example) a ScrollViewer in my control the visual update is not triggered or reflected in the ViewModel at all. But thanks for your suggestions.Whity

© 2022 - 2024 — McMap. All rights reserved.