Create guitar chords editor in WPF (from RichTextBox?)

Question

Main purpose of application I'm working on in WPF is to allow editing and consequently printing of songs lyrics with guitar chords over it.

You have probably seen chords even if you don't play any instrument. To give you an idea it looks like this:

E                 E6
I know I stand in line until you
E                  E6               F#m            B F#m B
think you have the time to spend an evening with me

But instead of this ugly mono-spaced font I want to have Times New Roman font with kerning for both lyrics and chords (chords in bold font). And I want user to be able to edit this.

This does not appear to be supported scenario for RichTextBox. These are some of the problems that I don't know how to solve:

• Chords have their positions fixed over some character in lyrics text (or more generally TextPointer of lyrics line). When user edits lyrics I want chord to stay over right character. Example:

.

E                                       E6
I know !!!SOME TEXT REPLACED HERE!!! in line until you

• Line wrapping: 2 lines (1th with chords and 2th with lyrics) are logically one line when it comes to wrapping. When a word wraps to next line all chords that are over it should also wrap. Also when chord wraps the word that it is over it also wrap. Example:

.

E                  E6
think you have the time to spend an
F#m            B F#m B
evening with me

• Chords should stay over right character even when chords are too near to each other. In this case some extra space is automatically inserted in lyrics line. Example:

.

                  F#m E6
  ...you have the ti  me to spend... 

• Say I have lyrics line Ta VA and chord over A. I want the lyrics to look like not like . Second picture is not kerned between V and A. Orange lines are there only to visualize the effect (but they mark x offsets where chord would be placed). Code used to produce first sample is <TextBlock FontFamily="Times New Roman" FontSize="60">Ta VA</TextBlock> and for second sample <TextBlock FontFamily="Times New Roman" FontSize="60"><Span>Ta V<Floater />A</Span></TextBlock>.

Any ideas on how to get RichTextBox to do this ? Or is there better way to do it in WPF? Will I sub-classing Inline or Run help? Any ideas, hacks, TextPointer magic, code or links to related topics are welcome.

---

Edit:

I'm exploring 2 major directions to solve this problem but both lead to another problems so I ask new question:

1. Trying to turn RichTextBox into chords editor - Have a look at How can I create subclass of class Inline?.

2. Build new editor from separate components like Panels TextBoxes etc. as suggested in H.B. answer. This would need a lot of coding and also led to following (unsolved) problems:

   • Components will change their Width/Height according to they layout position (white space removal at line beginning etc.)
   • Kerning will have to be inserted manually at components boundaries.
   • How to make RichTextBox look like TextBlock? (not elegant hack/workaround is known)

---

Edit#2

Markus Hütter's high quality answer has shown me that a lot more can be done with RichTextBox then I expected when I was trying to tweak it for my needs myself. I've had time to explore the answer in details only now. Markus might be RichTextBox magician I need to help me with this but there are some unsolved problems with his solution as well:

1. This application will be all about "beautifully" printed lyrics. The main goal is that the text looks perfect from the typographic point of view. When chords are too near to each other or even overlapping Markus suggests that I iteratively add addition spaces before its position until their distance is sufficient. There is actually requirement that the user can set minimum distance between 2 chords. That minimum distance should be honored and not exceeded until necessary. Spaces are not granular enough - once I add last space needed I'll probably make the gap wider then necessary - that will make the document look 'bad' I don't think it could be accepted. I'd need to insert space of custom width.
2. There could be lines with no chords (only text) or even lines with no text (only chords). When LineHeight is set to 25 or other fixed value for whole document it will cause lines with no chords to have "empty lines" above them. When there are only chords and no text there will be no space for them.

There are other minor problems but I either think I can solve them or I consider them not important. Anyway I think Markus's answer is really valuable - not only for showing me possible way to go but also as a demonstration of general pattern of using RichTextBox with adorner.

Seaton

Notes: User does not have to able to edit chords directly from editor (I'll can add some contextual menu to insert or change chord) nor even put caret but it would be nice. So the only thing the user have to be able to is editing lyrics lines. When user deletes character that has chord over it the chord might be deleted too or just shifted to next character. – Seaton

+1 very nice question. I have a lot of experience with WPFs RichTextBox. I'll give it some thought and see if I can come up with something. One question upfront though: Is Reflection allowed? – Safranine

Answer 1

I cannot give you any concrete help but in terms of architecture you need to change your layout from this

To this

Everything else is a hack. Your unit/glyph must become a word-chord-pair.

---

Edit: I have been fooling around with a templated ItemsControl and it even works out to some degree, so it might be of interest.

<ItemsControl Grid.IsSharedSizeScope="True" ItemsSource="{Binding SheetData}"
              Name="_chordEditor">
    <ItemsControl.ItemsPanel>
        <ItemsPanelTemplate>
            <WrapPanel/>
        </ItemsPanelTemplate>
    </ItemsControl.ItemsPanel>
    <ItemsControl.ItemTemplate>
        <DataTemplate>
            <Grid>
                <Grid.RowDefinitions>
                    <RowDefinition SharedSizeGroup="A" Height="Auto"/>
                    <RowDefinition SharedSizeGroup="B" Height="Auto"/>
                </Grid.RowDefinitions>
                <Grid.Children>
                    <TextBox Name="chordTB" Grid.Row="0" Text="{Binding Chord}"/>
                    <TextBox Name="wordTB"  Grid.Row="1" Text="{Binding Word}"
                             PreviewKeyDown="Glyph_Word_KeyDown" TextChanged="Glyph_Word_TextChanged"/>
                </Grid.Children>
            </Grid>
        </DataTemplate>
    </ItemsControl.ItemTemplate>
</ItemsControl>

private readonly ObservableCollection<ChordWordPair> _sheetData = new ObservableCollection<ChordWordPair>();
public ObservableCollection<ChordWordPair> SheetData
{
    get { return _sheetData; }
}

public class ChordWordPair: INotifyPropertyChanged
{
    private string _chord = String.Empty;
    public string Chord
    {
        get { return _chord; }
        set
        {
            if (_chord != value)
            {
                _chord = value;
                // This uses some reflection extension method,
                // a normal event raising method would do just fine.
                PropertyChanged.Notify(() => this.Chord);
            }
        }
    }

    private string _word = String.Empty;
    public string Word
    {
        get { return _word; }
        set
        {
            if (_word != value)
            {
                _word = value;
                PropertyChanged.Notify(() => this.Word);
            }
        }
    }

    public ChordWordPair() { }
    public ChordWordPair(string word, string chord)
    {
        Word = word;
        Chord = chord;
    }

    public event PropertyChangedEventHandler PropertyChanged;
}

private void AddNewGlyph(string text, int index)
{
    var glyph = new ChordWordPair(text, String.Empty);
    SheetData.Insert(index, glyph);
    FocusGlyphTextBox(glyph, false);
}

private void FocusGlyphTextBox(ChordWordPair glyph, bool moveCaretToEnd)
{
    var cp = _chordEditor.ItemContainerGenerator.ContainerFromItem(glyph) as ContentPresenter;
    Action focusAction = () =>
    {
        var grid = VisualTreeHelper.GetChild(cp, 0) as Grid;
        var wordTB = grid.Children[1] as TextBox;
        Keyboard.Focus(wordTB);
        if (moveCaretToEnd)
        {
            wordTB.CaretIndex = int.MaxValue;
        }
    };
    if (!cp.IsLoaded)
    {
        cp.Loaded += (s, e) => focusAction.Invoke();
    }
    else
    {
        focusAction.Invoke();
    }
}

private void Glyph_Word_TextChanged(object sender, TextChangedEventArgs e)
{
    var glyph = (sender as FrameworkElement).DataContext as ChordWordPair;
    var tb = sender as TextBox;

    string[] glyphs = tb.Text.Split(' ');
    if (glyphs.Length > 1)
    {
        glyph.Word = glyphs[0];
        for (int i = 1; i < glyphs.Length; i++)
        {
            AddNewGlyph(glyphs[i], SheetData.IndexOf(glyph) + i);
        }
    }
}

private void Glyph_Word_KeyDown(object sender, KeyEventArgs e)
{
    var tb = sender as TextBox;
    var glyph = (sender as FrameworkElement).DataContext as ChordWordPair;

    if (e.Key == Key.Left && tb.CaretIndex == 0 || e.Key == Key.Back && tb.Text == String.Empty)
    {
        int i = SheetData.IndexOf(glyph);
        if (i > 0)
        {
            var leftGlyph = SheetData[i - 1];
            FocusGlyphTextBox(leftGlyph, true);
            e.Handled = true;
            if (e.Key == Key.Back) SheetData.Remove(glyph);
        }
    }
    if (e.Key == Key.Right && tb.CaretIndex == tb.Text.Length)
    {
        int i = SheetData.IndexOf(glyph);
        if (i < SheetData.Count - 1)
        {
            var rightGlyph = SheetData[i + 1];
            FocusGlyphTextBox(rightGlyph, false);
            e.Handled = true;
        }
    }
}

Initially some glyph should be added to the collection, otherwise there will be no input field (this can be avoided with further templating, e.g. by using a datatrigger that shows a field if the collection is empty).

Perfecting this would require a lot of additional work like styling the TextBoxes, adding written line breaks (right now it only breaks when the wrap panel makes it), supporting selection accross multiple textboxes, etc.

Cesar

H.B. +1. Thank you for nice visualization it might be useful for others. I very well understand that I need to do this but I just don't know how to do it with FlowDocument. I do not even know where to start. How to create those double-line blocks? Well I can create those blocks as Figures that would contain Paragraph with 2 lines. But this does not work when there are chords over same word. You will have to have word spitted between 2 blocks and there will be no kerning between 2 letters at the boundary. This is the kerning issue presented in original question. – Seaton

H.B.: So this would normally mean to me that there is missing element that I need to implement. In this case implement custom Inline. But there is not documentation/blocks/tutorials on how to do it. At least non that I can find. Is it supported scenario at all ? – Seaton

I do not know, as you said there are no tutorials to be found, it does look like sub-classing and overriding is not intended... – Cesar

I added a draft for a method of how to approximate things with a templated ItemsControl and significant navigational logic. – Cesar

@H.B.: Thank you. Your code did not work as it was for some reason (even when I add new ChordWordPair to _sheetData in constructor it displayed empty ContentControl) but when I used ContentControl.Items instead of ContentControl.DataSource bind to _sheetData it was OK. It would need a lot of changes to be usable (and really a lot of code to work as I want it to), in the fact I'd have to re-implement RichtTextBox with some this 'chords-specific' blocking added. I think this can be all done but there is something that I don't know how to do: KERNING. – Seaton

(continue) Consider you have lyrics that consist of only one word like Ta VA that has chord over last letter. In this design you will have to split text Ta VA to 2 stings Ta V and A and place both to separate boxes (they will become Grids after templating). That means you loose kerning between V and following A character. It is exactly unwanted effect demonstrated in my question by those 2 images. It looks little as if V and A had some space in between them even when they belong to the same word. – Seaton

What a pain, good thing i do not have to write such an application :P – Cesar

H.B.: :D Well it will not be easy I guess and this question might need a bounty... But eventually there will be some good solution found or I'll just have to hack RichTextBox. But you showed me the other direction when not using RichtTextBox might not be hopeless at all. I'm Working on getting it work right now. – Seaton

@H.B.: So I'm trying to solve the kerning problem. You have seen my question about 'cropping' and you were able to give me answer really quickly. Now I need to know how much do I need to adjust (negative) space between glyphs (boxes) to simulate kerning. That would be the kerning value between characters a glyphs borders. I have another question about it here if you want to see progress or answer it again :). – Seaton

I already saw and upvoted that question some time ago. Messing with fonts is a pain though and i cannot answer that question just from experience, maybe someone else can help you. – Cesar

@H.B.: There is edit to my question that sums my latest "advances" and bounty opened. – Seaton

Answer 2

Soooo, I had a little fun here. This is how it looks like:

The lyrics is fully editable, the chords are currently not (but this would be an easy extension).

this is the xaml:

<Window ...>
    <AdornerDecorator>
        <!-- setting the LineHeight enables us to position the Adorner on top of the text -->
        <RichTextBox TextBlock.LineHeight="25" Padding="0,25,0,0" Name="RTB"/>
    </AdornerDecorator>    
</Window>

and this is the code:

public partial class MainWindow
{
    public MainWindow()
    {
        InitializeComponent();
        const string input = "E                 E6\nI know I stand in line until you\nE                  E6               F#m            B F#m B\nthink you have the time to spend an evening with me                ";
        var lines = input.Split('\n');

        var paragraph = new Paragraph{Margin = new Thickness(0),Padding = new Thickness(0)}; // Paragraph sets default margins, don't want those

        RTB.Document = new FlowDocument(paragraph);

        // this is getting the AdornerLayer, we explicitly included in the xaml.
        // in it's visual tree the RTB actually has an AdornerLayer, that would rather
        // be the AdornerLayer we want to get
        // for that you will either want to subclass RichTextBox to expose the Child of
        // GetTemplateChild("ContentElement") (which supposedly is the ScrollViewer
        // that hosts the FlowDocument as of http://msdn.microsoft.com/en-us/library/ff457769(v=vs.95).aspx 
        // , I hope this holds true for WPF as well, I rather remember this being something
        // called "PART_ScrollSomething", but I'm sure you will find that out)
        //
        // another option would be to not subclass from RTB and just traverse the VisualTree
        // with the VisualTreeHelper to find the UIElement that you can use for GetAdornerLayer
        var adornerLayer = AdornerLayer.GetAdornerLayer(RTB);

        for (var i = 1; i < lines.Length; i += 2)
        {
            var run = new Run(lines[i]);
            paragraph.Inlines.Add(run);
            paragraph.Inlines.Add(new LineBreak());

            var chordpos = lines[i - 1].Split(' ');
            var pos = 0;
            foreach (string t in chordpos)
            {
                if (!string.IsNullOrEmpty(t))
                {
                    var position = run.ContentStart.GetPositionAtOffset(pos);
                    adornerLayer.Add(new ChordAdorner(RTB,t,position));
                }
                pos += t.Length + 1;
            }
        }

    }
}

using this Adorner:

public class ChordAdorner : Adorner
{
    private readonly TextPointer _position;

    private static readonly PropertyInfo TextViewProperty = typeof(TextSelection).GetProperty("TextView", BindingFlags.Instance | BindingFlags.NonPublic);
    private static readonly EventInfo TextViewUpdateEvent = TextViewProperty.PropertyType.GetEvent("Updated");

    private readonly FormattedText _formattedText;

    public ChordAdorner(RichTextBox adornedElement, string chord, TextPointer position) : base(adornedElement)
    {
        _position = position;
        // I'm in no way associated with the font used, nor recommend it, it's just the first example I found of FormattedText
        _formattedText = new FormattedText(chord, CultureInfo.GetCultureInfo("en-us"),FlowDirection.LeftToRight,new Typeface(new FontFamily("Arial").ToString()),12,Brushes.Black);

        // this is where the magic starts
        // you would otherwise not know when to actually reposition the drawn Chords
        // you could otherwise only subscribe to TextChanged and schedule a Dispatcher
        // call to update this Adorner, which either fires too often or not often enough
        // that's why you're using the RichTextBox.Selection.TextView.Updated event
        // (you're then basically updating the same time that the Caret-Adorner
        // updates it's position)
        Dispatcher.CurrentDispatcher.BeginInvoke(DispatcherPriority.Loaded, new Action(() =>
        {
            object textView = TextViewProperty.GetValue(adornedElement.Selection, null);
            TextViewUpdateEvent.AddEventHandler(textView, Delegate.CreateDelegate(TextViewUpdateEvent.EventHandlerType, ((Action<object, EventArgs>)TextViewUpdated).Target, ((Action<object, EventArgs>)TextViewUpdated).Method));
            InvalidateVisual(); //call here an event that triggers the update, if 
                                //you later decide you want to include a whole VisualTree
                                //you will have to change this as well as this ----------.
        }));                                                                          // |
    }                                                                                 // |
                                                                                      // |
    public void TextViewUpdated(object sender, EventArgs e)                           // |
    {                                                                                 // V
        Dispatcher.CurrentDispatcher.BeginInvoke(DispatcherPriority.Loaded, new Action(InvalidateVisual));
    }

    protected override void OnRender(DrawingContext drawingContext)
    {
        if(!_position.HasValidLayout) return; // with the current setup this *should* always be true. check anyway
        var pos = _position.GetCharacterRect(LogicalDirection.Forward).TopLeft;
        pos += new Vector(0, -10); //reposition so it's on top of the line
        drawingContext.DrawText(_formattedText,pos);
    }
}

this is using an adorner like david suggested, but I know it is hard to find a how to out there. That's probably because there is none. I had spent hours before in reflector trying to find that exact event that signals that the layout of the flowdocument has been figured out.

I'm not sure if that dispatcher call in the constructor is actually needed, but I left it in for being bulletproof. (I needed this because in my setup the RichTextBox had not been shown yet).

Obviously this needs a lot more coding, but this will give you a start. You will want to play around with positioning and such.

For getting the positioning right if two adorners are too close and are overlapping I'd suggest you somehow keep track of which adorner comes before and see if the current one would overlap. then you can for example iteratively insert a space before the _position-TextPointer.

If you later decide, you want the chords editable too, you can instead of just drawing the text in OnRender have a whole VisualTree under the adorner. (here is an example of an adorner with a ContentControl underneath). Beware though that you have to handle the ArrangeOveride then to correctly position the Adorner by the _position CharacterRect.

Ruysdael

@drasto updated my code with some deeper explanation of what's going on – Safranine

+1 very nice answer. Lot of thanks for writing this! I was really busy during the week so I had time to have a look on it properly only now, once the bounty is over :(. It shown me a lot but when I was playing with a little I found 2 thinks that I could not solve. There is edit#2 describing the problems. – Seaton

@drasto, I'll see what I can do about the remaining issues, I just don't have the time right now, but I will stay at this. In case you were wondering why I suggested to use Spaces to separate two chords which are too close: to keep the whole solution simple, I would have otherwise suggested to use that InlineUIContainer from H.B.s Answer but I'm pretty sure you'd then have to handle all the editing functions yourself (typing backspace or del or cutting with contextmenu) – Safranine

@drasto ... (continued) after reading you question once again though, I wonder if that would be OK with "pretty print" (as that would mean you will have a large space between the lyrics under these chords) – Safranine

@Markus please feel free to take your time! There is no big hurry and even if there was this is great on SO - you can answer me half a year if you like or not answer at all. Your advices has already been very useful. I'm back on editor work again - I'm already using InlineContainer to insert custom width space when there are 2 chords too near... It seems to work quite well. InlineContainer gets deleted after hitting backspace or del automatically. Moving caret as if it was just a regular space is more tricky: What I'm trying to do now is: when the component inside InlineContainer gets.. – Seaton

... (continued) focus (the component is just a place holder) I want to give the caret back to RichTextBox and place the cursor just before or after the InlineContainer (depending on what was previous position of caret). As to long space between under chords that are too near: It might not be very beautiful but it has to be there. Otherwise I would change semantics of the lyrics and the way the song is played at the end. Some LaTeX classes fro chords printing solve this issue by placing - character in the middle of the inserted space - I might also do that. But only if the space was ... – Seaton

... (continued) inserted inside the word so that the word would appear as if it was slitted into two (by inserted space) - the - character in the middle of inserted space helps the guitarist and singer(s) to understand that it is single word. In the same time I'm also exploring the other road that @H.B. suggested. I've not given up that yet. But the big disadvantage there is that I'd have to simulate a lot of editing functionality on low level (listening to keystrokes). But if somebody has some ideas on this please share. – Seaton