When using MergeField FieldCodes in OpenXml SDK in C# why do field codes disappear or fragment?

Question

I have been working successfully with the C# OpenXml SDK (Unofficial Microsoft Package 2.5 from NuGet) for some time now, but have recently noticed that the following line of code returns different results depending on what mood Microsoft Word appears to be in when the file gets saved:

var fields = document.Descendants<FieldCode>();

From what I can tell, when creating the document in the first place (using Word 2013 on Windows 8.1) if you use the Insert->QuickParts->Field and choose MergeField from the Field names left hand pane, and then provide a Field name in the field properties and click OK then the field code is correctly saved in the document as I would expect.

Then when using the aforementioned line of code I will receive a field code count of 1 field. If I subsequently edit this document (and even leave this field well alone) the subsequent saving could mean that this field code no longer is returned in my query.

Another case of the same curiousness is when I see the FieldCode nodes split across multiple items. So rather than seeing say:

" MERGEFIELD  Author  \\* MERGEFORMAT "

As the node name, I will see:

" MERGEFIELD  Aut"
"hor  \\* MERGEFORMAT"

Split as two FieldCode node values. I have no idea why this would be the case, but it certainly makes my ability to match nodes that much more exciting. Is this expected behaviour? A known bug? I don't really want to have to crack open the raw xml and edit this document to work until I understand what is going on. Many thanks all.

Bugloss

I have noticed that my 'missing fields' appear to have been converted to 'SimpleField's does anyone understand why this would be? Or what the difference is? – Bugloss

Answer 1

I came across this very problem myself, and found a solution that exists within OpenXML: a utility class called MarkupSimplifier which is part of the PowerTools for Open XML project. Using this class solved all the problems I was having that you describe.

The full article is located here.

Here are some pertinent exercepts :

Perhaps the most useful simplification that this performs is to merge adjacent runs with identical formatting.

It goes on to say:

Open XML applications, including Word, can arbitrarily split runs as necessary. If you, for instance, add a comment to a document, runs will be split at the location of the start and end of the comment. After MarkupSimplifier removes comments, it can merge runs, resulting in simpler markup.

An example of the utility class in use is:

SimplifyMarkupSettings settings = new SimplifyMarkupSettings
{
    RemoveComments = true,
    RemoveContentControls = true,
    RemoveEndAndFootNotes = true,
    RemoveFieldCodes = false,
    RemoveLastRenderedPageBreak = true,
    RemovePermissions = true,
    RemoveProof = true,
    RemoveRsidInfo = true,
    RemoveSmartTags = true,
    RemoveSoftHyphens = true,
    ReplaceTabsWithSpaces = true,
};
MarkupSimplifier.SimplifyMarkup(wordDoc, settings);

I have used this many times with Word 2010 documents using VS2015 .Net Framework 4.5.2 and it has made my life much, much easier.

Update:

I have revisited this code and have found it clears upon runs on MERGEFIELDS but not IF FIELDS that reference mergefields e.g.

{if {MERGEFIELD When39} = "Y???" "Y" "N" }

I have no idea why this might be so, and examination of the underlying XML offers no hints.

Mislike

Very interesting, I'll try and have a go at this and report back – Bugloss

SimplifyMarkupSettings it is the best way for me. Thanks bro ;) – Annabal

Answer 2

Word will often split text runs with into multiple text runs for no reason I've ever understood. When searching, comparing, tidying etc. We preprocess the body with method which combines multiple runs into a single text run.

    /// <summary>
    /// Combines the identical runs.
    /// </summary>
    /// <param name="body">The body.</param>
    public static void CombineIdenticalRuns(W.Body body)
    {

        List<W.Run> runsToRemove = new List<W.Run>();

        foreach (W.Paragraph para in body.Descendants<W.Paragraph>())
        {
            List<W.Run> runs = para.Elements<W.Run>().ToList();
            for (int i = runs.Count - 2; i >= 0; i--)
            {
                W.Text text1 = runs[i].GetFirstChild<W.Text>();
                W.Text text2 = runs[i + 1].GetFirstChild<W.Text>();
                if (text1 != null && text2 != null)
                {
                    string rPr1 = "";
                    string rPr2 = "";
                    if (runs[i].RunProperties != null) rPr1 = runs[i].RunProperties.OuterXml;
                    if (runs[i + 1].RunProperties != null) rPr2 = runs[i + 1].RunProperties.OuterXml;
                    if (rPr1 == rPr2)
                    {
                        text1.Text += text2.Text;
                        runsToRemove.Add(runs[i + 1]);
                    }
                }
            }
        }
        foreach (W.Run run in runsToRemove)
        {
            run.Remove();
        }
    }

Atlas

Is this true for field codes too? I don't mind the runs being split out, but the fact that the field code seems to be rendered corrupt is what is bothering me. I don't truly comprehend how the fieldcode node is created, are you suggesting that the code itself is being split into runs? I shall have a look....! – Bugloss

Does two run properties nodes with the same properties have always the same outerxml? Or could it be that the properties inside have a different order, so the outerxml is different? – Kelwunn

@Kelwunn They can have a different order, but IIRC if Word or the OOXML library produced the XML they standardise the order for all runs (They standardise the order for all XML since there are several cases where the ordering is important) – Atlas

Answer 3

I tried to simplify the document with Powertools but the result was a corrupted word file. I make this routine for simplify only fieldcodes that has specifics names, works in all parts on the docs (maindocumentpart, headers and footers):

internal static void SimplifyFieldCodes(WordprocessingDocument document)
    {
        var masks = new string[] { Constants.VAR_MASK, Constants.INP_MASK, Constants.TBL_MASK, Constants.IMG_MASK, Constants.GRF_MASK };
        SimplifyFieldCodesInElement(document.MainDocumentPart.RootElement, masks);

        foreach (var headerPart in document.MainDocumentPart.HeaderParts)
        {
            SimplifyFieldCodesInElement(headerPart.Header, masks);
        }

        foreach (var footerPart in document.MainDocumentPart.FooterParts)
        {
            SimplifyFieldCodesInElement(footerPart.Footer, masks);
        }

    }

    internal static void SimplifyFieldCodesInElement(OpenXmlElement element, string[] regexpMasks)
    {
        foreach (var run in element.Descendants<Run>()
            .Select(item => (Run)item)
            .ToList())
        {
            var fieldChar = run.Descendants<FieldChar>().FirstOrDefault();
            if (fieldChar != null && fieldChar.FieldCharType == FieldCharValues.Begin)
            {
                string fieldContent = "";
                List<Run> runsInFieldCode = new List<Run>();

                var currentRun = run.NextSibling();
                while ((currentRun is Run) && currentRun.Descendants<FieldCode>().FirstOrDefault() != null)
                {
                    var currentRunFieldCode = currentRun.Descendants<FieldCode>().FirstOrDefault();
                    fieldContent += currentRunFieldCode.InnerText;
                    runsInFieldCode.Add((Run)currentRun);
                    currentRun = currentRun.NextSibling();
                }

                // If there is more than one Run for the FieldCode, and is one we must change, set the complete text in the first Run and remove the rest
                if (runsInFieldCode.Count > 1)
                {
                    // Check fielcode to know it's one that we must simplify (for not to change TOC, PAGEREF, etc.)
                    bool applyTransform = false;
                    foreach (string regexpMask in regexpMasks)
                    {
                        Regex regex = new Regex(regexpMask);
                        Match match = regex.Match(fieldContent);
                        if (match.Success)
                        {
                            applyTransform = true;
                            break;
                        }
                    }

                    if (applyTransform)
                    {
                        var currentRunFieldCode = runsInFieldCode[0].Descendants<FieldCode>().FirstOrDefault();
                        currentRunFieldCode.Text = fieldContent;
                        runsInFieldCode.RemoveAt(0);

                        foreach (Run runToRemove in runsInFieldCode)
                        {
                            runToRemove.Remove();
                        }
                    }
                }
            }
        }
    }

Hope this helps!!!

Tops