FOR XML PATH(''): Escaping "special" characters

Question

This code basically translates characters based on position in one string to the character at the same position in another string and it runs for all rows in the table.

When I run this (simplified version):

DECLARE @R           char(40)
DECLARE @U           char(40)
SET @R=' abcdefghijklmnopqrstuvwxyz!@#$%^&*()_+'+char(181)
SET @U=REVERSE(@R)

DECLARE @TestTable TABLE (RowID int identity(1,1) primary key, Unreadable  varchar(500))
INSERT INTO @TestTable VALUES ('+µt$zw!*µsu+yt!+s$xy')
INSERT INTO @TestTable VALUES ('%*!!xµpxu!(')
INSERT INTO @TestTable VALUES ('pxpµnxrµu+yµs%$t')


    ;WITH CodeValues AS
    (
    SELECT
        Number,SUBSTRING(@R,Number,1) AS R,ASCII(SUBSTRING(@U,Number,1)) AS UA
        FROM Numbers
        WHERE Number<=LEN(@R)
    )
    SELECT
        t.RowID
            ,(SELECT
                  ''+c.R
                  FROM Numbers               n
                      INNER JOIN CodeValues  c ON ASCII(SUBSTRING(t.Unreadable,n.Number,1))=c.UA
                  WHERE n.Number<=LEN(t.Unreadable) 
                  FOR XML PATH('') 
             ) AS readable
        FROM @TestTable t

I get the following:

RowID       readable
----------- ---------------------------------------
1           a simple translation
2           hello world
3           wow you ran this

But need:

RowID       readable
----------- ---------------------------------------
1           a simple translation
2           hello world
3           wow you ran this

Is there any way, other than REPLACE(), to have the spaces show up properly? This also happens on line breaks, in my actual code.

Can this be rewritten in a better way? I basically just used the FOR XML PATH('') to concatenate the individual row values together.

Appleton

I'm sure this is your real code, but you should really post a simpler example that illustrates the same problem. That's a fair amount of weird code to digest, though the problem is likely to be a lot simpler. – Faulk

@John Saunders, I wish that was my "real code", because I'd be done now! Sorry to say that the final code will more complex, as this is a small part of one query. – Appleton

Answer 1

The XML you get is correct. It is XML, not text, and readable as XML by an XML parser. Special characters are properly escaped, as they should be. Whatever client module you have that consumes that XML should parse it as XML, not as text, and then it will display properly.

Update:

In case is not clear, all you need to do in your query is to treat XML as XML and text as text, not mix XML as text, ie:

;WITH CodeValues AS
    (
    SELECT
        Number,SUBSTRING(@R,Number,1) AS R,ASCII(SUBSTRING(@U,Number,1)) AS UA
        FROM Numbers
        WHERE Number<=LEN(@R)
    )
, XmlValues AS (
SELECT
        t.RowID
            ,(SELECT
                  ''+c.R
                  FROM Numbers               n
                      INNER JOIN CodeValues  c ON ASCII(SUBSTRING(t.Unreadable,n.Number,1))=c.UA
                  WHERE n.Number<=LEN(t.Unreadable) 
                  FOR XML PATH(''), TYPE
             ) AS readable
        FROM @TestTable t)
SELECT x.RowId,
    x.readable.value('.', 'VARCHAR(8000)') as readable
    FROM XmlValues AS x

Superannuate

+1 - this answer just helped me. I missed the , TYPE for a while, though. – Tientiena

Thanks for the heads up about the client module parsing it - saved me from messing with the code! – Jessy

Additional reading: sqlblog.com/blogs/rob_farley/archive/2010/04/14/… – Juglandaceous