Escaping double-quote in `delims` option of `for /F`

Question

I'm having a bit of trouble with a batch script which needs to parse a value out of an config file into a variable.

Suitably anonymised, the relevant line of the file looks like

<?define ProductShortName="Foo" ?>

I want to set a variable to Foo. The string ProductShortName is unique enough to get the line with findstr, but then I have to extract the value. The correct approach seems to be for /F, but all of the following give errors:

for /F "delims=^" usebackq" %%G in (`findstr /L "ProductShortName" "%~dp0Installer\Branding.wxi"`)
for /F "delims="" usebackq" %%G in (`findstr /L "ProductShortName" "%~dp0Installer\Branding.wxi"`)
for /F "delims=\" usebackq" %%G in (`findstr /L "ProductShortName" "%~dp0Installer\Branding.wxi"`)
for /F 'delims=^" usebackq' %%G in (`findstr /L "ProductShortName" "%~dp0Installer\Branding.wxi"`)
for /F 'delims=" usebackq' %%G in (`findstr /L "ProductShortName" "%~dp0Installer\Branding.wxi"`)
for /F "delims=" usebackq" %%G in (`findstr /L "ProductShortName" "%~dp0Installer\Branding.wxi"`)

mostly along the lines of

usebackq" %G in (`findstr /L "ProductShortName" "C:\foo\bar\Installer\Branding.wxi"`) was unexpected at this time.

What's the correct way of escaping it to split the string on "?

Striation

Humor me and try this: for /F "USEBACKQ tokens=3 delims= =" %%G in (findstr /L ProductShortName "%~dp0Installer\Branding.wxi"`) DO (SET var=%%~G) – Dilworth

Answer 1

You can use the double quotation mark as a delimiter with syntax like:

FOR /F delims^=^"^ tokens^=2 %G IN ('echo I "want" a "pony"') DO @ECHO %G

When run on the command line, using tokens^=2 should give you want, and 4 tokens gets you a pony.

Applying the technique to your original question, this should work in your batch file:

FOR /F delims^=^"^ tokens^=2 %%G IN ('FINDSTR /L "ProductShortName" "data.txt"')

Details

I'm no expert on quirks of the command line parser, but it may help to think of the usual "delims=blah tokens=blah" as a single, combined argument passed to FOR. The caret escaping trick in delims^=blah^ tokens^=blah bypasses the need for enclosing quotes while still treating the sequence as a single argument. I've used a bit of creative analogy here, and the effect isn't universal across the shell. E.g. you can't do dir C:^\Program^ Files (which makes sense since ^ is a valid filename character).

Test Cases

With sufficient escaping, you can quickly check your original sample on the command line:

FOR /F delims^=^"^ tokens^=2 %G IN ('echo ^^^<?define ProductShortName="Foo" ?^^^>') DO @ECHO %G

Others playing with this may want to create a file testcases.txt:

blah blah "red"
     blah "green" blah
How about a "white" "unicorn"?

and run something like:

FOR /F delims^=^"^ tokens^=2 %G IN (testcases.txt) DO @ECHO %G

to check results for a variety of inputs. In this case it should yield:

red
green
white

One last example:

FOR /F delims^=^"^ tokens^=2 %G IN ('FINDSTR /L "unicorn" "testcases.txt"') ^
DO @ECHO The unicorn is %G.

Finally, note my testing for this was done on Windows Server 2003.

Centra

Thanks for answering, but am I missing something or is this just the same as jeb's final answer? – Striation

In fact, it's the same technique. I think perhaps I approached it a little more clearly (with an aim to help future programmers who stumble across this question), but I'll let you be the judge :-). – Centra

You are the man! It is the best = falst + easiest solution than others in that question!!! If I was the asker, I certanly change the answer to yours! – Foundling

OK +1 so far so good. Now, lets say I want to use both space and double-quote as delims. – Forgetmenot

Thanks a lot. Any clue how to insert the skip parameter into this solution? – Dwarfism

This technique enabled me to figure out how to make the double-quote my eol comment; but I noticed that as a consequence, tab-completion later in the command line was disabled (Windows 11). That's not the fault of the answer, though. Thanks! – Diverge

None of these answers appear to run for me in a batch file. Here is what I have: for /f delims^=^"^ tokens^=1-2 %%A in ('Run command') do (stuff) and it replies with %%A in ('Run command') do (stuff) was not expected at this time – Carbolated

Answer 2

EDIT: This is wrong, see my comment later: As Joey said, there seems no possibility to use the quote as delim, it can be only used as EOL character.
This seems to be an effect of the FOR-LOOP parser of cmd.exe, as it scans the options-part and stops scanning it after an quote, only the EOL= option breaks this, as it read always the next character without any expection.

You can solve it with a workaround like icabod.
The solution is to replace the quotes with an unused character, but if you want to accept any character inside the quotes there isn't an unused character.

So my solution first creates an unused character, by replacing all previous occurences.
I want to use the # to replace the quotes, ut to preserve all # inside the quotes a replace it before with $R, but then it can collides with an existing $R in the text, therefore I first replace all $ with $D, then it is absolutly collision free.
After extracting the "quoted" text, I have to replace the $R and $D back to their original values, that's all.

@echo off
setlocal EnableDelayedExpansion

for /F "tokens=1,2" %%1 in ("%% #") DO (
    for /f "tokens=* usebackq" %%a in ("datafile.txt") do (
        set "z=%%a"
        set "z=!z:$=$D!"
        set "z=!z:#=$R!"
        set "z=!z:"=#!"
        for /f "tokens=1-3 delims=#" %%a in ("!z!") do (
            set "value=%%b"
            if defined value (
                set "value=!value:$R=#!"
                set "value=!value:$D=$!"
                echo result='!value!'
            )
        )
    )
)

Sample text:
<?define ProductShortName="Two #$* $D $R" ?>
results to Two #$* $D $R as expected

EDIT: There is a way!
I always tested things like this (and it fails)

setlocal EnableDelayedExpansion
set "var=one"two"three"
FOR /F ^"tokens^=1-3^ delims^=^"^" %%a in ("!var!") do echo %%a--%%b--%%c

But removing the first quote, it works.

setlocal EnableDelayedExpansion
set "var=one"two"three"
FOR /f tokens^=1-3^ delims^=^" %%a in ("!var!") do echo %%a--%%b--%%c

Garbo

Answer 3

I don't believe this is possible - a quote (") can't be used as a delimiter.

However one solution is to store the whole line in an environment variable, and use the built-in "replace" functionality of set to replace the quote with something else - for example _. You can then use another for loop on just this line to split on the new delimiter:

setlocal EnableDelayedExpansion
for /f "tokens=* usebackq" %%a in (`...`) do (
    set z=%%a
    set z=!z:"=_!
    for /f "tokens=1-3 delims=_" %%a in ("!z!") do echo %%b
)

A little explanation... the first for loop gets the entire line into the %a variable. This is then copied into variable z. z is then set again using sets' built-in search/replace functionality (note that here we're referencing the variable using !z:"=_!, which does the replacement). Finally we parse this single line to get the item between the quotes.

I hope that makes some kind of sense.

Edible

A quick bit of googling brought up this link which does something similar, but replacing the quote with !, which may be of use: wiert.wordpress.com/2010/09/02/… – Edible

Yes, that makes sense. I had adapted Joey's approach using substitutions of " and " to the empty string, which is simpler but a bit less robust against whitespace changes. – Striation

The OP marked this as the answer, but it's wrong. As explained in two instances below, there is a way: using ^ to escape the equals sign, double quotes, and the space in-between tokens and delims. And by doing so, you can remove the double quotes around the entire tokens/delims parameter. – Racy

Answer 4

I haven't found a way for that to be possible. Maybe jeb chimes in with more in-depth knowledge than I have. Alternatively, chop up the line using = and space as delimiters and just remove the quotes around the result:

for /f "tokens=3 usebackq delims== " %G in (`...`) do @echo %~G

Botts

That looks like a promising approach, but the value I'm trying to get could potentially have spaces in it. I'll try going this route with delims==? and then trimming the space before using %~. – Striation

That was the intention ;-). Though I can't think of other or more robust approaches right now than either replace quotes and split afterwards (icabod's solution) or split around the quotes and remove them – Botts

Answer 5

I think that it is mostly easier to search for the characters that surround the quotes and strip the quote off in a later step. If we want to extract the values from a certain line in an XML file

<line x0="745" y0="1162" x1="1203" y1="1166"/>

We proceed like this

SETLOCAL ENABLEDELAYEDEXPANSION
FOR /F "tokens=3,5,7,9 delims==/ " %%i IN ('FINDSTR line %1') DO (
SET x0=%%~i
SET y0=%%~j
SET x1=%%~k
SET y1=%%~l
)

In general, quotes are no real delimiters for themselves, so this will do the trick in most cases.

Legume

Answer 6

I recently had as issue similar to this. The examples in the answers are overly complex and hard to read. I ended up wrapping the command and its functionality into another CMD script and then calling it from the FOR /F. Here is an example of the command:

wmic fsdir where name="C:\\some\\path\\to\\a\\folder" get creationdate

The path was extracted and passed in as a variable and the output captured and set in the DO section for the FOR /F of the calling script. This lead to a more readable approach and reduced complexity.

Hopes this helps someone in the future.

Unreason

Answer 7

Just avoid the double quote using ^ to escape all characters in the string (including spaces). This way you can add the double quote as parameter.

for /F Tokens^=1^,2^-5^*^ Delims^=^" %%i in ( ...

This should work.

Eiger