Grep and Awk in Windows Invalid Char in Expression Error

Question

I am new to grep and awk - using Windows 7 (I downloaded grep and awk for windows from GnuWin).

I am have having trouble running this script:

grep -Fwf dictionary.txt frequency.txt | awk '{print $2 "," $1}'

I get the error:

awk: '{print
awk: ^ invalid char ''' in expression

I believe it might have something to do with having to use double quotes in Windows, but I tried all the combinations I can think of and still it doesn't work.

Can anyone help? Thanks

Karleenkarlen

How about "{print $2 "","" $1}"? On Windows you escape double-quotes with double-quotes. – Heteroclite

@Heteroclite That changed the error to: awk: {print $2 ", awk: ^ unterminated string (pointing to the quote). – Karleenkarlen

@Luke: On Windows each process parses itself its command line. With some C runtimes, double quotes are escaped with backslash. – Malaria

Try awk "{print $2 "","" $1}" if you are on the cmd command line – Suprematism

I would try awk "{print $2 ',' $1}". – Tien

Awk doesn't recognise ' for strings :-( – Suprematism

Answer 1

Escaping command line items is always a pain on Windows. As a last resort you could probably use gawk -f!

So: your file script.awk contains:

print $2,$1

And you do grep -Fwf dictionary.txt frequency.txt | awk -f script.awk

Malaria

@Malaria Can you help me out with what the actual command line would be (I tried grep -Fwf dictionary.txt frequency.txt | gawk -f "'"{print $2 "","" $1}"'" and I get the error:gawk: fatal: can't open source file `'{print' for reading (No such file or directory) but print shouldn't be a file....)? Thanks, I'm new at this. – Karleenkarlen

gawk -f expect a filename. The file contains the awk script. – Malaria

@Malaria Ok, got the awk script in a file - it still doesn't like the format of the awk script though...I tried awk "{print $2 "","" $1}" in the text file that is called by gawk -f and it almost seems like it goes through, but the frequency.txt is not changed...is there a way to tell if anything happened? – Karleenkarlen

@Nathan: What happens if you just omit the comma, as in {print $2 $1}? What happens if you change from print to printf("%s, %s\n", $2, $1); – Salman

I got it to work - working with a bash script as well and Notepad ++ need to encode the file in UNIX (Edit->EOL Conversion->UNIX). Thanks so much for your help! I ended up using Cygwin as well. Thanks so much! – Karleenkarlen

Answer 2

On Windows, you need to use double quotes to quote your awk commands. So

grep -Fwf dictionary.txt frequency.txt | awk '{print $2 "," $1}'

needs to be changed to

grep -Fwf dictionary.txt frequency.txt | awk "{print $2 "," $1}"

But remember, you can't use double quotes inside double quotes, you need to escape them. On Windows you can't simply use \ to escape it. You need the following syntax:

grep -Fwf dictionary.txt frequency.txt | awk "{print $2 \"",\"" $1}"

Right, that's \"" to represent " inside double quotes.

Jansenism

Actually neither of these will work on windows. it should be awk "{print $2 \",\" $1}" – Anguiano

@DavidArenburg is correct. You need to escape the internal double quotes on windows command line. – Marcmarcano

OR, you could use the MINGW version of gawk where apostrophe / single quote works. – Emaemaciate

Answer 3

You need to use double quotes around your awk script and escape the embedded quotes in the print statement using a good old backslash: [g]awk "BEGIN {print \"Hello escape char!\"}"

Scout

Answer 4

Here is a short example which will accept an input.csv , then output new.csv:

gawk < input.csv -F, "{print $1 \"",\"" $5} ">new.csv

Surgery

Answer 5

Since the brackets must be within double quotes, three sets of double quotes are required inside the bracketed expression.

For example:

gawk "{print $2 """,""" $1}"

Caliginous

Answer 6

I have stuggled over the years to get AWK running under windows. There are problems with quoting and path delimiters. My final solution is to "let AWK fly free", that is free from the command line. I understand that it was developed as a glue for unix style command line juju, but I just wanted to use it as a scripting language.

All my AWK scripts contain a list of targets and a defined output file. They can be run by double clicking through an associated DOS batch file:

: AWK.BAT - place in the same directory as GAWK
@echo off

:Check %1 in not null
If [%1]==[] (
    cls
    Echo No parameters passed
    goto End
)

: Change to the parameter file location
cd /D "%~dp1"

: Set PrintFile - this will be the name of the script (not the target file) with ".out"
Set PrintFile=%~nx1.out

:Run AWK
:   -v PrintFile to allow renaming of output file
:   -f ScriptFile.awk the program
:   > Redirects output to a known destination
cls
P:\MyPrograms\EDITORS\Addins\gawk\gawk.exe  -v PrintFile=%PrintFile% -f %* >%PrintFile%

:End
pause

An example of my AWK scripts is presented below (extract all lines with ::tab and print them):

# AWK Template

BEGIN{
    ## Hard Code Target Files - Unix paths with / separators ##
    #   Realtive paths from the location of ScriptFileName.awk
    #   These will be added to the end of the ARG array - after any command line target files
    AddTarget("../APEdit.ahk")

    ## Hard Code Output Files - WinDos paths with \\ separators ##
    #   Realtive paths from the location of ScriptFileName.awk
    #   Default is ScriptFileName.awk.out passed in as a variable called PrintFile
    #   PrintFile will be copied to OutputFile after processing using the END section
    OutputFile = "Keys.txt"

    # Set input record sep and field sep
    RS="\n"
    FS=" "

    # Set output RS and FS
    ORS="\n"
    OFS=" " 

    # Write a header
    print "Key assignments from the source code"
    print " "
}

## MIDDLE - Once per matching record! ## 

# Find autohotkey key definitions
/::\t/ { 
    print $0
}

END{

    ## Rename output files
    if (OutputFile) {
        system("Copy /Y " PrintFile "  " OutputFile)
    }
}

## Functions ##
function AddTarget(FN){
    # Need to check file exists
    if (FileExists(FN)){
        ARGV[ARGC] = FN
        ARGC ++
    }
}

function FileExists(FN) {
    if ((getline < FN) > 0) {
        close(FN);
        return 1
    } else {
        print "Target file not found " FN > "error.awk.txt"
        return ""
    }
}

You can see this defines the input target within the script and defines the final output target within the script. It uses a temp ".out" file to avoid lots of print redirection, copying the file to the desired output in the END section of the script.

I have associated AWK files with this batch file and added an option in my editor to send AWK files to the batch file.

Kind Regards

Aglimmer

Answer 7

Yousui said:

... you can't use double quotes inside double quotes, you need to escape them. On Windows you can't simply use \ to escape it.

while ReluctantBIOSGuy has used just \" in his example.

I tried both \"" and \", and both works for me (gawk under Windows XP both on the command line and in a batch file).

Here is an example involving " in the output (a string literal in c code):

FCIV\fciv -add ..\07-Sources -type *.h -type *.c -bp ..\07-Sources | find /V "//" | sort /+33 > listof.md5
FCIV\fciv -add listof.md5 | find /V "//" | gawk "{print \"static const char md5[] = \\\"\" $1 \"\\\";\"}" > ..\07-Sources\generated\md5.c

Belongings