Python Argparse: Raw string input [duplicate]

Question

Apologies if this has been asked before, I did search for it but all hits seemed to be about python raw strings in general rather than regarding argparse.

Anyways, I have a code where the user feeds in a string and then this string is processed. However, I have a problem as I want my code to be able to differentiate between \n and \\n so that the user can control if they get a line break or \n appear in the output (respectively).

This in itself is quite simple, and I can get the logic working to check the string, etc. However, argparse doesn't seem to keep the input string raw. So if I were to write: Here is a list:\nItem 1 it gets parsed as Here is a list:\\nItem 1. As the exact same thing gets parsed if I were to replace \n with \\n in the input string, it becomes impossible to differentiate between the two.

I could include a bodge (for example, I could make the user enter say $\n for \n to appear in the output, or just \n for a line break). But this is messy and complicates the usage of the code.

Is there a way to ensure the string being parsed by argparse is raw? (I.e. if I enter \n it parses \n and not \\n)

Again, sorry if this has been asked before, but I couldn't find an answer and after over an hour of trying to find an answer, I'm out of ideas (bar the bodge). Cheers in advance for any and all help.

Example code (sorry if this doesn't work, not sure how best to do example code for argparse!):

import argparse

parser = argparse.ArgumentParser( description = 'Test.' )
parser.add_argument( 'text', action = 'store', type = str, help = 'The text to parse.' )

args = parser.parse_args( )

print( repr( args.text ) )

Canaigre

You need to separate what's handled by Python and what's handled by shell. argparse can only parse command line arguments passed to Python's process, but it cannot modify how these arguments are handled by shell. And some escaping may or may now happen there, e.g. python -c "import sys; print sys.argv" My\nMagic\nString prints different value in Windows' cmd.exe and different value in bash, even if same characters we're typed in prompt. – Organo

The answer is unrelated to argparse. – Instead

Ok, so by that do you mean that argparse is working how I want it to, however it's the terminal that's causing the issues (i.e. I've typed "\n" but it's told argparse "\\n")? Or am I misunderstanding you here? – Canaigre

Answer 1

Here's a possible solution to your problem:

import argparse

parser = argparse.ArgumentParser(description='Test.')
parser.add_argument('text', action='store', type=str, help='The text to parse.')

args = parser.parse_args()

print '-' * 80
raw_text = eval('"' + args.text.replace('"', '\\"') + '"')
print raw_text
print '-' * 80
print args.text

But be aware of one thing, eval really is dangerous

Kamerman

Sadly that doesn't seem to work. I think @Lukasz-rogalski might be correct (if I'm understanding him correctly that is) that it's actually the terminal that's the problem rather than argparse :( – Canaigre

@Canaigre Try to elaborate further because obviously I've tested my answer before posting – Kamerman

@BLP Sorry, I spoke poorly there, I didn't mean to say it doesn't work, rather when I used it, it didn't get the results I'm after. I think Lukasz has identified that it's the terminal side of things that is the issue. That is to say, if I write \\n in the terminal (to parse) what it gives to the parser is actually \n; so nothing can be done on the python side of things. I think it's a case of the terminal automatically ignoring the extra backslash. Thus, it looks like I need to use \\\n in the terminal to get it to give the parser \\n. – Canaigre

@Canaigre Your question asked Is there a way to ensure the string being parsed by argparse is raw? (I.e. if I enter \n it parses \n and not \\n), i think my answer gave you a possible solution. But now you say you didn't get the results you're after... I'm not gonna downvote your question because you're polite and newcomer in SO. One advice though, some people could get pissed off if you're not more precise next time ;) . Welcome to SO – Kamerman

@BLP, sorry about that, but in my defence, the question was the problem as I saw it. I was typing a string into the terminal, and when I looked at said string after going through argparse, it was modified (i.e. both \\n and \n became \n). Thus I assumed it was a case of argparse not parsing the raw string (only realising the terminal itself can/does alter inputs after Lukasz' comment). So I gave the best question I could at the time. I'm sorry the question ended up being more reflective of the problem I thought I had & not of the underlying problem. Thank you for your welcome however :) – Canaigre

Answer 2

As noted in the comments, argparse works with sys.argv, a list which is produced by the shell and Python interpreter.

With a simple argv echo script:

0928:~/mypy$ cat echo_argv.py
import sys
print(sys.argv)

I get (with a bash shell):

0929:~/mypy$ python echo_argv.py Here is a list:\nItem 1 
['echo_argv.py', 'Here', 'is', 'a', 'list:nItem', '1']
0931:~/mypy$ python echo_argv.py "Here is a list:\nItem 1 "
['echo_argv.py', 'Here is a list:\\nItem 1 ']
0931:~/mypy$ python echo_argv.py "Here is a list:\\nItem 1 "
['echo_argv.py', 'Here is a list:\\nItem 1 ']

argparse treats that argv as a list of strings. It does nothing to those strings, at least not with the default None type parameter.

Clericals

Answer 3

As @hpaulj has pointed out, that your problem is originated from the shell and the way sys.argv work. Your option is to handle the string with the escape characters that you got.

Take a look at this answer on SO: Process escape sequences in a string in Python . Basically, use string_escape or unicode_escape to process the string. It is better than hand-handle your string.

Dall