How do I pass a single string with multiple arguments to std::process::Command?

Question

Rust's std::process::Command type demands that process arguments be passed in individually via .arg("-arg1").arg("-arg2") or as a vector of strings via .args(&["-arg1", "-arg2"]). How do you split a string into a vector that can be passed as arguments?

Wisnicki

Answer 1

What shells do when splitting a command-line string into arguments is far from trivial, especially when you want to handle things like quoting. For example, your code should pass the following assertions:

assert_eq!(split(r#""foo\"bar""#), vec!["foo\"bar"]);
assert_eq!(split(r#""foo"#), vec!["\"foo"]);          // Or error

Unless you think simply splitting on whitespace is enough for your use-case, you should really use a crate like shell-words or shlex. shlex presents the advantage that it can return an iterator, thus avoiding unnecessary allocations, but on the other hand it makes it easy to miss the error in the second test above:

extern crate shell_words;
extern crate shlex;

use shell_words::split;
use shlex::Shlex;

fn main() {
    assert_eq!(split(r#"a b"#).unwrap(), vec!["a", "b"]);
    let mut lex = Shlex::new(r#"a b"#);
    assert_eq!(lex.by_ref().collect::<Vec<_>>(), vec!["a", "b"]);
    assert!(!lex.had_error);    // ← Don't forget this check!

    assert_eq!(split(r#"a "b c""#).unwrap(), vec!["a", "b c"]);
    let mut lex = Shlex::new(r#"a "b c""#);
    assert_eq!(lex.by_ref().collect::<Vec<_>>(), vec!["a", "b c"]);
    assert!(!lex.had_error);    // ← Don't forget this check!

    assert_eq!(split(r#""foo\"bar""#).unwrap(), vec!["foo\"bar"]);
    let mut lex = Shlex::new(r#""foo\"bar""#);
    assert_eq!(lex.by_ref().collect::<Vec<_>>(), vec!["foo\"bar"]);
    assert!(!lex.had_error);    // ← Don't forget this check!

    assert!(split(r#""foo"#).is_err());
    // assert_eq!(Shlex::new(r#""foo"#).collect::<Vec<_>>(), vec!["\"foo"]);

    let mut lex = Shlex::new(r#""foo"#);
    lex.by_ref().for_each (drop);
    assert!(lex.had_error);     // ← Don't forget this check!
}

Mweru

I really like how the syntax highlighting is the best example how hard proper escaping of strings is. The assert_eq! lines do not look right – Sneak

Answer 2

Implementation which does not support quoted arguments (but easy to add):

fn sh(command: &str) -> std::io::Result<std::process::Output> {
    let mut the_args = command.split(' '); // todo: support quoted strings
    let first: &str = the_args.next().unwrap();
    let rest: Vec<&str> = the_args.collect::<Vec<&str>>();
    std::process::Command::new(first).args(rest).output()
}

fn main() {
    let output = sh("ls -la").unwrap(); 
    let s = String::from_utf8_lossy(&output.stdout).to_string();
    println!("{:?}", s);
}

You have to do quite a bit of song and dance with iterators and string conversions. This tripped me up for a few days. I hope someone can chime in with a basic parser that handles quoted argument strings :).

Wisnicki

Or you can use the shell-words crate to do this for you… – Mweru

@Mweru shell-words takes a vector of strings, not a string. I was specifically interested in a quick hacky way to pass in a string from command line and have it "just run" so I can get on with it. Can downvoters post a better solution to my own problem? – Wisnicki

@Jmb, consider posting an alternative answer? Ideally don't want too require an additional dependency, but looking at the split function it's non-trivial. std::env::args() seems to deal with platform-specific arguments via sys::args::args(), but not sure if it can be used. – Wisnicki

shell_words::split takes a &str, not a vector of strings. – Mweru

You can at least avoid creating the vector. – Proa