How to increase depth limit of jq?

Question

When trying to minify a deeply nested JSON file (~10k deep) using jq I got a parse error:

$ jq -c . <input.json >minified.json
parse error: Exceeds depth limit for parsing at line 227263, column 355

How can I increase the depth limit? Is there a command line option? I couldn't find anything about this in the man page.

Tevet

If you're just using jq -c to convert to compact form, rather than doing anything fancier, I'd think about switching to an alternate implementation; there's a native-Go alternative at github.com/itchyny/gojq f/e, or python -mjson.tool --compact may suffice. Could still potentially hit a max stack depth on either of those, but I'd start by testing. – Lexicostatistics

There's now a feature request issue for increasing depth from 256 where you can add your support: github.com/jqlang/jq/issues/2846 – Tevet

Answer 1

Workaround is to use another tool, e.g. jj, which does the job perfectly, and probably also faster in general, even if jq doesn't error.

jj -u <input.json >minified.json

Binaries available at: https://github.com/tidwall/jj/releases/latest or via homebrew:

brew install tidwall/jj/jj

Tevet

Answer 2

Regarding the C implementation, you'd have to remake the binary after changing MAX_PARSING_DEPTH in jv_parse.c, in which the relevant line is currently:

#define MAX_PARSING_DEPTH (256)

As best I can tell, there does not seem to be any downside to increasing the limit to a very large number.

Bosworth

Thanks for looking into the source code! Would be nice if this was surfaced as a CLI option. – Tevet

Answer 3

gojq (the Go implementation of jq) has a limit on the maximum depth it can handle: it's greater than 1,024 but less than 10,000.

I've verified that jj and my "JSON Machine"-based utility, jm, can both handle a depth of 10,000. The following are equivalent ways of compactifying a JSON document, at least if the limit isn't breached:

   jj -u @this < input.json
   jm input.json

Bosworth