What's the meaning of the &, @ etc. operators in JOLT? [closed]

The answer is: look here and here

Correct me if I'm wrong, but: Wow, currently to my knowledge a single .java file on GitHub, last commit in 2017, holds relevant parts of the official documentation of the JOLT syntax. I had to use its syntax since I'm working with NiFi and applied its JoltTransformJSON processor (hence the SEO abuses in my question, so more people find the answer)

Here are some of the most relevant parts copied from https://github.com/bazaarvoice/jolt/blob/master/jolt-core/src/main/java/com/bazaarvoice/jolt/Shiftr.java and slightly edited. The documentation itself is more extensive and also shows examples.

'*' Wildcard

Valid only on the LHS ( input JSON keys ) side of a Shiftr Spec

The '*' wildcard can be used by itself or to match part of a key.

'&' Wildcard

Valid on the LHS (left hand side - input JSON keys) and RHS (output data path)

Means, dereference against a "path" to get a value and use that value as if were a literal key.

The canonical form of the wildcard is "&(0,0)".

The first parameter is where in the input path to look for a value, and the second parameter is which part of the key to use (used with * key).

There are syntactic sugar versions of the wildcard, all of the following mean the same thing; Sugar : '&' = '&0' = '&(0)' = '&(0,0)

The syntactic sugar versions are nice, as there are a set of data transforms that do not need to use the canonical form, eg if your input data does not have any "prefixed" keys.

'$' Wildcard

Valid only on the LHS of the spec.

The existence of this wildcard is a reflection of the fact that the "data" of the input JSON, can be both in the "values" and the "keys" of the input JSON

The base case operation of Shiftr is to copy input JSON "values", thus we need a way to specify that we want to copy the input JSON "key" instead.

Thus '$' specifies that we want to use an input key, or input key derived value, as the data to be placed in the output JSON.

'$' has the same syntax as the '&' wildcard, and can be read as, dereference to get a value, and then use that value as the data to be output.

There are two cases where this is useful

when a "key" in the input JSON needs to be a "id" value in the output JSON, see the ' "$": "SecondaryRatings.&1.Id" ' example above.

you want to make a list of all the input keys.

'#' Wildcard

Valid both on the LHS and RHS, but has different behavior / format on either side.

The way to think of it, is that it allows you to specify a "synthentic" value, aka a value not found in the input data.

On the RHS of the spec, # is only valid in the the context of an array, like "[#2]".

What "[#2]" means is, go up the three levels and ask that node how many matches it has had, and then use that as an index in the arrays.

This means that, while Shiftr is doing its parallel tree walk of the input data and the spec, it tracks how many matches it has processed at each level of the spec tree.

This useful if you want to take a JSON map and turn it into a JSON array, and you do not care about the order of the array.

On the LHS of the spec, # allows you to specify a hard coded String to be place as a value in the output.

The initial use-case for this feature was to be able to process a Boolean input value, and if the value is boolean true write out the string "enabled". Note, this was possible before, but it required two Shiftr steps.

'@' Wildcard

Valid on both sides of the spec.

The basic '@' on the LHS.

This wildcard is necessary if you want to put both the input value and the input key somewhere in the output JSON.

Thus the '@' wildcard is the mean "copy the value of the data at this level in the tree, to the output".

Advanced '@' sign wildcard

The format is lools like "@(3,title)", where "3" means go up the tree 3 levels and then lookup the key "title" and use the value at that key.

Thanks to https://stackoverflow.com/a/67821482/ (Barbaros Özhan, see comments) for pointing me into the correct direction.

Recommended topics

Hot tags