NiFi from overwriting values in nifi.properties

Question

I am running NiFi in docker with all relevant directories mounted as volumes. I am attempting to modify some settings in my nifi.properties file, specifically to add a custom properties file. However, when I restart NiFi, some of the properties are reverted to their original values.

Here is an example of my current nifi.properties file:

nifi.ui.autorefresh.interval=5 sec
...
nifi.variable.registry.properties=

If I then change the file to the following:

nifi.ui.autorefresh.interval=3 sec
...
nifi.variable.registry.properties=./conf/custom.properties

and then restart NiFi, it prints several lines of replacing target file /opt/nifi/nifi-current/conf/nifi.properties, and then starts the UI. When I check the nifi.properties file again, it looks like:

nifi.ui.autorefresh.interval=3 sec
...
nifi.variable.registry.properties=

For some reason, the nifi.ui.autorefresh.interval property will update successfully, but the nifi.variable.registry.properties property does not.

Why are some values refusing to take, and how can I get them to survive the startup process?

Pothole

maybe you running nifi in docker? – Pressing

@Pressing Yes I am, and the conf directory is mounted into the container – Pothole

Docker does not store the status. So, when you restart it, all inside docker returned to original state. Only mounted directories keep their statuses as soon as they located out of docker. – Pressing

@Pressing The directories are mounted, and since some settings in the same file are successfully saved, I do not believe a mount point is the issue – Pothole

Answer 1

There are some props which can be set only with ENV vars (beside hacking). If you look at the command bellow you can figure it out. As you can see the nifi.variable.registry.properties is one of them.

cat /opt/nifi/scripts/start.sh | grep prop_replace
prop_replace 'nifi.web.http.port'               "${NIFI_WEB_HTTP_PORT:-8080}"
prop_replace 'nifi.web.http.host'               "${NIFI_WEB_HTTP_HOST:-$HOSTNAME}"
prop_replace 'nifi.remote.input.host'           "${NIFI_REMOTE_INPUT_HOST:-$HOSTNAME}"
prop_replace 'nifi.remote.input.socket.port'    "${NIFI_REMOTE_INPUT_SOCKET_PORT:-10000}"
prop_replace 'nifi.remote.input.secure'         'false'
prop_replace 'baseUrl' "http://${NIFI_WEB_HTTP_HOST:-$HOSTNAME}:${NIFI_WEB_HTTP_PORT:-8080}" ${nifi_toolkit_props_file}
prop_replace 'nifi.variable.registry.properties'    "${NIFI_VARIABLE_REGISTRY_PROPERTIES:-}"
prop_replace 'nifi.cluster.is.node'                         "${NIFI_CLUSTER_IS_NODE:-false}"
prop_replace 'nifi.cluster.node.address'                    "${NIFI_CLUSTER_ADDRESS:-$HOSTNAME}"
prop_replace 'nifi.cluster.node.protocol.port'              "${NIFI_CLUSTER_NODE_PROTOCOL_PORT:-}"
prop_replace 'nifi.cluster.node.protocol.threads'           "${NIFI_CLUSTER_NODE_PROTOCOL_THREADS:-10}"
prop_replace 'nifi.cluster.node.protocol.max.threads'       "${NIFI_CLUSTER_NODE_PROTOCOL_MAX_THREADS:-50}"
prop_replace 'nifi.zookeeper.connect.string'                "${NIFI_ZK_CONNECT_STRING:-}"
prop_replace 'nifi.zookeeper.root.node'                     "${NIFI_ZK_ROOT_NODE:-/nifi}"
prop_replace 'nifi.cluster.flow.election.max.wait.time'     "${NIFI_ELECTION_MAX_WAIT:-5 mins}"
prop_replace 'nifi.cluster.flow.election.max.candidates'    "${NIFI_ELECTION_MAX_CANDIDATES:-}"
prop_replace 'nifi.web.proxy.context.path'                  "${NIFI_WEB_PROXY_CONTEXT_PATH:-}"
prop_replace 'nifi.security.user.login.identity.provider' 'ldap-provider'

Sicular

Thank you so much. I wouldn't expect properties in a property file to be overwritten no matter what, so that's some weird behavior – Pothole

Answer 2

You can specify it in docker-compose.yaml as follows:

environment:
    - NIFI_VARIABLE_REGISTRY_PROPERTIES={PATH HERE}

Antigone