Currently, I have Nifi running on an edge node that has 4 cores. Say I have 20 incoming flow files and I give concurrent tasks as 10 for ExecuteStreamCommand processor, does it mean I get only conc...

Background I have a workflow that uses a FetchFile processor, some intermediary processors (e.g. RouteOnAttribute, HashContent, etc.), and then a PutFile processor. Throughout it all, the content...

I got a secure cluster NIFI with 3 nodes, configured with truststore.jks and keystore.jks In my invokeHTTP, i've set "StandardSSLContextService" with keystore and trustore for https. invokeHTTP ...

We are struggling with trying to figure out the best approach for updating processor configurations as a flow progresses through the dev, test, and prod stages. We would really like to avoid manipu...

I wanted to convert my JSON string to SQL statement by using ConvertJSONtoSQL processor. example: JSON string - {"cpuwait":"0.0","servernamee":"mywindows","cpusys":"5.3","cpuidle":"77.6","datee"...

I am planning to do a class project and was going through few technologies where I can automate or set the flow of data between systems and found that there are couple of them i.e. Apache NiFi and ...

In a NiFi flow, I want to read a JSON structure, split it, use the payload to execute a SQL query, and finally output each result in a JSON file. However I am having problems retrieving the value ...

I am aware of how to delete a single attribute in Nifi, but what is the proper syntax to delete multiple attributes? Do I have to use multiple UpdateAttributes transformations or is there a delimit...

I am developing new Nifi processor for my data flow. I make code changes in eclipse , creates new .nar file and copy it to Nifi lib for testing it. On ever nar update Nifi needs a restart which t...

I'm trying to get the attribute values from a JSON file using EvaluvateJsonPath processer in NiFi. Below is the sample JSON file {"widget": { "debug": "on", "window": { "title": "Sample Konfa...

New to NiFi! I'm using InvokeHTTP nifi processor (using this link as an example to implement my flow http://www.tomaszezula.com/2016/10/30/nifi-and-http-post-configuration/). I currently need to d...

Can someone explain in details how NiFi processors like GetFile or QueryDatabaseTable store the rows when the next processor is not available to receive or process any data? Would the data gets pip...

I'm trying to solve one task and will appreciate any help - links to documentation, or links to forums, or other FAQs besides https://cwiki.apache.org/confluence/display/NIFI/FAQs, or any meaningfu...

How to clear the cache of FetchDistributedMapCache processor in Apache NiFi? I tried deleting the persisted directory and also tried giving a new directory all together but it still fetches old da...

What are the use cases for Apache Beam and Apache Nifi? It seems both of them are data flow engines. In case both have similar use case, which of the two is better?

I'm processing a file and I'd like to remove (trim) the first X header lines to keep only data, possibly avoiding using regular expressions. Thanks

I'm working on a flow where I get CSV files. I want to put the records into different directories based on the first field in the CSV record. For ex, the CSV file would look like this country,fir...

I'm learning Hadoop/Big data technologies. I would like to ingest data in bulk into hive. I started working with a simple CSV file and when I tried to use INSERT command to load each record by reco...

In Apache NiFi, using FetchS3Object to read from an S3 bucket, I see it can reads all the object in bucket and as they are added. Is it possible: To configure the processor to read only objects a...

What's the general recommendation for running Zookeeper in a clustered NiFi? Use the embedded or external Zookeeper? Starting from scratch so no existing Zookeeper running....

I am trying to stop the processor which is in running state using the PUT method in rest api /processors/{id}. I am able to start the processor by changing the state in the component as follows "...

Im using NIFI and i want to extract attributes of my file lines . My Filetext looks like this : DEV=A9E ,SEN=1 DEV=B9E ,SEN=2 ... And i want to split text by line and then extract dev and sen t...

Im new to Apache Nifi , and I'm having a use case which i need to parse and decode different kind of messages from Sensors, transform and load the data in Hbase all my sensors send data every 10 mi...

I am using Apache NiFi to copy some local files to HDFS. I've created a GetFile processor bound to a PutHDFS processor. The GetFile processor recursively queries a read-only directory. The problem ...

We are creating some flows in NiFi and there might be some cases where the queues are being build up but due to some reason the flow doesn't work as expected. At the end of the day, i would like ...