I'm trying to run a storm topology using intellij. Up until today, the topology ran perfectly fine but today everytime I try to run it I keep getting the following error: Error:maven-resources-te...

My project consists of some dependencies which are pulling the same common dependency. The common dependency storm-kafka has a new version 1.0.2 and an old version 0.10.0 On building a shaded jar...

I'm studying distributed systems and referring to this old question: stackoverflow link I really can't understand the difference between exactly-once, at-least-once and at-most-once guarantees, I ...

I deployed a long running Storm topology. After several hours running, the whole topology went down. I checked worker logs, and found these logs . As it says, zookeeper client session timed out and...

I have created a Storm topology which connects to Redis Cluster using Jedis library. Storm component always expects that Redis is up and running and only then it connects to Redis and subscribes th...

What would be the correct way to submit only the highest offset of every partion when batch bolt finishes proccessing a batch? My main concern is machines dying while proccessing batches as the who...

I Have Storm 1.1.1 installed on my machine and on my other machine I'm using Kafka version 0.10.0.1. Both services are connected with Zookeeper version 3.4.6 I succesfully deployed my topology, tha...

I got this error when I tried to run topology in local mode with storm using mvn compile exec:java -Dexec.classpathScope=compile -Dexec.mainClass=my.Topology the error is ERROR backtype.storm....

numpy.array.tostring doesn't seem to preserve information about matrix dimensions (see this question), requiring the user to issue a call to numpy.array.reshape. Is there a way to serialize a num...

Is there a clean way to stop storm without killing it with kill XXX where XXX is the PID? I run storm kill topology-name to kill the topology, but after that, is there a clean way to shutdown worke...

I've read some article, benchmarking the performance of stream processing engines like Spark streaming, Storm, and Flink. In the evaluation part, the criterion was 99th percentile and throughput. F...

Apache Kafka: Distributed messaging system Apache Storm: Real Time Message Processing How we can use both technologies in a real-time data pipeline for processing event data? In terms of real t...

Flink has been compared to Spark, which, as I see it, is the wrong comparison because it compares a windowed event processing system against micro-batching; Similarly, it does not make that much se...

I installed Apache Storm 1.0 by following this tutorial but I am not able to access to the Storm UI from the Internet. Accessing localhost:8080 gives the following error: org.apache.storm.utils.Ni...

When following the guide in the storm-starter repo, I'm unable to actually run any topology- like the ExclamationTopology. mvn clean install -DskipTests=true ran successfully, executed from the to...

I work with Storm for a while already, but want to get started with development. As suggested, I am using IntelliJ (up to now, I was using Eclipse and did only write topologies against Java API). ...

I'm having the following situation: There is a number of bolts that calculate different values This values are sent to visualization bolt Visualization bolt opens a web socket and sends values to...

How to store the temporary data in Apache storm? In storm topology, bolt needs to access the previously processed data. Eg: if the bolt processes varaiable1 with result as 20 at 10:00 AM. and a...

In one of our topologies containing 1 spout and 1 bolt - I had a hunch that bolts are completing fine (and acking) but spouts are still failing. I tried to confirm this by having a TaskHook as belo...

We’re using Storm with the Kafka Spout. When we fail messages, we’d like to replay them, but in some cases bad data or code errors will cause messages to always fail a Bolt, so we’ll get into an in...

I'm very naive about data engineering but it seems to me that a popular pipeline for data used to be Kafka to Storm to something.... but as I understand it Kafka now seems to have data proces...

I was looking for how to kill a topology from a storm bolt in storm 1.2.1 in my scala project. From this answer, I wrote following code to do it: private def shutTopology(){ import org.apache.sto...

There are a lot of questions comparing Flink vs Spark Streaming, Flink vs Storm and Storm vs Heron. The origin of this question is from the fact that both Apache Flink and Twitter Heron are true s...

I have an kafka environment which has 3 brokers and 1 zookeeper. I had pushed around >20K message in my topic. Apache Storm is computing the data in topic which is added by producer. After few hou...

I want to turn off the logging feature offered by default when we run from local cluster. Currently its logging so many information on the console. Below is the example of log: 261 [main] INFO b...