executor Questions
2
Spark memory overhead related question asked multiple times in SO, I went through most of them. However, after going through multiple blogs, I got confused.
Below are the questions I have
whether ...
Shrewmouse asked 24/8, 2020 at 12:39
17
Solved
I want to create a ThreadPoolExecutor such that when it has reached its maximum size and the queue is full, the submit() method blocks when trying to add new tasks. Do I need to implement a custom ...
Varityper asked 4/1, 2010 at 17:59
11
I am trying to execute lots of tasks using a ThreadPoolExecutor. Below is a hypothetical example:
def workQueue = new ArrayBlockingQueue<Runnable>(3, false)
def threadPoolExecutor = new Thre...
Surra asked 10/8, 2010 at 4:24
5
Solved
Our airflow installation is using CeleryExecutor.
The concurrency configs were
# The amount of parallelism as a setting to the executor. This defines
# the max number of task instances that shoul...
2
Solved
I'm new to Spark on YARN and don't understand the relation between the YARN Containers and the Spark Executors. I tried out the following configuration, based on the results of the yarn-utils.py sc...
Kokoschka asked 12/7, 2016 at 14:19
12
Solved
Say I have a queue full of tasks which I need to submit to an executor service. I want them processed one at a time. The simplest way I can think of is to:
Take a task from the queue
Submit it to...
8
Solved
Is there a possibility to set priority to tasks which are executed by Executors? I've found some statements in JCIP about it's possible but I cannot find any example and I cannot find anything rela...
Magma asked 7/7, 2010 at 20:46
2
I'm starting to be comfortable with Java CompletableFuture composition, having worked with JavaScript promises. Basically the composition just scheduled the chained commands on the indicated execut...
Hagioscope asked 5/3, 2021 at 16:0
3
Solved
On API level 28(Pie) a new method is introduced in the Context class to get Executor for the main thread getMainExecutor().
How to get this executor on API level below 28?
Hulen asked 4/10, 2018 at 8:26
1
Solved
Can someone please explain the difference between Kotlin Coroutine's ExecutorCoroutineDispatcher and CoroutineDispatcher from practical point of view, i.e. in which scenarios to use one against ano...
Hexagram asked 14/11, 2019 at 10:36
3
I have a problem with my JVM Running on a CentOS 6.0 with openJDK 1.7.0_51 64Bit.
My System is a 4-Core System with 8GB Ram.
I'm running a Java multithread application that I wrote myself. It's su...
Selfinduction asked 22/1, 2014 at 17:35
7
I've searched a lot but could not find a solutuion to my problem.
I have my own class, BaseTask, that uses a ThreadPoolExecutor to handle tasks. I want task prioritization, but when I try to use ...
Debose asked 23/8, 2010 at 7:52
3
Solved
My Java application works on music files within folders, it is designed to process multiple folders in parallel and independently. To do this each folder is processed by an ExecutorService that has...
Octant asked 16/6, 2019 at 7:53
1
Solved
I am wondering if there is any reason to use an Executor instead of an ExecutorService.
As far as I know there is no implementation of the Executor interface in the JDK which is not also an Execut...
Neilson asked 8/3, 2019 at 8:40
3
Solved
What is the low-level difference among using:
ForkJoinPool = new ForkJoinPool(X);
and
ExecutorService ex = Executors.newWorkStealingPool(X);
Where X is the desired level of parallelism, i.e. thre...
Violetavioletta asked 27/12, 2016 at 0:2
3
I'm creating an ExecutorService to execute some tasks which under normal conditions is expected to take one minute to complete but under no circumstances should be allowed to run for more than two ...
Chloramine asked 7/11, 2018 at 19:23
2
Solved
I am developing some code that will eventually be multithreaded, using a thread pool Executor. The tasks executed by the thread pool will make call-backs and (sometimes) submit further tasks to the...
Subsolar asked 6/11, 2018 at 10:23
10
Solved
i have this rather simple question about the ThreadPoolExecutor. I have the following situation: I have to consume objects from a queue, create the appropiate worker tasks for them and submit them ...
Bield asked 5/11, 2009 at 10:16
2
Solved
In spark2.0, I have two dataframes and I need to first join them and do a reduceByKey to aggregate the data. I always got OOM in executor. Thanks in advance.
Data
d1 (1G, 500 million rows, cached...
Oakland asked 28/10, 2016 at 16:19
3
Solved
I'm new to multi-threading in Python and am currently writing a script that appends to a csv file. If I was to have multiple threads submitted to an concurrent.futures.ThreadPoolExecutor that appen...
Chiro asked 13/10, 2015 at 15:43
2
I'm running parallel tests with Jenkins.
The way I have it set up is I have a build flow job that executes three other jobs, in parallel. The three other jobs are connected to separate Test XML fi...
Oppugn asked 29/6, 2016 at 8:24
5
Solved
Below is the snippet of the pseudo code.
Does the below code not defeat the very notion of parallel asynchronous processing?
The reason I ask this is because in the below code the main thread would...
Til asked 27/6, 2015 at 18:27
2
Solved
I wrote a Spark application, which sets sets some configuration stuff via SparkConf instance, like this:
SparkConf conf = new SparkConf().setAppName("Test App Name");
conf.set("spark.driver.cores...
Lechery asked 4/5, 2017 at 13:49
1
Solved
I'm working on a project where the following line is used to create a test Executor member variable instance:
private Executor executor = Runnable::run;
The code runs and compiles but I don't und...
Winger asked 6/10, 2017 at 13:24
2
Solved
I would like to start a blocking function in an Executor using the asyncio call loop.run_in_executor and then cancel it later, but that doesn't seem to be working for me.
Here is the code:
import...
Winger asked 16/10, 2014 at 20:47
1 Next >
© 2022 - 2024 — McMap. All rights reserved.