I have a strange issue that comes and goes randomly and I really can't figure out when and why. I am running a snakemake pipeline like this: conda activate $myEnv snakemake -s $snakefile --config...

I'm testing a simple MPI program on my desktop (Ubuntu LTS 16.04/ Intel® Core™ i3-6100U CPU @ 2.30GHz × 4/ gcc 4.8.5 /OpenMPI 3.0.0) and mpirun won't let me use all of the cores on my machine (4). ...

I am using slurm on a cluster to run jobs and submit a script that looks like below with sbatch: #!/usr/bin/env bash #SBATCH -o slurm.sh.out #SBATCH -p defq #SBATCH --mail-type=ALL #SBATCH [email...

In SYCL, there are three types of memory: host memory, device memory, and Unified Shared Memory (USM). For host and device memory, data exchange requires explicit copying. Meanwhile, data movement ...

Consider massiveley SIMD-vectorized loops on very large amounts of floating point data (hundreds of GB) that, in theory, should benefit from non-temporal ("streaming" i.e. bypassing cache...

I'm trying to set up a remote environment (PyCharm preferred, VSCode also okay) so I can run and debug all my experiments on a compute node in a SLURM-managed cluster. The issue is that I have to r...

I'm attempting to benchmark the memory bandwidth on a ccNUMA system with 2x Intel(R) Xeon(R) Platinum 8168: 24 cores @ 2.70 GHz, L1 cache 32 kB, L2 cache 1 MB and L3 cache 33 MB. As a reference, ...

Trying to run slurmd: sudo systemctl start slurmd I display the status of the daemon and an error is displayed on the screen: >>sudo systemctl status slurmd ● slurmd.service - Slurm node dae...

I am using Nvidia's HPC compiler nvc++. Is there a way to detect that the program is being compile with this specific compiler and the version? I couldn't find anything in the manual https://docs.n...

My python program prepares inputs, runs an external FORTRAN code, and processes the outputs in a Windows HPC 2008 environment. It works great, unless the code executes the external program between ...

I've come across several instances where it would be really helpful to containerize a conda environment for long-term reproducibility. As I'm normally running in high-performance computing systems,...

According to this Quora forum, One of the simplest rules of thumb is to remember that hardware loves arrays, and is highly optimized for iteration over arrays. A simple optimization for many probl...

I'm running the following job array on SLURM: #!/bin/bash #SBATCH --array=1-1000 #SBATCH --partition=scavenge #SBATCH --mem=2g #SBATCH --time=1:00:00 module load Python/3.6.4-iomkl-2018a python r...

I submitted several jobs via SLURM to our school's HPC cluster. Because the shell scripts all have the same name, so the job names appear exactly the same. It looks like [myUserName@rclogin06 ~]$ ...

I'm currently trying to compile software for the use on a HPC-Cluster using Intel compilers. The login-node, which is where I compile and prepare the computations uses Intel Xeon Gold 6148 Processo...

I have the following Chapel code. proc update(x: int(32)) { return 2*x; } proc dynamics(x: int(32)) { return update(x); } writeln(dynamics(7)); I would like to send some kind of callback to ...

When I run singularity exec foo.simg whoami I get my own username from the host, unlike in Docker where I would get root or the user specified by the container. If I look at /etc/passwd inside thi...

I've written a python script that requires two arguments and works just fine when I run it on the command line with: pythonscript.py arg1 arg2 I need to run this in a SLURM batch script, but wh...

I seek to parallize an R file on an SLURM HPC using the future.batchtools packages. While the script is executed on multiple nodes, it only use 1 CPU instead of 12 that are available. So far, I t...

I would like to run a job on the cluster. There are a different number of CPUs on different nodes and I have no idea which nodes will be assigned to me. What are the proper options so that the job ...

i saw that some people use Kubernetes co-exist with slurm, I was just curious as to why you need kubernetes with slurm? what is the main difference between kubernetes and slurm?

I have a fairly complex computational code that I'm trying to speed up and multi-thread. In order to optimize the code, I'm trying to work out which functions are taking the longest or being called...

datatable is a relatively fresh high performance DataFrame/data.table alternative for Python. The datatable documentation states: It focuses on: big data support, high performance, both in-memory ...

I want to do block matrix-matrix multiplication with the following C code.In this approach, blocks of size BLOCK_SIZE is loaded into the fastest cache in order to reduce memory traffic during calcu...

I am optimising an MPI code and I am working with Gprof. The problem is that the results I obtained are completely unreasonable. My workflow is the following: compiling the code adding -pg as a ...