SLURM sbatch output buffering

Question

I created some slurm scripts and then tried to execute them with sbatch. But the output file is updated not frequently (once a minute maybe).

Is there a way to change the output buffering latency in sbatch? I know stdbuf is used in such situations but I could not make it work with sbatch.

Breastplate

see #25171263 – Nautical

I am aware of that question and answer. but it looks like the buffering of output in sbatch is not related to stdbuf. It looks like sbatch command has some internal buffering. – Breastplate

Sbatch just runs the shell stated on the first line of the script. Usually bash, so there is no buffering by Slurm itself. What exactly are you trying to run? – Nautical

if you use srun within sbatch you could try to use the -u option (u=unbuffered – Ortrude

Answer 1

The issue is certainly with buffering. If you are trying to run python code, add flush=True in print command like print(...,flush=True).

Monostich

A better alternative is python -u or something like sys.stdout.reconfigure(line_buffering=True, write_through=True). – Flashboard

@Flashboard Thanks! This was far easier and did the trick for me indeed. – Moravia