How do I create a temporary directory using mktemp in my current working directory?

Question

I created a pipeline that strings multiple programs together, unfortunately these programs are creating a huge amount of temporary files in the /tmp folder and when using large datasets my pipeline crashes because the /tmp folder fills up.

How do I export temporary files so that they are created in my current working directory where the pipeline is being run and not in the /tmp folder?

Currently I have tried to export the TMPDIR env variable to an already created directory /work in my current working directory, but the temporary files are still being created in the /tmp folder:

export TMPDIR=$(mktemp -d --tmpdir=/work)
<script>
rm -rf $TMPDIR

The programs do not have the option to set different output folders for temporary files created.

Datha

What error message do you get? Please always mention error messages when they are available. – Ichthyo

The error message for the pipeline? It's a typical IOError: No space available on device error that's very common when the /tmp file is full. – Datha

How much space is left on your hard drive (i.e. df -h . )? – Ichthyo

I have two 4TB hard-drives where I do most of the scratch work. One is completely empty and the other is 50% full. My /tmp directory only has 9gb of free space which is where the problem stems from. – Datha

Answer 1

Just change /work to work if the directory work is in your current directory. /work means that you have a top-level directory named /work. Without the forward slash, it will be a relative directory.

I just tested this code on my computer. No files were written to /tmp that I noticed:

mkdir work
export TMPDIR=$(mktemp -d --tmpdir=work)
ls work
# tmp.AWA4dTERha
rm -rf $TMPDIR
ls work
# --no output--

Ichthyo

Great, let me give this a shot! – Datha

Doesn't seemed to have solved the problem. Temporary files still being created in the /tmp directory. – Datha

@Datha Just updated my answer to include code that I tested. – Ichthyo