dup2 / dup - Why would I need to duplicate a file descriptor?

Question

I'm trying to understand the use of dup2 and dup.

From the man page:

DESCRIPTION

dup and dup2 create a copy of the file descriptor oldfd. After successful return of dup or dup2, the old and new descriptors may be used interchangeably. They share locks, file position pointers and flags; for example, if the file position is modified by using lseek on one of the descriptors, the position is also changed for the other.

The two descriptors do not share the close-on-exec flag, however. dup uses the lowest-numbered unused descriptor for the new descriptor.

dup2 makes newfd be the copy of oldfd, closing newfd first if necessary.

RETURN VALUE

dup and dup2 return the new descriptor, or -1 if an error occurred (in which case, errno is set appropriately).

Why would I need that system call? What is the use of duplicating the file descriptor? If I have the file descriptor, why would I want to make a copy of it? I'd appreciate it if you could explain and give me an example where dup2 / dup is needed.

Contraband

How would you implement the piping functionality of shells without dup or dup2 ? You need to call pipe(2) and then to have one of the file descriptors dup-ed to e.g. STDIN_FILENO – Bloat

Possible duplicate of practical examples use dup or dup2 – Cullis

Answer 1

The dup system call duplicates an existing file descriptor, returning a new one that refers to the same underlying I/O object.

Dup allows shells to implement commands like this:

ls existing-file non-existing-file > tmp1  2>&1

The 2>&1 tells the shell to give the command a file descriptor 2 that is a duplicate of descriptor 1. (i.e stderr & stdout point to same fd).
Now the error message for calling ls on non-existing file and the correct output of ls on existing file show up in tmp1 file.

The following example code runs the program wc with standard input connected to the read end of a pipe.

int p[2];
char *argv[2];
argv[0] = "wc";
argv[1] = 0;
pipe(p);
if(fork() == 0) {
    close(STDIN); //CHILD CLOSING stdin
    dup(p[STDIN]); // copies the fd of read end of pipe into its fd i.e 0 (STDIN)
    close(p[STDIN]);
    close(p[STDOUT]);
    exec("/bin/wc", argv);
} else {
    write(p[STDOUT], "hello world\n", 12);
    close(p[STDIN]);
    close(p[STDOUT]);
}

The child dups the read end onto file descriptor 0, closes the file de scriptors in p, and execs wc. When wc reads from its standard input, it reads from the pipe.
This is how pipes are implemented using dup, well that one use of dup now you use pipe to build something else, that's the beauty of system calls,you build one thing after another using tools which are already there , these tool were inturn built using something else so on .. At the end system calls are the most basic tools you get in kernel

Cheers :)

Lennyleno

Sodup is helpful for the caller and not the ls program itself? Is there any benefit of having dup used in program like ls itself if already has access to the file? Here for example, ls writes errors to 2 which is is hardcoded, so I have a way to over-ride it as a consumer of ls. I think thats a subtle point no? – Madonia

Your example program seems to have a bug; you're calling dup(p[STDIN]) but then throwing away the result. Did you mean to use dup2(p[STDIN], 0)? – Gand

@Gand dup returns the "lowest numbered descriptor currently not in use by the process." Since fd 0 was just closed, dup should return 0. dup2 is explicit about which fd should be used, instead of just using the lowest free fd, so I would prefer that. – Dortheydorthy

@Wodin: Ah, I bet you're right about what OP was thinking. Am I also right, though, that "just closed" is relative, and OP's code could break in the presence of, for example, concurrent threads that might be opening files too? – Gand

@Gand I suspect you're right, but don't know for sure. But in that case you'll have other problems. If you close stdin because you want to read from somewhere else and then another thread opens a file before you do, it will get fd 0. If you then use dup2, it will close the fd that the other thread opened, so the other thread will now be reading from (and writing to?) the file you open. Anyway, if I remember correctly you shouldn't call exec* from a multithreaded process. But I am no threading expert :) – Dortheydorthy

@Gand or maybe it was fork I was thinking of: linuxprogrammingblog.com/… – Dortheydorthy

@Dortheydorthy JFI: with dup2 you would not call close before. – Crib

Answer 2

Another reason for duplicating a file descriptor is using it with fdopen. fclose closes the file descriptor that was passed to fdopen, so if you don't want the original file descriptor to be closed, you have to duplicate it with dup first.

Insalubrious

fdopen() seems don't duplicate a file descriptor, it just create buffer in user space. – Pullulate

You misread my answer. The point is that you may want to dup the fd before passing it to fdopen since fclose will close it. – Insalubrious

Why would you call fclose if you don't want the file descriptor to be closed? Are you talking about just in the case of like a library function you don't control that calls fclose, and you would call dup before calling the library function to keep it open? – Vas

@theferrit32: If you allocate a FILE handle to access a preexisting open file via stdio interfaces, you need to call fclose to deallocate that FILE handle. If you want to keep using the underlying open file, or if your software architecture is such that the original "owner" code for the file descriptor will close it, the fact that fclose also closes the underlying file descriptor you handed to fdopen is a problem. You can avoid this problem by using dup to make a new file descriptor for the same open file to pass to fdopen, so that fclose does not close the original one. – Insalubrious

The point is that fdopen() moves ownership of the fd to the FILE, rather than copying it. That's something users should be aware of. Consumers that need to retain a usable fd handle in addition to the FILE object must duplicate the fd. That's all. – Saxecoburggotha

@ConradMeyer: Yes, that's a very good way of putting it, with a note that there is no operation to "move ownership" away from the FILE once you move ownership to it. – Insalubrious

Answer 3

dup is used to be able to redirect the output from a process.

For example, if you want to save the output from a process, you duplicate the output (fd=1), you redirect the duplicated fd to a file, then fork and execute the process, and when the process finishes, you redirect again the saved fd to output.

Vogul

Answer 4

Some points related to dup/dup2 can be noted please

dup/dup2 - Technically the purpose is to share one File table Entry inside a single process by different handles. ( If we are forking the descriptor is duplicated by default in the child process and the file table entry is also shared).

That means we can have more than one file descriptor having possibly different attributes for one single open file table entry using dup/dup2 function.

(Though seems currently only FD_CLOEXEC flag is the only attribute for a file descriptor).

http://www.gnu.org/software/libc/manual/html_node/Descriptor-Flags.html

dup(fd) is equivalent to fcntl(fd, F_DUPFD, 0);

dup2(fildes, fildes2); is equivalent to 

   close(fildes2);
   fcntl(fildes, F_DUPFD, fildes2);

Differences are (for the last)- Apart from some errno value beteen dup2 and fcntl close followed by fcntl may raise race conditions since two function calls are involved.

Details can be checked from http://pubs.opengroup.org/onlinepubs/009695399/functions/dup.html

An Example of use -

One interesting example while implementing job control in a shell, where the use of dup/dup2 can be seen ..in the link below

http://www.gnu.org/software/libc/manual/html_node/Launching-Jobs.html#Launching-Jobs

Undone