C read binary stdin

Question

I'm trying to build an instruction pipeline simulator and I'm having a lot of trouble getting started. What I need to do is read binary from stdin, and then store it in memory somehow while I manipulate the data. I need to read in chunks of exactly 32 bits one after the other.

How do I read in chunks of exactly 32 bits at a time? Secondly, how do I store it for manipulation later?

Here's what I've got so far, but examining the binary chunks I read further, it just doesn't look right, I don't think I'm reading exactly 32 bits like I need.

char buffer[4] = { 0 }; // initialize to 0
unsigned long c = 0;
int bytesize = 4; // read in 32 bits
while (fgets(buffer, bytesize, stdin)) {
  memcpy(&c, buffer, bytesize); // copy the data to a more usable structure for bit manipulation later
  // more stuff
  buffer[0] = 0; buffer[1] = 0; buffer[2] = 0; buffer[3] = 0; // set to zero before next loop
}
fclose(stdin);

How do I read in 32 bits at a time (they are all 1/0, no newlines etc), and what do I store it in, is char[] okay?

EDIT: I'm able to read the binary in but none of the answers produce the bits in the correct order — they are all mangled up, I suspect endianness and problems reading and moving 8 bits around ( 1 char) at a time — this needs to work on Windows and C ... ?

Telestich

actually you can just read either byte by byte or 4 bytes per call, because you need to store the data to a buffer. once got buffer stores whatever length of data, you could process the buffer 4 byte by 4 byte. since stdin/stdout is though as streamy, so byte by byte is natural too. unlike socket I/O, normally you can ignore byte-order upon stdin etc, of course if you need , do byte-order convert yourself. because of no newline (e.g. '\n'), use read or fread would be better. please think it as a stream. – Isbell

None of these solutions read in the binary in correct order! Help please, deadline approaching. – Telestich

x86 platforms are little endian, you cant expect to have in c the bits in the rigth order. – Atlee

+1 yes I've often wondered about this ... what's the question again? – Doggett

How to read in binary bits and then segment those bits into (perhaps oddly sized) chunks, bitmasks for example. – Telestich

Answer 1

I had it right the first time, except, I needed ntohl ... C Endian Conversion : bit by bit

Telestich

Answer 2

What you need is freopen(). From the manpage:

If filename is a null pointer, the freopen() function shall attempt to change the mode of the stream to that specified by mode, as if the name of the file currently associated with the stream had been used. In this case, the file descriptor associated with the stream need not be closed if the call to freopen() succeeds. It is implementation-defined which changes of mode are permitted (if any), and under what circumstances.

Basically, the best you can really do is this:

freopen(NULL, "rb", stdin);

This will reopen stdin to be the same input stream, but in binary mode. In the normal mode, reading from stdin on Windows will convert \r\n (Windows newline) to the single character ASCII 10. Using the "rb" mode disables this conversion so that you can properly read in binary data.

freopen() returns a filehandle, but it's the previous value (before we put it in binary mode), so don't use it for anything. After that, use fread() as has been mentioned.

As to your concerns, however, you may not be reading in "32 bits" but if you use fread() you will be reading in 4 chars (which is the best you can do in C - char is guaranteed to be at least 8 bits but some historical and embedded platforms have 16 bit chars (some even have 18 or worse)). If you use fgets() you will never read in 4 bytes. You will read in at least 3 (depending on whether any of them are newlines), and the 4th byte will be '\0' because C strings are nul-terminated and fgets() nul-terminates what it reads (like a good function). Obviously, this is not what you want, so you should use fread().

During

There's no need to even try to assign the return value of freopen to stdin - freopen either returns NULL or the previous value of stdin (it changes the pointed-to FILE but doesn't change the FILE * value itself) – Friction

Ah. I didn't realized it returned the old value. Edited to fix. – During

As always, Windows is different. It doesn't permit NULL for the path parameter. See this comment. – Indult

Unfortunately this won't work on Windows, for the reason given by @schieferstapel: msdn.microsoft.com/en-us/library/wk2h68td.aspx. -1. – Stanstance

Answer 3

Consider using SET_BINARY_MODE macro and setmode:

#ifdef _WIN32
# include <io.h>
# include <fcntl.h>
# define SET_BINARY_MODE(handle) setmode(handle, O_BINARY)
#else
# define SET_BINARY_MODE(handle) ((void)0)
#endif

More details about SET_BINARY_MODE macro here: "Handling binary files via standard I/O"

More details about setmode here: "_setmode"

Moraine

thanks for your mentioning _setmode. This it was actually worked for me (I was trying to open stdout for binary). – Medieval

# define SET_BINARY_MODE(handle) ((void)0) doesn't work for me. I'm using an Atmega 128A1 and it seems to be impossible to set stdout into binary mode. Can anyone help? – Eisenhart

Answer 4

I had to piece the answer together from the various comments from the kind people above, so here is a fully-working sample that works - only for Windows, but you can probably translate the windows-specific stuff to your platform.

#include "stdafx.h"
#include "stdio.h"
#include "stdlib.h"
#include "windows.h"
#include <io.h>
#include <fcntl.h>

int main()
{
    char rbuf[4096];
    char *deffile = "c:\\temp\\outvideo.bin";
    size_t r;
    char *outfilename = deffile;
    FILE *newin;

    freopen(NULL, "rb", stdin);
    _setmode(_fileno(stdin), _O_BINARY);

    FILE *f = fopen(outfilename, "w+b");
    if (f == NULL)
    {
        printf("unable to open %s\n", outfilename);
        exit(1);
    }

    for (;; )
    {
        r = fread(rbuf, 1, sizeof(rbuf), stdin);
        if (r > 0)
        {
            size_t w;
            for (size_t nleft = r; nleft > 0; )
            {
                w = fwrite(rbuf, 1, nleft, f);
                if (w == 0)
                {
                    printf("error: unable to write %d bytes to %s\n", nleft, outfilename);
                    exit(1);
                }
                nleft -= w;
                fflush(f);
            }
        }
        else
        {
            Sleep(10); // wait for more input, but not in a tight loop
        }
    }

    return 0;
}

Lymphatic

Answer 5

For Windows, this Microsoft _setmode example specifically shows how to change stdin to binary mode:

// crt_setmode.c
// This program uses _setmode to change
// stdin from text mode to binary mode.

#include <stdio.h>
#include <fcntl.h>
#include <io.h>

int main( void )
{
   int result;

   // Set "stdin" to have binary mode:
   result = _setmode( _fileno( stdin ), _O_BINARY );
   if( result == -1 )
      perror( "Cannot set mode" );
   else
      printf( "'stdin' successfully changed to binary mode\n" );
}

Dewhurst

Answer 6

fgets() is all wrong here. It's aimed at human-readable ASCII text terminated by end-of-line characters, not binary data, and won't get you what you need.

I recently did exactly what you want using the read() call. Unless your program has explicitly closed stdin, for the first argument (the file descriptor), you can use a constant value of 0 for stdin. Or, if you're on a POSIX system (Linux, Mac OS X, or some other modern variant of Unix), you can use STDIN_FILENO.

Miniaturize

This won't work for non-POSIX systems, of course, which defeats the purpose, because on POSIX systems there is no difference between binary reading and text reading for filehandles. – During

Answer 7

fread() suits best for reading binary data.

Yes, char array is OK, if you are planning to process them bytewise.

Sunshade

This is not the answer. stdin is a buffered input stream, and fread() will read buffered data, and on Windows it will be reading in text mode and convert \r\n to a single character, which will be bad for binary data. – During

Answer 8

I don't know what OS you are running, but you typically cannot "open stdin in binary". You can try things like

int fd = fdreopen (fileno (stdin), outfname, O_RDONLY | OPEN_O_BINARY);

to try to force it. Then use

uint32_t opcode;
read(fd, &opcode, sizeof (opcode));

But I have no actually tried it myself. :)

Way

You don't have to do the fdreopen(). stdin is always open - see my answer. – Miniaturize

WTF?? I know its open, that's the only kind of file you can FEED into fdreopen. But it has the wrong MODE. stdin is opened as a TEXT file. The guy wants to read raw binary data, and text isn't going to cut it. – Way