C++: Store read binary file into buffer

Question

I'm trying to read a binary file and store it in a buffer. The problem is, that in the binary file are multiple null-terminated characters, but they are not at the end, instead they are before other binary text, so if I store the text after the '\0' it just deletes it in the buffer.

Example:

char * a = "this is a\0 test";
cout << a;

This will just output: this is a

here's my real code:

this function reads one character

bool CStream::Read  (int * _OutChar)
{
    if (!bInitialized)
        return false;

    int iReturn = 0;

     *_OutChar = fgetc (pFile);

    if (*_OutChar == EOF)
        return false;

    return true;
}

And this is how I use it:

    char * SendData = new char[4096 + 1];

    for (i = 0; i < 4096; i++)
    {
        if (Stream.Read (&iChar))
            SendData[i] = iChar;
        else
            break;
    }

Felon

Is your problem about doing something with the read data ? Because the reading seems ok (assuming open mode is std::binary), and the use of the buffer as well. – Jemena

@Jemena To open the the file i'am using fopen and as mode parameter "ab+", I think this is binary. What I do next, is sending the buffer to a socket with ssl_write, there i'am write the buffer back to to a file, but this fails, because the buffer that is recieved, is inclomplete, it just writes the buffer until a '\0' – Felon

Answer 1

I just want to mention that there is a standard way to read from a binary file into a buffer.

Using <cstdio>:

char buffer[BUFFERSIZE];

FILE * filp = fopen("filename.bin", "rb"); 
int bytes_read = fread(buffer, sizeof(char), BUFFERSIZE, filp);

Using <fstream>:

std::ifstream fin("filename.bin", ios::in | ios::binary );
fin.read(buffer, BUFFERSIZE);

What you do with the buffer afterwards is all up to you of course.

Edit: Full example using <cstdio>

#include <cstdio>

const int BUFFERSIZE = 4096;    

int main() {
    const char * fname = "filename.bin";
    FILE* filp = fopen(fname, "rb" );
    if (!filp) { printf("Error: could not open file %s\n", fname); return -1; }

    char * buffer = new char[BUFFERSIZE];
    while ( (int bytes = fread(buffer, sizeof(char), BUFFERSIZE, filp)) > 0 ) {
        // Do something with the bytes, first elements of buffer.
        // For example, reversing the data and forget about it afterwards!
        for (char *beg = buffer, *end=buffer + bytes; beg < end; beg++, end-- ) {
           swap(*beg, *end);
        }
    }

    // Done and close.
    fclose(filp);

    return 0;
}

Denishadenison

If I want to read a file over 1GB, i can't use fread, because the buffer size would be too big. This is because I'am using fgets, to read character by character – Felon

@Felon You would still probably get better performance reading into your smaller buffers with fread rather than fget. There is nothing about fread saying you have to read the whole file in one go. – Denishadenison

I've already tried this, see this: #24712927 – Felon

@Felon Aye, I took a look at your code there, I don't see anything wrong at first glance, unless fseek actually returns an error. Please note that as long as you are not jumping around the file the call to fseek is not needed. – Denishadenison

But how can i tell fread that it shouldn't read from beginning. As you say fseek isn't needed here? – Felon

The current position in the file is stored in the FILE object, the next call to fread will therefore continue where the last read ended. (Just in the same way as fget does) The only difference is that fread reads more data at a time and therefore induces less overhead. – Denishadenison

The only problem I have now is, that if I write a mp3 file, windows media player says that the file is corrupted. But when I compare the original and the copied file, they look exactly the same, additional they have the same size, too. The file mode to write is "wb+". What am I doing wrong? – Felon

That is probably best asked as a new question. But before you get that far, I would suggest making sure that the files are in fact equal. The simplest way to check this is to use a checksum-utility that compares the files for you – Denishadenison

Answer 2

static std::vector<unsigned char> read_binary_file (const std::string filename)
{
    // binary mode is only for switching off newline translation
    std::ifstream file(filename, std::ios::binary);
    file.unsetf(std::ios::skipws);

    std::streampos file_size;
    file.seekg(0, std::ios::end);
    file_size = file.tellg();
    file.seekg(0, std::ios::beg);

    std::vector<unsigned char> vec;
    vec.reserve(file_size);
    vec.insert(vec.begin(),
               std::istream_iterator<unsigned char>(file),
               std::istream_iterator<unsigned char>());
    return (vec);
}

and then

auto vec = read_binary_file(filename);
auto src = (char*) new char[vec.size()];
std::copy(vec.begin(), vec.end(), src);

Spellman

For anyone who wants to use this: The read_binary_file function has the line std::vector<unsigned char> vec(file_size); It should just be std::vector<unsigned char> vec ; Otherwise, the function returns a vector with the file contents at the beginning x bytes followed by a dummy x bytes. The vector has 2x bytes ! If you want to optimize the memory allocs as you read the file into the vector, use vec.reserve(file_size) right after you create the vector. – Abolition

fyi - the code is updated for the above comment – Spellman

Answer 3

The problem is definitievely the writing of your buffer, because you read a byte at a time.

If you know the length of the data in your buffer, you could force cout to go on:

char *bf = "Hello\0 world"; 
cout << bf << endl;
cout << string(bf, 12) << endl;

This should give the following output:

Hello
Hello  world

However this is a workaround, as cout is foreseent to output printable data. Be aware that the output of non printable chars such as '\0' is system dependent.

Alternative solutions:

But if you manipulate binary data, you should define ad-hoc data structures and printing. Here some hints, with a quick draft for the general principles:

struct Mybuff {   // special strtucture to manage buffers of binary data
    static const int maxsz = 512; 
    int size;
    char buffer[maxsz]; 
    void set(char *src, int sz)  // binary copy of data of a given length
    { size = sz; memcpy(buffer, src, max(sz, maxsz)); }
} ; 

Then you could overload the output operator function:

ostream& operator<< (ostream& os, Mybuff &b)
{
    for (int i = 0; i < b.size; i++) 
        os.put(isprint(b.buffer[i]) ? b.buffer[i]:'*');  // non printables replaced with *
    return os;
}

ANd you could use it like this:

char *bf = "Hello\0 world"; 
Mybuff my; 
my.set(bf, 13);   // physical copy of memory
cout << my << endl;   // special output 

Jemena

Answer 4

I believe your problem is not in reading the data, but rather in how you try to print it.

char * a = "this is a\0 test";
cout << a;

This example you show us prints a C-string. Since C-string is a sequence of chars ended by '\0', the printing function stops at the first null char. This is because you need to know where the string ends either by using special terminating character (like '\0' here) or knowing its length.

So, to print whole data, you must know the length of it and use a loop similar to the one you use for reading it.

Antre

Answer 5

Are you on Windows? If so you need to execute _setmode(_fileno(stdout), _O_BINARY);

Include <fcntl.h> and <io.h>

Weinman

Is this for the fopen function or for the socket? – Felon