Performance Difference Between C and C++ Style File IO

Question

I've always heard that C++ file I/O operations are much much slower than C style I/O. But I didn't find any practical references on comparatively how slow they actually are, so I decided to test it in my machine (Ubuntu 12.04, GCC 4.6.3, ext4 partition format).

First I wrote a ~900MB file in the disk.

C++ (ofstream): 163s

ofstream file("test.txt");
    
for(register int i = 0; i < 100000000; i++) 
    file << i << endl;

C (fprintf): 12s

FILE *fp = fopen("test.txt", "w");
    
for(register int i = 0; i < 100000000; i++) 
    fprintf(fp, "%d\n", i);

I was expecting such output, it shows that writing to a file is much slower in C++ than in C. Then I read the same file using C and C++ I/O. What made me exclaimed that there is almost no difference in performance while reading from file.

C++ (ifstream): 12s

int n;
ifstream file("test.txt");

for(register int i = 0; i < 100000000; i++) 
    file >> n;

C (fscanf): 12s

FILE *fp = fopen("test.txt", "r");
    
for(register int i = 0; i < 100000000; i++) 
    fscanf(fp, "%d", &n);

So, why is taking so long to execute writing using stream? Or, why reading using stream is so fast compared to writing?

Conclusion: The culprit is the std::endl, as the answers and the comments have pointed out. Changing the line file << i << endl; to file << i << '\n'; has reduced running time to 16s from 163s.

Rudderpost

Did you turn off optimization? Maybe in ifstream case compiler sees that you keep overwriting n and just moves file pointer without actually reading from file? – Cooee

Sigh. There is no difference, your benchmark is flawed. If anything, C++ formatted input/output is faster. Aren’t there a bazillion duplicates of this? – Kymry

I don't deny that fstream is slower than cstdio, but these differences seem a little larger than I'd expect from my measurements. It's not the case that you are running with low (or no) optimization levels? Since stream is largely implemented through templates, it gets compiled into your code, where cstdio type functions are compiled into a library and have higher level of optimization. – Cogency

@Cooee No, I did'n do that, I'll make a test turning optimization. – Rudderpost

You are cheating. std::endl is more than the \n you output in fprintf. It leads to a stream flush. So try file << i << '\n' – Pickard

possible duplicate of Why C++ output is too much slower than C? – Series

codeforces.com/blog/entry/5217 benchmarks – Feltner

Answer 1

You're using endl to print a newline. That is the problem here, as it does more than just printing a newline — endl also flushes the buffer which is an expensive operation (if you do that in each iteration).

Use \n if you mean so:

file << i << '\n';

And also, must compile your code in release mode (i.e turn on the optimizations).

Outside

Thanks, that was the problem. Changing to '\n' reduces the time to 16s! – Rudderpost

So there actually is no "Performance Difference Between C and C++ Style File IO" ? – Markley

@AlexandruBarbarosie: Yes, there is no reason to be. – Outside

Hmm, still, 16s vs 12s, a 25% difference is not quite negligible? – Searching

No prob, here you go. Tried on MSVC2015U3, Windows 10 x64, 30% performance difference. But! The same code on FreeBSD 11 (clang 3.4.1) showed exactly the same results in both cases. So I guess it's implementation-dependent. – Searching

@RustyX: Impl-dependent, as you said. Also, could you please test it with std::ofstream instead of std::fstream? (apart from that, you should use unique_ptr<char[]>, not unique_ptr<char>; though it is not related to performance, but currently your code invokes UB, so fix that as well). – Outside

@Nawaz - here's the updated version. ofstream: 580MB/s, fopen: 1040MB/s (I have an M.2 SSD with max write speed of ~1200MB/s). And on FreeBSD it's the same, it just has a slower disk. But CPU usage there is 50% with fstream vs 4% with fopen. – Searching

@RustyX: Interesting. In theory, there should not be any significant difference, but practically there is. Seems like MSVC has not optimised this.. Or maybe, you have not used correct set of optimization flags? Have you used them at all? If yes, what are they? – Outside

-O2 in both MSVC and FreeBSD. C++ streams just seem to use a lot more CPU, even when buffering is used. That leads to seemingly similar performance, until a CPU saturates. – Searching

@RustyX: I think you meant /O2 for VC++. Did you try /Ox? – Outside

Let us continue this discussion in chat. – Searching

Answer 2

No, C++ input/output is not substantially slower than C’s – if anything, a modern implementation should be slightly faster on formatted input/output since it doesn’t need to parse a format string, and the formatting is instead determined at compile time through the chaining of the stream operators.

Here are a few caveats to consider in a benchmark:

• Compile with full optimisations (-O3) to get a fair comparison.
• A proper benchmark needs to estimate biases – in practice this means that you need to repeat your tests and interleave them. At the moment your code isn’t robust to disturbances from background processes. You should also report a summary statistic of the repeated runs to catch outliers that distort the estimates.
• Disable C++ stream synchronisation with C streams (std::ios_base::sync_with_stdio(false);)
• Use '\n' instead of the (flushing) std::endl
• Don’t use register declarations – it simply makes no difference and modern compilers probably ignore it anyway.

Kymry

Great points. Thank you for elaborating on your original comment. – Hodman

Answer 3

When working with large files with fstream, make sure to set a stream buffer >0.

Counterintuitively, disabling stream buffering dramatically reduces performance. At least the MSVC 2015 implementation copies 1 char at a time to the filebuf when no buffer was set (see streambuf::xsputn), which can make your application CPU-bound, which will result in lower I/O rates.

const size_t bufsize = 256*1024;
char buf[bufsize];
mystream.rdbuf()->pubsetbuf(buf, bufsize);

You can find a complete sample application here.

Searching