What is the purpose of std::byte?

Question

Now that c++17 has std::byte, I was looking for a way to convert code that reads files to char into code that reads files into byte. A file contains bytes, not a bunch of integers.

Then I read this question and this other question where people argued that reading files into byte is wrong, and that reading files into char is right.

If byte is not designed for the purpose of accessing memory, and by analogy, files, then what is the its purpose? As is quoted in the other two questions:

Like char and unsigned char, it can be used to access raw memory occupied by other objects (object representation), but unlike those types, it is not a character type and is not an arithmetic type. A byte is only a collection of bits, and only bitwise logic operators are defined for it.

This sounds like the exact type that should be used for reading files, not characters.

Assai

Reading files into byte is correct, but C++ iostreams are not direct access to files, they automatically perform conversion. Your OS file access API will probably work fine with byte pointers. – Frumpy

Consider it a vocabulary type. If you intend to manipulate the bytes of your object, which is more expressive, an unsigned char* or a std::byte*? – Epi

To be honest, I don't really see its purpose either. The allowed operations on it seems arbitrary. What's the difference between bit manipulations, and for example, addition? Why is one allowed, while the other is forbidden? But I agree with StoryTeller, std::byte is a more expressive name when you just want to store/read/write some bytes. I don't think I'll ever use it (except if someone could give us an enlightening answer), as unsigned char is fine for me – Cochabamba

@geza: well, in theory, things like addition assign meanings to the individual bits whereas bit manipulations don't. If your byte is representing a number (why else would you use addition), use an integral type. In practice, that breaks some bittwiddeling hacks like i & (i-1) which also only want to perform bit operations. – Dicotyledon

@MikeMB: good example! That's another reason I won't use std::byte. (one could argue that there should be a clearLowestBit(std::byte) function for this, though, which casts internally - but I don't really see the benefit of forbidding usual operations on a byte, like addition) – Cochabamba

@geza: As I said, I see the benefit, because it makes it clear that a std::byte doesn't represent a numerical value. However, I'd whish for a much better integration with the rest of the standard library (e.g. file/network I/O, bitset, or helpers like your clearLowestBit or a special as_bytes cast (preferably with an array_view type like gsl::span) – Dicotyledon

@MikeMB: okay :) for me, it adds nothing. The name std::byte already represents well, that it's not a numerical value. I think, there's no need to forbid certain operations on it. Where's the benefit? Makes us to make less mistakes? I don't think so. And your example makes sense. Here, i-1 isn't actually a numerical operation, but used as a bit operation (though a complex one: it sets the trailing zeros to one (here, we don't need this behavior, but it's harmless), and the lowest set bit to zero)... – Cochabamba

@MikeMB: ... There are a lot of bit tricks, where numerical operations are needed. You cannot cover them all with std functions. – Cochabamba

Answer 1

You are perhaps misunderstanding things.

byte is very much intended for "accessing memory". You are intended to use the type when the storage is just a sequence of bytes rather than an array of characters.

Iostream types cannot be specialized with byte, since they're designed around characters as their interface. That is, they do not think of files as sequences of bytes; they think of them as sequences of characters. Now, you can certainly read directly into a byte array by using a cast or two. But that's not the way iostream natively thinks.

You have to make a distinction between the way iostream works and the way files work. Iostream is just one file IO library, after all; it is hardly the end-all, be-all of file APIs.

Most file APIs for reading binary data take void* rather than character arrays. std::fread/fwrite, and so forth.

That is, you should think of this, not as a problem with std::byte, but as a problem with iostream. Just another in a long line of them.

Wadley

Iostreams strike again,just stop using it – Calipee

As far as I can tell, iostream (std::fstream) is the only cross-platform way to work with Unicode filenames - using the std::filesystem::path overload. Are there plans to add a similar overload to std::fopen or to provide a different way to read binary data from files? – Expansible

@Nikolai: The committee is unwilling to make modifications to C-standard library APIs. But ifstream API is perfectly serviceable at reading binary data. – Wadley

@Nicol Bolas: this comment suggests that iostreams are not suitable for raw access to files - is that incorrect? Perhaps you could clarify what exactly the difference between a byte and a character is? – Expansible