Is there any "standard" htonl-like function for 64 bits integers in C++?

Question

I'm working on an implementation of the memcache protocol which, at some points, uses 64 bits integer values. These values must be stored in "network byte order".

I wish there was some uint64_t htonll(uint64_t value) function to do the change, but unfortunately, if it exist, I couldn't find it.

So I have 1 or 2 questions:

• Is there any portable (Windows, Linux, AIX) standard function to do this ?
• If there is no such function, how would you implement it ?

I have in mind a basic implementation but I don't know how to check the endianness at compile-time to make the code portable. So your help is more than welcome here ;)

Thank you.

---

Here is the final solution I wrote, thanks to Brian's solution.

uint64_t htonll(uint64_t value)
{
    // The answer is 42
    static const int num = 42;

    // Check the endianness
    if (*reinterpret_cast<const char*>(&num) == num)
    {
        const uint32_t high_part = htonl(static_cast<uint32_t>(value >> 32));
        const uint32_t low_part = htonl(static_cast<uint32_t>(value & 0xFFFFFFFFLL));

        return (static_cast<uint64_t>(low_part) << 32) | high_part;
    } else
    {
        return value;
    }
}

Girlish

possible duplicate of this one https://mcmap.net/q/18626/-64-bit-ntohl-in-c – Ireland

@ereOn: I also have similar question here. If possible can you take a look and let me know what wrong I am doing here? – Squint

Instead of including your answer inside the question, you should let your answer with answers. It is more readable. – Anthelmintic

Answer 1

You are probably looking for bswap_64 I think it is supported pretty much everywhere but I wouldn't call it standard.

You can easily check the endianness by creating an int with a value of 1, casting your int's address as a char* and checking the value of the first byte.

For example:

int num = 42;
if(*(char *)&num == 42)
{
   //Little Endian
}
else
{
   //Big Endian
} 

Knowing this you could also make a simple function that does the swapping.

---

You could also always use boost which contains endian macros which are portable cross platform.

Dunlin

Thanks ;) The boost macros seem interesting. Do you have link ? – Girlish

Technically, just a byte-swap is not always enough. There are systems that are middle-endian (though no modern systems are). Also, bswap_64 is definitely NOT supported everywhere; just look at the long list of systems with bswap_64 missing: gnu.org/software/gnulib/manual/html_node/bswap_005f64.html – Buettner

Note that bswap_64 is not POSIX compliant. – Bevel

bswap_64 is not present on Solaris i86pc. Does anyone know if it is present on AIX? – Gerent

to ensure others are aware, bswap_64 always assume swapping is required whereas htobe64, like htonl, will just return their argument if network byte order (which is big endian) matches computer system's byte ordering/endian. – Pericarp

Answer 2

#define htonll(x) ((1==htonl(1)) ? (x) : ((uint64_t)htonl((x) & 0xFFFFFFFF) << 32) | htonl((x) >> 32))
#define ntohll(x) ((1==ntohl(1)) ? (x) : ((uint64_t)ntohl((x) & 0xFFFFFFFF) << 32) | ntohl((x) >> 32))

The test (1==htonl(1)) simply determines (at runtime sadly) if the hardware architecture requires byte swapping. There aren't any portable ways to determine at compile-time what the architecture is, so we resort to using "htonl", which is as portable as it gets in this situation. If byte-swapping is required, then we swap 32 bits at a time using htonl (remembering to swap the two 32 bit words as well).

---

Here's another way to perform the swap that is portable across most compilers and operating systems, including AIX, BSDs, Linux, and Solaris.

#if __BIG_ENDIAN__
# define htonll(x) (x)
# define ntohll(x) (x)
#else
# define htonll(x) (((uint64_t)htonl((x) & 0xFFFFFFFF) << 32) | htonl((x) >> 32))
# define ntohll(x) (((uint64_t)ntohl((x) & 0xFFFFFFFF) << 32) | ntohl((x) >> 32))
#endif

The important part is to use __BIG_ENDIAN__ or __LITTLE_ENDIAN__; and not __BYTE_ORDER__, __ORDER_BIG_ENDIAN__ or __ORDER_LITTLE_ENDIAN__. Some compilers and operating systems lack __BYTE_ORDER__ and friends.

Hiram

I'm sure this answer is great, but could you please add something other than code to explain what and why? – Cortex

as mentioned @ereon you cannot use 32 bits shifts on ntohl or ntohl because they return uint32_t, hence triggering UB. – Broida

With C++11 I really would have preferred inline functions instead of macros... – Leontineleontyne

Answer 3

You can try with uint64_t htobe64(uint64_t host_64bits) & uint64_t be64toh(uint64_t big_endian_64bits) for vice-versa.

Doublebreasted

Well, no I can't because it's been 6+ years since I touched that codebase ;) – Girlish

Answer 4

This seems to work in C; did I do anything wrong?

uint64_t htonll(uint64_t value) {
    int num = 42;
    if (*(char *)&num == 42) {
        uint32_t high_part = htonl((uint32_t)(value >> 32));
        uint32_t low_part = htonl((uint32_t)(value & 0xFFFFFFFFLL));
        return (((uint64_t)low_part) << 32) | high_part;
    } else {
        return value;
    }
}

Bogtrotter

Answer 5

To reduce the overhead of the "if num == ..." Use the pre-processor defines:

#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
#else
#endif

Truss

Not portable. __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ is a Linux thing. AIX lacks it, and will drop into the big-endian code path. – Gerent

Answer 6

Well, I think it's better to use the endian switching at compile time as much as possible, but I prefer to use function instead of macro, because in a macro the parameters are just replaced by the arguments; so arguments can be evaluated multiple times and create weird result if they are present multiple times in the macro (as done in some of the previous provided solutions).

uint64_t htonll(uint64_t x)
{
#if __BIG_ENDIAN__
    return x;
#else
    return ((uint64_t)htonl((x) & 0xFFFFFFFFLL) << 32) | htonl((x) >> 32);
#endif
}

uint64_t ntohll(uint64_t x)
{
#if __BIG_ENDIAN__
    return x;
#else
    return ((uint64_t)ntohl((x) & 0xFFFFFFFFLL) << 32) | ntohl((x) >> 32);
#endif
}

So this allow calling htonll(x++) without increment x several time like it will be done using the previous macros.

Clancy

Answer 7

I suggest to use the functions which are defined in endian.h

As network byte order is big endian by convention you just have to call the appropriate function from host to "be". It includes 16, 32 and 64 bits versions.

I've created the following single ntoh templated fuctions which work for all the integral types so I don't have to worry about which function to call.

template <typename T> std::enable_if_t<sizeof(T) == 1, T> ntoh(T v) { return v; }
template <typename T> std::enable_if_t<sizeof(T) == 2, T> ntoh(T v) { return be16toh(v); }
template <typename T> std::enable_if_t<sizeof(T) == 4, T> ntoh(T v) { return be32toh(v); }
template <typename T> std::enable_if_t<sizeof(T) == 8, T> ntoh(T v) { return be64toh(v); }

template <typename T> std::enable_if_t<sizeof(T) == 1, T> hton(T v) { return v; }
template <typename T> std::enable_if_t<sizeof(T) == 2, T> hton(T v) { return htobe16(v); }
template <typename T> std::enable_if_t<sizeof(T) == 4, T> hton(T v) { return htobe32(v); }
template <typename T> std::enable_if_t<sizeof(T) == 8, T> hton(T v) { return htobe64(v); }

On Linux, the call to that function has no penalty as no transformation need to be done. On Windows, who cares about windows ;-P

Hope it helps someone.

Subcutaneous

Answer 8

EDIT: combining the two (used Brian's code):

uint64_t htonll(uint64_t value)
{
     int num = 42;
     if(*(char *)&num == 42)
          return (htonl(value & 0xFFFFFFFF) << 32LL) | htonl(value >> 32);
     else 
          return value;
}

Warning: untested code! Please test before using.

Hildegardehildesheim

@Pavel Still doesn't work. htonl() returns a 32 bits value on which you cannot call << 32LL (because you cannot shift 32 bits left on an only 32 bits value). – Girlish

I think shifting by 32LL will do promotion of the left side, no? – Hildegardehildesheim

@Pavel: no, the bit-size of the shift value doesn't change anything. gcc emits a warning: "left shift count >= width of type". And your function says that htonll(0x0102030405060708ULL) == 0xc070605. – Polythene

Change it to return ((uint64_t)htonl(value & 0xFFFFFFFF) << 32LL) | htonl(value >> 32); and it works right. – Polythene