Linux timestamping for TCP sockets

Question

I am working in a project to get the receive and transmit time stamping from the NIC for TCP socket as mentioned in the document Linux timestamping. But all the documentation and test coding are done for UDP sockets. But I am getting the Transmit timestamping for the NIC and not getting the time stamping for the receive TCP packets.

My Interface support the following time stamps

    Time stamping parameters for enp4s0:
Capabilities:
    hardware-transmit     (SOF_TIMESTAMPING_TX_HARDWARE)
    software-transmit     (SOF_TIMESTAMPING_TX_SOFTWARE)
    hardware-receive      (SOF_TIMESTAMPING_RX_HARDWARE)
    software-receive      (SOF_TIMESTAMPING_RX_SOFTWARE)
    software-system-clock (SOF_TIMESTAMPING_SOFTWARE)
    hardware-raw-clock    (SOF_TIMESTAMPING_RAW_HARDWARE)
PTP Hardware Clock: 3
Hardware Transmit Timestamp Modes:
    off                   (HWTSTAMP_TX_OFF)
    on                    (HWTSTAMP_TX_ON)
Hardware Receive Filter Modes:
    none                  (HWTSTAMP_FILTER_NONE)
    all                   (HWTSTAMP_FILTER_ALL)

I enable the timestamping for NIC after bind() using ioctl(sockfd, SIOCSHWTSTAMP, &net_device); with

memset(&net_device, 0, sizeof(net_device));
strncpy(net_device.ifr_name, interface_name, sizeof(net_device.ifr_name));
net_device.ifr_data = (void *)&tstconfig;
memset(&tstconfig, 0, sizeof(tstconfig));

tstconfig.tx_type = HWTSTAMP_TX_OFF;
tstconfig.rx_filter = HWTSTAMP_FILTER_ALL;

then enable the time stamping in the NIC through setsockopt()

int opt= 0;
opt |= SOF_TIMESTAMPING_RX_HARDWARE;
opt |= SOF_TIMESTAMPING_RAW_HARDWARE;
if (setsockopt(sockfd, SOL_SOCKET, SO_TIMESTAMPING,
           (char *)&opt, sizeof(opt))) {
    error(1, 0, "setsockopt timestamping");
    bail("setsockopt SO_TIMESTAMPING");
}

After the listen() and accept(), I do select(), and check whether fd is rfds then call the recvmsg() with following options

int rc;
struct iovec vec[1];
struct msghdr msg;
char data[8192];
struct cmsghdr *cmsg;

union {
    struct cmsghdr cm;
    char control[256];
} cmsg_un;

vec[0].iov_base = data;
vec[0].iov_len = sizeof(data);

memset(&msg, 0, sizeof(msg));
memset(&from_addr, 0, sizeof(from_addr));
memset(&cmsg_un, 0, sizeof(cmsg_un));

msg.msg_name = NULL;
msg.msg_namelen = 0;
msg.msg_iov = vec;
msg.msg_iovlen = 1;
msg.msg_control = cmsg_un.control;
msg.msg_controllen = sizeof(cmsg_un.control);   
rc = recvmsg(flow->fd, &msg, 0);

printf("tried reading %d bytes, got %d", bytes, rc);
if (msg.msg_flags & MSG_TRUNC) {
    printf("received truncated message\n");
    return 0;
}

if (msg.msg_flags & MSG_CTRUNC) {
    printf("received truncated ancillary data\n");
    return 0;
}

if (msg.msg_controllen <= 0) {
    printf("`received short ancillary data (%ld/%ld)`\n",
            (long)msg.msg_controllen, (long)sizeof(cmsg_un.control));
    return 0;
}

But I am always getting the following message,

received short ancillary data (0/256)

I am not getting the ancillary data from the recvmsg(), I like to know whether linux support TCP receive hardware time stamping for the NIC.

Endosteum

Answer 1

Linux timestamping doesn't support the receive hardware or software timestamping for TCP. Documentation of linux timestamping mentioned only regarding the in terms of "packets". This refers UDP, which is used for the implementation of synchronization of PTP hardware clock in NIC, look the code of PTP daemon and linux ptp for more understanding. Linux 3.18 support only the timestamping in transmit. So basically you can't achieve the linux timestamping for TCP in receiver.

Pulmonary

anyway to get timestamps for TCP then? – Mature

Worth adding is that some NICs and Linux drivers support general timestamping of UDP packets, not intended just for PTP. – Gi

I know this is old, but I think libpcap supports TCP hardware timestamping. – Mathias

As of today, 2017 April 18, support staff from Solarflare (network card manufacturer) report that the Linux kernel does not support TCP hardware timestamping. You could get it from an offloaded network card driver, with a card such as the SFN8522-PLUS, but that card is not fully baked either :-( . – Brolly

Quoting from kernel.org/doc/Documentation/networking/timestamping.txt: "SO_TIMESTAMPING Generates timestamps on reception, transmission or both. Supports multiple timestamp sources, including hardware. Supports generating timestamps for stream sockets. " Are you sure that Linux does not support TCP timestamping ? It seems that this is not the case, although I am still trying to get the timestamp for TCP. – Namnama

My question is even if it is supported what does it mean? TCP is a stream and generally the segmentation of the stream is not exposed, so the timestamp you get corresponds to...? – Crumpton

Answer 2

I was able to use SO_TIMESTAMPING with a TCP socket using a recent kernel version (4.18). This works for both TX and RX timestamps. I am still working on this but I can try to write a minimal proof of concept if you want.

Enough

If you read data that came in multiple segments which timestamp do you get? Or is there some way to expose the segmented view? – Crumpton