How can I check the data transfer on a network interface in python?

Question

There is a socket method for getting the IP of a given network interface:

import socket
import fcntl
import struct

def get_ip_address(ifname):
    s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    return socket.inet_ntoa(fcntl.ioctl(
        s.fileno(),
        0x8915,  # SIOCGIFADDR
        struct.pack('256s', ifname[:15])
    )[20:24])

Which returns the following:

>>> get_ip_address('lo')
'127.0.0.1'

>>> get_ip_address('eth0')
'38.113.228.130'

Is there a similar method to return the network transfer of that interface? I know I can read /proc/net/dev but I'd love a socket method.

Duggins

'network transfer'? Byte counts? network protocol? Wiring type? – Almeida

A socket != network interface. If you mean that you want TX/RX bytes/packets, that's at the link level. You should be able get the info from the kernel via a NETLINK socket (which is how the ip command works), but it would be much easier to simply parse /proc/net/dev and get the same information. – Rosen

Answer 1

The best way to poll ethernet interface statistics is through SNMP...

• It looks like you're using linux... if so, load up your snmpd with these options... after installing snmpd, in your /etc/defaults/snmpd (make sure the line with SNMPDOPTS looks like this):

  SNMPDOPTS='-Lsd -Lf /dev/null -u snmp -I -smux,usmConf,iquery,dlmod,diskio,lmSensors,hr_network,snmpEngine,system_mib,at,interface,ifTable,ipAddressTable,ifXTable,ip,cpu,tcpTable,udpTable,ipSystemStatsTable,ip,snmp_mib,tcp,icmp,udp,proc,memory,snmpNotifyTable,inetNetToMediaTable,ipSystemStatsTable,disk -Lsd -p /var/run/snmpd.pid'

• You might also need to change the ro community to public ^{See Note 1} and set your listening interfaces in /etc/snmp/snmpd.conf (if not on the loopback)...

• Assuming you have a functional snmpd, at this point, you can poll ifHCInBytes and ifHCOutBytes ^{See Note 2} for your interface(s) in question using this...

poll_bytes.py:

from SNMP import v2Manager
import time

def poll_eth0(manager=None):
    # NOTE: 2nd arg to get_index should be a valid ifName value
    in_bytes = manager.get_index('ifHCInOctets', 'eth0')
    out_bytes = manager.get_index('ifHCOutOctets', 'eth0')
    return (time.time(), int(in_bytes), int(out_bytes))

# Prep an SNMP manager object...
mgr = v2Manager('localhost')
mgr.index('ifName')
stats = list()
# Insert condition below, instead of True...
while True:
    stats.append(poll_eth0(mgr))
    print poll_eth0(mgr)
    time.sleep(5)

SNMP.py:

from subprocess import Popen, PIPE
import re

class v2Manager(object):
    def __init__(self, addr='127.0.0.1', community='public'):
        self.addr      = addr
        self.community = community
        self._index = dict()

    def bulkwalk(self, oid='ifName'):
        cmd = 'snmpbulkwalk -v 2c -Osq -c %s %s %s' % (self.community,
            self.addr, oid)
        po = Popen(cmd, shell=True, stdout=PIPE, executable='/bin/bash')
        output = po.communicate()[0]
        result = dict()
        for line in re.split(r'\r*\n', output):
            if line.strip()=="":
                continue
            idx, value = re.split(r'\s+', line, 1)
            idx = idx.replace(oid+".", '')
            result[idx] = value
        return result

    def bulkwalk_index(self, oid='ifOutOctets'):
        result = dict()
        if not (self._index==dict()):
            vals = self.bulkwalk(oid=oid)
            for key, val in vals.items():
                idx = self._index.get(key, None)
                if not (idx is None):
                    result[idx] = val
                else:
                    raise ValueError, "Could not find '%s' in the index (%s)" % self.index
        else:
            raise ValueError, "Call the index() method before calling bulkwalk_index()"
        return result

    def get_index(self, oid='ifOutOctets', index=''):
        # This method is horribly inefficient... improvement left as exercise for the reader...
        if index:
            return self.bulkwalk_index().get(index, "<unknown>")
        else:
            raise ValueError, "Please include an index to get"

    def index(self, oid='ifName'):
        self._index = self.bulkwalk(oid=oid)

END NOTES:

1. SNMP v2c uses clear-text authentication. If you are worried about security / someone sniffing your traffic, change your community and restrict queries to your linux machine by source ip address. The perfect world would be to modify the SNMP.py above to use SNMPv3 (which encrypts sensitive data); most people just use a non-public community and restrict snmp queries by source IP.

2. ifHCInOctets and ifHCOutOctets provide instantaneous values for the number of bytes transferred through the interface. If you are looking for data transfer rate, of course there will be some additional math involved.

Camilia

Incidentally, once you have snmpd running on your server, you can leverage the all the power of cacti and nagios in your server management endeavors – Camilia

I'd strongly suggest using self.cmdGen all over the script. There are still places where cmdgen.CommandGenerator is re-created on each .get() and .walk() etc. call. That really hurts performance. Especially when you load up all MIBs. – Westerly