How can I programmatically manage iptables rules on the fly?

Question

I need to query existing rules, as well as being able to easily add and delete rules. I haven't found any API's for doing this. Is there something that I'm missing?

The closest I've come to a solution is using iptables-save | iptables-xml for querying and manually calling the iptables command itself to add/delete rules. Another solution I've considered is simply regenerating the entire ruleset out of my application's database and flushing the whole chain, then applying it again. But I want to avoid this as I don't want to drop any packets -- unless there's a way to atomically do this. I'm wondering if there's a better way.

An API in C would be great; however, as I'm planning to build this into a stand-alone suid program, libraries that do this in ANY language are fine too.

Participate

Apparently it's possible to go from XML to iptables-restore xsltproc iptables.xslt my-iptables.xml | iptables-restore. See manpage of iptables-xml. – Tumblebug

Answer 1

From the netfilter FAQ:

The answer unfortunately is: No.

Now you might think 'but what about libiptc?'. As has been pointed out numerous times on the mailinglist(s), libiptc was NEVER meant to be used as a public interface. We don't guarantee a stable interface, and it is planned to remove it in the next incarnation of linux packet filtering. libiptc is way too low-layer to be used reasonably anyway.

We are well aware that there is a fundamental lack for such an API, and we are working on improving that situation. Until then, it is recommended to either use system() or open a pipe into stdin of iptables-restore. The latter will give you a way better performance.

Fruitless

I wonder why the FAQ doesn't address the issue of atomicity. It should; I've gone to the trouble of looking at the implementation of iptables-restore just to make sure it's atomic. It's important to the OP here, and I've had a project that required it too. – Anallese

This netfilter mailing list post says iptables-restore is atomic: mail-archive.com/[email protected]/msg00456.html – Fruitless

For the very brave that still want to do this, there's some info here: netfilter.org/documentation/HOWTO/… – Antepast

Programmatic use of netfilter/iptables suffers from two things: lack of API and lack of atomicity in concurrent access. These are obstacles to changing rules on the fly. To overcome this problem we developed rfw which serves as a system wide iptables guard with REST API. See the full answer: https://mcmap.net/q/359135/-how-can-i-programmatically-manage-iptables-rules-on-the-fly – Beefeater

Answer 2

Using iptables-save and iptables-restore to query and regenerate rules is easily the most efficient way of doing it. These used to, once, be shell scripts, but now they are C programs that work very efficiently.

However, I should point out that there is a tool that you can use which will make maintaining iptables much easier. Most dynamic rulesets are really the same rule repeated many times, such as:

iptables -A INPUT -s 1.1.1.1 -p tcp -m --dport 22 -j ACCEPT
iptables -A INPUT -s 2.2.2.0/24 -p tcp -m --dport 22 -j ACCEPT
iptables -A INPUT -p tcp -m tcp --dport 22 -j REJECT

Instead of replacing those rules every time you want to change what ports can access port 22 (useful for say, port knocking), you can use ipsets. Viz:

ipset -N ssh_allowed nethash
iptables -A ssh_allowed -m set --set ssh_allowed src -p tcp -m --dport 22 -j ACCEPT
ipset -A ssh_allowed 1.1.1.1
ipset -A ssh_allowed 2.2.2.0/24

Sets can hold ip addresses, networks, ports, mac addresses, and have timeouts on their records. (Ever wanted to add something for just an hour?).

There is even an atomic way of swapping one set with another, so a refresh means creating a new temporary set, then swapping it in as the name of the existing set.

Cysto

Aye, the top answer mentioned ipsets, but as I said in a comment there - it requires a kernel module which isn't in Ubuntu by default, and it's not something I can install on any of the VMs that I use. – Participate

Yep, I reported that bug in January '07. bugs.launchpad.net/ubuntu/+source/ipset/+bug/79182 – Cysto

Ubuntu supports ipset adequately now. – Cysto

Answer 3

You may consider using rfw which is the REST API for iptables. It is serializing iptables commands from various potentially concurrent sources and remotely executes iptables on the fly.

rfw is designed for distributed systems that try to update firewall rules on multiple boxes but it can be run also on a single machine on localhost interface. Then it allows avoiding the SSL and authentication overhead as it can be run on plain HTTP in this case.

Sample command:

PUT /drop/input/eth0/11.22.33.44

which corresponds to:

iptables -I INPUT -i eth0 -s 11.22.33.44 -j DROP

You can insert and delete rules as well as query for current status to get the existing rules in JSON format:

GET /list/input

Disclaimer: I started that project. It's open source under the MIT license.

Beefeater

If it's over HTTP, doesn't that make it dangerous for a firewall? – Shiloh

As stated above, default is HTTPS. There is an option to disable SSL and use plain HTTP, useful when running on localhost in single user environment. – Beefeater

Anything that manipulates the firewall "programmatically"/remotely and/or automatically is dangerous. However, proper security in place can mitigate this, ie sso tokens/2fa behind an api gateway(like kong) would go a long way to meet ones security needs. – Ivie

Good job!!! I think it could be "core" tech for other platforms. Some extensions missing, but it's ok. – Enginery

Your answer to de OP is, I think, "NO! it's not possible, unless you use subprocess (or direct OS invocation)" – Enginery

Answer 4

As far as I understand (although no reference seems to mention it), iptables-restore is atomic. At the end, when the COMMIT line is read, iptables calls iptc_commit in libiptc (which in an internal interface you aren't supposed to use), which then calls setsockopt(SO_SET_REPLACE) with your new rulesets.

That sounds about as atomic as you can get: with one kernel call. However, more knowledgeable parties are invited to dispute this. :-)

Edit: I can confirm that your description is correct. iptables-restore is done as an atomic operation in the kernel.

To be even more specific the operation "only" is atomic on a per CPU basis. As we store the entire ruleset blob per CPU (due to cache optimizations).

Anallese

Answer 5

There is deliberately no API to manage these rules. You're not supposed to want to do so. Or something.

If you need rules which are sufficiently dynamic you care about the performance of executing /sbin/iptables, there are other ways to do it:

• Using something like the "recent" match or ip set matching, you can add/remove IP addresses from black/white lists without changing the rule set.
• You can pass packets into userspace for filtering using NFQUEUE

Curio

It seems silly to me that there's no API for this. I don't really care about the performance as such, but calling iptables feels like a horribly hacky way of doing things. – Participate

Hmm, I could use ipsets - and from a performance perspective it's a really good idea. Unfortunately I'd have to roll my own kernel, and this isn't possible as some of the places this software will run is on VMs where I can't mess with the kernel easily. And they still don't provide a nice API. – Participate

ipt_recent is a standard iptables match target which allows you to dynamically add/remove IP addresses from a set by writing to a file in /proc without changing the rules. On the other hand, it's not intended for large sets of IPs and seems to have a fixed maximum limit. – Curio

Answer 6

This morning I woke up to find that was getting a Denial Of Service (DOS) attack from Russia. They were hitting me from dozens of IP blocks. They must have either had a large pool of IPs or some sort of proxy list/service. Every time I blocked an IP, another one popped up. Finally, I looked for a script, and found I needed to write my own solution. The following is a bit agressive, but they were running my TOP LOAD LEVEL to over 200.

Here is a quick script I wrote to block the DOS in realtime.

cat  **"output of the logs"** | php ipchains.php **"something unique in the logs"**

==> PHP Script:

<?php

$ip_arr = array();

while(1)
{
   $line = trim(fgets(STDIN)); // reads one line from STDIN
   $ip = trim( strtok( $line, " ") );

   if( !array_key_exists( $ip, $ip_arr ) )
      $ip_arr[$ip] = 0;

   $regex = sprintf( "/%s/", $argv[1] );

   $cnt = preg_match_all( $regex, $line );

   if( $cnt < 1 ) continue;

   $ip_arr[$ip] += 1;

   if( $ip_arr[$ip] == 1  )
     {
//     printf( "%s\n", $argv[1] );
//     printf( "%d\n", $cnt );
//     printf( "%s\n", $line );

       printf( "-A BLOCK1 -s %s/24 -j DROP\n", $ip );

       $cmd = sprintf( "/sbin/iptables  -I BLOCK1  -d %s/24 -j DROP", $ip );
       system( $cmd );
     }
}

?>

Assumptions:

1) BLOCK1 is a Chain already created. 
2) BLOCK1 is a Chain that is run/called from the INPUT CHAIN 
3) Periodically you will need to run "ipchains -S BLOCK1" and put output in /etc/sysconfig file. 
4) You are familiar with PHP 
5) You understand web log line items/fields and output.

Miele

Have a look at fail2ban. It should the same. – Karinakarine

Answer 7

This is an example of using bash and iptables to dynamically block hackers abusing sshd on CentOS. In this case, I configured sshd to disallow password login (allows keys). I look in /var/log/secure for entries of "Bye Bye", which is sshd's polite way of saying f-off...

IP=$(awk '/Bye Bye/{print $9}' /var/log/secure |
     sed 's/://g' |sort -u | head -n 1)

[[ "$IP" < "123" ]] || {

  echo "Found $IP - blocking it..." >> /var/log/hacker.log

  /sbin/iptables -A INPUT -s $IP -j DROP

  service iptables save

  sed -i "/$IP/d" /var/log/secure

}

I run this in a loop every second, or minute, or whatever makes me happy. I test the value of $IP to verify it found a useful value, if so I invoke iptables to drop it, and I use sed to purge the log file of $IP so the entry doesn't get added again.

I do a little pre-processing (not shown) to white list some important IPs that are always valid and that might have had trouble connecting (due to user error).

From time-to-time I sort the iptables filter list and create IP ranges from them (using a different script - and when checked, they are usually IP ranges from india, china and russia). Thus, my overall iptables filter rule set stays between 50 and 500 entries; ipset doesn't really improve much on a list that short.

Pascual

What's the significance of the number 123? If you want to check that you have a valid IP address by examining the value of the first octet, it could be anything up to 223 I believe. – Villeneuve

This is not a test of any octets. If when parsing /var/log/secure you end up with an empty or corrupt field, you test the value to not run the iptables command. The '123' value was quite arbitrary. In testing this a bit more, I found you probably should replace '123' with '1' to include a complete IP range, though. When testing an IP less than '123', it will be true for IP addresses in the range 2.0.0.0 to 255.255.255.255, so it will not block IPs in the 1.x.x.x range. When testing IP is less than '1', it matches 0.0.0.1 to 255.255.255.255. – Pascual

Answer 8

MarkR's right, you're not supposed to do this. The easiest way is to call iptables from the script or to write the iptables config and 'restore' it.

Still, if you want to, read the source of iptables. iptables uses matches and tables as shared objects. You can use the source or them.

The Linux netfilter also has some include files under /usr/include/netfilter*. These are somewhat low-level functions. It is what iptables uses. This is as near an API as one can get without iptables.

But this API is 'messy'. Bear in mind that it was designed to be used only by iptables. It's not very well documented, you can hit very specific problems, the API can change fairly quick without any warning, so an upgrade propably will break your code, etc.

Amenra

I can accept that using internal APIs is bad, but why is this so bad that they would deliberately not include a public API? I might go with doing a restore if I can do a restore of a single chain - I'll have to do some testing. – Participate

Yes, you can restore a single chain. :-) – Anallese

Answer 9

I know its a short term solution, per the netfilter discussion, but in the short term you can use iptc wrapped in python with this:

https://github.com/ldx/python-iptables

I played with it some in a recent project of mine and found it quite effective.

Outrigger

This link may answer the question, but it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes. – Fatso