Best way to simulate a WAN network

Question

Simplified, I have an application where data is intended to flow over the internet between two servers. Ideally, I'd like to test at what point the software ceases to function. At what lowerbound limit (bandwidth, latency, dropped packets) do things stop working to test the reliability of the software.

What I thought I would do was the following:

1. Setup up 3 machines (VMware instances)
2. Install the 2 applications on two of the servers.
3. Setup up the 3rd server to sit between the two machines by doing some sort of magic with Routing and Remote Access on Windows 2003
4. Install either Traffic Shaper XP or NetLimiter to limit the bandwidth
5. Run something like TMnetSim Network Simulator to simulate a bad connection.

Does this sound like a good idea or are there easier/better ways of doing this? I'm not that comfortable on Linux and my team mates are even less so.

Dejection

Could you elaborate on what sort of performance testing you're trying to do? – Dubuffet

I think performance was the wrong word. What I really want to test is at what lowerbound limit (bandwidth, latency, dropped packets) do things stop working to test the reliability of the software. – Dejection

Answer 1

WANem does exactly this. We have used it both in a virtual machine on the desktop and on a dedicated old pc and it worked great. It can simulate all sorts of broken connectivity.

Kleeman

Answer 2

FreeBSDs ipfw has provisions to simulate links with a given bandwith, latency or error rate. You could use that FreeBSD machine as your machine "in the middle" in your above setup.

You probably can also run at least one of the endpoints on the same machine if you want to reduce the amount of servers involved.

Kutenai

A colleague and I used ipfw with great success for proxy load testing, many years ago. It replaced an expensive solution (which shall remain nameless) which worked so poorly that I tried to get our money back. – Reclusion

Answer 3

Someone actually packaged up the settings and whatnot necessary for the FreeBSD solution to this problem and they call it DUMMYNET.

It simulates/enforces queue and bandwidth limitations, delays, packet losses, and multipath effects. It also implements a variant of Weighted Fair Queueing called WF2Q+. It can be used on user's workstations, or on FreeBSD machines acting as routers or bridges.

It can simulate exactly what you want, and its free and will boot onto commodity hardware. They even have a canned install of it that is small enough to put on a floppy disk (!) that you can download at that link.

Enfold

Can DummyNet work on VMware machines? I downloaded and looked at it and seems to only work on physical machines. The examples I've seen use two crossover cables between the machines and freebsd machine and I'm not sure how to setup freebsd as a gateway. – Dejection

I haven't tried dummynet on VMware, but that is certainly an interesting idea. Potentially if you built a cheap fast box with a bunch of network cards you could simulate a whole topology with it. – Enfold

Answer 4

Maybe it is time to learn a bit about Linux because adding a 50ms delay on every outgoing packet can be done in typing just one line:

tc qdisc add dev eth0 root netem delay 50ms

For more see the Linux Traffic Control HOWTO

Millepore

Answer 5

We had a similar requirement some ten years ago - I'll see if I can recall how we managed it.

If I remember, we wrote a socket proxy program which was controlled by inetd on a UNIX box. This socket would accept connections from a client and open equivalent sessions through to the server. It would then loop, passing messages in both directions.

The way we achieved WAN characteristics was to introduce random delays (with upper and lower limits) in both the connection establishment and the passing of data once the link was up.

It also had the feature to drop the link occasionally as WAN links were less reliable for us than local traffic.

I recall we had to make it threaded to stop the delays from affecting reverse traffic on the link.

Bandsman

Answer 6

There is a very good (and free) Microsoft solution for that, we use it for quite some time and it works great, it can very easily simulate every thing(packet loss, low bandwidth, disconnection, latency....) This is the best solution i found for a windows environment

More information and a download link can be found here: MARCO blog post

this product has gone some evolution and it is now integrated into visual studio as part of the automation testing, but i found the use of the standalone(that is quite hard to find, so keep a local copy) to work much better. keep in mind that you need at least two computers(or VMs) since you need to pass through a network adapter in order for the application to work its magic.

Glacial