SimulateLowBandwidth

The need

Our test environments sometimes differ from production server environments. Sometimes the network performance is radically different - with much higher latency and/or much lower bandwidth. This affects page load time as well as the appearance of a page whilst it is loading, and the result can be really awful.

We need to be able to model these conditions in our test environments in order to be able to test properly. It would be very helpful to the developer community if there are effective, straightforward and repeatable ways to do this that don't involve financial outlay.

Options

There is more than one way to do that - each with its own pros and cons.

Use someone else's server

The easiest way to test the effect of real-world internet conditions is to use the internet. For example: foswiki.org and trunk.foswiki.org.

The advantage of using someone else's server is that no setup is required. However, you have no control over the network performance and it is hard to make changes on the server when addressing performance issues.

Use a router

If you put a router (e.g. a linux box with two ethernet ports) between your browser and your server, then you can limit the bandwidth and latency. This isn't easy, but it is do-able. However, not everyone has spare hardware lying around to make a router - and several developers use the same machine for browser and server.

Here would be an ideal place to link to yours or someone else's instructions on how to do this

Limit throughput on the server

If you have a linux server, then you could limit throughput via the server's kernel using tc. This gives good control for bandwidth and latency (e.g. as documented here).

In principle, this approach is accessible to most foswiki devs, because most use a linux-based server. However, it can be difficult to configure which means that in practice, most of the time, it is not worth the trouble. This is certainly the case in my (MichaelTempest's) experience. If there were a recipe for doing it, then this might be practical.

I (DarrylGreen) don't know how much of a cookbook for tc you want - but the attached slownet script should be fine for testing with browser and server on the same linux host, and should not impact any traffic that isn't directed to/from localhost on port 80 ie. you can use the browser to access web at full speed and access to apache from other hosts is not impacted.

To undo the slowness run fastnet.

Obviously the choice of bandwidth, using other queue strategies/parameters to simulate different impairments etc is a subject in itself, but as it happens putting a severe rate limit introduces delay. Probably enough for basic testing? You can tweak the script to use netem delay emulation as well (you need to make a qdisc with delay under the root one unless you want all localhost traffic delayed though....)

On Mac and FreeBSD you should be able to use dummynet / ipfw. In fact, this is the most portable solution - you can install and use it on Windows as well (as I do for testing web and other network apps at work) and it is also possible to use it on Linux (but may depending on the distro/version you are using require building a kernel). Configuration is probably simpler than using netem / tc on linux (not that thats saying much....).

If you are installing dummynet on windows be aware that the install of the network packet filter is not something you want to do while you have active network connections (network shares open etc) - close them first!

All the above is quite independent of issues related to running Foswiki itself on Windows. which has its own issues, just one of which is performance. How does Foswiki perform on windows and is this adequately modelled by controlling latency and bandwidth? Please update this if you know

Proxy server

It should be possible to simulate additional latency and lower bandwidth by using a web proxy. The advantage of this approach is that you don't need to adjust your server's kernel routing settings, you don't need additional hardware and you can return to normal operation simply by changing browser settings.

The proxy server approach may not give the same level of control as a kernel-based approach.

Also, it is not obvious what proxy-server software one should use. We might be able to base something on Perl's HTTP::Proxy library.

Other ways

Please contribute :)

Contributors: -- MichaelTempest - 01 Aug 2010

References:

• http://www.linuxfoundation.org/collaborate/workgroups/networking/netem
• http://tldp.org/HOWTO/Traffic-Control-HOWTO/
• http://info.iet.unipi.it/~luigi/dummynet/
• http://www.freebsd.org/cgi/man.cgi?query=ipfw&sektion=8
• Tasks.Item5045 has a tc command to reliably reproduce strange IE6 behaviour (timeout event never fires)
• slownet: linux shell script using tc (netfilter/lartc) traffic control to make localhost port 80 slow....
• fastnet: undoes the slowness of the slownet script...