Mahimahi

A set of lightweight tools for browser developers, website authors, and network protocol designers that provides accurate measurements when recording and replaying HTTP content over emulated network conditions.

Mahimahi is free software and is available on Ubuntu (version 14.04 or higher).

Getting Mahimahi »

Record and replay HTTP traffic

Using Mahimahi's RecordShell, you can completely record and store real HTTP content (such as Web pages) to disk. Recorded sites can then be repeatably replayed using ReplayShell. ReplayShell accurately models a Web application's multi-server structure by locally mirroring a page's server distribution.

Emulate numerous network conditions

Mahimahi's network emulation tools can be used to emulate many different link conditions for replaying recorded HTTP traffic. Mahimahi supports emulating fixed propagation delays (DelayShell), fixed and variable link rates (LinkShell), stochastic packet loss (LossShell). LinkShell also supports a variety of queueing disciplines such as DropTail, DropHead, and active queue management schemes like CoDel. Each of Mahimahi's network emulation tools can be arbitrarily nested within one another providing more experimental flexibility.

Use unmodified client applications

Recording and replaying sites can be done using commercial browsers such as Google Chrome, Mozilla Firefox, and Safari. Additionally, other client applications including virtual machines and mobile-device emulators can be run unmodified within each of Mahimahi's tools.

Network Emulation Tools

DelayShell

Every packet entering or leaving DelayShell's container is delayed by the specified one-way delay (in milliseconds). This technique enforces a fixed delay on a per-packet basis.

LinkShell

LinkShell is a network emulation tool that emulates links using packet delivery trace files. An uplink trace file emulates the link from LinkShell's container to the Internet and a downlink trace file emulates the link from the Internet to LinkShell's container.

LinkShell can emulate both time-varying links, such as cellular links, and links with fixed link speeds. When a packet arrives at the link (from either the Internet or from LinkShell), it is directly placed into one of two packet queues depending on its intended direction: the uplink queue or the downlink queue. LinkShell releases packets from each queue based on the corresponding input packet-delivery trace.

Each line in a trace file represents a packet delivery opportunity: the time at which an MTU-sized packet can be delivered in the emulation. Accounting is done at the byte-level, and each delivery opportunity represents the ability to deliver 1500 bytes. Thus, a single line in the trace file can delivery several smaller packets whose sizes sum to 1500 bytes. Delivery opportunities are wasted if bytes are unavailable at the instant of an opportunity. When LinkShell reaches the end of an input trace file, it wraps around to the beginning of the trace file. LinkShell can be nested within DelayShell to flexibly create links with a user-supplied one-way delay and a user-supplied link rate.

LinkShell supports live graphing to visualize a process's use of the network. Supported plots include the transfer rate as well as the per-packet queueing delay for traffic leaving (uplink) and entering (downlink) LinkShell's container.

LossShell

Packets are lost at a given rate (between 0 and 1) both when leaving (uplink) and entering (downlink) the LossShell's container.

HTTP Record-and-Replay Tools

RecordShell

RecordShell transparently proxies outgoing HTTP and HTTPS connections, saving the requests, corresponding responses, and IP address of each Web server contacted in the given directory. RecordShell uses a self-signed TLS certificate in its HTTPS proxy, causing typical Web browsers to reject it. For testing or debugging purposes, this behavior can usually be turned off, e.g.: with the --no-check-certificate option to wget or the --ignore-certificate-errors option to the Chromium Browser.

ReplayShell

Replays a saved session from a previous run of RecordShell. Unlike most mahimahi tools, the ReplayShell container does not have a network connection to the outside world. Instead, it has dummy network interfaces bound to each IP address on which a Web server in the saved session had answered a request. ReplayShell runs an Apache2 Web server bound to each such IP address inside the container. Each Web server emulates the corresponding server from the saved session. When receiving a request that matches one in the directory, the corresponding Apache2 server replies with the same reply as previously captured.

ReplayShell can be used to measure the performance of Web browsers on complex websites and the effect of changes in Web protocols (e.g. HTTP, HTTP/2, SPDY, QUIC). Unlike tools like web-page-replay, ReplayShell preserves the sharded structure of a website, binds to the actual IP addresses that the real website used, and serves requests from real Web servers.

Technical Paper

The Mahimahi research paper, which provides more details about the design and evaluation of Mahimahi, was presented at the 2015 USENIX Annual Technical Conference.

To spawn a shell with a delayed, lossy link to the Internet:

mm-delay 50 mm-loss uplink 0.2

Running this command will result in a shell prompt of "[delay 50 ms] [loss up=0.1]" and all programs run inside this environment will have a 100 ms minimum RTT and the emulated link has a loss rate of 20%.

To record a page load from www.nytimes.com in Chromium:

mm-webrecord /tmp/nytimes chromium-browser --ignore-certificate-errors --user-data-dir=/tmp/nonexistent$(date +%s%N) www.nytimes.com

The use of "--user-data-dir=/tmp/nonexistent$(date +%s%N)" is to prevent the browser from reusing an existing chromium-browser process.

To emulate a variable cellular network and visualize a process's use of the network:

mm-delay 20 mm-link --meter-all /usr/share/mahimahi/traces/Verizon-LTE-short.up /usr/share/mahimahi/traces/Verizon-LTE-short.down

This will result in the following shell prompt: "[delay 20 ms] [link] $" and both per-packet queueing delay and uplink/downlink throughput will be plotted in real-time.

To make Chromium retrieve the saved website from the example above over a delayed, lossy link with throughput of 1 full-sized packet per millisecond:

mm-webreplay /tmp/nytimes mm-delay 50 mm-loss uplink 0.1 mm-link <(echo 1) <(echo 1) -- chromium-browser --ignore-certificate-errors --user-data-dir=/tmp/nonexistent$(date +%s%N) www.nytimes.com

Exiting

To exit any shell, simply use exit. When nesting shells, exit will only exit the inner-most shell.

You can verify that you have exited a given shell by checking that there is no prompt in the terminal (eg. [delay 10 ms] ).

Manual

More details can be found in the mahimahi(1) manual pages.

Mahimahi was written by Ravi Netravali, Anirudh Sivaraman, Greg D. Hill, and Keith Winstein.

For any questions or suggestions, please e-mail mahimahi@mit.edu