800 runs of dig: what one semester of internet measurement taught me
In the English-language course Introduction to Internet Measurements I designed, ran and analysed a measurement every week. The last one was my own RIPE Atlas study.
Five domains, four DNS resolvers, UDP and TCP, 20 runs each. That’s 800 runs of dig, and at the end a heatmap that shows at a glance which resolver gets slow where.
That was the fourth of five assignments in Introduction to Internet Measurements, a course at HPI taught entirely in English. Every week worked the same way: design a measurement, run it, analyse the data and write down what it shows and what it doesn’t. All individual work. Over the term that added up to about 5,900 lines of Python, all mine.
It started small. My own ping and traceroute series every five minutes, a Wireshark capture. Then locating speedtest servers with GeoIP and checking whether distance and throughput are related. Then passive data: NetFlow with SiLK, who talks the most on a network and how the volume per minute is spread. And MAWI traces, real captures from a backbone link, from which I pulled HTTP status codes and the split of DNS over UDP and TCP.
The DNS study is the part the demo on this page replays, using the 800 measurements from the course. The pipeline is simple. A shell script calls dig for every combination of domain, resolver and protocol and writes the response time to a CSV. A Python script reads the CSV, computes percentiles and draws the heatmap: rows are domains, columns are resolvers, colour is time.
- 01dnsLookUp.sh
- 02dnsTimes.csv
- 03analyseDNS.py
- 04RIPE Atlas
Chapters 1 to 3 use my real course measurements (800 dig queries from one machine, one run; not a general benchmark). Chapter 4 shows only the study design from my homework, with synthetic probe positions and no measurement results.
Why percentiles? Because the average lies when it comes to network times. One single outlier of a second pulls the mean up even though 19 of 20 queries were fast. The 95th percentile says something else: in 95 out of 100 cases you wait at most this long. That’s the number a user actually feels. The distribution view in the demo shows exactly that, every measurement sorted instead of one number.
The fifth assignment was open. I designed my own study with RIPE Atlas, a network of measurement probes around the world. The question: how do Cloudflare 1.1.1.1 and DNS4EU compare when you ask from the Starlink network? The nasty part about DNS is caching. Ask the same thing twice and the second time you’re measuring the cache, not the resolver. So every probe gets its own subdomain with its probe ID in it, a name nobody has ever asked for before. On top of that, three scripts that create the measurements, download the results and analyse them, and a README with research questions, a measurement matrix and a calculation of how many credits the whole thing costs.
What I’m taking away. Confounders first, then the measurement. Without the subdomain trick I would mostly have measured the cache. Second, percentiles instead of means, at least for anything that involves waiting. Third, writing it up is part of measuring. I handed in the DNS study as a LaTeX report in English, and only while writing do you find out which plot actually shows something.
Tools and skills: Python with pandas and matplotlib, shell and dig, the RIPE Atlas API, MaxMind GeoLite2, SiLK, Wireshark, LaTeX. And experiment design, meaning the question of what you’re actually measuring before you measure it.