How do Download Managers accelerate file downloads?
Although broadband Internet access is getting more and more common, people's demand for faster file downloads has never stopped. It is this desire for speed that gave birth to Download Managers, which accelerate file downloads for people like you.
Plenty of Download Managers are out there on the market for you to choose from. Just to name a few, there are GetRight, Download Accelerator, FlashGet...etc. They may use different strategies to speed up file downloads, but their basic principle is the same. In this article, we are going to discuss how they work.
Before we delve into the technical details, let's have some water fun first. How would you fill up 10 buckets with water within the shortest time? One of the many ways of doing it is to line them all up in front of a faucet, turn on the faucet at full blast and then fill up the buckets one by one. Another way of doing it might be to put them under 10 different faucets and fill them up at the same time. Which way would you choose? I would go for the second strategy.
When it comes to downloading files, the situation is essentially the same. Say, you are going to download a file whose size is 100 MB (Mega Bytes). If you use a regular browser to download the file, it will take 100 seconds to download the file if the server delivers a maximum download bandwidth of 1MB per second per connection. However, if you can somehow break up the file into 2 halves and download them simultaneously, you can cut the download time to 50 seconds. That is a 100% increase in speed! By doing the math, you know that the more pieces you break up the file into, the more time you will save. Does it sound like a crazy idea?
That is not a crazy idea at all. In fact, that is exactly how most (if not all) Download Managers accelerate file downloads. Let's take a look at a real-life example:
I used a Download Manager to download the file "gaim-0.79.exe" (size: 6.2MB) from one of the sourceforge servers (in this case, it is belnet.dl.sourceforge.net). The very first thing it did was to make an ordinary HTTP request to retrieve the file as shown below:
There was nothing special in the HTTP request made by the Download Manager. From the Content-Length header of the HTTP response, the Download Manager learned that the file size was 6,284,421 bytes:
Content-Length: 6284421
Once the Download Manager knew the file size, it calculated how many pieces it wanted to break the file into and how big each piece should be. Then, it opened up another connection (c.f. turning on another faucet) to the sourceforge server to download the "second piece":
This time, the HTTP request made by the Download Manager was a bit unusual. There was a Range header:
Range: bytes=1413878-
It instructed the sourceforge server to return the file from the 1,413,878 th byte onwards. You may ask: The Download Manager was already downloading the entire file via its first connection, would doing this actually increase the download time? That was a good question. Although the Download Manager requested the entire file in its first connection, it would however terminate the connection as soon as it obtained the first 1,413,877 bytes so that it wouldn't waste any time and resources downloading the same portion of data twice.
As you can imagine, the Download Manager would make several HTTP requests to download different portions of the same file simultaneously. The following screenshot shows you what it did to obtain the "third piece":
Being a smart dude that you are, you must have spotted that the Range header was different from last time:
Range: bytes=2109669-
This process continued until the Download Manager finished breaking up the file into its desired number of pieces. When the Download Manager obtained all the pieces, it would assemble them into one single file that you had requested -- i.e. "gaim-0.79.exe" in this case. Now that you know how Download Managers work, do you agree that accelerating file downloads isn't actually that mysterious.
That's all. I hope you enjoyed the discussion. I found out the above with HttpRevealer. You can explore the web yourself too!
Steven Chau
GetRight® is a registered trademark of Headlight Software, Inc. All other company and product names may be trademarks of the respective companies with which they are associated.