Here on the ship, we have satellite Internet. This means that, at a good moment, I’ve got round-trip ping times to google.com at just over half a second (~660ms) and download speeds that seem to average around a megabyte per minute. Furthermore, access to the satellite Internet is on a paid basis, so you have to authenticate with the ship’s portal before you can get to the rest of the world. Oh, and the portal requires you to re-authenticate after two hours.
Now, when we’re ashore, we can get data on our phones. We have a swell (expensive, but swell) international roaming plan, which means that when we get into port we get this swell text:
Welcome to Indonesia. You have TravelPass, which lets you use your domestic plan for $10 a day. Pay only on the days you use your mobile service. Use talk, text, or data to start your session. You’ll get 2GB of high speed data. Then enjoy unlimited 3G data for the remainder of the session.
Which is great-ish, when the cell network isn’t overloaded by, say, a couple thousand other people trying to download stuff.
Meanwhile, my phone operating system is a couple of point releases down-rev from the latest. At least one of these releases is a security update, and I’d really like to get that applied since I’m far from my home network. Oh, and how big is that software update I want to download? And it’s competing with what other baloney the apps on my phone are trying to download at the same time? There’s the “solve this particular problem” approach, which is to force my phone to download the update as soon as we hit port, and leave it plugged into a backup battery so that it thinks it’s docked and doesn’t abort the process. But that’s really ignoring the fundamental problem: sometimes, you really do need to receive a large file sent over a slow and/or unreliable link.
Back in the days of dial-up, this was handled by the transport layer and we were all a lot closer to that transport layer. An xmodem file transfer would fail at 98% of the way through if there was suddenly a bunch of line noise (like your roommate picking up the extension) and you’d have to start all over again. So, we used zmodem and were able to resume a download. You can still see this kind of feature in a browser, when you’re downloading some godawful huge file and then, in the middle of the download, you unplug your network. You can often resume the download.
This “resume download” feature isn’t part of the operating system, though (or, not for Linux, iOS, Android, Windows, or MacOS, anyway) but rather a feature of the program initiating the download — in the above example, the web browser. So, what if the thing that’s trying to do this massive download (or upload) isn’t a web browser, but a standalone application? Unless the developers of both the application you’re interacting with and the remote application it’s talking to have already thought about this problem, then the answer is probably, “You’re stuck.”
Now, it’s easy to start decrying the popular frameworks (Electron, Flutter, et al.) that hide even the possibility of handling this well deep inside some barely, if at all, accessible network library, but that’s neither productive nor even fair. It’s still a hell of a lot of work even when you have access to the file system and the low-level networking calls to send/receive a small buffer of bytes. The real situation is, most developers don’t even consider what this somewhat-connected experience is like.
When HTTP was the protocol for data transfer, you could run a caching proxy and then do silly tricks like fire off requests through the proxy for the big things and then unplug your computer. Weaknesses of this hack aside, that’s not how we do things anymore. Nowadays, we assume that there are bad guys warping all the data we send and receive, so we encrypt everything. That means that sticking a proxy in the middle just won’t work — to the apps on both ends, the proxy looks the same as a Bad Guy.
So. Developers. When you’re coming up with a server API that allows for transfer of large blobs of data, please consider a RESUME endpoint.
And, developers, when you’re writing some client for an API that provides a RESUME endpoint, make use of it.