[11]entry 11 of 34
Trav
this entry is written twice, in full
- 1.in brief
BitTorrent client for the Mac on an async Rust engine
- 2.in general
A fast, quiet BitTorrent client for the Mac, around an engine written from scratch.
- 3.in particular
A headless async Rust engine that speaks TCP and µTP, punches through NAT and pulls from web seeds, inside a native Mac app that updates itself on every green push.
- 1.in brief
a BitTorrent app for the Mac, written from scratch
- 2.in general
A quick, quiet way to download torrents on a Mac, with the downloading engine built by hand.
- 3.in particular
The engine runs on its own inside a proper Mac app, reaches people other downloaders cannot, and updates itself every time a change passes its checks.
context
Trav is a BitTorrent client for the Mac: a native Tauri3 app hosting trav-core, a headless async engine in Rust1, behind the Nova UI, a Next.js4 static export in this site’s design language with three themes (Ink, Paper and Phosphor). It began as one engine with three front-ends; the terminal dashboard, the CLI and the Docker image have since been removed so the work goes into one app. It ships as a universal DMG for Apple Silicon and Intel (macOS 11+), and every push to main that passes CI is released as a new version, numbered by its commit count on main.
the problem
A client that only speaks plain TCP to the peers a tracker hands it is slow on the real internet. Much of a swarm sits behind home routers and carrier-grade NAT where nothing can dial it, some peers refuse unencrypted traffic, an old torrent may have no peers online at all, and a client that seeds flat out fills the queue in front of a home uplink until its own downloads crawl. All of it has to happen without the window ever stalling.
what i built
- 1.
Downloading: bencode, metainfo and magnet parsing, rarest-first or sequential picking over per-peer bitfields, adaptive request pipelining, per-file priorities, SHA-1 checks and disk writes on spawn_blocking, and an end-game where up to four peers race any block that has waited more than 1.5 s
- 2.
TCP and µTP (BEP 29) on one port, µTP with LEDBAT congestion control, selective acks and fast retransmit, checked against libtorrent in both directions; TCP goes first and µTP joins only when TCP cannot connect
- 3.
NAT hole punching (BEP 55) reaches peers behind home routers and carrier-grade NAT through a peer connected to both, and the Fast extension (BEP 6) gets allowed-fast pieces out of peers that are choking
- 4.
Web seeds (BEP 19): HTTP mirrors fetched in whole-piece range requests where the swarm is thinnest, hash-checked like any peer, and dropped when they serve bad data
- 5.
Discovery: HTTP and UDP trackers in tiers, Mainline DHT sharing the µTP socket, PEX, local peer discovery (BEP 14), UPnP and NAT-PMP port mapping, and MSE/PE encryption with a plain fallback
- 6.
An automatic upload limit that, like LEDBAT, reads each connection’s smoothed round-trip time from the kernel (TCP_CONNECTION_INFO) and eases off when seeding starts to queue the uplink
- 7.
Safety: torrent paths sanitised, made unique and kept inside the save folder, files opened O_NOFOLLOW, every parser run against seeded random and mutated input, peer frames and tracker replies size-capped, and peers that send bad data banned
- 8.
The Mac app: a full menu bar with Mac shortcuts, a menu bar icon that fills with overall progress, a Dock progress bar, removals that go to the Trash, and signed self-updates checked hourly
outcome
BEP 3 and twelve extensions to it (5, 6, 9, 10, 11, 12, 14, 15, 19, 29, 47 and 55) plus MSE/PE, in 16,066 lines of Rust1 in the engine crate, with 114 tests including a loopback swarm and the hostile-input suite. In the project’s own runs, TCP first with µTP as the fallback averaged 4.4 to 4.6 MiB/s on one Ubuntu swarm, against 2.9 TCP-only and 1.8 µTP-first; the end-game took the last 5% of a download from 36 s to 12 s; and magnets start in about 3 to 5 s rather than 18 to 24. Not implemented yet: BitTorrent v2 and the IPv6 DHT (BEP 32).
Trav is the Mac app now and nothing else: one engine hosted inside it, and every push to main that passes CI is published as a signed update the app checks for every hour.
context
Trav is a BitTorrent app for the Mac. Underneath is an engine that does all the downloading and knows nothing about windows; around it is a real Mac app, with an interface in the same style as this site and three colour schemes: dark, paper, and a green terminal look. It started as one engine with three different faces; the terminal version, the command line and the version for running on a home server have been removed so that everything goes into the one app. It comes as a single installer for both kinds of Mac, and every change that passes its checks is published as a new version.
the problem
A downloader that only makes ordinary connections to the computers a coordinating server tells it about is slow in practice. Many of the people sharing a file are behind a home router or their provider’s shared address, where nobody can connect to them; some refuse to talk unencrypted; an old file may have nobody sharing it at all; and an app that uploads as fast as it can clogs a home connection until its own downloads crawl. All of that has to happen without the window ever freezing.
what i built
- 1.
The downloading itself: reading the torrent file or magnet link, the rarest pieces first or everything in order for something you want to watch as it arrives, each peer kept busy without being overloaded, individual files skipped or put first, every piece checked and saved off to one side so nothing waits on the disk, and, near the end, faster peers allowed to rescue the last pieces stuck on slow ones.
- 2.
Two ways of connecting on one port: the ordinary kind, tried first because it is faster, and a lighter kind that backs off when the line is busy and reaches peers the ordinary kind cannot. Both were checked against the engine inside the most widely used downloaders, in both directions.
- 3.
Reaching people behind home routers whom nobody could connect to before, by having a peer connected to both introduce them so both sides dial at once, and getting a few pieces even out of peers that are refusing to send.
- 4.
Plain web servers as extra sources: some torrents list ordinary download mirrors, and Trav fetches whole pieces from them where peers are scarce, checks them like anything else, and drops a mirror that sends bad data.
- 5.
Every way of finding peers: the coordinating servers, a shared directory with no server at all, peers passing on other peers, other computers on the same network, a port opened on the router automatically, and the traffic encrypted whenever the other side allows it.
- 6.
An automatic upload limit that watches how long each connection takes to answer, read straight from the operating system, and eases off the moment sharing starts to clog the line.
- 7.
Safety throughout: a torrent cannot write outside its folder or through a shortcut planted inside it, two files cannot overwrite each other, everything that reads data from strangers is tested against random and deliberately broken input, oversized messages are refused, and peers that send bad data are banned.
- 8.
A proper Mac app around it: a full menu bar with the usual shortcuts, an icon in the menu bar that fills up as things download, progress on the Dock icon, removals that go to the Trash, and updates that install themselves once their signature checks out.
outcome
The base protocol and twelve of its published extensions, plus encryption, in about sixteen thousand lines of the engine’s own code, checked by 114 automated tests, including a whole simulated swarm on one machine and the suite of broken inputs. In the project’s own runs, trying the ordinary connection first and the lighter one as a fallback averaged 4.4 to 4.6 megabytes a second on one swarm, against 2.9 with ordinary connections only and 1.8 with the lighter one first; the last 5 per cent of a download went from 36 seconds to 12; and a magnet link starts in about 3 to 5 seconds instead of 18 to 24. Not done yet: the second version of the protocol, and the newer kind of internet address in the serverless directory.
It is only the Mac app now: one engine running inside it, and every change that passes the automated checks is published as a signed update, which the app looks for every hour.