A Detailed Walk Through the Process

  1. The BitTorrent tracker is started on the image server by the command /etc/init.d/systemimager-server-bittorrent start. The tracker allows the distribution of all the torrents that are in the directory /var/lib/systemimager/torrents. Torrents needed for the imaging will be generated in the next step.

  2. The BitTorrent protocol has been designed to transfer only regular files and directories, but it's not natively able to transfer all the UNIX metadata and special files (like your /dev/ for example). For this reason is necessary to "map" all the files in a image to a single regular file. During the systemimager-server-bittorrent startup, SystemImager provides to generate a tarball and a ".torrent" file for each image and override to be distributed by this transport. Tarballs are stored in /var/lib/systemimager/tarballs and ".torrent"s in /var/lib/systemimager/torrents.

  3. After the tarballs and torrents generation, the first seeder is started on the image server (after the tracker) by the script /etc/init.d/systemimager-server-bittorrent. The first seeder is the only peer that owns all the chunks of the files to be distributed to the other peers from the beginning. During the first phase of the imaging process it constitutes the only bottleneck (like in the client-server approach), but when the first chunks of data are distributed to some peers (peer set) they begin to talk together, exploiting the advantages of the peer-to-peer network and freeing the image server from the load. In this way, after this transition phase, the image server becomes like another peer in the swarm and it does not constitute a bottleneck anymore.

  4. Clients start with a boot package (kernel + initrd.img) that includes the BitTorrent client. The first step for a client is to download the needed torrents. This is done using the rsync protocol (torrents are really small compared to the whole image and this doesn't represent a problem in terms of scalability, also with a huge number of clients downloading them at the same time).

  5. When the clients have the needed torrents they can start to use the BitTorrent protocol to download anything they need. The first files downloaded via BitTorrent are the BOEL binaries (distributed in a tarball and extracted in the client after the download). The clients continue to seed (upload) the BOEL binaries tarball during the whole auto-installation process, also when they have extracted it, giving their upload band availability to the newer approaching clients.

  6. After the download of the BOEL binaries tarball and a first system initialization (module autodetection, disk partitioning, filesystem creation, etc.) it's time to download the image tarball via BitTorrent. This operation works in the same way as the BOEL binaries distribution.

  7. The image tarball is extracted in the client's filesystem. During the extraction the client continues to act as a seeder (uploader) for the image tarball. After the extraction the image tarball is removed from the client host and the BitTorrent client is stopped.

  8. The overrides are downloaded in the same way.

  9. The client is rebooted/kexec-ed/powered-off or starts to beep, according to the post-install action.