Saturday, June 25, 2011

Bittorrent stub files done through Sparse files

When a bittorrent client reads the metainfo file it will scan the download path to see what files have already been downloaded. If the files don't exist it's common practice to create "stub" files. The client crams the pieces into these stub files until they can be validated by a SHA1 sum. I found two ways of making stub files:

1. Naive Method
for (ii=0; ii<filesize; ii += stridelength)
  char string[stridelength];

2. Sparse files method

This is insanely quick compared to the method 1! The man page for lseek() explains this nicely:
"The lseek() function allows the file offset to be set beyond the end of the file (but this does not change the size of the file). If data is later written at this point, subsequent reads of the data in the gap (a "hole") return null bytes ('\0') until data is actually written into the gap.".
Does this work on OSes other than Linux?