largefiles: use filechunkiter for iterating largefile when serving getlfile The default file iterator is line based and will give odd chunk sizes - often very short and relatively expensive.

largefiles: stat all largefiles in one batch before downloading This avoids a lot of expensive roundtrips to remote repositories ... but might be slightly slower for local operations. This will also change some aborts on missing files to warnings. That will in some situations make it possible to continue working on a repository with missing largefiles.

largefiles: 'put' should store 'source' file in under 'hash', also in localstore

largefiles: getlfile must hit end of HTTP chunked streams to reuse connections We did read the exactly the right number of bytes from the response body. But if the response came in chunked encoding then that meant that the HTTP layer still hadn't read the last 0-sized chunk and expected the app layer to read more data from the stream. The app layer was however happy and sent another request which had to be sent on another HTTP connection while the old one was lingering until some other event closed the connection. Adding an extra read where we expect to hit the end of file makes the HTTP connection ready for reuse. This thus plugs a real socket leak. To distinguish HTTP from SSH we look at self's class, just like it is done in putlfile.

largefiles: drop limitreader, use filechunkiter limit filechunkiter.close was a noop.

largefiles: move protocol conversion into getlfile and make it an iterable Avoid the intermediate limitreader and filechunkiter between getlfile and copyandhash - return the right protocol and put the complexity where it better can be managed.

largefiles: don't close the fd passed to store._getfile

largefiles: remove blecch from lfutil.copyandhash - don't close the passed fd

largefiles: drop lfutil.blockstream - use filechunkiter like everybody else The old chunk size is kept - just to avoid changing it.

largefiles: refactoring - use findfile in localstore._getfile