caches: invalidate store caches when lock is taken The fncache was not being properly invalidated each time the lock was taken, so in theory it could contain old data from prior to the caller having the lock. This changes it to be invalidated as soon as the lock is taken (same as all our other caches).

fncache: move fncache writing to be in a transaction Previously the fncache was written at lock.release time. This meant it was not tracked by a transaction, and if an error occurred during the fncache write it would fail to update the fncache, but would not rollback the transaction, resulting in an fncache that was not in sync with the files on disk (which causes verify to fail, and causes streaming clones to not copy all the revlogs). This uses the new transaction backup mechanism to make the fncache transacted. It also moves the fncache from being written at lock.release time, to being written at transaction.close time.

transaction: add support for non-append files This adds support for normal, non-append-only files in transactions. For example, .hg/store/fncache and .hg/store/phaseroots should be written as part of the transaction, but are not append only files. This adds a journal.backupfiles along side the normal journal. This tracks which files have been backed up as part of the transaction. transaction.addbackup() creates a backup of the file (using a hardlink), which is later used to recover in the event of the transaction failing. Using a seperate journal allows the repository to still be used by older versions of Mercurial. A future patch will use this functionality and add tests for it.

transaction: add onclose/onabort hook for pre-close logic Adds an optional onclose parameter to transactions that gets called just before the transaction is committed. This allows things that build up data over the course of the transaction (like the fncache) to commit their data. Also adds onabort. It's not used, but will allow extensions to hook into onclose and onabort to provide transaction support.

clone: put streaming clones in a transaction Streaming clones were writing to files outside of a transaction. Soon the fncache will be written at transaction close time, so we need streaming clones to be in a transaction.

fncache: remove the rewriting logic The fncache could rewrite itself during a read operation if it noticed any entries that were no longer on disk. This was problematic because it caused Mercurial to perform write operations outside the scope of a lock or transaction, which could interefere with any other pending writes. This will be replaced in a future patch by logic that cleans up the fncache as files are deleted during strips.

pull: prevent duplicated entry in `op.pulledsubset` In the bare pull case we could add the same node multiple time to the `pulloperation.pulledsubset`. Beside being a bit wrong this confused the new revset implementation of `revset._revancestor` into giving bad result. This changeset fix the pull operation part. The fix for the revset itself will come in another changeset.

bundle2: part params

bundle2: support for bundling and unbundling payload We add the ability to bundle and unbundle a payload in parts. The payload is the actual binary data of the part. It is used to convey all the applicative data. For now we stick to very simple implementation with all the data fit in single chunk. This open the door to some bundle2 testing usage. This will be improved before bundle2 get used for real. We need to be able to stream the payload in multiple part to exchange any changegroup efficiently. This simple version will do for now. Bundling and unbundling are done in the same changeset because the test for parts is less modular. However, the result is not too complex.

merge with stable