fuzz: new fuzzer for dirs.c This found a six-year-old bug immediately, and then I put it through a few CPU-days of time before sending it. Differential Revision: https://phab.mercurial-scm.org/D7031

dirs: fix trivial over-read of input data This code, introduced in 8c0a7eeda06d, was intentionally over-reading an input string to avoid getting a shared string object for a one-byte input. Unfortunately with an empty input (like in the case of a fuzzer getting started) this was a trivial over-read and triggered an AddressSanitizer failure. I went out of my way to make sure the code still does the copy-avoidance tricks. I don't think this change will cost us much performance since the one-character strings should be cached aggressively anyway. Differential Revision: https://phab.mercurial-scm.org/D7030

sidedatacopies: deal with upgrading and downgrading to that format This is quite useful to test this on real life data. Differential Revision: https://phab.mercurial-scm.org/D6955

sidedatacopies: move various copies related function to the copies modules We will need to access these logic form the copies module. So we move them from their higher level module to the lower level `copies` module. We cannot use them from their top level module as it would create cycles. Differential Revision: https://phab.mercurial-scm.org/D6954

sidedatacopies: read rename information from sidedata Repository using the new format now use changeset centric algorithm and read the copies information from the changelog sidedata. Differential Revision: https://phab.mercurial-scm.org/D6953

changelog: make copies related function return None or a valid value With the previous code, existing but empty value were not "decoded", leading to the method returning one of `None`, some valid value (`list` or `dict`) or `b''`. On a general basis, not explicitly checking for None is a source of bugs. Having a clean return types will help the side-data copies code in future changesets. Differential Revision: https://phab.mercurial-scm.org/D7037

test: fix zstd related output in pure tests Differential Revision: https://phab.mercurial-scm.org/D7036

sidedatacopies: get and store sidedata in the changelogrevision object The object provide a simple way to access changelog entry, we need it to also bear the sidedata value. Since the sidedata are retrieved at the same time as the revision, we can do that without extra cost. Differential Revision: https://phab.mercurial-scm.org/D6951

sidedatacopies: write copies information in sidedata when applicable If the format of the repository indicate it stores copies information into changeset's sidedata, then we actually write that information into sidedata at commit time. It will be put to use in later changesets. Currently, we store all field unconditionally, but that is likely to change in the future for the sake of efficiency. Differential Revision: https://phab.mercurial-scm.org/D6950

sidedatacopies: introduce a sidedata testcase for test-copies-in-changeset.t For now it is equivalent to the filelog case, but introducing this early helps make the comings changesets clearer. Differential Revision: https://phab.mercurial-scm.org/D6949

sidedatacopies: introduce a sidedata testcase for test-copies-unrelated.t For now it is equivalent to the filelog case, but introducing this early helps make the comings changesets clearer. Differential Revision: https://phab.mercurial-scm.org/D6948

sidedatacopies: introduce a sidedata testcase for test-copies.t For now it is equivalent to the filelog case, but introducing this early helps make the comings changesets clearer. Differential Revision: https://phab.mercurial-scm.org/D6947

sidedatacopies: teach upgrade about the new requirement The `debugformat` and `debugupgraderepo` command now detect the requirement. (upgrade to and from are not currently possible). Differential Revision: https://phab.mercurial-scm.org/D6946

sidedatacopies: add a new requirement for storing copies into sidedata The end goal is to have changesets centric sidedata information stored into changelog sidedata. This make it possible to use the changeset based copy tracing algorithm on any repository without affecting hashes. The actual implementation is coming. The feature is marked as experimental (do not use in production) until we stabilise details about the format. Differential Revision: https://phab.mercurial-scm.org/D6945

debugsidedata: small doc improvement The command has a mode to actually show the sidedata, but it wasn't documented. Differential Revision: https://phab.mercurial-scm.org/D6944

upgrade: allow for `sidedata` removal If the side-date feature is removed, we need to drop them. Differential Revision: https://phab.mercurial-scm.org/D6943

upgrade: allow upgrade to repository using sidedata Repository can now be migrated to support sidedata. More requirements and migration will be needed to actual side-data usage. This is a step in that direction. To test the feature, we leverage the test extension. It make sure the `update` part of the side-data migration actually works. Differential Revision: https://phab.mercurial-scm.org/D6942

revlog: add a way to control sidedata changes during revlog.clone We introduce a new argument to pass a callable dealing with the side data logic. Since the side data are usually associated to higher level logic, the revlog code itself is unlikely to know that to do itself. As a result the higher level upgrade code will be responsible to decide what needs to changes. The lower level revlog.clone just get simple instructions to apply. Differential Revision: https://phab.mercurial-scm.org/D6941

sidedata: use only changegroup3 if sidedata is in use This is necessary to exchange the flags over the wire (and within bundles). Differential Revision: https://phab.mercurial-scm.org/D6940

sidedata: apply basic but tight security around exchange We don't currently have code to deal with exchange between repository using sidedata and repository not using sidedata. Until we implement such code (eg: dropping side data when pushing to a non-sidedata repo) we prevent the two kind of repo to speak to each other. This is somewhere similar to what 'treemanifest' does. Note that sidedata exchange is broken unless one use changegroup v3 anyway. See next changeset for details. Differential Revision: https://phab.mercurial-scm.org/D6939

perf: use `setup` function in `perfdirstatewrite` The command seems to pre-date the introduction of the `setup` support in timer. We move the line that is obviously about benchmark setup in such `setup` function.

perf: document `perfdirstatewrite`

perf: use `setup` function in `perfdirfoldmap` The command seems to pre-date the introduction of the `setup` support in timer. We move the line that is obviously about benchmark setup in such `setup` function.

perf: document `perfdirfoldmap`

perf: use `setup` function in `perfdirstatefoldmap` The command seems to pre-date the introduction of the `setup` support in timer. We move the line that is obviously about benchmark setup in such `setup` function.

perf: document `perfdirstatefoldmap`

perf: use `setup` function in `perfdirstatedirs` The command seems to pre-date the introduction of the `setup` support in timer. We move the line that is obviously about benchmark setup in such `setup` function.

perf: document `perfdirstatedirs`

perf: use `setup` function in `perfdirstate` The command seems to pre-date the introduction of the `setup` support in timer. We move the line that is obviously about benchmark setup in such `setup` function.

perf: document `perfdirstate`