bundle: add an experimental knob to include obsmarkers in bundle The "hg bundle" command is a good place to test if the inclusion of obsmarkers within a bundle is working well (part exists, content is correct etc). So we add a way to have them included. Ideally, this would be controlled by a change around bundlespec (bundlespec "v3" + arguments). However, my main goal is to have obsmarkers included in bundle created by the 'hg strip' command, not the 'hg bundle' so for now I'm avoiding the detour through bundlespec rework territory. Better debug output for obsmarkers in 'debugbundle' will be added in later changesets. The 'test-obsolete-bundle-strip.t' test will also get updated in a later changeset to keep the current changeset smaller.

bundle2: move function building obsmarker-part in the bundle2 module We move it next to similar part building functions. We will need it for the "writenewbundle" logic. This will allow us to easily include obsmarkers in on-disk bundle, a necessary step before having `hg strip` also operate on markers. (Yes, the bundle2 module was already too large, but there any many interdependencies between its components so it is non-trivial to split, this is a quest for another adventure.)

policy: remove unused policynoc and policynocffi constants

cffi: remove superfluous "if True" blocks

cffi: split modules from pure The copyright lines are updated per change history. cffi/osutil.py isn't tested since I have no access to OS X machine right now, sorry.

policy: extend API version checks for cffi This is just a stub for future extension. I could add a version constant to CFFI modules by putting it to both ffi.set_source() and ffi.cdef(), but that doesn't seem right. So for now, cffi modules will be explicitly unversioned (i.e. version constant must be undefined or set to None.) We can revisit it later when we need to consider CFFI support more seriously.

filterpyflakes: allow reexporting pure symbols from cffi modules cffi modules will do 'from ..pure.<module> import *'.

import-checker: guess names of C extension modules Since extension modules aren't included in the list of source files, they need to be populated somehow. Otherwise the import from cext/cffi would be treated as a global one.

import-checker: convert localmods to a set of module names This makes it easy to add a source-less module name to the set.

import-checker: allow importing symbols from pure modules This allows us to re-export pure functions from cffi modules: # mercurial/cffi/base85.py from ..pure.base85 import *

cffi: put compiled modules into mercurial.cffi package Don't pollute the top-level namespace.

cffi: rename build scripts This frees up cffi package for modules to be split from pure.

match: remove support for includes from patternmatcher Includes (and excludes) are now delegated to the includematcher.

match: simplify includematcher a bit The "include" we have in symbols is redundant and the double negative in visitdir() can be removed.

match: remove support for non-include patterns from includematcher The includematcher will always get at least one include pattern and will never get any non-include patterns, so we can remove most of the code in it. This patch does mostly straight-forward deletions of code. We will clean up further later.

match: split up main matcher into patternmatcher and includematcher At this point the includematcher is an exact copy of the main matcher class. We will specialize and simplify both classes in the following patches. This initial unmodified copy is just to make the differences clearer. We also rename the main matcher to "patternmatcher" for consistency. I may eventually merge this new includematcher back into the main matcher, but I think doing it this way makes the intermediate steps clearer regardless.

match: remove support for exact matching from main matcher class Exact matching is now handled by the exactmatcher class. We can safely remove _files from the __repr__() implementation, because even though the field is set, the patternspat field is enough for the representation to be unambiguous (which was not the case when the matcher could handle exact matches).

match: handle exact matching using new exactmatcher

merge: use intersectmatchers() in "m2-vs-ma optimization" It doesn't seem like this can actually happen, but seems like cleaner anyway.

match: handle includes using new intersectionmatcher

match: move entire uipath() implementation to basematcher Even though most matchers will always want to use the relative path in uipath(), when we add support for intersecting matcher, we will want to control which form to use for any kind of matcher without knowing the type (see next patch), so we need the implementation on the base class. Also rename the attribute from "pathrestricted" to "relativeuipath" since there actually are cases where we match everything but still use relative paths (like when the user runs "hg files .." from inside mercurial/).

local-clone: also copy tags related caches This caches provide a large speedup for some repositories. Keeping it around is valuable.

local-clone: also copy revs-branch-cache files This cache provides a large speedup for some repositories. Keeping it around is valuable.

local-clone: extract the listing of caches to copy Right now, the clone only copies the branchmap caches. There are multiple other valuable caches that we should copy and extensions might add their own. So we add a function to list the cache files to copy from the repository. The repository argument is unused but extensions will want it.

local-clone: extract the closure copying caches Closures often get on the way. They are not much value in having that as a closure so I'm extracting it at the module level.

test: add isolated prune case (to test-obsolete-bundle-strip.t) This adds a test where the prune marker is not related to any other obsmarkers.

test-obsolete-bundle-strip: add a complex split and fold case This is a more complex case that checks the logic used when split and fold gets into play.

test-obsolete-bundle-strip: add cases with prune on missing revs Same as the previously added case, but the prune is no longer known locally. This will mostly matter for the strip testing. Introducing the test early will help clarify patches related to strip.

obsolete: fix relevant-obsmarkers computation on pruned changeset The markers pruning a node was not directly considered relevant for the pruned node, only to its parents. This went unnoticed during obsmarkers exchange because all ancestors of the pruned node would be included in the computation. This still affects obsmarkers exchange a bit since "inline" prune markers would be ignored (see second test case). This went unnoticed, because in such case, we always push another obsolescence markers for that node. We add explicit tests covering this case. (The set of relevant changeset is use in the obsmarkers discovery protocol used in the evolve experimental extension, the impact will be handled on the extension side).

test: add a test file for relevant obsmarkers and its usage The logic around obsmarkers "relevant" to a set of revs have a couple of test around in other places but no systematic testing. In addition, all the current testing focus on the exchange case (we looks at relevant markers for '::heads'). For bundles, we'll need something a bit different. We'll no longer have set of revision going down to the repository roots. So we'll have to test these cases too. In addition, stripping obsmarkers will introduce new logic around obsmarkers that will need testing too. So a new test file make sense here. We start with a simple tests, more advanced cases are coming in the next changesets. The extra testing catch a minor bug (later in the series).