merge with stable

py3: call hgweb.hgweb() with bytes values # skip-blame because just b'' prefixes I believe this should fix some tests. Differential Revision: https://phab.mercurial-scm.org/D4594

py3: use '%d' for integers instead of '%s' Differential Revision: https://phab.mercurial-scm.org/D4593

py3: use "%f" for floats instead of "%s" Differential Revision: https://phab.mercurial-scm.org/D4592

py3: suppress the return value from .write() call Differential Revision: https://phab.mercurial-scm.org/D4591

py3: add b'' prefixes in tests/test-diff-color.t # skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D4590

py3: slice through bytes to prevent getting ascii value I still don't know why python-dev thought it was a nice idea to do this. Differential Revision: https://phab.mercurial-scm.org/D4589

censor: use a reasonable amount of memory Before this change, trying to censor some random revision uses an ever increasing amount of memory (I stopped at 20GB, but it was by no means finished), presumably because these contexts have a lot of information that is kept alive. After this change, the memory usage plateaus quickly. Differential Revision: https://phab.mercurial-scm.org/D4582

help: add internals.wireprotocolrpc to the table

setup: exclude vendored futures package on Python 3 The vendored future can't live on Python 3.

py3: whitelist another passing test Differential Revision: https://phab.mercurial-scm.org/D4562

py3: prevent the win32 ctype _fields_ from being transformed to bytes Otherwise, any hg invocation dies with TypeError: '_fields_' must be a sequence of (name, C type) pairs # skip-blame just a r prefix

cext: fix warnings when building for py3 on Windows MSVC++ 14 now has standard int types that don't need to be redefined (I didn't go back to see when they came along since the build system wants either 2008 or 2015), but doesn't have ssize_t. The FILE pointer in posixfile is only used on python2.

cext: stop preprocessing a partial function call MSVC++ 14 yelled: mercurial/cext/revlog.c(1913): fatal error C1057: unexpected end of file in macro expansion At this point, the C extensions build (with warnings), and it dies in win32.py because the `_fields_` strings in the ctypes classes are being converted to bytes by the source translator.

py3: add b'' to some setup.py strings for Windows These were things found trying to do `make PYTHON="py -3" local`. The following is dumped out, before dying while compiling the C extensions: C:\Program Files\Python37\lib\site-packages\setuptools\dist.py:406: UserWarning: The version specified (b'4.7.1') is an invalid version, this may not work as expected with newer versions of setuptools, pip, and PyPI. Please see PEP 440 for more details. "details." % self.metadata.version running build_py byte-compiling .\mercurial\thirdparty\concurrent\futures\_base.py to _base.cpython-37.pyc File "mercurial\thirdparty\concurrent\futures\_base.py", line 416 raise exception_type, self._exception, self._traceback ^ SyntaxError: invalid syntax # skip-blame since these are just converting to bytes literals

dagop: fix typo spotted while doing unrelated investigation Differential Revision: https://phab.mercurial-scm.org/D4584

hg: don't reuse repo instance after unshare() Unsharing a repository is a pretty invasive procedure and fundamentally changes the behavior of the repository. Currently, hg.unshare() calls into localrepository.__init__ to re-initialize the repository instance. This is a bit hacky. And future commits that refactor how localrepository instances are constructed will make this difficult to support. This commit changes unshare() so it constructs a new repo instance once the unshare I/O has completed. It then poisons the old repo instance so any further use will result in error. Surprisingly, nothing in core appears to access a repo instance after it has been unshared! .. api:: ``hg.unshare()`` now poisons the repo instance so it can't be used. It also returns a new repo instance suitable for interacting with the unshared repository. Differential Revision: https://phab.mercurial-scm.org/D4557

unionrepo: dynamically create repository type from base repository This is basically the same thing we just did for bundlerepo except for union repositories. .. api:: ``unionrepo.unionrepository()`` is no longer usable on its own. To instantiate an instance, call ``unionrepo.instance()`` or ``unionrepo.makeunionrepository()``. Differential Revision: https://phab.mercurial-scm.org/D4556

bundlerepo: dynamically create repository type from base repository Previously, bundlerepository inherited from localrepo.localrepository. You simply instantiated a bundlerepository and its __init__ called localrepo.localrepository.__init__. Things were simple. Unfortunately, this strategy is limiting because it assumes that the base repository is a localrepository instance. And it assumes various properties of localrepository, such as the arguments its __init__ takes. And it prevents us from changing behavior of localrepository.__init__ without also having to change derived classes. Previous and ongoing work to abstract storage revealed these limitations. This commit changes the initialization strategy of bundle repositories to dynamically create a type to represent the repository. Instead of a static type, we instantiate a new local repo instance via localrepo.instance(). We then combine its __class__ with bundlerepository to produce a new type. This ensures that no matter how localrepo.instance() decides to create a repository object, we can derive a bundle repo object from it. i.e. localrepo.instance() could return a type that isn't a localrepository and it would "just work." Well, it would "just work" if bundlerepository's custom implementations only accessed attributes in the documented repository interface. I'm pretty sure it violates the interface contract in a handful of places. But we can worry about that another day. This change gets us closer to doing more clever things around instantiating repository instances without having to worry about teaching bundlerepository about them. .. api:: ``bundlerepo.bundlerepository`` is no longer usable on its own. The class is combined with the class of the base repository it is associated with at run-time. New bundlerepository instances can be obtained by calling ``bundlerepo.instance()`` or ``bundlerepo.makebundlerepository()``. Differential Revision: https://phab.mercurial-scm.org/D4555

bundlerepo: factor out code for instantiating a bundle repository This code will soon become a bit more complicated. So extract to its own function. And change both instantiators of bundlerepository to use it. Differential Revision: https://phab.mercurial-scm.org/D4554

bundlerepo: pass create=True I don't want to know how this came to be. Maybe a holdover from the days before Python had a bool type? Differential Revision: https://phab.mercurial-scm.org/D4553

shelve: use bundlerepo.instance() to construct a repo object The instance() functions are preferred over cls.__init__ for creating repo instances. It doesn't really matter now. But future commits will refactor the bundlerepository class in ways that will cause the old way to break. Differential Revision: https://phab.mercurial-scm.org/D4552

templatekw: add experimental {status} keyword This is another example of fctx-based keywords. I think this is somewhat useful in log templates.

templatekw: add option to include ignored/clean/unknown files in cache They will be necessary to provide {status} of files.

templatekw: keep status tuple in cache dict and rename cache key accordingly There's no point to drop tail elements, which are mostly empty lists.

templatekw: extract function that computes and caches file status

py3: use sysstr() to convert ProgrammingError bytes with no unicode error risk msg.decode('utf8') may fail if msg isn't an ASCII string, and that's possible as we sometimes embed a filename in the error message for example.

revlog: reuse cached delta for identical base revision (issue5975) Since 8f83a953dddf, we skip over empty deltas when choosing a delta base. Such delta happens when two distinct revisions have the same content. The remote might be sending a delta against such revision within the bundle. In that case, the delta base is no longer considered, but the cached one could still, be used with the equivalent revision. Not reusing the delta from the bundle can have a significant performance impact, so we now make sure with doing so when possible.

snapshot: fix line order when skipping over empty deltas The code movement in 37957e07138c introduced an error. Since 8f83a953dddf, we discarded some revisions because they are identical to their delta base (and use that delta base instead). That logic is good, however, in 37957e07138c we mixed up the order of two line, adding the "new" revision to the set of already tested one, instead of the discarded one. So in practice, we were never investigating any revisions in a chain starting with an empty delta. Creating significantly worst delta chain (eg: Mercurial's manifest move goes from about 60MB up to about 80MB).

tests: stabilize change for handling not quoting non-empty-directory The change originated in cb1329738d64. I suspect the problem is with the combination of (re) and the '\' to '/' retry on Windows. I've no idea if py3 on Windows needs the quoting, since it can't even run `hg` with no arguments. (It's dying somewhere on the ctype declarations when win32.py is imported.)