tests: perform an ast parse with Python 3 Previously, test-check-py3-compat.t parsed Python files with Python 2 and looked for known patterns that are incompatible with Python 3. Now that we have a mechanism for invoking Python 3 interpreters from tests, we can expand check-py3-compat.py and its corresponding .t test to perform an additional AST parse using Python 3. As the test output shows, we identify a number of new parse failures on Python 3. There are some redundant warnings for missing parentheses for the print function. Given the recent influx of patches around fixing these, the redundancy shouldn't last for too long.

run-tests: add --with-python3 to define a Python 3 interpreter Currently, very few parts of Mercurial run under Python 3, notably the test harness. We want to write tests that run Python 3. For example, we want to extend test-check-py3-compat.t to parse and load Python files. However, we have a problem: finding appropriate files requires running `hg files` and this requires Python 2 until `hg` works with Python 3. As a temporary workaround, we add --with-python3 to the test harness to allow us to define the path to a Python 3 interpreter. This interpreter is made available to the test environment via $PYTHON3 so tests can run things with Python 3 while the test harness and `hg` invocations continue to run from Python 2. To round out the feature, a "py3exe" hghave check has been added.

crecord: add docblock to handlekeypressed This information is pretty useful when reading the code.

crecord: fix docblock indentation

crecord: clean up empty lines at ends of docblocks

filemerge: indicate that local/other are p1/p2

sslutil: use preferred formatting for import syntax

largefiles: add some docstrings

largefiles: drop partial support for not having a user cache 971c55ce03b8 introduced support for not having a "global" user cache. In the rare cases where the environment didn't provide the location of the current home directory, the usercachepath function could return None. That functionality has since bitrotten and several code paths did not correctly check for usercachepath returning None: $ HOME= XDG_CACHE_HOME= hg up --config extensions.largefiles= getting changed largefiles abort: unknown largefiles usercache location Dropping the partial support for it is thus not really a backward compatibility breaking change. Thus: consistently fail early if the usercache location is unknown. It is relevant to be able to control where the largefiles are stored and how they propagate, but that should probably be done differently. The dysfunctional code just gets in the way.

largefiles: refactor usercachepath - extract user cache path function It is convenient to have the user cache location explicitly.

shelve: adds restoring newly created branch (issue5048) (BC) Before this patch shelve never preserved branch information, so after applying unshelve branch was the same as it was on working copy no matter in which branch shelve took place. This patch makes bare shelving(with no files specified, without interactive,include and exclude option) remembers information if the working directory was on newly created branch ,in other words working directory was on different branch than its first parent. In this situation unshelving restores branch information to the working directory.

shelve: make non bare shelve not saving branch information in bundle This patch prepares for restoring newly created branch only on bare shelve later because information about new-branch will be preserved only when shelve was bare and working copy branch was different than branch of its parent. In other case information about new-branch will be gone, so unshelve will not recognise that shelve was made on new-branch and it will not restore branch information from the bundle to the working directory.

shelve: preserve newly created branch on non-bare shelve in wctx (BC) Before this patch current branch in working context wasnt preserved after shelve, this patch makes it restore after update.

crecord: rewrite a comment about filtering patches I couldn't parse the previous version. I think the rewrite better expresses the intent of that comment.

run-tests: teach _processoutput to handle multiple lines of churn Instead of treating expected output as happening in a precise order, and assuming that if a line is missing it will never happen, assume that expected output is a prioritized list of likely matching lines. This means that if: foo/bar (glob) baz/bad (glob) changes to: baz/bad foo/bar instead of generating: baz/bad foo/bar For which we've lost both (glob) markers, we will match both lines and generate: baz/bad (glob) foo/bar (glob) This retains any special annotations we have for lines.

run-tests: indent _processoutput to aid readability for next patch The next commit will loop over the expected[pos] list, this change makes that change easier to review.

namespaces: fix name/node confusion There was a lot of copy/paste here.

tests: add test for "log -r wdir() -p" (issue4871) The issue has been fixed by 93bcc73df8d5, "cmdutil.changeset_printer: pass context into showpatch()". This patch adds test to prevent future regression.

tests: include modified/added/removed files in "log -r wdir()" output This patch prepares for the test of "hg log -r wdir() -p".

contrib: revsetbenchmarks use absolute_import and print_function

contrib: synthrepo use absolute_import

contrib: python-hook-examples use absolute_import

contrib: make perf.py use absolute_import

largefiles: replace invocation of os.path module by vfs in lfutil.py Replaces invocations os.path functions to methods in vfs. Unfortunately (in my view) this makes code less readable, because instead of using clear variable names with path it needs to replace them with vfs(..). I need guidance how to make such transition look more readable. For example in this patch there is example with few places with wvfs.join(standindir), standindir before this patch was absolute path, in this it is changed to relative because it is used also in expression wvfs.join(standindir, pat).

largefiles: replace invocation of os.path module by vfs in lfcommands.py

cache: rebuild branch cache from scratch when inconsistencies are detected This should recover automatically from some corruptions that for unknown reasons are seen in the wild.

cache: safer handling of failing seek when writing revision branch cache If the seek for some reason fails (perhaps because the file is too short to search to the requested position), make sure we seek to the start and rewrite everything. It is unknown if this fixes a real problem that ever happened.

cache: remove branch revision file before rewriting the branch name file New branch names are usually appended to the branch name file. If that fails or the file has been modified by another process, it is rewritten. That left a small opportunity that there could be references to non-existent entries in the file while it was rewritten. To avoid that, remove the revision branch cache file with the references to the branch name file before rewriting the branch name file. Worst case, when interrupted at the wrong time, the cache will be lost and rebuilt next time. It is unknown if this fixes a real problem that ever happened.

chg: do not redirect stdout to /dev/null Redirecting stdout to /dev/null has unwanted side effects, namely ui.write will stop working. This patch removes the redirection code and helps chg to pass test-bad-extension.t.

pager: skip uisetup if chg is detected chg has its own pager implementation that it wants to skip pager's uisetup. It is currently done by redirecting stdout to /dev/null, which has unintended side effects. This patch makes pager aware of chg and skip uisetup directly from pager. We may want to merge chg and pager's pager implementation to make this unnecessary in the future.

chgserver: add an explicit gc to trigger __del__ SocketServer.ForkingMixIn uses os._exit which will skip all cleanup handlers. We want to run __del__ to make things like transactions, {ssh,http}peer, atomictempfile, dirstateguard, etc. work. This patch adds a "gc.collect()" to trigger __del__. It is helpful for chg to pass some test cases in test-devel-warnings.t.

blackbox: do not assume self._bb{vfs,repo,fp} are set in blackboxui.__init__ It's possible for the blackboxui code to do a "del self._bbvfs", then ui.copy() or similar attempt will fail. It will also fail when constructing a blackboxui from a non-blackbox ui. This patch fixes the issue by not assuming any _bb* attr is set.

chg: downgrade "failed to read channel" from abortmsg to debugmsg If the server has an uncaught exception, it will exit without being able to write the channel information. In this case, the client is likely to complain about "failed to read channel", which looks inconsistent with original hg. This patch silences the error message and makes uncaught exception behavior more like original hg. It will help chg to pass test-fileset.t.

histedit: do not close stdin Closing stdin is unexpected by chgserver and is not a good idea generally. This patch refactors related code a bit and make sure stdin is not closed. It will make chg much happier on test-histedit*.t.

tests: reorder hg serve commands chg currently does not support hg serve -d. It has a quick path testing if the command is hg serve -d and fallbacks to hg if so. But the test only works if "serve" is the first argument since the test wants to avoid false positives (for example, "-r serve" is different). This patch reorders "hg server" commands in tests, making them chg friendly.

templater: add debugtemplate command This is useful for debugging template parsing. Several tests are ported to this command.

templater: expand list of parsed templates to template node This patch eliminates a nested data structure other than the parsed tree. ('template', [(op, data), ..]) -> ('template', (op, data), ..) New expanded tree can be processed by common parser functions. This change will help implementing template aliases. Because a (template ..) node should have at least one child node, an empty template (template []) is mapped to (string ''). Also a trivial string (template [(string ..)]) node is unwrapped to (string ..) at parsing phase, instead of compiling phase.

templater: relax type of mapped template Now compiled template fragments are packed into a generic type, (func, data), a string can be a valid template. This change allows us to unwrap a trivial string node. See the next patch for details.

templater: lift parsed and compiled templates to generic data types Before this patch, parsed and compiled templates were kept as lists. That was inconvenient for applying transformation such as alias expansion. This patch changes the types of the outermost objects as follows: stage old new -------- -------------- ------------------------------ parsed [(op, ..)] ('template', [(op, ..)]) compiled [(func, data)] (runtemplate, [(func, data)]) New templater.parse() function has the same signature as revset.parse() and fileset.parse().

tests: python executable path should always be globbed Although this is coming in under the guise of consistency, part of the desire for this is that at least as part of the official Solaris builds, we build with a versioned python interpreter, such as "python2.7", which doesn't match "*python".

crecord: use ui.interface to choose curses interface use ui.interface to select curses mode, instead of experimental.crecord

ui: add new config flag for interface selection This patch introduces a new config flag ui.interface to select the interface for interactive commands. It currently only applies to chunks selection. The config can be overridden on a per feature basis with the flag ui.interface.<feature>. features for the moment can only be 'chunkselector', moving forward we expect to have 'histedit' and other commands there. If an incorrect value is given to ui.interface we print a warning and use the default interface: text. If HGPLAIN is specified we also use the default interface: text. Note that we fail quickly if a feature does not handle all the interfaces that we permit in ui.interface; in future, we could design a fallback path (e.g. blackpearl to curses, curses to text), but let's leave that until we need it.

extensions: also search for extension in the 'hgext3rd' package Mercurial extensions are not meant to be normal python package/module. Yet the lack of an official location to install them means that a lot of them actually install as root level python package, polluting the global Python package namespace and risking collision with more legit packages. As we recently discovered, core python actually support namespace package. A way for multiples distinct "distribution" to share a common top level package without fear of installation headache. (Namespace package allow submodule installed in different location (of the 'sys.path') to be imported properly. So we are fine as long as extension includes a proper 'hgext3rd.__init__.py' to declare the namespace package.) Therefore we introduce a 'hgext3rd' namespace packages and search for extension in it. We'll then recommend third extensions to install themselves in it. Strictly speaking we could just get third party extensions to install in 'hgext' as it is also a namespace package. However, this would make the integration of formerly third party extensions in the main distribution more complicated as the third party install would overwrite the file from the main install. Moreover, having an explicit split between third party and core extensions seems like a good idea. The name 'hgext3rd' have been picked because it is short and seems explicit enough. Other alternative I could think of where: - hgextcontrib - hgextother - hgextunofficial

hgext: use templatekeyword to mark a function as template keyword This patch replaces registration of template keyword function in bundled extensions by registrar.templatekeyword decorator all at once.

templatekw: use templatekeyword to mark a function as template keyword Using decorator can localize changes for adding (or removing) a template keyword function in source code. This patch also removes leading ":KEYWORD:" part in help document of each keywords, because using templatekeyword makes it useless. For similarity to decorator introduced by subsequent patches, this patch uses 'templatekeyword' instead of 'keyword' as a decorator name, even though the former is a little redundant in 'templatekw.py'. file name reason =================== ================= ================================== templatekw.py templatekeyword for similarity to others templatefilters.py templatefilter 'filter' hides Python built-in one templaters.py templatefunc 'func' is too generic

registrar: add templatekeyword to mark a function as template keyword (API) _templateregistrarbase is defined as a super class of templatekeyword, for ease of adding template common features between "keyword", "filter" and "function". This patch also adds loadkeyword() to templatekw, because this combination helps to figure out how they cooperate with each other. Listing up loadkeyword() in dispatch.extraloaders causes implicit loading template keyword functions at loading (3rd party) extension. This change requires that "templatekeyword" attribute of (3rd party) extension is registrar.templatekeyword or so.

chgserver: do not keep repo object The current chgserver design is to use one server to handle multiple repos which has same [extensions] config. Previously the client uses --cwd / to avoid creating a repo object. Now we need to set repo to None before we have code to make "serve" command norepo when it's chg.

chgserver: invalidate the server if extensions fail to load Previously, if extensions fail to load, chg server will just keep working without those extensions. It will print a warning message but only if a new server starts. This patch invalidates the server if any extension failed to load, but still serve the client (hopefully just) once. It will help chg pass some test cases of test-bad-extension.t.

chgserver: add an explicit "reconnect" instruction to validate In some rare cases (next patch), we may want validate to do "unlink" without forcing the client reconnect. This patch addes a new "reconnect" instruction and makes "unlink" not to reconnect by default.

dispatch: flush ui before returning from dispatch A chg client may exit after received the result from runcommand. It is necessary to do a flush to make sure the warning message is printed out and the process waiting for the chg client will actually see the output. This helps chg to pass test-alias.t.

tests: make tests for convert with svn portable svn 1.6.x (at least, 1.6.12 or 1.6.17) might display empty lines, even though svn 1.9.x (at least, 1.9.3) doesn't. To make tests for convert with svn portable, this patch adds "|(^$)" regexp to egrep in filter_svn_output. To avoid similar future issue, this patch adds "|(^$)" regexp to all filter_svn_output (and adjusts test-subrepo-svn.t), even though only test-convert-svn-source.t fails with svn 1.6.x, AFAIK.

merge with stable

test-pager: add a test for pager with color enabled

http: support sending hgargs via POST body instead of in GET or headers narrowhg (for its narrow spec) and remotefilelog (for its large batch requests) would like to be able to make requests with argument sets so absurdly large that they blow out total request size limit on some http servers. As a workaround, support stuffing args at the start of the POST body. We will probably want to leave this behavior off by default in servers forever, because it makes the old "POSTs are only for writes" assumption wrong, which might break some of the simpler authentication configurations.

fsmonitor: flag msc_stdint as no-check-code I'd rather not modify code that we're vendoring, so I'm just marking it this way.

fsmonitor: use custom stdint.h file when compiling with Visual C Visual C/C++ 9, which Python 2.7 is compatible with, doesn't have C99 support and thus doesn't contain a stdint.h file. This changeset adds a custom version of stdint.h, created specifically for Visual C, and uses it when building with that compiler.

tests: handle getaddrinfo reporting "No address associated with hostname" This has been seen on some Fedora 23 systems.

httpconnection: remove obsolete comment about open() When httpsendfile was moved from url.py into httpconnection.py in e7525a555a64 (url: use new http support if requested by the user, 2011-05-06), the comment about not being able to just call open() became obsolete.

sslutil: allow multiple fingerprints per host Certificate pinning via [hostfingerprints] is a useful security feature. Currently, we only support one fingerprint per hostname. This is simple but it fails in the real world: * Switching certificates breaks clients until they change the pinned certificate fingerprint. This incurs client downtime and can require massive amounts of coordination to perform certificate changes. * Some servers operate with multiple certificates on the same hostname. This patch adds support for defining multiple certificate fingerprints per host. This overcomes the deficiencies listed above. I anticipate the primary use case of this feature will be to define both the old and new certificate so a certificate transition can occur with minimal interruption, so this scenario has been called out in the help documentation.

help: add empty lines to hostfingerprints section I think this is now much easier to read.

help: document requirements We didn't have unified documentation of the various repository requirements. This patch changes that.

showstack: use absolute_import

contrib: use absolute_import in win32/hgwebdir_wsgi.py

dispatch: catch KeyboardInterrupt more broadly Because _runcatch() can run long operations in its exception handler, it wasn't enough to catch KeyboardInterrupt at the same level. For example, "hg unknown" will load all extension modules, so we could easily make it crashed by Ctrl-C.

histedit: have dropmissing abort on empty plan We noticed that many users have the intuition of laving the editor empty when they want to abort the operation. The fact that dropmissing allows user to delete all edited commits is not intuitive even for users that asked for it. Let's prevent people from this footgun.

streamclone: fix error when store files grow while stream cloning Effectively a backout of 9fea6b38a8da, but updated to using 'with'.

tests: add test of stream clone of repo that is changing This reveals an error introduced by 9fea6b38a8da.

chgserver: handle ParseError during validate Currently the validate command in chgserver expects config can be loaded without issues but the config can be broken and chg will print a stacktrace instead of the parsing error, if a chg server is already running. This patch adds a handler for ParseError in validate and a new instruction "exit" to make the client exit without abortmsg. A test is also added to make sure it will behave as expected.

dispatch: extract common logic for handling ParseError The way ParseError is handled at two different places in dispatch.py is the same. Move common logic into _formatparse.

chgserver: resolve relative path before sending via system channel The chgserver may have a different cwd from the client because of the side effect of "--cwd" and other possible os.chdir done by extensions. Therefore relative paths can be misunderstood by the client. This patch solves it by expanding relative cwd path to absolute one before sending them via the 'S' channel. It can help chg to pass a testcase in test-alias.t later.

mercurial: use pure Python module policy on Python 3 The C extensions don't yet work with Python 3. Let's minimize the work required to get Mercurial running on Python 3 by always using the pure Python module policy on Python 3.

chg: provide early exception to user See the previous patch for details. Since the socket will be closed by the server, handleresponse() will never return: Traceback (most recent call last): ... chg: abort: failed to read channel

cmdserver: write early exception to 'e' channel in 'unix' mode In 'unix' mode, the server is typically detached from the console. Therefore a client couldn't see the exception that occurred while instantiating the server object. This patch tries to catch the early error and send it to 'e' channel even if the server isn't instantiated yet. This means the error may be sent before the initial hello message. So it's up to the client implementation whether to handle the early error message or error out as protocol violation. The error handling code is also copied to chgserver.py. I'll factor out them later if we manage to get chg passes the test suite.

contrib: make memory.py use absolute_import

check-code: use absolute_import and print_function

encoding: use range() instead of xrange() Python 3 doesn't have xrange(). Instead, range() on Python 3 is a generator, like xrange() is on Python 2. The benefits of xrange() over range() are when there are very large ranges that are too expensive to pre-allocate. The code here is only creating <128 values, so the benefits of xrange() should be negligible. With this patch, encoding.py imports safely on Python 3.

encoding: make HFS+ ignore code Python 3 compatible unichr() doesn't exist in Python 3. chr() is the equivalent there. Unfortunately, we can't use chr() outright because Python 2 only accepts values smaller than 256. Also, Python 3 returns an int when accessing a character of a bytes type (s[x]). So, we have to ord() the values in the assert statement.

extensions: factor import error reporting out To clarify third party extensions lookup, we are about to add a third place where extensions are searched for. So we factor the error reporting logic out to be able to easily reuse it in the next patch.

extensions: extract the 'importh' closure as normal function There is no reason for this to be a closure so we extract it for clarity.

zeroconf: remove leftover camelcase identifier eb9d0e828c30 (zeroconf: remove camelcase in identifiers, 2016-03-01) forgot one occurrence of "numAuthorities", which makes test-paths.t fail for me. I don't even know what zeroconf is, but this patch seems obviously correct and it fixes the failing test case.

hg: acquire wlock while updating the working directory via updatetotally updatetotally() might be invoked outside wlock scope (e.g. invocation via postincoming() at "hg unbundle" or "hg pull"). In such case, acquisition of wlock is needed for consistent view, because parallel "hg update" and/or "hg bookmarks" might change working directory status while executing updatetotally(). Strictly speaking, truly consistent updating should acquire also store lock, because active bookmark might be moved to another one outside wlock scope (e.g. pulling from other repository causes updating current active one). Acquisition of wlock in this patch ensures consistency in as same level as past "hg update".

commands: add postincoming docstring for explanation of arguments

commands: centralize code to update with extra care for non-file components This patch centralizes similar code paths to update the working directory with extra care for non-file components (e.g. bookmark) into newly added function updatetotally(). 'if True' at the beginning of updatetotally() is redundant at this patch, but useful to reduce amount of changes in subsequent patch.

update: omit redundant activating message for already active bookmark This patch also adds "hg bookmarks" invocation into tests, where redundant message is omitted but bookmark activity isn't clear from context.

tests: make test-verify-repo-operations.py not run by default test-verify-repo-operations.py currently starts way too late and extends the running time with -j50 on my machine from around 3:48 min to 6:30 min. We could of course make it run earlier, but the test case seems unlikely to find bugs not covered by other tests, so let's mark it "slow" instead. I think this type of test is better suited to running separately in a long-running job.

ui: log devel warnings

util: refactor getstackframes

util: reword debugstacktrace comment

changelog: avoid slicing raw data until needed Before, we were slicing the original raw text and storing individual variables with values corresponding to each field. This is avoidable overhead. With this patch, we store the offsets of the fields at construction time and perform the slice when a property is accessed. This appears to show a very marginal performance win on its own and the gains are so small as to not be worth reporting. However, this patch marks the end of our parsing refactor, so it is worth reporting the gains from the entire series: author(mpm) 0.896565 0.795987 89% desc(bug) 0.887169 0.803438 90% date(2015) 0.878797 0.773961 88% extra(rebase_source) 0.865446 0.761603 88% author(mpm) or author(greg) 1.801832 1.576025 87% author(mpm) or desc(bug) 1.812438 1.593335 88% date(2015) or branch(default) 0.968276 0.875270 90% author(mpm) or desc(bug) or date(2015) or extra(rebase_source) 3.656193 3.183104 87% Pretty consistent speed-up across the board for any revset accessing changelog revision data. Not bad! It's also worth noting that PyPy appears to experience a similar to marginally greater speed-up as well! According to statprof, revsets accessing changelog revision data are now clearly dominated by zlib decompression (16-17% of execution time). Surprisingly, it appears the most expensive part of revision parsing are the various text.index() calls to search for newlines! These appear to cumulatively add up to 5+% of execution time. I reckon implementing the parsing in C would make things marginally faster. If accessing larger strings (such as the commit message), encoding.tolocal() is the most expensive procedure outside of decompression.

changelog: parse description last Before, we first searched for the double newline as the first step in the parse then moved to the front of the string and worked our way to the back again. This made sense when we were splitting the raw text on the double newline. But in our new newline scanning based approach, this feels awkward. This patch updates the parsing logic to parse the text linearly and deal with the description field last. Because we're avoiding an extra string scan, revsets appear to demonstrate a very slight performance win. But the percentage change is marginal, so the numbers aren't worth reporting.

changelog: lazily parse files More of the same. Again, modest revset performance wins: author(mpm) 0.896565 0.822961 0.805156 desc(bug) 0.887169 0.847054 0.798101 date(2015) 0.878797 0.811613 0.786689 extra(rebase_source) 0.865446 0.797756 0.777408 author(mpm) or author(greg) 1.801832 1.668172 1.626547 author(mpm) or desc(bug) 1.812438 1.677608 1.613941 date(2015) or branch(default) 0.968276 0.896032 0.869017

changelog: lazily parse date/extra field This is probably the most complicated patch in the parsing refactor. Because the date and extras are encoded in the same field, we stuff the entire field into a dedicated variable and add a property for accessing the sub-components of each. There is some duplicated code here. But the code is relatively simple, so it shouldn't be a big deal. We see revset performance wins across the board: author(mpm) 0.896565 0.876713 0.822961 desc(bug) 0.887169 0.895514 0.847054 date(2015) 0.878797 0.820987 0.811613 extra(rebase_source) 0.865446 0.823811 0.797756 author(mpm) or author(greg) 1.801832 1.784160 1.668172 author(mpm) or desc(bug) 1.812438 1.822756 1.677608 date(2015) or branch(default) 0.968276 0.910981 0.896032 author(mpm) or desc(bug) or date(2015) or extra(rebase_source) 3.656193 3.516788 3.265024 We see a speed-up on revsets accessing date and extras because the new parsing code only parses what you access. Even though they are stored the same text field, we avoid parsing dates when accessing extras and vice-versa. But strangely revsets accessing both date and extras appeared to speed up as well! I'm not sure if this is due to refactoring the parsing code or due to an optimization in revsets. You can't argue with the results!

changelog: lazily parse user Same strategy as before. Revsets not accessing the user demonstrate a slight performance win: desc(bug) 0.887169 0.910400 0.895514 date(2015) 0.878797 0.870697 0.820987 extra(rebase_source) 0.865446 0.841644 0.823811 date(2015) or branch(default) 0.968276 0.945792 0.910981

changelog: lazily parse manifest node Like the description, we store the raw bytes and convert from hex on access. This patch also marks the beginning of our new parsing method, which is based on newline offsets and doesn't rely on str.split(). Many revsets showed a performance improvement: author(mpm) 0.896565 0.869085 0.868598 desc(bug) 0.887169 0.928164 0.910400 extra(rebase_source) 0.865446 0.871500 0.841644 author(mpm) or author(greg) 1.801832 1.791589 1.731503 author(mpm) or desc(bug) 1.812438 1.851003 1.798764 date(2015) or branch(default) 0.968276 0.974027 0.945792

changelog: lazily parse description Before, the description field was converted to a localstr at parse time. With this patch, we store the raw description and convert to a localstr when it is first accessed. We see a revset speedup for revsets that don't access the description: author(mpm) 0.896565 0.914234 0.869085 date(2015) 0.878797 0.891980 0.862525 extra(rebase_source) 0.865446 0.912514 0.871500 author(mpm) or author(greg) 1.801832 1.860402 1.791589 date(2015) or branch(default) 0.968276 0.994673 0.974027 author(mpm) or desc(bug) or date(2015) or extra(rebase_source) 3.656193 3.721032 3.643593 As you can see, most of these revsets are already faster than from before this refactoring: we have already offset the performance loss from the introduction of the new class representing parsed changelog entries!

context: use changelogrevision Upcoming patches will make the changelogrevision object perform lazy parsing. Let's switch to it. Because we're switching from a tuple to an object, everthing that accesses the internal cached attribute needs to be updated to access via attributes. A nice side-effect is this makes the code easier to read! Surprisingly, this appears to make revsets accessing this data slightly faster (values are before series, p1, this patch): author(mpm) 0.896565 0.929984 0.914234 desc(bug) 0.887169 0.935642 0.921073 date(2015) 0.878797 0.908094 0.891980 extra(rebase_source) 0.865446 0.922624 0.912514 author(mpm) or author(greg) 1.801832 1.902112 1.860402 author(mpm) or desc(bug) 1.812438 1.860977 1.844850 date(2015) or branch(default) 0.968276 1.005824 0.994673 author(mpm) or desc(bug) or date(2015) or extra(rebase_source) 3.656193 3.743381 3.721032

changelog: add class to represent parsed changelog revisions Currently, changelog entries are parsed into their respective components at read time. Many operations are only interested in a subset of fields of a changelog entry. The parsing and storing of all the fields adds avoidable overhead. This patch introduces the "changelogrevision" class. It takes changelog raw text and exposes the parsed results as attributes. The code for parsing changelog entries has been moved into its construction function. changelog.read() has been modified to use the new class internally while maintaining its existing API. Future patches will make revision parsing lazy. We implement the construction function of the new class with __new__ instead of __init__ so we can use a named tuple to represent the empty revision. This saves overhead and complexity of coercing later versions of this class to represent an empty instance. While we are here, we add a method on changelog to obtain an instance of the new type. The overhead of constructing the new class regresses performance of revsets accessing this data: author(mpm) 0.896565 0.929984 desc(bug) 0.887169 0.935642 105% date(2015) 0.878797 0.908094 extra(rebase_source) 0.865446 0.922624 106% author(mpm) or author(greg) 1.801832 1.902112 105% author(mpm) or desc(bug) 1.812438 1.860977 date(2015) or branch(default) 0.968276 1.005824 author(mpm) or desc(bug) or date(2015) or extra(rebase_source) 3.656193 3.743381 Once lazy parsing is implemented, these revsets will all be faster than before. There is no performance change on revsets that do not access this data. There /could/ be a performance regression on operations that perform several changelog reads. However, I can't think of anything outside of revsets and `hg log` (basically the same as a revset) that would be impacted.

httppeer: compute header names only once This also helps make the code a little more readable.

httppeer: indent existing argument handling with if True I'm about to add another case, so it makes sense to split this out to make the semantic changes in the next change more obvious.

httppeer: move size computation later in _callstream A future change will alter some of the arg-sending logic in a way that matters for request body size. Centralizing the logic now will make later patches easier to review.

httppeer: do less splitting on httpheader value We only care about the first value split off, so only split off the first value.

chgserver: remove outdated comment about setvbuf() I've replaced setvbuf() by fdopen() before merging chg into the core, so this comment is wrong. https://bitbucket.org/yuja/chg/commits/5c24e72bc447

transplant: use absolute_import

transplant: switch to using nodemod for hex+short

convert: bzr use absolute_import

chgserver: include [extdiff] in confighash extdiff's uisetup will register new commands. If we do not include it in confighash, changes to [extdiff] will not get new commands registered. This patch adds extdiff to confighash and makes it possible for chg to pass test-extdiff.t.

chg: silently inherit server exit code If chgserver aborts during startup, for example, error.ParseError when parsing a config file, chg client probably just wants to exit with a same exit code without printing other unrelated text. This patch changes the text "cmdserver exited with status" from abortmsg to debugmsg and exits with a same exit code.

debugshell: use absolute_import

test: make check-py3-compat.py ignore empty code more reliably It couldn't exclude an empty file containing comments. That's why hgext/__init__.py had been listed in test-check-py3-compat.t before "hgext: officially turn 'hgext' into a namespace package".

patchbomb: specify unit for ui.progress when sending emails

streamclone: specify unit for ui.progress when handling data

churn: specify unit for ui.progress when analyzing revisions

convert: specify unit for ui.progress when scanning paths

convert: specify unit for ui.progress when operating on files

tests: stabilize svn output With 1.9.3 extra bits were appearing...

similar: specify unit for ui.progress when operating on files

verify: specify unit for ui.progress when checking files

repair: specify unit for ui.progress in rebuildfncache()

repair: use 'rebuilding' progress topic in rebuildfncache()

largefiles: use revisions as a ui.progress unit Using plural form is consistent with other progress units, and "1 out of 5 revisions" sounds more correct. Also, tests don't show this, but if you have 'speed' item in progress.format config, it shows e.g. '100 revisions/sec', which also seems better.

largefiles: specify unit for ui.progress when operating on files Also make it available for translation. It could already be translated, because it's used as a unit in archival.py and subrepo.py, for example.

templater: make label() just fail if ui object isn't available Silent failure hides bugs and makes it harder to track down the issue. It's worse than raising exception. In future patches, I plan to sort out template functions that require 'ui', 'ctx', 'fctx', etc. so that incompatible functions are excluded and the doc can say in which context these functions are usable. @templatefunc('label', requires=('ui',)) def label(context, mapping, args): ...

convert: fix "stdlib import follows local import" problem in transport Before this patch, import-checker reports error below for importing subversion python binding libraries. stdlib import "svn.*" follows local import: mercurial

convert: fix relative import of stdlib module in subversion Before this patch, import-checker reports "relative import of stdlib module" error for importing Pool and SubversionException from svn.core in subversion.py. To fix this relative import of stdlib module, this patch adds prefix 'svn.core.' to Pool and SubversionException in source. These 'svn.core.' relative accessing shouldn't cause performance impact, because there are much more code paths accessing to 'svn.core.' relative properties. BTW, in transport.py, this error is avoided by assignment below. SubversionException = svn.core.SubversionException But this can't be used in subversion.py case, because: - such assignment in indented code block causes "don't use camelcase in identifiers" error of check-code.py - but it should be placed in indented block, because svn is None at failure of importing subversion python binding libraries (= examination of 'svn' is needed)

convert: make subversion import transport locally Introducing "absolute_import" feature in previous patch caused failure of importing local module "transport".

keyword: compact writing of temporary kwdemo hgrc

dockerdeb: add Ubuntu Trusty One problem reported by lintian is "bad-distribution-in-changes-file unstable" in changelog, but the current changelog for the official package in Ubuntu also uses that distribution name (unstable), because they import from Debian. This certainly doesn't stop the build process.

pushoperation: fix language issues in docstring

chg: do not write pidfile Current pidfile logic will only keep the pid of the newest server, which is not very useful if we want to kill all servers, and will become outdated if the server auto exits after being idle for too long. Besides, the server-side pidfile writing logic runs before chgserver gets confighash so it's not trivial to append confighash to pidfile basename like we did for socket file. This patch removes --pidfile from the command starting chgserver and switches to an alternative way (unlink socket file) to stop the server.

chg: remove manual reload logic chgserver now validates and reloads configs automatically. Manually reloading is no longer necessary. Besides, we are deprecating pid files since the server will periodically check its ownership of the socket file and exit if it does not own the socket file any longer, which works more reliable than a pid file. This patch removes the SIGHUP reload logic from both chg server and client.

chg: use --daemon-postexec chdir:/ instead of --cwd / The chgserver is designed to load repo config from current directory. "--cwd /" will prevent chgserver from loading repo config and generate a wrong confighash, which will result in a redirect loop. This patch removes "--cwd /" and uses "--daemon-postexec chdir:/" instead.

serve: add chdir command for --daemon-postexec For chgserver, it probably needs a chdir to /. This patch adds chdir command support for --daemon-postexec so chg client can make use of it.

serve: accept multiple values for --daemon-postexec The next patch will add another postexec command: chdir, which can be used together with unlink. This patch changes the option type of --daemon-postexec from string to list to accept multiple commands. The error message of invalid --daemon-postexec value is also changed to include the actual invalid value.

hgext: officially turn 'hgext' into a namespace package Actually since Python 2.3, there is some way to turn top level package into "namespace package" so that multiple subpackage installed in different part of the path can still be imported transparently. This feature was previously thought (at least by myself) to be only provided by some setuptool black magic. Turning hgext into such namespace package allows third extensions to install themselves inside the "hgext" namespace package to avoid polluting the global python module namespace. They will now be able to do so without making it a pain to use a Mercurial "installed" in a different way/location than these extensions. The only constrains is that the extension ship a 'hgext/__init__.py' containing the same call to 'pkgutil.extend_path' and nothing else. This seems realistic. The main question that remains is: should we introduce a dedicated namespace for third party extension (hgext3rd?) to make a clearer distinction between what is officially supported and what is not? If so, this will be introduced in a follow up patch.

merge with stable

fileset: replace predicate by filesetpredicate of registrar (API) To make all built-in predicates be known to hggettext, loading built-in predicates by loadpredicate() should be placed before fixing i18nfunctions but after all of predicate decorating.

registrar: add filesetpredicate to mark a function as fileset predicate filesetpredicate is used to replace fileset.predicate in subsequent patch. This patch also adds loadpredicate() to fileset, because this combination helps to figure out how the name of "status caller" (or "existing caller") predicate is put into _statuscallers (or _existingcallers). Listing up loadpredicate() in dispatch.extraloaders causes implicit loading fileset predicate functions at loading (3rd party) extension.

registrar: remove useless base classes (API) Previous patches make these classes useless by removing classes derived from them.

revset: remove useless extpredicate class (API) Previous patch makes this classes useless by replacing it with revsetpredicate of registrar. BTW, extpredicate itself has already been broken by that patch, because revsetpredicate of registrar doesn't have compatibility with original predicate (derived from funcregistrar of registrar), in fact.

hook: filter out unstable output in tests This output is different between Python 2.6 and Python 2.7. It's also pretty irrelevant, so just filter it out.

fsmonitor: hook up state-enter, state-leave signals Keeping the codebase in sync with upstream: Watchman 4.4 introduced an advanced settling feature that allows publishing tools to notify subscribing tools of the boundaries for important filesystem operations. https://facebook.github.io/watchman/docs/cmd/subscribe.html#advanced-settling has more information about how this feature works. This diff connects a signal that we're calling `hg.update` to the mercurial update function so that mercurial can indirectly notify tools (such as IDEs or build machinery) when it is changing the working copy. This will allow those tools to pause their normal actions as the files are changing and defer them until the end of the operation. In addition to sending the enter/leave signals for the state, we are able to publish useful metadata along the same channel. In this case we are passing the following pieces of information: 1. destination revision hash 2. An estimate of the distance between the current state and the target state 3. A success indicator. 4. Whether it is a partial update The distance is estimate may be useful to tools that wish to change their strategy after the update has complete. For example, a large update may be efficient to deal with by walking some internal state in the subscriber rather than feeding every individual file notification through its normal (small) delta mechanism. We estimate the distance by comparing the repository revision number. In some cases we cannot come up with a number so we report 0. This is ok; we're offering this for informational purposes only and don't guarantee its accuracy. The success indicator is only really meaningful when we generate the state-leave notification; it indicates the overall success of the update.

largefiles: add abstract methods in remotestore class Methods _put, _get, _stat were used in remotestore class as abstract expecting that subclass would implement them. This commit makes this fact explicit.

test-parse-date: defines explicit start/end dates for DST Prior to this patch, DST times where tested by specifying a custom TZ environment variable that didn't defined DST transition times. Due to a bug in glibc, the test fail on 32bits platforms that use timezone files generated by zic from tzcode >= 2014c (glibc >= 2.20). See https://sourceware.org/bugzilla/show_bug.cgi?id=19738 By defining explicit transition times for DST in the TZ environment variable, the test is now independant to how the system guess those transition times.

debuginstall: convert to formatter commit editor now reports its editor default template is now reported a broken vi editor (vi not in path) is still not considered a problem (!!)

largefiles: use iterbatch instead of batch This actually makes the code a little cleaner to read.

wireproto: make iterbatcher behave streamily over http(s) Unfortunately, the ssh and http implementations are slightly different due to differences in their _callstream implementations, which prevents ssh from behaving streamily. We should probably introduce a new batch command that can stream results over ssh at some point in the near future. The streamy behavior of batch over http(s) is an enormous win for remotefilelog over http: in my testing, it's saving about 40% on file fetches with a cold cache against a server on localhost.

setdiscovery: use iterbatch interface instead of batch It's a little more concise, and gives us some simple test coverage.

peer: add an iterbatcher interface This is very much like ordinary batch(), but it will let me add a mode for batch where we have pathologically large requests which are then handled streamily. This will be a significant improvement for things like remotefilelog, which may want to request thousands of entities at once.

wireproto: document quirk of _callstream between http and ssh This tripped me up when trying to use it, so it feels like we should document this to avoid future pain.

peer: raise NotImplementedError for abstract submit() method Nothing should ever call this submit method directly as it should be overridden by concrete batcher implementations.

fsmonitor: new experimental extension Extension to plug into a Watchman daemon, speeding up hg status calls by relying on OS events to tell us what files have changed. Originally developed at https://bitbucket.org/facebook/hgwatchman

fsmonitor: dependencies for new experimental extension In preparation for the filesystem monitor extension, include the pywatchman library. The fbmonitor extension relies on this library to communicate with the Watchman service. The library is BSD licensed and is taken from https://github.com/facebook/watchman/tree/master/python. This package has not been updated to mercurial code standards.

setup: show how to set the module policy for imports This is not technically needed, since mercurial.__version__ does not exist as a native module, but, without this style wrappings, if something else had a native flavor, the module loader would get upset. In principle, the `env` object is trying to set HGMODULEPOLICY for children, so, conceptually we should set it for this in-process child.

setup: create a module for the modulepolicy Instead of rewriting __init__ to define the modulepolicy, write out a __modulepolicy__.py file like __version__.py This should work for both system-wide installation and in-place build. Therefore we can avoid relying on two separate modulepolicy rules, '@MODULELOADPOLICY@' and 'mercurial/modulepolicy'.

rebase: turn rebaseskipobsolete on by default Consider the following use case. User has a set of commits he wants to rebase onto some destination. Some of the commits in the set are already rebased and their new versions are now among the ancestors of destination. Traditional rebase behavior would make the rebase and effectively try to apply older versions of these commits on top of newer versions, like this: a` --> b --> a` (where both 'a`' and 'a``' are rebased versions of 'a') This is not desired since 'b' might have made changes to 'a`' which can now result in merge conflicts. We can avoid these merge conflicts since we know that 'a``' is an older version of 'a`', so we don't even need to put it on top of 'b'. Rebaseskipobsolete allows us to do exactly that. Another undesired effect of a pure rebase is that now 'a`' and 'a``' are both successors to 'a' which is a divergence. We don't want that and not rebasing 'a' the second time allows to avoid it. This was not enabled by default initially because we wanted to have some more experience with it. After months of painless usages in multiple places, we are confident enough to turn it on my default.

graphlog: bring back color to node symbol template Follows up 3356bf61fa25. A ui object is required to render labels.

revset: add inspection data to max() and min() functions We are likely to be interested in how these functions build a result set.

revset: add inspection data to limit() and last() functions We are likely to be interested in how these functions calculate a result set.

revset: stub to add extra data to baseset for better inspection We sometimes construct a baseset from filtering result. In that case, a baseset can provide more precise information how it is constructed.

revset: add inspection data to all filter() calls This is useful for debugging revset construction.

revset: add extra data to filteredset for better inspection A filteredset is heavily used, but it cannot provide a printable information how given set is filtered because a condition is an arbitrary callable object. This patch adds an optional "condrepr" object that is used only by repr(). To minimize the maintaining/runtime overhead of "condrepr", its type is overloaded as follows: type example -------- --------------------------------- tuple ('<not %r>', other) str '<branch closed>' callable lambda: '<branch %r>' % sorted(b) object other

zeroconf: replace reduce+add with itertools.chain

zeroconf: replace has_key with in

zeroconf: compare singleton using is

zeroconf: remove camelcase in identifiers

setup: switch to with open as We're leaving the modulepolicy bit alone, because it's being rewritten in the next commit.

win32mbcs: use absolute_import

notify: use absolute_import

patchbomb: use absolute_import

convert: __init__ use absolute_import

convert: cvs use absolute_import

convert: transport use absolute_import

convert: bzr use absolute_import

convert: common use absolute_import

convert: convcmd use absolute_import

convert: subversion use absolute_import

blackbox: guard against recursion from dirty check

tests: divorce blackbox test from test-dispatch.py I used test-dispatch.py to demonstrate what would happen if a log file changed from being readonly to writable, by having it replace a directory (proxy for readonly/not-writable) with a log file in between transactions of a running python process (proxy for Mercurial). This commit makes it easier for people to follow what the test is doing, by creating a real file that people can read.

tests: test-dispatch use print_function

tests: test-dispatch use absolute_import

templater: ignore orig/rej files If your mercurial/templates/ directory is dirty, then the template system would otherwise import duplicate templates from the .orig files and potentially try to parse .rej files. Since editing/reverting these templates isn't an unexpected action, and since they're in .hgignore, it's best that the template system know to skip them."

resolve: when pats do not match, hint about path: Suggest a command that would probably work.

shelve: changes getting opts values by get method When shelve is used by another extension that doesn't provide all necessary values in opts shelve raises KeyError exception. This patch fixes this by getting values from opts dictionary with get method.

contrib: remove references to 2to3 The custom porting fixers are removed. A comment related to 2to3 has been removed from the import checker. After this patch, no references to 2to3 remain.

check-code: remove redundant import style check We have a dedicated tool that checks for import conventions. Remove a redundant and less powerful check.

setup: remove support for 2to3 We want to run unaltered source on multiple Python versions. We won't be using 2to3 for Python 3 support.

run-tests: remove 2to3 support Our goal is to have 1 code base that works on 2.6, 2.7, and 3.5+. 2to3 won't be used. Stop passing it to setup.py.

histedit: adds hint how to reorder changesets at editor (issue3766)

revset: replace predicate by revsetpredicate of registrar To make all built-in predicates be known to hggettext, loading built-in predicates by loadpredicate() should be placed before fixing i18nfunctions but after all of predicate decorating.

revset: replace extpredicate by revsetpredicate of registrar This patch consists of changes below (these can't be applied separately). - replace revset.extpredicate by registrar.revsetpredicate in extensions - remove setup() on an instance named as revsetpredicate in uisetup()/extsetup() of each extensions registrar.revsetpredicate doesn't have setup() API. - put new entry for revsetpredicate into extraloaders in dispatch This causes implicit loading predicate functions at loading extension. This loading mechanism requires that an extension has an instance named as revsetpredicate, and this is reason why largefiles/__init__.py is also changed in this patch. Before this patch, test-revset.t tests that all decorated revset predicates are loaded by explicit setup() at once ("all or nothing"). Now, test-revset.t tests that any revset predicate isn't loaded at failure of loading extension, because loading itself is executed by dispatch and it can't be controlled on extension side.

registrar: define revsetpredicate to decorate revset predicate revsetpredicate is used to replace revset.predicate and revset.extpredicate in subsequent patches. This patch also adds loadpredicate() to revset, because this combination helps to figure out how the name of safe predicate is put into safesymbols. This patch still uses safesymbols set to examine whether the predicate corresponded to the 'name' is safe from DoS attack or not, because just setting func._safe property needs changes below for such examination. before: name in revset.safesymbols after: getattr(revset.symbols.get(name, None), '_safe', False) "automatic registration" described in help doc of revsetpredicate class will be achieved by the subsequent patch, which lists loadpredicate() up in dispatch.extraloaders.

registrar: introduce new class for registration to replace funcregistrar _funcregistrarbase differs from funcregistrar in points below: - every code paths should use same class derived from _funcregistrarbase to register functions in a same category funcregistrar expects (3rd party) extensions to use (a class derived from) delayregistrar. - actual extra setup should be executed in another function For example, marking revset predicate as "safe" is executed in a class derived from _funcregistrarbase, but putting name of "safe" predicate into safesymbols is executed in another function for it. funcregistrar expects derived classes to do so. New class is named as module private one, because code paths, which register functions, should use not it directly but one derived from it.

dispatch: make loading extra information from extension extensible This patch makes loading extra information from extension module at dispatching extensible. Factoring 'loadcmdtable()' into commands.py is a part of generalization of loading extra information. This extensibility assumes registration of new function like below, for example: - revset predicate - fileset predicate - template keyword - template filter - template function - internal merge tool - web command This patch requires not loader function itself but container module and the name of it, because listing loader function directly up implies actual loading module of it, even if it isn't used at runtime (for example, extensions don't always define revset predicate)

obsolete: fix n^2 marker computation behavior Previously, if you ran obsolete.createmarkers with a bunch of markers that did not have successors (like when you do a prune), it encountered a n^2 computation behavior because the loop would read the changelog (to get ctx.parents()), then add a marker, in a loop. Adding a marker invalidated the computehidden cache, and reading the changelog recomputed it. This resulted in pruning 150 commits taking 150+ seconds in a large repo. The fix is to break the reading part of the loop to be separate from the writing part.

bdiff: (pure) support array.array arrays (issue5130)

mq: restrict generated patch name to 75 characters (issue5117) Super long first line in description lead to very long file name that windows is unhappy about. We restrict the name to 75 char to avoid the issue. 75 seems fine and leave some extra room for '__#' suffix in case of conflict. I does not seems worthwhile to add a dedicated config option to configure the length. It can be done in the future if there is an actual user demand for it.

files: don't recurse into subrepos without a path or -S (issue5127) The 'm.always()' check was needed for when a path to 'sub1' is given, and 'sub1' contains a subrepo itself. But that also caused the automatic recursion when no path was given. Instead, force -S when printing a subrepo if the subpath is an exact match (which will unconditionally recurse once in the nested subrepo).

parsers: optimize filtered headrevs logic The old native head revs logic would iterate over every node, starting from 0, and check if every node was filtered (by testing it against the filteredrevs python set). On large repos with hundreds of thousands of commits, this could take 150ms. This new logic iterates over the nodes in reverse order, and skips the filtered check if we've seen an unfiltered child of the node. This saves approximately a bagillion filteredrevs set checks, which shaves the time down from 150ms to 20ms during every branch cache write.

destutil: choose non-closed branch head at first (BC) Before this patch, destupdate() returns the tipmost (descendant) branch head regardless of closed or not. But updating to closed branch head isn't reasonable for ordinary workflow, because: - "hg heads" doesn't show closed heads (= updated parent itself) by default - subsequent committing on it re-opens closed branch even if inactivation of closed head is needed, update destination isn't it, because it should be merged into to another branch in such case. This patch chooses non-closed descendant branch head as default update destination at first. If all descendant branch heads are closed, destupdate() returns the tipmost closed branch head. For simplicity, this patch chooses adding _destupdatebranchfallback() instead largely changing _destupdatebranch(). This patch changes not only normal lookup code path, but also the "no default branch" code path, for consistency.

formatter: make labels work with templated output To describe the bug this fix is addressing, one can do ``$ hg status -T "{label('red', path)}\n" --color=debug`` and observe that the label is not applied before my fix and applied with it.

hghave: improve description of Hypothesis

purge: use absolute_import

record: use absolute_import

relink: use absolute_import

schemas: use absolute_import

shelve: use absolute_import

strip: use absolute_import

graphmod: augment the graph to include more information about the edges The walker knows when an edge leads to a direct parent, a grandparent (skipping revisions not part of the revset) and parents that are missing altogether (neither it nor a grandparent is in the revset). Add this information to the parents sequence yielded.

graphmod: refactor state handling Move ASCII graph state to a dictionary, to clarify what is being tracked. Move the 'seen' state (tracking currently active edges) into this structure.

templater: move label() function from color extension ui.label() is no-op by default, so we can just call ui.label() by label() template function no matter if the color is enabled or not.

templater: make label() take unknown symbol as color literal Instead of the mapping hack introduced by b775a2029e8d, this patch changes the way how a label symbol is evaluated. This is still hackish, but should be more predictable in that it doesn't depend on the known color effects. This change is intended to eliminate the reference to color._effects so that color.templatelabel() can be merged with templater.label().

convert: monotone use absolute_import

convert: p4 use absolute_import

convert: hg use absolute_import

convert: cvsps use absolute_import

convert: darcs use absolute_import

convert: filemap use absolute_import

convert: gnuarch use absolute_import

convert: git use absolute_import

branchmap: check node against changelog instead of repo Testing 'node in repo' requires constructing a changectx, which is a little expensive. Testing 'repo.changelog.hasnode(node)' is notably faster. This saves 10-20ms off of every command, when testing a few thousand nodes from the branch cache. I considered changing the implementation of localrepository.__contains__ so every place would benefit from the change, but since localrepository.__contains__ uses changectx to check if the commit exists, it means it supports a wider range of possible inputs (like revs, hashes, '.', etc), so it seemed unnecessarily risky.

changegroup: clear progress callback after changelog processing The progress callback is replaced by one for manifests after changelog processing is done, but let's not depend on manifests replacing the value and instead explicitly clear it.

tests: adjust for code move in Hypothesis 2.0.0 It appears that the Settings class was renamed to settings, and because of this, the settings module was renamed to configuration. https://github.com/DRMacIver/hypothesis/commit/a0e663b

changegroup: progress for added files is not measured in "chunks" The "prog" class cg1unpacker.apply() has the unit set to "chunks". This is not correct for files, where the file itself is the unit. The unit is not usually printed, which is probably why this has not been fixed yet. It can be show with e.g. "--config progress.format='topic number unit'".

changegroup: exclude submanifests from manifest progress The progress callback for manifests is cleared outside of _unpackmanifests(), which means it will remain in effect while pulling subdirectory manifests when using treemanifests. Since the total number of revisions used for the progress is the number of changesets, the total number of treemanifest revisions is usually larger than that. One effect of this is that the ETA is negative. It's hard to estimate the number of subdirectory revisions, so let's just exclude them from progress for now.

commit: block amend while histedit is in progress (issue4800)

chg: limit reconnect attempts Some users may have hg as a wrapper script which sets sensitive environment variables (like setting up virtualenv). This will make chg redirect forever because the environment variables are never considered up to date. This patch adds a limit (10) for reconnect attempts and warn the user with a possible solution if the limit is exceeded.

chg: use validate to make sure the server is up to date This patch uses the newly added validate method to make sure the server has loaded the up-to-date config and extensions. If the server cannot validate itself, the client will receive instructions and follow them to try to reach another server that is more likely to validate itself. The instructions can be a redirect (connect to another server address) and/or an unlink (stops an out-dated server).

chg: implement validate in hgclient This patch implements the corresponding validate method in hgclient. It will return instruction strings as is without taking any real action.

hgclient: use absolute_import and print_function

fixpax: use absolute_import and print_function

debugcmdserver: use absolute_import and print_function

check-config: use absolute_import and print_function

casesmash: use absolute_import

chgserver: implement validate command validate will load the repo config and check if the server has up-to-date config to continue serve the client. In case it does not, the server will send instructions to the client about what to do next, including to retry with a different address or to unlink an outdated socket file to stop an old server.

templater: handle exception when applying map operator to non-iterable object Before this, "{noniterable % template}" raised an exception. This tries to provide a better indication for the common case, where a left-hand-side expression is a keyword.

templater: factor out thin helper that evaluates argument as string This function is trivial, but it can make it clear that args[i] is a (func, data) pair.

color: make label() template function stringify "expr" argument first Since label() should return a string (or a thunk to be evaluated to a string), this change is okay. This helps porting to evalstring() helper. See the next patch for details.

templater: fix shortest() to evaluate int argument and handle error

templater: fix pad() to evaluate int argument and handle error

templater: drop redundant type conversion when evaluating integer argument A function argument may be an integer. In this case, it isn't necessary to convert a value to string and back to integer. Because an argument may be an arbitrary object (e.g. date tuple), TypeError should be caught as well.