tests: run import-checker with tests .t files

import-checker: parse python code from .t files

import-checker: track filenames for SyntaxErrors

import-checker: track SyntaxErrors We don't really need to report SyntaxErrors, since in theory docchecker or a test will catch them, but they happen, and we can't just have the code crash, so for now, we're reporting them.

import-checker: refactor source reading This will allow .t files to generate multiple sources. It will also allow .py doctests to generate additional sources.

tests: make test-pathencode use print_function

tests: make test-run-tests use absolute_import

tests: make test-symlink-os-yes-fs-no use absolute_import

tests: make test-ui-color use absolute_import

tests: make test-url use absolute_import

tests: make test-trusted use absolute_import

templater: add parsing and expansion rules to process "templatealias" section The debugtemplate command is updated to show expanded tree, but still the template engine doesn't support alias expansion. That's why the test says "parse error" for now.

templater: add function to parse whole string as template expression This will be a parser of template aliases, and it can also be used for processing quoted string templates in map files. That's why this function isn't defined in the upcoming _aliasrules class.

parser: factor out _trygetfunc() that extracts function name and arguments This provides a customization point for templater. In templater, there are two ways to call a unary function: func(x) and x|func. They are processed differently in templater due to historical reasons, but they should be handled in the same way while expanding aliases. In short, x|func should be processed as syntactic sugar for func(x). _funcnode and _getlist() are replaced by _trygetfunc().

parser: make _getalias() return (alias, pattern-args) pair This allows us to factor out a function that extracts a function (name, args) pair. See the next patch for why.

parser: drop redundant comparison between alias declaration tree and pattern Since _getalias() explicitly tests the type and name of the pattern tree, we don't need to compare "a.tree == tree" for 'symbol', and "a.tree == tree[:2]" for 'func', where tree is either ('symbol', name) or ('func', ('symbol', name)). This change helps implementing better handling of template aliases. See the subsequent patches for details. The alias.tree field is removed as it is no longer used.

patchbomb: fix public-is-missing hint Without this, there is no space between a hash and the -r preceding the next line in the use hg push hint

tests: clarify patchbomb repo is public not remote

run-tests: set HGMODULEPOLICY for --pure Without this, my python 2.6 virtualenv test run with --pure and --local fails with: + ImportError: Python minor version mismatch: The Mercurial extension modules were compiled with Python 2.7.8, but Mercurial is currently using Python with sys.hexversion=33950192: Python 2.6.9 (unknown, Apr 13 2016, 12:40:12) + [GCC 4.9.2 20141101 (Red Hat 4.9.2-1)] + at: ~/hg/py26/bin/python

cmdutil: avoid recycling variable name "name" in namespaces code This just feels like asking for future trouble and confusion.

update: fix bare update to work on new branch So far bare update on new branch results in 'abort: branch new-branch not found'. This commit fixes this by updating to the parent of wctx. The effect of updating to the parent of wctx is to move to the paren't branch - this means that it is no longer necessary to prevent you from updating if you would lose your newly created branch.

remove: fix --force option help description (issue5177) Before this commit --force option help description stated that file was removed and deleted even if file was added or modified which is not true. Force option removes added file only from dirstate, it doesn't delete it from the filesystem.

logtoprocess: new experimental extension This extension lets you direct specific ui.log() information to shell commands, spawned in the background. This can be used to, say, capture timings, commandfailure tracebacks, etc., making the output available to other command-line tools.

convert: keep converted hg parents that are outside convert.hg.revs (BC) Before, when converting revisions without also including their already converted parents in convert.hg.revs, the parents would no longer be parents. That seems unfortunate and we dare to assume that nobody ever wants that. Instead, preserve parents that are outside the current convert range but already have been converted. The parents returned in getcommit() are unconditionally converted, so we introduce a separate optparents with optional parents.

tests: coverage of ancestry with convert in multiple non-overlapping steps This exposes that parent information is lost in cases where it is possible to preserve it - and where it thus would make sense if that was what happened.

revset: rename findaliases() to expandaliases() This function returns a full tree of alias expansion applied, which sounds different from what "findaliases" would do.

parser: add short comment how aliases are expanded in phases

parser: reorder alias expansion routine to return early I think it improves readability to move trivial cases first, and unindent blocks.

parser: move functions that process alias expansion to rule-set class They will be commonly used by revset and templater. It isn't easy to understand how _expand() works, so I'll add comments by a follow-up patch. The local variable 'alias' is renamed to 'a' to avoid shadowing the global 'alias' class.

revset: unindent codes in _getalias() function We generally do return early if tree isn't a tuple.

parser: extract helper that creates a dict of aliases This will be common between revset and templater. The local variable 'alias' is renamed to 'a' to avoid shadowing the global 'alias' class.

parser: construct alias object by rule-set class It was odd that the revsetalias did the whole parsing stuff in __init__(). Instead, this patch adds a factory function to the aliasrules class, and makes the alias (= revsetalias) class a plain-old value object.

graphmod: shorten graph Shorten the graph, cutting the all vertical (not oblique) edges rows. Activate with 'graphshorten = true' in [experimental] section. Example graph with deactivated option: $ hg log --graph --template '{rev} {desc|firstline}' --rev 1035:1015 o 1035 Merge with BOS |\ | o 1034 Fix help output, and a few broken tests. | | | o 1033 Merge with MPM. | |\ | | o 1032 Get patchbomb working with tip again. | | | | | o 1031 Rewrite log command. New version is faster and more featureful. | | | | | o 1030 Merge with MPM. | | |\ | | | o 1029 Emacs: implement hg-incoming, hg-outgoing and hg-push. | | | | | | | o 1028 Add commands.debugconfig. | | | | | | | o 1027 Emacs: fix up hg-log and hg-diff to operate more uniformly. | | | | | | | o 1026 Merge with MPM. | | | |\ | | | | o 1025 Merge with MPM. | | | | | | | | | ~ | | | o 1024 Sync buffers prior to doing a diff. | | | | | | | ~ o | | 1023 Minor tweak to the revgen algorithm |/ / o | 1022 Minor hgwebdir tweaks | | o | 1021 Add Makefile to the manifest | | o | 1020 Add default make rule | | o | 1019 Create helper functions for I/O to files in the working directory | | o | 1018 Add some aliases | | o | 1017 Fix up help for binary options |/ o 1016 Teach annotate about binary files | o 1015 Add automatic binary file detection to diff and export | ~ Example graph with activated option: $ hg log --graph --template '{rev} {desc|firstline}' --rev 1035:1015 o 1035 Merge with BOS |\ | o 1034 Fix help output, and a few broken tests. | o 1033 Merge with MPM. | |\ | | o 1032 Get patchbomb working with tip again. | | o 1031 Rewrite log command. New version is faster and more featureful. | | o 1030 Merge with MPM. | | |\ | | | o 1029 Emacs: implement hg-incoming, hg-outgoing and hg-push. | | | o 1028 Add commands.debugconfig. | | | o 1027 Emacs: fix up hg-log and hg-diff to operate more uniformly. | | | o 1026 Merge with MPM. | | | |\ | | | | o 1025 Merge with MPM. | | | | | | | | | ~ | | | o 1024 Sync buffers prior to doing a diff. | | | | | | | ~ o | | 1023 Minor tweak to the revgen algorithm |/ / o | 1022 Minor hgwebdir tweaks o | 1021 Add Makefile to the manifest o | 1020 Add default make rule o | 1019 Create helper functions for I/O to files in the working directory o | 1018 Add some aliases o | 1017 Fix up help for binary options |/ o 1016 Teach annotate about binary files o 1015 Add automatic binary file detection to diff and export | ~

tests: ensure that 'hg update' is disabled during histedit (issue3655)

test-remove: drop a useless Windows specific conditional The Windows branch didn't pick up the 'deleting' progress bar addition from 62e73d42bd14. But since the Windows branch already globbed the error message, let's just drop the other branch.

test-blackbox: add missing glob for Windows The test warns instead of completes without this.

test-largefiles: stabilize output for Windows Systems with unix-permissions have a test above this that adds an additional head.

test-import: fix output on Windows There's a symlink conditionalized test above this that causes the rev to be 1. It isn't important to this test, so ignore it.

test-install: fix output on Windows See 1ff28873830e.

check-code: reject import urllib

pycompat: switch to util.urlreq/util.urlerr for py3 compat

pycompat: add util.urlerr util.urlreq classes for py3 compat python3 url.request and url.error are mapped as util.urlreq/util.urlerr python2 equivalents from urllib/urllib2 are mapped according to the py3 hierarchy

test-progress: disable mocking-time tests on chg It's hard to make these tests compatible with chg because a mocked time.time() is recorded and accessed by progbar at random timing. I don't think it's worth fixing this test as it is considered a unit test of time estimates, so just ignores on chg.

hghave: add "chg" flag to skip tests that can't be compatible with chg Several tests fail with chg for several reasons such as loaded chgserver extension, running uisetup() per server instead of per runcommand, etc. Since these tests can't/shouldn't be changed to be chg friendly, we need a flag to skip them. This patch explicitly drops CHGHG environment if chg isn't involved. This way, hghave can just check if CHGHG exists.

tests: add new test for #! shebang lines * use #!/bin/sh not e.g. #!/usr/bin/sh * use #!/usr/bin/env python not e.g. #!/usr/bin/python

largefiles: introduce push --lfrev to control which revisions are pushed The default of pushing all largefiles referenced in outgoing revisions is safe, but also expensive and sometimes not what is needed. We thus introduce a --lfrev option, similar to what pull already has. By specifying an empty set of revisions (or null), it is possible to get lazy (and insecure!) pushes of revisions without referenced largefiles, similar to how pull works.

largefiles: don't access repo.changelog directly in getlfilestoupload Make it possible to pass both nodes and revisions to getlfilestoupload.

localrepo: refactor prepushoutgoinghook to take a pushop prepushoutgoinghook was introduced in 6c383c871fdb and largefiles is the only in-tree use of it. Refactor it to be more useful for other use cases in largefiles.

parser: unify parser function of alias declaration and definition We no longer have to keep them separately.

revset: unify function that parses alias declaration and definition We no longer need separate parsers. Only difference between _parsealiasdecl() and _parsealiasdefn() is whether or not to flatten 'or' tree. Since alias declaration should have no 'or' operator, there was no practical difference.

parser: move alias definition parser to common rule-set class The original _parsealiasdefn() function is split into common _builddefn() and revset-specific _parsealiasdefn(). revset._relabelaliasargs() is removed as it is no longer used. The doctests are ported by using the dummy parse().

parser: move _relabelaliasargs() to common rule-set class This has no doctest because it will be covered by _builddefn() introduced by the next patch. revset._relabelaliasargs() will be removed soon.

parser: move alias declaration parser to common rule-set class The original _parsealiasdecl() function is split into common _builddecl() and revset-specific _parsealiasdecl(). And the original _parsealiasdecl() call is temporarily replaced by rules._builddecl(), which should be eliminated later. The doctests are mostly ported by using the dummy parse(), but the test for 'foo bar' is kept in _parsealiasdecl() as it checks if "pos != len(decl)" is working. Also, 'foo($1)' test is added to make sure the alias tokenizer can handle '$1' symbol, which is the only reason why we need _parsealiasdecl().

parser: add stub class that will host alias parsing and expansion This class will keep syntax rules that are necessary to parse and expand aliases. The implementations will be extracted from the revset module. In order to make the porting easier, this class keeps parsedecl and parsedefn separately, which will be unified later. Also, getlist and funcnode will be refactored by future patches for better handling of the template aliases. The following public functions will be added: aliasrules.build(decl, defn) -> aliasobj parse decl and defn into an object that keeps alias name, arguments and replacement tree. aliasrules.buildmap(aliasitems) -> aliasdict helper to build() a dict of alias objects from a list of (decl, defn) aliasrules.expand(aliasdict, tree) -> tree expand aliases in tree recursively Because these functions aren't introduced by this series, there would remain a few wrapper functions in the revset module. These ugly wrappers should be eliminated by the next series. This class is considered an inheritable namespace, which will host only class/static methods. That's because it won't have no object-scope variables. I'm not a big fan of using class as a syntax sugar, but I admit it can improve code readability at some level. So let's give it a try.

revset: narrow scope of "except ParseError" block in _parsealiasdecl() This helps to factor out a common function. "if True" will be removed soon.

obsstore: move delete function from obsstore class to repair module Since one of the original patches was accepted already and people on the mailing list still have suggestions as to how this should be improved, I'm implementing those suggestions in the following patches (this and the ones that might follow).

debugobsolete: style fixes to debugobsolete that slipped from original commit

import: document --exact behavior in more detail

util: add doctest to datestr()

date: fix boundary check of negative integer

chg: server exited with code 0 without being connectable is an error Before this patch, if the server started by chg has exited with code 0 without creating a connectable unix domain socket at the specified address, chg will exit with code 0, which is not the correct behavior. It can happen, for example, CHGHG is set to /bin/true. This patch addresses the issue by checking the exit code of the server and printing a new error message if the server exited normally but cannot be reached.

shelve: refactor directory name into constant Shelve directory name extracted into constant to avoid typos/duplication.

pycompat: switch to util.stringio for py3 compat

py3: use multi-line import in test-wireproto.py The reason I did it is that I had a later commit that was adding to the list.

py3: use absolute_import in test-hgweb-non-interactive.t

py3: use absolute_import in test-hgweb-no-request-uri.t

py3: use absolute_import in test-hgweb-no-path-info.t

chg: use fsetcloexec instead of closing lockfd manually Since we have the fsetcloexec utility function, use it instead of closing lockfd manually.

chg: extract the logic of setting FD_CLOEXEC to a utility function Setting FD_CLOEXEC is useful for other fds such like lockfd and sockdirfd, move the logic from hgc_open to util.

chg: add fchdirx as a utility function As part of the series to support long socket paths, we need to use fchdir and check its result in several places. Make it a utility function.

chg: check lockfd at freecmdserveropts We check for sockdirfd at freecmdserveropts but not lockfd, which is a bit strange to people new to the code. Add a comment and an assert to make it clear that lockfd should be closed earlier.

chg: add sockdirfd to cmdserveropts As part of the series to support long socket paths, we need to add the fd of the directory to the cmdserveropts structure so we can use basenames instead of full paths for sockname, redirectsockname, and lockfile.

chg: fix spelling in the error message about error waiting for cmdserver This is a trivial spelling and grammar fix.

sslutil: document and slightly refactor validation logic This main purpose of this patch is to make it clearer that fingerprint pinning takes precedence over CA verification. This will make subsequent refactoring to the validation code easier to read.

sslutil: require a server hostname when wrapping sockets (API) All callers appear to be passing the hostname. So this shouldn't break anything. By specifying the hostname, more validation options from the ssl module are available to us. Although this patch stops short of using them.

sslutil: move and document verify_mode assignment Consolidating all the SSLContext options setting makes the code a bit easier to read.

tests: use --insecure instead of web.cacerts=! --insecure is the proper and documented way to do this. The end result is the same: dispatch will set web.cacerts to ! when --insecure is passed. This patch is necessary to refactor handling of web.cacerts in upcoming patches.

help: remove references to "Python 2.6 or later" We require Python 2.6. So there is no value to these docs.

commands: make --rev and --index compatible in debugobsolete

tests: enable import checker for tests/**.py files Several known-bad files are excluded as they couldn't be trivially fixed. In principle, we should fix them first, however, it would have more risk to keep Py3k porting going without the test coverage. Still contrib/**.py aren't covered, which needs another round.

tests: stop direct symbol import of mercurial modules in test-status-inprocess

tests: alias ui as uimod in test-revlog-ancestry/test-ui-verbosity

tests: move stdlib imports before mercurial modules in test-parseindex2

tests: stop direct symbol import of pprint.pprint in tests-minirst

tests: import mercurial modules by name in test-propertycache This is our convention, and silences import-checker.py that would say imports weren't lexically sorted.

tests: remove unused import of mercurial.repoview from test-propertycache I don't see any reason to import it, but if there is a reason, please disregard this and the next patch.

templater: drop deprecated handling of KeyError from changeset_templater It's been superseded by 09cde75e0613 and the previous patch. templater.mapfile is no longer used and removed.

test-commandserver: handle cStringIO.StringIO/io.StringIO divergence

pycompat: add util.stringio to handle py3 divergence util.stringio = cStringIO.StringIO / io.StringIO

pycompat: alias xrange to range in py3

pycompat: fix demand import handling of Queue When demandimport is enabled, simply importing a non existent module does not trigger ImportError, a property access is necessary.

util: use __code__ (available since py2.6)

templater: give better error message for invalid engine type Before, KeyError was caught at changeset_templater._show(), which said "no key named '%s'" as it was intended to catch the KeyError of unknown map key. Instead, we should catch KeyError explicitly for better error indication. For those who don't know what the template engine is (read "everyone"), it is hidden extension feature that allows switching template syntax in map file. See d8c5a7f25a40 for details.

transaction: allow running file generators after finalizers Previously, transaction.close would run the file generators before running the finalizers (see the list below for what is in each). Since file generators contain the bookmarks and the dirstate, this meant we made the dirstate and bookmarks visible to external readers before we actually wrote the commits into the changelog, which could result in missing bookmarks and missing working copy parents (especially on servers with high commit throughput, since pulls might fail to see certain bookmarks in this situation). By moving the changelog writing to be before the bookmark/dirstate writing, we ensure the commits are present before they are referenced. This implementation allows certain file generators to be after the finalizers. We didn't want to move all of the generators, since it's important that things like phases actually run before the finalizers (otherwise you could expose commits as public when they really shouldn't be). For reference, file generators currently consist of: bookmarks, dirstate, and phases. Finalizers currently consist of: changelog, revbranchcache, and fncache.

run-tests: move install.err into test area Without this, sometimes installerrs generated errors about no such file. It also did not work well when you had multiple tests runners running around. It also did not make sense to pollute the repository test directory with the log file.

help: report source of aliases

compact: add color labels to -Tcompact

util: fix doc for datestr() timezone parameter was removed with c3182eeb70ea

date: reallow negative timestamp, fix for Windows buggy gmtime() (issue2513) DVCS are very useful to store various texts (as legislation) written before Unix epoch. Fri, 13 Dec 1901 is a nice gain over Thu, 01 Jan 1970. Revert dd24f3e7ca9e and e1002cf9fe54, fix c208dcd0f709. Add tests.

tests: use /usr/bin/env python for test-status-inprocess.py everyone else uses it, and the next commit will add a rule for it

run-tests: fix broken regular expression The regular expression in use passed tests because the test repo only has single-digit changesets present. When I tried to use this for real today, it broke, because the regular expression would only match a single digit. https://xkcd.com/1171/, or something like that.

test: don't rely on __del__ in test-devel-warnings.t Whatever the future of __del__ in Mercurial is, that devel-warning test is not about testing the automatic transaction rollback and we should explicitly call release. This change make this tests pass with pypy, as pypy try less hard to call __del__ at program exit.

dispatch: split out warning message generation to separate function Allow for patching warning message generation, or for patching out the ui.log / ui.warn behaviour (but still generate the warning message).

check-code: reject import Queue, suggest util.queue class for py3 compat

scmutil: use util.queue/util.empty for py3 compat

pycompat: add empty and queue to handle py3 divergence While the pycompat module will actually handle divergence, please access these properties from the util module: util.queue = Queue.Queue / queue.Queue util.empty = Queue.Empty / queue.Empty

convert: kill dead code gitread is unused with the new commandline-based code.

convert: don't ignore errors from git diff-tree

crecord: check for untracked arguments hg commit tracked untracked -- fails complaining about untracked prior to this commit, hg commit -i tracked untracked -- did not fail This is corrected by calling the refactored localrepo.checkcommitpatterns

localrepo: drop force check from checkcommitpatterns It was retained to make the code movement clearer

localrepo: refactor commit argument check as checkcommitpatterns

run-tests: handle empty tests

docchecker: try to reject single quotes

docchecker: report context line at most once

tests: splitting test-gendoc.t into per file tests Localizers can now run test-gendoc-$LOCALE.t instead of test-gendoc.t. After this change, test-gendoc.t only checks whether there is *some* localization for the expected set of languages and no others. Whenever a locale i18n/$LOCALE.po is added, someone needs to add test-gendoc-$LOCALE.t

test-hgweb-auth: stop direct symbol import of mercurial.error.Abort

test-hgweb-auth: alias ui as uimod

test-hg-parseurl: stop direct symbol import of mercurial.hg.parseurl

test-filelog: alias ui as uimod

test-duplicateoptions: alias ui as uimod