debugignore: eliminate inconsistencies with `hg status` (issue5222) Using a matcher for this command allows processing the named file(s) as relative to cwd. It also leverages the icasefs normalization logic the same way the status command does. (However, a false indicator is given for a nonexistent file in some cases, e.g. passing 'foo.REJ' when that file doesn't exist, and the rule is '*.rej'. Maybe the regex itself needs to be case insensitive on these platforms, at least for the debug command.) Finally, the file printed is relative to cwd and uses platform specific slashes, so a few (glob)s were needed in seemingly unrelated tests.

commandserver: close selector explicitly The selector does not have a __del__ method and needs a manual close. We can also use "with selector" but that makes the code too indented. Therefore append a "selector.close()" after the end of the main loop for now.

scmutil: remove duplicated import of i18n._()

obsstore: let read marker API take a range of offsets This allows us to read a customized range of markers, instead of loading all of them. The condition of stop is made consistent across C and Python implementation so we will still read marker when offset=a, stop=a+1.

commandserver: use selectors2 Previously, commandserver was using select.select. That could have issue if _sock.fileno() >= FD_SETSIZE (usually 1024), which raises: ValueError: filedescriptor out of range in select() We got that in production today, although it's the code opening that many files to blame, it seems better for commandserver to work in this case. There are multiple way to "solve" it, like preserving a fd with a small number and swap it with sock using dup2(). But upgrading to a modern selector supported by the system seems to be the most correct way.

selector2: vendor selector2 library This library was a backport of the Python 3 "selectors" library. It is useful to provide a better selector interface for Python2, to address some issues of the plain old select.select, mentioned in the next patch. The code [1] was ported using the MIT license, with some minor modifications to make our test happy: 1. "# no-check-code" was added since it's foreign code. 2. "from __future__ import absolute_import" was added. 3. "from collections import namedtuple, Mapping" changed to avoid direct symbol import. [1]: https://github.com/SethMichaelLarson/selectors2/blob/d27dbd2fdc48331fb76ed431f44b6e6956de7f82/selectors2.py # no-check-commit

context: name files relative to cwd in warning messages I was several directories deep in the kernel tree, ran `hg add`, and got the warning about the size of one of the files. I noticed that it suggested undoing the add with a specific revert command. The problem is, it would have failed since the path printed was relative to the repo root instead of cwd. While here, I just fixed the other messages too. As an added benefit, these messages now look the same as the verbose/inexact messages for the corresponding command. I don't think most of these messages are reachable (typically the corresponding cmdutil function does the check). I wasn't able to figure out why the keyword tests were failing when using pathto()- I couldn't cause an absolute path to be used by manipulating the --cwd flag on a regular add. (I did notice that keyword is adding the file without holding wlock.)

run-tests: disable color on Windows More Windows sadness. Maybe someone can figure out how to make win32 color work, but I think we avoid importing stuff from the mercurial package in this module. On the plus side, this conditionalizes away a test failure.

codemod: register core configitems using a script This is done by a script [2] using RedBaron [1], a tool designed for doing code refactoring. All "default" values are decided by the script and are strongly consistent with the existing code. There are 2 changes done manually to fix tests: [warn] mercurial/exchange.py: experimental.bundle2-output-capture: default needs manual removal [warn] mercurial/localrepo.py: experimental.hook-track-tags: default needs manual removal Since RedBaron is not confident about how to indent things [2]. [1]: https://github.com/PyCQA/redbaron [2]: https://github.com/PyCQA/redbaron/issues/100 [3]: #!/usr/bin/env python # codemod_configitems.py - codemod tool to fill configitems # # Copyright 2017 Facebook, Inc. # # This software may be used and distributed according to the terms of the # GNU General Public License version 2 or any later version. from __future__ import absolute_import, print_function import os import sys import redbaron def readpath(path): with open(path) as f: return f.read() def writepath(path, content): with open(path, 'w') as f: f.write(content) _configmethods = {'config', 'configbool', 'configint', 'configbytes', 'configlist', 'configdate'} def extractstring(rnode): """get the string from a RedBaron string or call_argument node""" while rnode.type != 'string': rnode = rnode.value return rnode.value[1:-1] # unquote, "'str'" -> "str" def uiconfigitems(red): """match *.ui.config* pattern, yield (node, method, args, section, name)""" for node in red.find_all('atomtrailers'): entry = None try: obj = node[-3].value method = node[-2].value args = node[-1] section = args[0].value name = args[1].value if (obj in ('ui', 'self') and method in _configmethods and section.type == 'string' and name.type == 'string'): entry = (node, method, args, extractstring(section), extractstring(name)) except Exception: pass else: if entry: yield entry def coreconfigitems(red): """match coreconfigitem(...) pattern, yield (node, args, section, name)""" for node in red.find_all('atomtrailers'): entry = None try: args = node[1] section = args[0].value name = args[1].value if (node[0].value == 'coreconfigitem' and section.type == 'string' and name.type == 'string'): entry = (node, args, extractstring(section), extractstring(name)) except Exception: pass else: if entry: yield entry def registercoreconfig(cfgred, section, name, defaultrepr): """insert coreconfigitem to cfgred AST section and name are plain string, defaultrepr is a string """ # find a place to insert the "coreconfigitem" item entries = list(coreconfigitems(cfgred)) for node, args, nodesection, nodename in reversed(entries): if (nodesection, nodename) < (section, name): # insert after this entry node.insert_after( 'coreconfigitem(%r, %r,\n' ' default=%s,\n' ')' % (section, name, defaultrepr)) return def main(argv): if not argv: print('Usage: codemod_configitems.py FILES\n' 'For example, FILES could be "{hgext,mercurial}/*/**.py"') dirname = os.path.dirname reporoot = dirname(dirname(dirname(os.path.abspath(__file__)))) # register configitems to this destination cfgpath = os.path.join(reporoot, 'mercurial', 'configitems.py') cfgred = redbaron.RedBaron(readpath(cfgpath)) # state about what to do registered = set((s, n) for n, a, s, n in coreconfigitems(cfgred)) toregister = {} # {(section, name): defaultrepr} coreconfigs = set() # {(section, name)}, whether it's used in core # first loop: scan all files before taking any action for i, path in enumerate(argv): print('(%d/%d) scanning %s' % (i + 1, len(argv), path)) iscore = ('mercurial' in path) and ('hgext' not in path) red = redbaron.RedBaron(readpath(path)) # find all repo.ui.config* and ui.config* calls, and collect their # section, name and default value information. for node, method, args, section, name in uiconfigitems(red): if section == 'web': # [web] section has some weirdness, ignore them for now continue defaultrepr = None key = (section, name) if len(args) == 2: if key in registered: continue if method == 'configlist': defaultrepr = 'list' elif method == 'configbool': defaultrepr = 'False' else: defaultrepr = 'None' elif len(args) >= 3 and (args[2].target is None or args[2].target.value == 'default'): # try to understand the "default" value dnode = args[2].value if dnode.type == 'name': if dnode.value in {'None', 'True', 'False'}: defaultrepr = dnode.value elif dnode.type == 'string': defaultrepr = repr(dnode.value[1:-1]) elif dnode.type in ('int', 'float'): defaultrepr = dnode.value # inconsistent default if key in toregister and toregister[key] != defaultrepr: defaultrepr = None # interesting to rewrite if key not in registered: if defaultrepr is None: print('[note] %s: %s.%s: unsupported default' % (path, section, name)) registered.add(key) # skip checking it again else: toregister[key] = defaultrepr if iscore: coreconfigs.add(key) # second loop: rewrite files given "toregister" result for path in argv: # reconstruct redbaron - trade CPU for memory red = redbaron.RedBaron(readpath(path)) changed = False for node, method, args, section, name in uiconfigitems(red): key = (section, name) defaultrepr = toregister.get(key) if defaultrepr is None or key not in coreconfigs: continue if len(args) >= 3 and (args[2].target is None or args[2].target.value == 'default'): try: del args[2] changed = True except Exception: # redbaron fails to do the rewrite due to indentation # see https://github.com/PyCQA/redbaron/issues/100 print('[warn] %s: %s.%s: default needs manual removal' % (path, section, name)) if key not in registered: print('registering %s.%s' % (section, name)) registercoreconfig(cfgred, section, name, defaultrepr) registered.add(key) if changed: print('updating %s' % path) writepath(path, red.dumps()) if toregister: print('updating configitems.py') writepath(cfgpath, cfgred.dumps()) if __name__ == "__main__": sys.exit(main(sys.argv[1:]))

phabricator: allow specifying reviewers on phabsend Sometimes people want to specify reviewer explicitly for a stack. The webpage only allows changing reviewer for one revision at a time. This patch adds a `--reviewer` flag to make it easier to specify reviewers. Test Plan: On a test Phabricator instance, enable `differential.allow-self-accept`, assign myself as a reviewer and make sure it works. Also try an invalid username and make sure it raises. Differential Revision: https://phab.mercurial-scm.org/D38