pycompat: alias urllib symbols directly urllib.request imports a bunch of symbols from other urllib modules. We should map to the original symbols not the re-exported ones because this is more correct. Also, it will prevent an import of urllib.request if only one of the lower-level symbols/modules is needed.

tests: clean up bad extension The presence of the "babar" extension breaks subsequent tests. So delete the file and create an empty one to return the config to sanity.

perf: perform a garbage collection before each iteration Currently, no explicit garbage collection is performed when running the microbenchmarks in `hg perf`. I think this is wrong because garbage collection can have a significant impact on execution times. And, if gc is triggered via the default heuristics, it will fire effectively randomly during subsequent benchmark iterations due to variable amount of garbage left over from previous runs. Running a gc before invoking the measured function will help ensure state is more consistent across all iterations.

formatter: support json formatting of long type By luck, we appear to not pass any long instances into the JSON formatter. I suspect this will change with all the Python 3 porting work. Plus I have another series that will convert some ints to longs that triggers this.

rebase: don't use mutable default argument value

mq: don't use mutable default argument value

util: don't use mutable default argument value I don't think this is any tight loops and we'd need to worry about PyObject creation overhead. Also, I'm pretty sure strptime() will be much slower than PyObject creation (date parsing is surprisingly slow).

match: don't use mutable default argument value There shouldn't be a big perf hit creating a new object because this function is complicated and does things that dwarf the cost of creating a new PyObject.

hgweb: don't use mutable default argument value

hgweb: don't use mutable default argument value

filemerge: don't use mutable default argument value

context: don't use mutable default argument value Mutable default argument values are a Python gotcha and can represent subtle, hard-to-find bugs. Lets rid our code base of them.

heads: enable pager

branches: enable pager

py3: fix slicing of bytes in revset.formatspec()

py3: make set of revset operators and quotes in bytes

py3: convert set of revset initial symbols back to bytes Otherwise tokenize() would fail due to comparison between unicode and bytes.

pycompat: add helper to iterate each char in bytes

branchmap: fix python 2.6 by using util.buffer() instead of passing bytearray

rebase: allow rebasing children of wd to wd if a new branch has been set (BC) The named branch of the leaf changeset can be changed by updating to it, setting the branch, and amending. But previously, there was no good way to *just* change the branch of several linear changes. If rebasing changes with another parent to '.', it would pick up a pending branch change up. But when rebasing changes that have the same parent, it would fail with 'nothing to rebase', even when the branch name was set differently. To fix this, allow rebasing to same parent when a branch has been set.

merge: check current wc branch for 'nothing to merge', not its p1 The working directory will usually be clean or very clean, and wc will usually have the same branch as its parent. This change will thus usually not make any difference and is done as a separate change to show that. It will be used in a later change.

lock: do not encode result of gethostname on Python 2 If a hostname contained non-ascii character, str.encode() would first try to decode it to a unicode and raise UnicodeDecodeError.

py3: prove `hg files --rev` works

tests: make a variable for hg binary location in test-check-py3-commands The number of which calls in here is starting to get silly.

lock: encode result of gethostname into a bytestring

config: avoid using a mutable default Nothing *currently* mutates this list, but the moment something does it'll be shared between all config instances. Avoid this eventuality.

localrepo: deprecate 'repo.join' in favor of 'repo.vfs.join' localrepo have an insane amount of method. Accessing the feature through the vfs is not really harder and allow us to schedule that method for removal.

pycompat: move imports of cStringIO/io to where they are used There's no point to import cStringIO as io since we have to select StringIO or BytesIO conditionally.

rbc: empty (and invalid) rbc-names file should give an empty name list An empty file (if it somehow should exist) used to give a list with an empty name. That didn't do any harm, but it was "wrong". Fix that.

rbc: use struct unpack_from and pack_into instead of unpack and pack These functions were introduced in Python 2.5 and are faster and simpler than the old ones ... mainly because we can avoid intermediate buffers: $ python -m timeit -s "_rbcrecfmt='>4sI'" -s 's = "x"*10000' -s 'from struct import unpack' 'unpack(_rbcrecfmt, buffer(s, 16, 8))' 1000000 loops, best of 3: 0.543 usec per loop $ python -m timeit -s "_rbcrecfmt='>4sI'" -s 's = "x"*10000' -s 'from struct import unpack_from' 'unpack_from(_rbcrecfmt, s, 16)' 1000000 loops, best of 3: 0.323 usec per loop $ python -m timeit -s "from array import array" -s "_rbcrecfmt='>4sI'" -s "s = array('c')" -s 's.fromstring("x"*10000)' -s 'from struct import pack' -s "rec = array('c')" 'rec.fromstring(pack(_rbcrecfmt, "asdf", 7))' 1000000 loops, best of 3: 0.364 usec per loop $ python -m timeit -s "from array import array" -s "_rbcrecfmt='>4sI'" -s "s = array('c')" -s 's.fromstring("x"*10000)' -s 'from struct import pack_into' -s "rec = array('c')" -s 'rec.fromstring("x"*100)' 'pack_into(_rbcrecfmt, rec, 0, "asdf", 7)' 1000000 loops, best of 3: 0.229 usec per loop

revlog: use bytes() instead of str() to get data from memoryview Fixes `files -v` on Python 3.

util: teach url object about __bytes__ __str__ tries to do something reasonable, but someone else more familiar with encoding bugs should check my work.

manifest: ensure paths are bytes (not str) in pure parser

manifest: now that node.bin is available, use it directly Previously we were getting it through revlog, which is a little unusual.

manifest: use node.bin instead of .decode('hex') The latter doesn't work in Python 3.

manifest: add __next__ methods for Python 3 Python 3 renamed .next() in the iterator protocol to __next__().

files: use native string type to load rev opt from dict

store: fix many single-byte ops to use slicing in _auxencode

py3: add "b" prefix to string literals related to module policy String literals without explicit prefix in __init__.py and policy.py are treated as unicode object on Python3, because these modules are loaded before setup of our specific code transformation (the later module is imported at the beginning of __init__.py). BTW, "modulepolicy" in __init__.py is initialized by "policy.policy". This causes issues below; - checking "policy" value in other modules causes unintentional result For example, "b'py' not in (u'c', u'py')" returns True unintentionally on Python3. - writing "policy" out fails at conversion from unicode to bytes 62939e0148f1 fixed this issue for default code path, but "policy" can be overridden by HGMODULEPOLICY environment variable (it should be rare case for developer using Python3, though). This patch does: - add "b" prefix to all string literals, which are related to module policy, in modules above. - check existence of HGMODULEPOLICY, and overwrite "policy" only if it exists For simplicity, this patch omits checking "supports_bytes_environ", switching os.environ/os.environb, and so on (Yuya agreed this in personal talking)

py3: drop unused aliases to array.array which are replaced with bytearray

pycompat: default to BytesIO instead of StringIO

repoview: specify setattr values as native strings

revlog: use bytes() to ensure text from _chunks is a reasonable type

revlog: extract first byte of revlog with a slice so it's portable

revsetlang: slice out single bytes instead of indexing For portability with Python 3.

lock: use %d to format integer into a bytestring

parser: use %d instead of %s for interpolating error position Error position is an int, so we should use %d instead of %s. Fixes failures on Python 3.

manifest: unbreak pure-python manifest parsing on Python 3

context: use portable construction to verify int parsing

ui: portably bytestring-ify url object

scmutil: fix key generation to portably bytestringify integer

branchmap: stringify int in a portable way We actually need a bytes in Python 3, and thanks to our nasty source loader this will portably do the right thing.

branchmap: don't use buffer() on Python 3 This is certainly slower than the Python 2 code, but it works, and we can revisit it later if it's a problem.

py3: use bytearray() instead of array('c', ...) constructions Portable from 2.6-3.6.

summary: don't explicitly str() something we're about to %s str() is wrong on Python 3 here, and %s implicitly calls str() anyway, so this was just extra dancing for no reason.

context: implement both __bytes__ and __str__ for Python 3 They're very similar, for obvious reasons.

context: work around `long` not existing on Python 3 I can't figure out what this branch is even trying to accomplish, and it was introduced in ac89a23ca814 which doesn't really shed any insight into why longs are treated differently from ints.

phases: explicitly evaluate list returned by map On Python 3 map() returns a generator, which bool()s to true even if it had an empty input set. Work around this by using list() on the map() result.

ui: check for --debugger in sys.argv using r-string to avoid bytes on py3 Our source loader was errantly turning this --debugger into a bytes, which was then causing me to still get a pager when I was using the debugger on py3. That made life hard.

minirst: use bytes.strip instead of str.strip bytes.strip exists in Python 2.6 and Python 2.7 also.

smcposix: pass unicode as first argument to array.array This is an instance where we can safely convert the first argument, rest are the cases except one where we are using 'c' which is not there in Python 3. So that needs to be handled differently. This will help in making `hg help` run on Python 3.

util: pass encoding.[encoding|encodingmode] as unicodes We need to pass str to encode() and decode().

win32text: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

transplant: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

shelve: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

share: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

mq: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

largefiles: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

keyword: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

journal: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

histedit: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

eol: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

convert: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

test-bundle2-remote-changegroup: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

subrepo: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

repair: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

merge: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

hg-mod: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

commands: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

cmdutil: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

localrepo: directly use repo.vfs.join The 'repo.join' method is about to be deprecated.

minirst: make encoding.encoding unicodes to pass into encode() and decode()

minirst: make regular expressions bytes

setup: convert setupversion to unicode Something deep in the bowels of distutils expects "version" passed to setup() to be a str/unicode. So, convert the type. This still works on Python 2 because the string is ascii and an implicit coercion back to str/bytes should work without issue. If it does cause problems, we can always make the unicode conversion dependent on running Python 3. This change makes `python3.5 setup.py install` work.

util: add debugstacktrace depth limit Useful when you don't care about the start of the stack, but only want to see the last entries.

util: strip trailing newline from debugstacktrace message This makes the function more convenient to use as drop-in replacement for ui.write & co.

tests: rework util.debugstacktrace tests Prepare for adding another test.

vfs: use repo.vfs.rename

vfs: use repo.vfs.unlinkpath

merge: use repo.wvfs.unlinkpath

vfs: use repo.wvfs.unlinkpath

policy: try and always have a bytes for module policy debuginstall now runs cleanly in Python 3.

init: zstd is already python3-ready, so don't run it through our importer

config: guard against setconfig specifying unicode values on py3 This was leading to some difficult to trace problems because the values were set in one place, but then blew up much later in the program. Exploding violently with an assertion seems reasonable here.

dispatch: enforce bytes when converting boolean flags to config items This fixes --verbose on Python 3.

py3: add a test that proves %include and missing-extension code works

extensions: tapdance to get reasonable import error formatting I'm not thrilled with this, but it seems to work.

rebase: abort if *any* commit in rebase set is public

merge with stable

httpconnection: rename config to groups Because that is what it is.

httpconnection: don't use dict() Not sure the history here. But we don't do this elsewhere.

formatter: set _first on formatter, not ui The _first field is used for tracking when to emit a separator between items. It seems like it's clearly formatter state, not ui state, so let's move it there.

rebase: unhide original working directory node as well (issue5219) By including the working directory revision at the start of rebase in the repo._rebaseset, we make sure it's not hidden when we update back to it at the end of the rebase. This feels like abusing the set a bit given its name (_rebaseset), but I couldn't think of another name that's clearly better.

rebase: pass in a regular set to _setrebasesetvisibility() Trivial refactoring to simplify the next patch.

tests: demonstrate broken rebase from obsolete commit This shows the bug I reported in issue 5219.

treemanifest: make node reuse match flat manifest behavior In a flat manifest, a node with the same content but different parents is still considered a new node. In the current tree manifests however, if the content is the same, we ignore the parents entirely and just reuse the existing node. In our external treemanifest extension, we want to allow having one treemanifest for every flat manifests, as a way of easeing the migration to treemanifests. To make this possible, let's change the root node treemanifest behavior to match the behavior for flat manifests, so we can have a 1:1 relationship. While this sounds like a BC breakage, it's not actually a state users can normally get in because: A) you can't make empty commits, and B) even if you try to make an empty commit (by making a commit then amending it's changes away), the higher level commit logic in localrepo.commitctx() forces the commit to use the original p1 manifest node if no files were changed. So this would only affect extensions and automation that reached passed the normal localrepo.commit() logic straight into the manifest logic.

help: update help.internalstable for new censor docs

parsers: drop old nonnormalentries method This is okay to do because the Python will fall back transparently if the method is missing.

parsers: avoid leak of nonnset and otherpset Py_BuildValue increments the refcount, rather than stealing the reference, which I missed in code review.

sslutil: issue warning when [hostfingerprint] is used Mercurial 3.9 added the [hostsecurity] section, which is better than [hostfingerprints] in every way. One of the ways that [hostsecurity] is better is that it supports SHA-256 and SHA-512 fingerprints, not just SHA-1 fingerprints. The world is moving away from SHA-1 because it is borderline secure. Mercurial should be part of that movement. This patch adds a warning when a valid SHA-1 fingerprint from the [hostfingerprints] section is being used. The warning informs users to switch to [hostsecurity]. It even prints the config option they should set. It uses the SHA-256 fingerprint because recommending a SHA-1 fingerprint in 2017 would be ill-advised. The warning will print itself on every connection to a server until it is fixed. There is no way to suppress the warning. I admit this is annoying. But given the security implications of sticking with SHA-1, I think this is justified. If this patch is accepted, I'll likely send a follow-up to start warning on SHA-1 certificates in [hostsecurity] as well. Then sometime down the road, we can drop support for SHA-1 fingerprints. Credit for this idea comes from timeless in issue 5466.

setup: use setuptools on Windows (issue5400) We've had a long, complicated history with setuptools. We want to make it the universal default. But when we do, it breaks things. `python setup.py build` is broken on Windows today. Forcing the use of setuptools via FORCE_SETUPTOOLS=1 unbreaks things. Since the previous bustage with making setuptools the default was on !Windows, it seems safe to move ahead with the setuptools transition on Windows. So this patch does that.

schemes: use br'' literal to define bytes regexp

help: fix layout of pre-formatted text

help: fix example of revs() fileset

filecache: make 'join' abstract All subclasses redefine this method, so we can make it abstract.

filecache: explicitly test 'repofilecache' The tests is actually about testing a repofilecache (it uses a fake repo). We make this clear to prevent blockers while cleaning theses API.

repofilecache: directly use 'repo.vfs.join' The 'vfs' attribute already have all methods we need, the value of going through the repository for this is low. so we removes it.

repofilecache: define a 'join' method We are about to turn the 'join' method of the base class Abstract, so we need on to be defined in the localrepo. The ultimate goal here is to be able to stop relying for the 'localrepo' class to have a 'join' methods (there is above one hundred methods on 'localrepo'. This change make te 'repo' file cache have its own code so that we can prepare this change to the repostory class. explicite join

wix: add censor docs to installer script Spotted by Matt Harbison.

internals: add some brief documentation about censor