py3: add alternate output on Python 3 This is basically the same deal as D5806. Python 3's exception printing output is different. We had to tweak the regular expression to match appropriately, hence the added line of output. Differential Revision: https://phab.mercurial-scm.org/D5807

py3: add Python 3 output for test-flagprocessor.t The exception is being generated from ui.traceback(), which simply calls traceback.format_*() to format exceptions. Since the output from the standard library has changed, there is seemingly not much we can do about it. So this commit adds conditional output depending on the Python version. I'm not thrilled about b'' appearing in user-facing output. Can we do something better here? Differential Revision: https://phab.mercurial-scm.org/D5806

zeroconf: Python 3 porting of vendored library A quick glance through this module reveals that most of it "just works" on Python 3 with the source transformer active. There are a few places where we need to ensure we're using str. Differential Revision: https://phab.mercurial-scm.org/D5804

relnotes: more improvements

py3: don't use universal_newlines in svnsubrepo On Python 3, it means text=True without any specific encoding, which is pretty bad. Instead, use util.fromnativeeol() to translate CR+LF to LF without encoding conversion.

py3: remove unneeded fsencode() from gitsubrepo Here p.stdout isn't a TextIO. read() must return bytes.

debugcommands: add a debugpathcopies command I've been working on storing copy metadata in the changelog instead of the filelog and this has been useful for debugging. Do we usually add tests for these? Differential Revision: https://phab.mercurial-scm.org/D5791

py3: record several more passes from the buildbot ratchet Differential Revision: https://phab.mercurial-scm.org/D5799

grep: respect ui.relative-paths Differential Revision: https://phab.mercurial-scm.org/D5777

grep: move writing of path outside of column loop This will make the next patch simpler. Differential Revision: https://phab.mercurial-scm.org/D5776

resolve: respect ui.relative-paths Differential Revision: https://phab.mercurial-scm.org/D5748

merge: respect ui.relative-paths We print file paths in a lot of places in this code and I've probably missed a few places. We can fix them as we discover them (I'm also happy to fix anything reviewers notice, of course). Differential Revision: https://phab.mercurial-scm.org/D5747

histedit: add templating support to histedit's rule file generation This will allow users to customize the display of the rule list for the free-form segment that we don't interpret. We've had users want to add things like bookmark names or similar to the rule list as a convenience, which seems reasonable. Differential Revision: https://phab.mercurial-scm.org/D5742

py3: account for demand import difference between Python versions Our lazy importer for Python 3 will validate that modules are loadable before returning a stub module object. This is different from Python 2, which will always return a stub module object. While we could change behavior of the Python 3 demand importer, that seems like a problem for another day. This commit teaches test-extension.t about that difference in behavior. Differential Revision: https://phab.mercurial-scm.org/D5798

tests: use unimported modules in test-demandimport.py contextlib isn't a good module to test because it is likely already imported by code above. Let's use modules that shouldn't have been imported. And let's verify that with asserts. Differential Revision: https://phab.mercurial-scm.org/D5797

py3: conditionalize test-demandimport.py for Python 3 The Python 3 lazy importer uses the LazyLoader that is part of importlib. On Python 3 and later, LazyLoader is implemented using a custom module type that defines a __getattribute__ which triggers module loading. Furthermore, there are additional differences as well. For example, it appears that Python 3 will return an existing sys.modules entry instead of constructing a new module object. This commit adds additional test coverage for lazy importing behavior to cover the differences between Python 2 and 3. This reveals that the test and some lazy import functionality is kinda busted. For example, the test assumes "contextlib" will be lazy. But in reality an import before it has already imported contextlib! There's definitely room to improve the behavior of the demand importer code, both for Python 2 and 3. But at least the test passes on Python 3 now. Differential Revision: https://phab.mercurial-scm.org/D5796

py3: replace print() with assert in test-demandimport.py Behavior of demand imports behaves differently between Python 2 and 3. .out files do not support conditional output the way that .t files do. In order to make this test work on Python 3, we'll need to make the test itself conditional. The first step of this is to port the test to not use a .out file to compare output. Unfortunately, we can't easily use the unittest framework for defining this test because putting import statements in functions changes the behavior of the demand importer (at least on Python 2). So, we effectively replace a bunch of print() with assert statements. This makes the test a bit annoying to debug, as the test will stop at first assertion failure. But we don't exactly have a good alternative. Differential Revision: https://phab.mercurial-scm.org/D5795

py3: pass str into grp.getgrnam grp.getgrnam expects str on Python 3. This fixes test-acl.t on Python 3. Differential Revision: https://phab.mercurial-scm.org/D5794

montone: fix addition to list by using .append() instead of '+' Differential Revision: https://phab.mercurial-scm.org/D5739

tests: give up and make setsockopt() calls optional in the output I can't figure out what causes these calls to happen or not, and I weary of dealing with them. End the madness for now by marking them all as optional lines of output. Differential Revision: https://phab.mercurial-scm.org/D5762

py3: fix test-remotefilelog-repack.t This is uglier, but more obviously correct in my opinion. I guess Python 3 doesn't hang on to the exception as long, which seems reasonable. Differential Revision: https://phab.mercurial-scm.org/D5781

tests: alter email `From` line to a value that's consistently parsed Python2: >>> email.header.decode_header('=?UTF-8?q?Rapha=C3=ABl=20Hertzog?= <hertzog@debian.org>') [('Rapha\xc3\xabl Hertzog', 'utf-8'), ('<hertzog@debian.org>', None)] Python3: >>> email.header.decode_header('=?UTF-8?q?Rapha=C3=ABl=20Hertzog?= <hertzog@debian.org>') [(b'Rapha\xc3\xabl Hertzog', 'utf-8'), (b' <hertzog@debian.org>', None)] So alter the input to an input that parses to the same result consistently. After skimming the relevant RFC (1342), I'm not sure if what we had was valid, or how I could modify it to be more consistent. Differential Revision: https://phab.mercurial-scm.org/D5769

diff: drop duplicate filter of copies by destination I'm pretty sure we don't need to filter copies by destination, at least since the previous patch. Differential Revision: https://phab.mercurial-scm.org/D5790

diff: use match.intersectmatchers() Differential Revision: https://phab.mercurial-scm.org/D5789

tests: convert ParseError arguments to str on Python 3 Arguments internally are bytes. Printing the exception on Python 3 will include b'' prefixes. This test file uses a .out file, which doesn't support conditional output. The easiest way to get this to pass on Python 3 is to normalize the exception before printing so there are no b'' prefixes. Differential Revision: https://phab.mercurial-scm.org/D5793

tests: quote $PYTHON for py3 support on Windows

py3: byteify the --retest path of run-tests.py

subrepo: clean up lingering bytes/str issues in svn support Path encoding is a little suspect in here, but it's not worse than it was when we were on Python 2 only. Subversion subrepo tests now pass in Python 3. Differential Revision: https://phab.mercurial-scm.org/D5774

tweakdefaults: set ui.relative-paths instead of command.status.relative Differential Revision: https://phab.mercurial-scm.org/D5775

status: introduce higher-level ui.relative-paths The existing commands.status.relative trumps the new config. We need to keep the existing config around for compatibility. However, I don't think we need to introduce similar command-specific options for other commands when they learn to respec ui.relative-paths. Differential Revision: https://phab.mercurial-scm.org/D5746

status: extract helper for producing relative or absolute path for UI I put the helper in scmutil so it can be used by most modules. I would have put it in utils.pathutil, but I think that's supposed to be unaware of repos. Differential Revision: https://phab.mercurial-scm.org/D5745

tags: avoid generating commit for an already deleted tag (issue5752) `$ hg tag --remove <tagname>` will keep on generating commits regardless of checking whether the tag is deleted or not. I added a a fix for `global` tags by aborting if `nullid` is already mapped to the tag that is to be deleted. Differential Revision: https://phab.mercurial-scm.org/D5780

watchman: detect nested mercurial repositories and abort Disable fsmonitor on nested mercurial repositories. Only depth one repositories are detected at the moment. Differential Revision: https://phab.mercurial-scm.org/D5587

watchman: add verbose config knob This new config knob allows to silent watchman log and warning messages when watchman is unavailable. Differential Revision: https://phab.mercurial-scm.org/D5586

extdiff: add --per-file and --confirm options The new options lets the user control how the external program is run. By default, Mercurial passes the 2 snapshot directories as usual, but it can also run the program repeatedly on each file's snapshot pair, and optionally prompt the user each time.

run-tests: sort the skip, failure and error lists in the final output This will help keep the lists consistent, for comparison across runs.

tests: add Python 3 output for test-remotefilelog-gc.t Python 3 raises a slightly different error on invalid paths. Differential Revision: https://phab.mercurial-scm.org/D5773

hg: raise Abort on invalid path Currently, some os.path functions when opening repositories may raise an uncaught TypeError or ValueError if the path is invalid. Let's catch these exceptions and turn them into an Abort for convenience. Differential Revision: https://phab.mercurial-scm.org/D5772

subrepo: bytes/str cleanups on Git support Git subrepo tests now pass on Python 3. Differential Revision: https://phab.mercurial-scm.org/D5768

tests: compare against a bytes in test-lock.py skip-blame: just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D5771

tests: perform a shallow copy instead of a deep copy Python 3 can't perform a deep copy because it looks like symbols in the locals() namespace are not deep-copyable. For the curious, somehow the deepcopy() is attempting to copy objects attached to the unittest.* functions for the running test! We don't use deepcopy() anywhere in the code base and a shallow object copy should be sufficient to test lock copying. Actually, I'm not sure why we even test this, as I couldn't find copy.copy() being used for lock copying either. Who knows. Differential Revision: https://phab.mercurial-scm.org/D5770

tests: make and use a new `svnurlof.py` helper for constructing svn urls The previous trick of a Python oneliner and some subshells is too hard to make portable, and this lets us consolidate some Windows-specific logic down to a single place. Differential Revision: https://phab.mercurial-scm.org/D5766

server: skip logging of ECONNRESET I believe this was exposed by 5492dc20, because the sending of the 500 would have already failed and prevented this logging. On Python 3, this will be a ConnectionResetError, which is a subtype of OSError, which is why we check for both OSError and socket.error. Bonus: this fixes a race in test-hgweb.t where sometimes the ECONNRESET wouldn't happen, because now we just don't log those errors. Differential Revision: https://phab.mercurial-scm.org/D5764

git: a little pycompat.bytestring() love to make this code work in py3 Differential Revision: https://phab.mercurial-scm.org/D5765

py3: have test-revset2.t write test scripts in a more portable way Fixes the test on Python 3. Differential Revision: https://phab.mercurial-scm.org/D5763

py3: fix up test-remotefilelog-cacheprocess.t to not depend on a repr It looks like the repr() of Exceptions is different from Python 2 to Python 3.7 (but not 3.5?), but the str() is still stable. Sigh. Differential Revision: https://phab.mercurial-scm.org/D5761

remotefilelog: cast division result to an int Otherwise mid is a float and this confuses __slice__ on Python 3. Differential Revision: https://phab.mercurial-scm.org/D5760

tests: cast division result to int Otherwise it is a float on Python 3 and code later compares about casting a float to an int. Differential Revision: https://phab.mercurial-scm.org/D5759

tests: various Python 3 ports for test-remotefilelog-datapack.py Use bytes I/O. Use byteschr(). Convert temporary path to bytes. Differential Revision: https://phab.mercurial-scm.org/D5758

tests: use items() in test-remotefilelog-datapack.py Performance doesn't matter in tests. iteritems() doesn't exist in Python 3. Differential Revision: https://phab.mercurial-scm.org/D5757

tests: use bytes and %d formatting in test-remotefilelog-datapack.py There were numerous failures on Python 3 due to str/bytes mismatch and '%s' not working for ints. Differential Revision: https://phab.mercurial-scm.org/D5756

perf: add a --[no-]clear-caches option to `perfnodemap` The option is useful to look at pure lookup performance on a warm data structure.

perf: add a perfnodemap command The command focus on timing of the nodemap object itself.

wireprotov1server: use binascii.unhexlify The "hex" codec doesn't exist in Python 3. We could use `codecs.decode(h, 'hex_codec')`. But `binascii.unhexlify()` exists and should work the same on Python 2 and 3. Differential Revision: https://phab.mercurial-scm.org/D5755

tests: conditionalize test-http-bad-server.t for Python 3.5 It appears that Python 3 introduced output buffering in the HTTP response stack. And Python 3.6 switched from sock.makefile().write() to sock.sendall(). So, we need to conditionalize test-http-bad-server.t to account for the difference in behavior between Python 3.5 and 3.6. Differential Revision: https://phab.mercurial-scm.org/D5754

tests: log sendall() operations and port test-http-bad-server.t Python 3's HTTP server layer buffers output and uses sendall() instead of write(). In order to make test-http-bad-server.t pass on Python 3, we needed to teach our socket proxy to log sendall() events and to abort future sends if we reached our send limit. The tests using `tail` were difficult to port with inline output conditionals since the number of lines varied. So we now use `#if py3` for these tests. test-http-bad-server.t now passes on Python 3.6 and 3.7 on at least Linux. However, it does not yet pass on Python 3.5 because of low-level differences to how the HTTP server is implemented. Differential Revision: https://phab.mercurial-scm.org/D5753

tests: glob away readline(-1) Most of these are readline(65537) on Python 3. I don't think it is worth the readability hit to use (re), as it would require escaping parenthesis. Differential Revision: https://phab.mercurial-scm.org/D5752

tests: change how sockets are closed Python 3 uses a different type to represent a socket file object than Python 2. We need to conditionalize how the socket is closed accordingly. While we're here, we switch to use socket.shutdown() to close the socket. This is because socket.close() may not actually close the socket until it is GCd. socket.shutdown() forces an immediate shutdown. I suspect Python 3 changed semantic behavior here, as I can't get test-http-bad-server.t to work with socket.close(). socket.shutdown() does appear to work, however. Differential Revision: https://phab.mercurial-scm.org/D5751

tests: add b'' prefixes to badserverext.py This avoids a handful of failures due to missing str and bytes. # skip-blame: just a bunch of b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D5750

hgweb: log error before attempting I/O Previously, an uncaught exception during HTTP request serving would attempt to send an error response then log the exception. If an exception occurred during I/O, this exception would be raised and the original exception wouldn't be logged. This commit changes behavior so the original exception is logged first, before we attempt to do anything else. This ensures the exception is logged. This change resulted in new tracebacks appearing in various tests. Because tracebacks can vary between Python versions, we added a simple script to filter the stack part of traceback lines. This makes testing much simpler, as we don't need to glob over lines and make lines conditional. Differential Revision: https://phab.mercurial-scm.org/D5749

tests: write commit message using file I/O Python 2.7 will print() \x94\x5c\x0a whereas Python 3 will print() \xc2\x94\x5c\x0a. Why, I'm not sure. It probably has to do with print() being Unicode aware on Python 3 and Python attempting some kind of encoding before emitting the output. This difference results in a different bytes making it to the commit message and the JSON output varying. We work around this by writing bytes to a commit message file. Differential Revision: https://phab.mercurial-scm.org/D5741

Added signature for changeset 83377b4b4ae0

Added tag 4.9 for changeset 83377b4b4ae0

subrepo: reject potentially unsafe subrepo paths (BC) (SEC) In addition to the previous patch, this prohibits '~', '$nonexistent', etc. for any subrepo types. I think this is safer, and real-world subrepos wouldn't use such (local) paths.

subrepo: prohibit variable expansion on creation of hg subrepo (SEC) It's probably wrong to expand path at localrepo.*repository() layer, but fixing the layering issue would require careful inspection of call paths. So, this patch adds add a validation to the subrepo constructor. os.path.realpath(util.expandpath(root)) is what vfsmod.vfs() would do.

subrepo: extend path auditing test to include more weird patterns (SEC) While reviewing patches for the issue 5739, "$foo in repository path expanded", I realized that subrepo paths can also be cheated. This patch includes various subrepo paths which are potentially unsafe. Since an expanded subrepo path isn't audited, this bug allows symlink check bypass. As a result, a malicious subrepository could be checked out to a sub tree of e.g. $HOME directory. The good news is that the destination directory must be empty or nonexistent, so the existing ~/.bashrc wouldn't be overwritten. See the last part of the tests for details.

copyright: update to 2019 Differential Revision: https://phab.mercurial-scm.org/D5779

patch: handle 0 context lines (diff.unified=0) when parsing patches Previously, if there were no context lines, we would just keep updating the ranges and the hunk, but not actually storing the hunk (just overwriting it each time). Thus a diff like this: $ hg diff --config diff.unified=0 diff --git a/bar b/bar --- a/bar +++ b/bar @@ -1,0 +2,1 @@ 1 +change1 @@ -3,0 +5,1 @@ 3 +change2 would come out of the parser like this (change1 is lost): bar: @@ -3,0 +5,1 @@ 3 +change2 This had some really weird side effects for things like commit --interactive, split, etc. Differential Revision: https://phab.mercurial-scm.org/D5743

py3: pass str into RuntimeError() to prevent b'' in output # skip-blame as just r'' prefixes Differential Revision: https://phab.mercurial-scm.org/D5740

py3: use '%d' instead of '%s' for integers Differential Revision: https://phab.mercurial-scm.org/D5738

tests: port test-hgweb-auth.py to Python 3 Differential Revision: https://phab.mercurial-scm.org/D5736

url: convert some variables back to bytes Differential Revision: https://phab.mercurial-scm.org/D5735

url: add some defensive asserts on expected incoming types Our type handling is a nightmare here, and we're loading passwords to do network IO, so we can afford to be potentially-slow but pedantic here. Differential Revision: https://phab.mercurial-scm.org/D5734

changegroup: don't try to build changelog chunks if not required When we extend a narrow clone without ellipsis, we don't download changelog information because that's already present with the client. However we still try to build that chunk stream. Building that chunk stream involves calling a lookup function and store.emitrevisions() API. The lookup function is called len(cl) number of times. On large repositories, where len(cl) is in millions, calling that lookup function is not a good idea. Also it's not required to use the store.emitrevisons() API because we already have nodes present which we can use. This patch short-circuits state building logic if we are processing a non-ellipsis case and changelog is not required. This saves up ~20 seconds on our internal repo for a single extend call. Differential Revision: https://phab.mercurial-scm.org/D5733

revlog: make sure we never use sparserevlog without general delta (issue6056) We are getting user report where the delta code tries to use `sparse-revlog` logic on repository where `generaldelta` is disabled. This can't work so we ensure the two booleans have a consistent value. Creating this kind of repository is not expected to be possible the current bug report point at a clonebundle related bug that is still to be properly isolated (Yuya Nishihara seems to a have done it). Corrupting a repository to reproduce the issue is possible. A test using this method is included in this fix.

sparserevlog: document the config option This was overlooked when this graduated from experimental.

changegroup: initialize the state variable a bit earlier This will make the next patch much easier. Differential Revision: https://phab.mercurial-scm.org/D5732

tests: conditionalize test output on Python 3.7 Python 3.7 changed behavior of urllib.parse.quote() from RFC 2396 to RFC 3986 and ~ is now in the set of reserved characters and isn't escaped. We conditioanlize test output accordingly. Differential Revision: https://phab.mercurial-scm.org/D5717

hghave: add pyXY features for Python version numbers This will allow us to sniff for Python >= versions in tests. Differential Revision: https://phab.mercurial-scm.org/D5088

py3: whitelist couple more passing tests found by buildbot Differential Revision: https://phab.mercurial-scm.org/D5731

keepalive: implement _close_conn() so closes are known Keepalives were not working on Python 3 because http.client.HTTPResponse was refactored to call _close_conn() instead of close(). Our custom close() is what returns inactive connections to the available state. We better support Python 3 by implementing a _close_conn(). Differential Revision: https://phab.mercurial-scm.org/D5720

lfs: explicitly add the Content-Length header when uploading blobs, for py3 This was the reason for test-lfs-test-server.t#git-server complaining about an "invalid byte in chunk length". For some reason if this isn't explicitly added, py3.7.1 is adding `transfer-encoding: chunked` as well as `Content-length: x`. Wireshark flagged this as malformed. However, if this is set, it doesn't bother with `transfer-encoding`. Before this patch with py3: PUT /objects/31cf46fbc4ecd458a0943c5b4881f1f5a6dd36c53d6167d5b69ac45149b38e5b HTTP/1.1 Accept-Encoding: identity Content-length: 12 accept: application/vnd.git-lfs content-type: application/octet-stream host: localhost:20062 transfer-encoding: chunked user-agent: git-lfs/2.3.4 (Mercurial 4.9rc0+149-7eb7637e34bf) Before this patch with py27: PUT /objects/31cf46fbc4ecd458a0943c5b4881f1f5a6dd36c53d6167d5b69ac45149b38e5b HTTP/1.1 Accept-Encoding: identity accept: application/vnd.git-lfs content-type: application/octet-stream content-length: 12 host: localhost:20062 user-agent: git-lfs/2.3.4 (Mercurial 4.9rc0+149-7eb7637e34bf+20190128) With this patch and py3, the content is the same as the py27 example. RFC2616 says to ignore `Content-Length` if `Transfer-Encoding` is present, so maybe there's nothing to do in the hg-server side (though I'm not sure which it is using if presented both). Maybe chunked encoding is better to do? If someone knows how to suppress the `Content-Length`, we can try that instead.

perf: support looking up multiple revisions The nodemap code has optimisations around the number of lookup we actually made. As a result, being able to specify multiple revisions to look up is important when measuring performances. One can now specify full revspecs with the --rev arguments.

perf: add a no-lookup variant to perfindex It is useful to check how long it takes to create a index object without doing anything with it. We add a new flag dedicated to that.

perf: add some documentation to perfindex It seems useful to document how the arguments can affect the benchmark.

perf: move cache clearing in the `setup` step of `perfheads` The cache clearing is pretty fast, but this seems more "correct".

revlog: document cext oddities in terms of object/caches This clarify why we just call clearcaches on a single object.

perf: document perfheads

py3: stabilize the output of lfs commandserver tests The print() statements were being output at the very end, so write to the same stdout sink as runcommand, and explicitly flush.

py3: conditionalize some LFS test output I'm not sure why the one stackframe is py2 only, but that seems harmless. The remaining failure is LfsCorruptionError printing the fully qualified name, as well as b'' around its message.

lfs: strip the response headers from the Batch API before printing For reasons unknown, py3 is adding an extra '\n' before the headers print out. This makes the output the same as py2.

py3: force hgweb.server error log to internally write unicode Otherwise, there's a lot of py2/py3 divergence in the LFS tests because of the "HG error" lines picking up a b'' prefix. wsgicgi.py uses procutil.stderr, so I assume the input was meant to be bytes.

py3: byteify the decoded JSON responses upon receipt in the LFS blobstore It got too confusing juggling r'' vs b'' across several functions.

hgweb: ensure Content-Length and Content-Type are not promoted to HTTP_ on py3 In stabilizing test-lfs-serve-access.t for py3, the server started asserting on blob upload: Environment should not have the key: HTTP_CONTENT_LENGTH (use CONTENT_LENGTH instead) It could be avoided by explicitly setting the Content-Length header on the client side. I didn't go back to py2, but printing the original header here in py37 revealed 'Content-length' when sent to the error log.

py3: raw stringify various JSON and HTTP headers in the LFS blobstore module This is (almost?) entirely from Augie's work. I'm a bit surprised that the JSON data is being encoded with ASCII via `pycompat.bytesurl()`- I would have thought UTF-8.

py3: byteify the LFS blobstore module This is almost entirely b'' prefixing, with a couple of exceptions forced to bytes. Much of this is also borrowed from Augie's code. There's an HTTPError.read() that I flagged that I assume needs to be converted to bytes, but I can't find confirmation. Handling the deserialized JSON object over several functions made r'' vs b'' accesses confusing, so this assumes that the JSON object will be converted to bytes immediately. That will be done in the following commits, so it's not buried in these trivial changes.

py3: raw stringify various things in the LFS server module Some of this is based on code written by Augie. I'm slightly unsure if these are the correct pycompat bytes <-> str conversion methods.

py3: add b'' prefixes to the LFS server module There are a ton of these changes in the blobstore, as well as r'' prefixing. So separating these out hopefully makes review easier. # skip-blame for b'' prefixing

py3: add b'' and r'' prefixes to LFS tests

py3: more tests from the ratchet Differential Revision: https://phab.mercurial-scm.org/D5730

dispatch: unify handler of IOError and OSError These exceptions were merged in Python 3.

dispatch: quote filename in IOError as well It's explicitly added for OSError at 720308f741cb "dispatch: show empty filename in OSError aborts". Let's do the same for IOError.

dispatch: add inline comment about possible IOError subtypes It's hard to tell which "if" would handle which exception.

convert: print exception message directly Otherwise the default repr() implementation will b'' prefix the returned str on Python 3. Differential Revision: https://phab.mercurial-scm.org/D5729

crecord: always return a str from uihunk.__repr__ Otherwise Python 3 complains about it returning bytes. Differential Revision: https://phab.mercurial-scm.org/D5728

tests: handles bytes/str mismatch in test-commit-interface-curses.t This fixes various test failures on Python 3. Differential Revision: https://phab.mercurial-scm.org/D5727

githelp: format with %d if an integer Python 3 doesn't allow us to format an int with %s like Python 2 did. So handle that. Differential Revision: https://phab.mercurial-scm.org/D5726

githelp: make argument parsing more compatible with Python 3 There were various mixing of str and bytes in here. This change fixes most of the failures in test-githelp.t. Differential Revision: https://phab.mercurial-scm.org/D5725

tests: add b'' prefixes to flagprocessorext.py Otherwise a part of test-flavprocessor.t fails due to not setting the proper key. Differential Revision: https://phab.mercurial-scm.org/D5724

tests: add b'' to config file name Without this, things die in the bowels of the config system due to mixing str and bytes. # skip-blame: just b'' prefix Differential Revision: https://phab.mercurial-scm.org/D5723

sslutil: ensure serverhostname is bytes when formatting It will likely be a str on Python 3. Differential Revision: https://phab.mercurial-scm.org/D5722

sslutil: use raw strings for exception reason compare Otherwise we attempt to compare a bytes to a str on Python 3 and it always fails. Differential Revision: https://phab.mercurial-scm.org/D5721

keepalive: track ready state with a bool This code may have been written before Python had a bool type. Differential Revision: https://phab.mercurial-scm.org/D5719

keepalive: use collections.defaultdict for host map Cleaning up the code as part of debugging Python 3 issues. Differential Revision: https://phab.mercurial-scm.org/D5718

statichttprepo: use str to appease Python 3 The URL fed into urllib and HTTP headers need to be str on Python 3. Differential Revision: https://phab.mercurial-scm.org/D5716

statichttprepo: use URLError.reason directly 0b3f4be5c5bf changed str(inst) to inst.reason[0] all the way back in 2006. URLError.reason is a str and we should have taken that attribute in its entirety. I think the code was supposed to be inst.args[1] for compatibility with ancient Python versions. Python 2.7 always sets .reason, so it should be safe to use directly. Differential Revision: https://phab.mercurial-scm.org/D5715

mail: document behavior of Python 3 test-notify.t (and possibly other tests) are failing on Python 3 because email.message.Message is now aware of encodings and attempts to roundtrip values with the specified message encoding. Python 2 doesn't perform this roundtripping. We have tests with non-ascii data being serialized to a message that claims to use ascii encoding. I /think/ Mercurial's behavior may be buggy here. But I'm not sure. I'm documenting the behavior so the next person who looks into this doesn't start from scratch like I did. Differential Revision: https://phab.mercurial-scm.org/D5714

notify: be more defensive aboute None values encoding.strtolocal is the identity function on Python 2 but an actual string manipulation routine on Python 3. In some cases, we were passing None, which caused Python 3 to barf. Let's change the code to react properly when the value is None. Differential Revision: https://phab.mercurial-scm.org/D5713

wireprotov2server: use our JSON encoder Python's json module doesn't like to encode bytes instances. This makes this code difficult to work with Python 3. We simply swap in Mercurial's JSON encoder to work around it. Differential Revision: https://phab.mercurial-scm.org/D5712

tests: add optional setsockopt() lines for Python 3 Differential Revision: https://phab.mercurial-scm.org/D5711