heptapod-ci: colorize output The run result are nicer to read with color. Differential Revision: https://phab.mercurial-scm.org/D8012

heptapod-ci: add a basic file to be able to run tests with heptapod Having this yaml file somewhere in the main mercurial repository makes it trivial for contributors using heptapod to run CI on their in-progress work. There are alot of different combination (python2/python3 pure/cext/rust/pypy) to be tested and making sure all of them are covered manually is cumbersome. Automatic CI runnig on draft really helps in that matters. We start small bu later changesets will add more step testing more of the variants. The series is targetted on stable to make it available to the widest amount of contribution possible. The definition of the docker files used for this are available here: https://dev.heptapod.net/octobus/ci-dockerfiles Differential Revision: https://phab.mercurial-scm.org/D8011

worker: manually buffer reads from pickle stream My previous fix (D8051, cb52e619c99e, which added Python's built-in buffering to the pickle stream) has the problem that the selector will ignore the buffer. When multiple pickled objects are read from the pipe into the buffer at once, only one object will be loaded. This can repeat until the buffer is full and delays the processing of completed items until the worker exits, at which point the pipe is always considered readable and all remaining items are processed. This changeset reverts D8051, removing the buffer again. Instead, on Python 3 only, we use a wrapper to modify the "read" provided to the Unpickler to behave more like a buffered read. We never read more bytes from the pipe than the Unpickler requests, so the selector behaves as expected. Also add a test case for "pickle data was truncated" issue. https://phab.mercurial-scm.org/D8051#119193 Differential Revision: https://phab.mercurial-scm.org/D8076

py3: __repr__ needs to return str, not bytes Differential Revision: https://phab.mercurial-scm.org/D8089

config: also respect HGRCSKIPREPO in the zeroconf extension Differential Revision: https://phab.mercurial-scm.org/D8075

config: also respect HGRCSKIPREPO in hgwebdir_mod Differential Revision: https://phab.mercurial-scm.org/D8074

config: also respect HGRCSKIPREPO in `dispatch._getlocal` For some reason, we are also reading the local config in that function. Differential Revision: https://phab.mercurial-scm.org/D8073

config: add a function in `rcutil` to abstract HGRCSKIPREPO We wil need to respect this environment variable in more place. Differential Revision: https://phab.mercurial-scm.org/D8072

packaging: make the path to Win32 requirements absolute when building WiX Otherwise this broke automation when not launched from `contrib/packaging`. Differential Revision: https://phab.mercurial-scm.org/D8068

resourceutil: blacken

merge with stable

rebase: abort if the user tries to rebase the working copy I think it's more correct to treat `hg rebase -r 'wdir()' -d foo` as `hg co -m foo`, but I'm instead making it error out. That's partly because it's probably what the user wanted (in the case I heard from a user, they had done `hg rebase -s f` where `f` resolved to `wdir()`) and partly because I don't want to think about more complicated cases where the user specifies the working copy together with other commits. Differential Revision: https://phab.mercurial-scm.org/D8057

tests: add tests for rebasing wdir() revision Differential Revision: https://phab.mercurial-scm.org/D8056

merge: when rename was made on both sides, use ancestor as merge base When both sides of a merge have renamed a file to the same place, we would treat that as a "both created" action in merge.py. That means that we'd use an empty diffbase. It seems better to use the copy source as diffbase. That can be done by simply dropping code that prevented us from doing that. I think I did it that way in 57203e0210f8 (copies: calculate mergecopies() based on pathcopies(), 2019-04-11) only to preserve the existing behavior. I also suspect it was just an accident that it behaved that way before that commit. Note that until fa9ad1da2e77 (merge: start using the per-side copy dicts, 2020-01-23), it was non-deterministic (depending on iteration order of the `allsources` set in `copies._fullcopytracing()`) which source was used in the affected test case in test-rename-merge1.t. We could easily have fixed that by sorting them, but now we can instead detect the case (the TODO added in the previous patch). Differential Revision: https://phab.mercurial-scm.org/D7974

absorb: graduate -i flag from experimental The interactive mode seems to work well. I have previously thought that `-i` should be what `-e` does, but the current behavior matches what other `-i` flags do (select a subset of the hunks), so I think that is what we want. Differential Revision: https://phab.mercurial-scm.org/D8055

rust-cpython: remove PySharedRefCell and its companion structs Also updates py_shared_iterator!() documentation accordingly.

rust-cpython: switch to upstreamed version of PySharedRefCell Our PyLeaked is identical to cpython::UnsafePyLeaked. I've renamed it because it provides mostly unsafe functions.

rust-cpython: rename inner_shared() to inner() The "shared" accessor will be automatically generated, and will have the same name as the data itself.

rust-cpython: use PyList.insert() instead of .insert_item() Silences the deprecated warning. https://github.com/dgrunwald/rust-cpython/commit/e8cbe864841714c5555db8c90e057bd11e360c7f

rust-cpython: bump cpython to 0.4 to switch to upstreamed PySharedRef

rust: update dependencies For no particular reason, but just because I'll bump the rust-cpython version.

Added signature for changeset 7f5410dfc8a6

Added tag 5.3 for changeset 7f5410dfc8a6

rust-dirstatemap: add missing @propertycache While investigating a regression on `hg update` performance introduced by the Rust `dirstatemap`, two missing `@propertycache` were identified when comparing against the Python implementation. This adds back the first one, that has no observable impact on behavior. The second one (`nonnormalset`) is going to be more involved, as the caching has to be done from the Rust side of things. Differential Revision: https://phab.mercurial-scm.org/D8047

worker: Use buffered input from the pickle stream On Python 3, "pickle.load" will raise an exception ("_pickle.UnpicklingError: pickle data was truncated") when it gets a short read, i.e. it receives fewer bytes than it requested. On our build machine, Mercurial seems to frequently hit this problem while updating a mozilla-central clone iff it gets scheduled in batch mode. It is easy to trigger with: #wipe the workdir rm -rf * hg update null chrt -b 0 hg update default I've also written the following program, which demonstrates the core problem: from __future__ import print_function import io import os import pickle import time obj = {"a": 1, "b": 2} obj_data = pickle.dumps(obj) assert len(obj_data) > 10 rfd, wfd = os.pipe() pid = os.fork() if pid == 0: os.close(rfd) for _ in range(4): time.sleep(0.5) print("First write") os.write(wfd, obj_data[:10]) time.sleep(0.5) print("Second write") os.write(wfd, obj_data[10:]) os._exit(0) try: os.close(wfd) rfile = os.fdopen(rfd, "rb", 0) print("Reading") while True: try: obj_copy = pickle.load(rfile) assert obj == obj_copy except EOFError: break print("Success") finally: os.kill(pid, 15) The program reliably fails with Python 3.8 and succeeds with Python 2.7. Providing the unpickler with a buffered reader fixes the issue, so let "os.fdopen" create one. https://bugzilla.mozilla.org/show_bug.cgi?id=1604486 Differential Revision: https://phab.mercurial-scm.org/D8051

packaging: lowercase the `contrib` and `templates` directories with Inno I have no idea why these (and `contrib/vim`) were leading with uppercase with Inno, but not WiX. It probably doesn't matter too much, but might be a problem with `templates` if the user enabled case sensitivity on NTFS. Differential Revision: https://phab.mercurial-scm.org/D8063

packaging: merge the requirements.txt files for WiX and Inno Now that the content is common, there's no need to have separate files. The content still differs from the non-Windows platforms though. Differential Revision: https://phab.mercurial-scm.org/D8066

packaging: bundle dulwich, keyring, and pywin32-ctypes with WiX too TortoiseHg installs these, which is possibly where they originated (though I would have thought it more likely to be in the WiX installer, given its heritage). When I was working on the TortoiseHg app for Mac (which uses the similar `py2app`), it wasn't possible to use the keyring extension (even externally) without bundling this keyring package into the app. Assuming the same principle applies here, these would enable some common extensions. One of the things that the TortoiseHg packager on macOS does now is it adds the user's local `site-packages` directory to `sys.path`. That would allow the user to install these critical modules in cases like this. But that can probably wait for py3 packaging. The only difference in the installed packages that I see now is WiX also bundles distutils for some reason. I suppose that's not harming anything, so I'm not touching it. The only orphans in the install directories when comparing WiX and Inno now is the Copying.txt vs COPYING.rtf, the two uninstaller files for Inno, and a `Mercurial.url` file in Inno. I have no idea what that is, and it has *.ini syntax with a single field pointing to the Mercurial homepage. Differential Revision: https://phab.mercurial-scm.org/D8062

packaging: bundle the default mercurial.ini template with Inno also This is a step towards converging on the same installer content on Windows. Differential Revision: https://phab.mercurial-scm.org/D8061

packaging: set the FileVersion field in the Inno installer executable Previously, Properties > Details > File version showed "0.0.0.0". This appears to be a longstanding issue, and not part of the refactoring this cycle. Differential Revision: https://phab.mercurial-scm.org/D8060