largefiles: rely on the higher level `changing_giles` in `mergerecordupdates` Now that context open on the main dirstate also affect the underlying one, we can skip opening our own in `mergerecordupdates`

dirstate: use wlock and `dirstate.change_files` to scope the change in `mq` This is the way.

subrepo: use `changing_files` context in subrepository code This is better, not ideal, but better.

subrepo: let black expand some call on multiple lines early newer version of black are righfully doing this, so I apply it before more semantic change to reduce noise in the next changeset.

dirstate: use `dirstate.change_files` to scope the change in `import` This is the way.

dirstate: use `dirstate.change_files` to scope the change in `automv` This is... mostly... the way.

dirstate: use `dirstate.change_files` to scope the change in `amend` This is the way.

dirstate: use the `changing_files` context in the `keyword` demo This is the way.

dirstate: wrap repository change in appropriate context in `test-context` We need the `wlock` (to add files), the `lock` (to commit), a `transaction` (to commit) and a `changing_files` context (to add files). Strictly speaking, we could let the commit take the `lock` and create a `transaction`, but it seems more consistent that way.

dirstate: use wlock and changing_files context in `test-revlog-ancestry` This is the way.

dirstate: use `dirstate.change_files` to scope the change in `revert` This is the way.

dirstate: use `dirstate.change_files` to scope the change in `gpg` This is the way.

dirstate: use `dirstate.change_files` to scope the change in `tag` This is the way.

dirstate: use `dirstate.change_files` to scope the change in `rename` This is the way, unless we are not actually touching the working copy. In such cases we don't need to do something.

dirstate: use `dirstate.change_files` to scope the change in `copy` This is the way, unless we are not actually touching the working copy. In such cases we don't need to do something.

dirstate: use `dirstate.change_files` to scope the change in `remove` This is the way.

dirstate: use `dirstate.change_files` to scope the change in `forget` This is the way.

dirstate: use `dirstate.change_files` to scope the change in `addremove` This is the way.

dirstate: use `dirstate.change_files` to scope the change in `add` This is the way.

commit: use `dirstate.change_files` to scope the associated `addremove` This was significantly more complicated than I expected, because multiple extensions get in the way. I introduced a context that lazily open the transaction and associated context to work around these complication. See the inline documentation for details. Introducing the wrapping transaction remove the need for dirstate-guard (one of the ultimate goal of all this), and slightly affect the result of a `hg rollback` after a `hg commit --addremove`. That last part is deemed fine. It aligns the behavior with what happens after a failed `hg commit --addremove` and nobody should be using `hg rollback` anyway. The small output change in the test come from the different transaction timing and fact the transaction now backup the dirstate before the addremove, which might mean "no file to backup" when the repository starts from an empty state.

commit: move the addremove logic around to make the next changeset clearer Lets do the noise now, without changing any thing. So the new changeset can focus on the actual semantic changes.

largefiles: link the core dirstate._changing context to the lfdirstate one This will be much cleaner and safer to make sure the two dirstates are in sync. This way, the large-files dirstate will simply inherit the state of the main dirstate, so if the core code does the right thing, the large-files code should be right too.

dirstate: add a context for tracking files change Let us start to use it. We will enforce it later.

dirstate: invalidate the dirstate change on transaction failure If the change context lives inside a transaction, the change are not flushed to disk on exit as this is delegated to the transaction. As a result we should also delegate the part that do cleanup on failure. The issue was caught by tests with other change, but it seems useful to fix this as soon as possible.

dirstate: factor the "changing" context logic out This makes it reusable for other types of changes.

dirstate: introduce a `is_changing_any` property This will embrace other cases than changing parents.

dirstate: rename `pendingparentchange` to `is_changing_parents` This is clearer and more inline witht he other change we did.

dirstate: rename _parentwriters to _changing_level We will have context for changing more than parents, so lets be prepared.

largefiles: remove the `changing_parents` context in `openlfdirstate` This code might run in any kind of change context, we should rely on the higher level context instead of opening our own.

largefiles: remove the second `changing_parents` in `updatelfiles` Now that the `update_file` call have been migrated, we can drop the semantically-wrong `changing_parents` context.

largefiles: remove the first `changing_parents` in `updatelfiles` Now that the `update_file` call have been migrated, we can drop the semantically-wrong `changing_parents` context.

largefiles: use `hacky_extension_update_file` in `updatelfiles` This is what the function is meant for.

largefiles: use `hacky_extension_update_file` in `synclfdirstate` This is what the function is meant for.

largefiles: use `hacky_extension_update_file` in `openlfdirstate` This is what the function is meant for.

win32text: make the hacky call cover more cases Now that I understand what this is doing, It seems like the case for `p1_tracked` were covered by neither the code nor the test. Now the code cover it at least.

win32text: drop the `changing_parents` context in revert upgrade We are not changing parents here, so let us not pretend we do.

win32text: clean up and clarify the post-revert hack of dirstate The change is unrelated to changing parents and should not be a in a "changing_parents" context. This call is quite hacky, but now it is at least explicitly hacky. We will drop the `changing_parents` context in the next changesets, but we kept this change simple to help readability.

dirstate: introduce a `hacky_extension_update_file` method See inline documentation for details.

mq: properly take the wlock during the full qfold operation Otherwise the operation could be raced… for unknown result.

locking: hold the wlock for the full duration of the "keyword demo" The risk of racing the demo is low, since it seems to create its own repository on the fly. However it is clearer and more consistent.

locking: grab the wlock before touching the dirstate in `perfdirstatewrite` If we touch the dirstate, we should hold the `wlock`.

locking: take the `wlock` for the full `hg addremove` duration Otherwise, there is a race condition window between the time we resolve the file to addremove with the matcher and the time we lock the repo and modify the dirstate. For example, the working copy might have been updated away, or purged, and the matched files would no longer be correct.

locking: take the `wlock` for the full `hg forget` duration Otherwise, there is a race condition window between the time we resolve the file to forget with the matcher and the time we lock the repo and modify the dirstate. For example, the working copy might have been updated away, or purged, and the matched files would no longer be correct.

locking: take the `wlock` for the full `hg remove` duration Otherwise, there is a race condition window between the time we resolve the file to remove with the matcher and the time we lock the repo and modify the dirstate. For example, the working copy might have been updated away, or purged, and the matched files would no longer be correct.

locking: take the `wlock` for the full `hg add` duration Otherwise, there is a race condition window between the time we resolve the file to add with the matcher and the time we lock the repo and modify the dirstate. For example, the working copy might have been updated away, or purged, and the matched files would no longer be correct.

dirstate: drop some very fishy looking piece of code This piece of code is marking the **real** dirstate file as a temporary transaction file. This means it get deleted on transaction rollback. This this quite wrong, especially as the comment points out some `dirstate.pending` motivation and the `.pending` file should already be fully managed by the transaction. The only ready I can think of this behavior not having awful results right now is because other transaction logic restore backed up content above the one that got wrongfully deleted. Let us stop doing this anyway, All tests seems happy.

dirstate: do not write an empty dirstate just for backup This will get in the way when we get more strict about holding the lock when writing the dirstate. Instead, we simply don't copy dirstate files around if there are None at backup time. A couple of tests are impacted they no longer need to backup such "empty" dirstate.

dirstate: pre-indent some of the backup code This will make the next changeset clearer.

debugrebuilddirstate: double check that no transaction is open Since transaction impact dirstate write, we make sure nobody is trying anything strange with this internal command.

dirstate: explicitly write the dirstate after `debugrebuilddirstate` I am working on making the dirstate write patterns more predictable. This patch is part of a small series of similar patches that adds a explicit dirstate write in a handful of location where the dirstate is updated "a bit in a strange way". With this explicit write, we are no longer relying on implicite write of the dirstate on `wlock` release. This make the world a better place.

dirstate: explicitly write the dirstate after `keyword` "overwrite" I am working on making the dirstate write patterns more predictable. This patch is part of a small series of similar patches that adds a explicit dirstate write in a handful of location where the dirstate is updated "a bit in a strange way". With this explicit write, we are no longer relying on implicite write of the dirstate on `wlock` release. This make the world a better place.

dirstate: explicitly write the dirstate after `eol` dirstate manipulation I am working on making the dirstate write patterns more predictable. This patch is part of a small series of similar patches that adds a explicit dirstate write in a handful of location where the dirstate is updated "a bit in a strange way". With this explicit write, we are no longer relying on implicite write of the dirstate on `wlock` release. This make the world a better place.

dirstate: explicitly write the dirstate after mq dirstate rebuild I am working on making the dirstate write patterns more predictable. This patch is part of a small series of similar patches that adds a explicit dirstate write in a handful of location where the dirstate is updated "a bit in a strange way". With this explicit write, we are no longer relying on implicite write of the dirstate on `wlock` release. This make the world a better place.

transaction: quietly rollback if no other changes than temporary files If no actual change have been made, we don't really need to roll them back. We only have to cleanup some temporary files and it seems reasonable to do that quietly. This will help us to use the transaction in wider context¹ without impacting the user experience. [1] as in Python context managers that lives longer.

transaction: run abort callback in all cases Previously, these possibly important callback were "forgotten" when running a quick rollback. This is now fixed, as the tests shown.

transaction: clarify the "quick abort" scenario Right now, the transaction has a code-pass to do a "quick abort" that skip most¹ (too much) of the logic when the right condition are detected² We are about to improve this logic in multiple aspect. We clarify the code first. The conditional return in `_can_quick_abort` looks a bit weird because we are about to make them more complex very soon. [1] actually too much [2] actually not often enough

test: use a more direct form of interruption in fncache "recover" testing The previous test was relying on implementation details and harder to maintain. The new version is closer to the initial intend : "What happens if the process die without cleanup". This change is motivated by further changes around the transaction and dirstate logic that would break the fragile equilibrium that existed before this patch. Making this change early make it easier to review on its own and remove noise in future larger changes.

test: use a more direct approach to test racy mutation The previous test was relying on implementation details and harder to maintain. The new version is closer to the initial intend : "What happens the file get overwritten from under the current process" This change is motivated by further changes around the transaction and dirstate logic that would break the fragile equilibrium that existed before this patch. Making this change early make it easier to review on its own and remove noise in future larger changes.

test: create some history in test-dirstate-backup An empty repository, based on `null` is quite a corner cases. We create a more "natural" setup for this tests to make sure it can keep testing what it intend to test in the future.

test: explicitly "add" file before some commit in test-keyword.t `hg commit -A` will revert the `hg addremove` step if the commit fails. However `hg rollback` currently does not. We are about to improve internal consistency around transaction and dirstate and the behavior of `hg rollback` will align on the other behavior in the process. Before doing so, we make sure the test is using a separate call to `hg add` to avoid the test scenario to be affected by that future change. note: the behavior change for `hg rollback` seems fine as it affect a niche usecase and `hg rollback` usage have been strongly discouraged for a while.

test: explicitly "add" file before some commit in test-filecache.py `hg commit -A` will revert the `hg addremove` step if the commit fails. However `hg rollback` currently does not. We are about to improve internal consistency around transaction and dirstate and the behavior of `hg rollback` will align on the other behavior in the process. Before doing so, we make sure the test is using a separate call to `hg add` to avoid the test scenario to be affected by that future change. note: the behavior change for `hg rollback` seems fine as it affect a niche usecase and `hg rollback` usage have been strongly discouraged for a while.

test: explicitly "add" file before some commit in test-bookmark.t `hg commit -A` will revert the `hg addremove` step if the commit fails. However `hg rollback` currently does not. We are about to improve internal consistency around transaction and dirstate and the behavior of `hg rollback` will align on the other behavior in the process. Before doing so, we make sure the test is using a separate call to `hg add` to avoid the test scenario to be affected by that future change. note: the behavior change for `hg rollback` seems fine as it affect a niche usecase and `hg rollback` usage have been strongly discouraged for a while.

test: explicitly "add" file before some commit in test-rollback.t `hg commit -A` will revert the `hg addremove` step if the commit fails. However `hg rollback` currently does not. We are about to improve internal consistency around transaction and dirstate and the behavior of `hg rollback` will align on the other behavior in the process. Before doing so, we make sure the test is using a separate call to `hg add` to avoid the test scenario to be affected by that future change. note: the behavior change for `hg rollback` seems fine as it affect a niche usecase and `hg rollback` usage have been strongly discouraged for a while.

rhg-files: add support for narrow when specifying a revision This makes it so that `rhg files -r NODE` works properly when using narrow.

rust-narrow: enable narrow support for plain `rhg files` Support for `rhg files -r NODE` in a future changeset.

rhg-files: make signature of `display_files` more flexible This allows the callers to use any error type that converts to `CommandError` instead of a particular concrete type.

rhg: fix user-facing error message so it matches Python implementation

rust-ui: refactor ui code for printing narrow/sparse warnings This will be used elsewhere in the code, starting from the next commit.

rhg-files: reuse centralized dirstate logic The `files` logic predates the centralized dirstate logic. It was duplicated, an didn't receive bugfixes along the way.

rust: move `filter_map_results` to public util This is a useful general-purpose function. It will be used in the next changesets.

rhg: fix typo

run-tests: stop ignoring venv-installed packages This will allow test dependencies to be installed within a venv, instead of tampering with sys/user sites. One thing to note here is that the `VIRTUAL_ENV` path takes precedence over system-site, unlike when run directly with an activated `venv`. For example, `sys.path` as viewed from a feature test in `hghave.py`, when running `hghave vcr` directly with an activated `venv`: sys.path: [ '/home/jenkins/hg-committed/tests', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/jenkins/test_venv/lib/python3.8/site-packages' ] vs `sys.path` from the same feature test, when run by `run-tests.py` with this change: sys.path: [ '/home/jenkins/hg-committed/tests', '/home/jenkins/hg-committed', '/home/jenkins/hg-committed/tests', '/home/jenkins/test_venv/lib/python3.8/site-packages', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/jenkins/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages' ]

fix: add more information to the debug output The previous debug output did not help to understand which file where fixed in which revision. The new output should cover that.

bundlerepo: raise `error.LookupError` instead of `LookupError` I'm pretty sure that this was the intention, given `error.LookupError` is raised a few lines above with the same arguments. Note that PyCharm complains about `self.display_id` (and the use above this too) being `() -> Any` instead of `bytes`, but it may be confused about the `util.propertycache` annotation on it.

unionrepo: resync several methods to actually override superclass methods Not sure how/when these skewed, but these were flagged by PyCharm as having different signatures.

hgweb: unbyteify the 100-continue check The environment seems to be `str` or `tuple[int]`, and the same check in `wsgicgi.py` is already `str` based, so I suspect this was wrong. I think what happened here is that the string wasn't `r''` prefixed in 482d6f6dba91 like `wsgicgi.py`, so it got wrongly byteified in 687b865b95ad when converting to byte literals en masse.

branching: merge with stable

setup: treat error output and non-zero return code differently Choking on Mercurial's stderr mean armless warning can derails the whole install process. We do like to be able to issue warning to people so we stop considering any stderr is a fatal mistake. This allow us to actually boostrap a version of Mercurial without loosing to many sanity point. For example in case where evolve is not loaded, something that happens when the recorded version is malformatted. Creating a wonderful ouroboros.

setup: make the version computation process more resistant The tag fetching might return and empty string¹, from there everything derails quickly. As setup tools becomes more picky about version format we make the whole seems a bit more robust. The resulting version will be obviously weird, but at least it will actually install itself. [1] This is a problem we will address in the next changesets.

bundlerepo: enforce the requirements declared by the underlying repository Previously, `hg log -r 'bundle()' -R bundle.hg` was failing for me when run from source, complaining about an unknown parent, when the system installed `hg` didn't. Some debugging showed the index was 0 length. It turned out that I didn't have the C extensions compiled, which a simple `hg log -r .` was able to indicate. The problem being that the RequirementError got handled by RepoError, which uses an empty directory as a fallback to process the bundle.

scmutil: make checknewlabel() allow "_" in otherwise numeric names (issue6737)

storageutil: match node length with repository

rust-narrow: fix loop that never loops This was caught by `clippy`. I guess the narrow tests leave something to be desired, since this previously only checked the first valid pattern.

dirstate: enforce holding the lock while doing any changes This seems like a sensible sanity check. This already caught the issue in largefile. This will catch more issue when we start using similar context manager for operation other than the parent changes.

largefile: make sure we hold the lock when updating the second dirstate The largefile extension uses a second dirstate file (and object) to track some states. In some situations, it is lazily updated when needed. These operations might not have the lock taken. This means they might conflict and race with other ongoing operations. So we now take the lock to do these operations. This was caught by the next commit.

dirstate: rename `@requires_no_parents_change` too This match the rename of the context manager.

dirstate: rename `@requires_parents_change` to `@requires_changing_parents` This match the rename of the context manager.

dirstate: write dirstate on successful exit of changing_parents context This is the first step toward having more sensible and predicatable write patterns for the dirstate. Having better write/rollback patterns will greatly reduce and clarify the needs to backup the dirstate.

dirstate: rename parentchange to changing_parents Since the new argument breaks the API anyway, we can rename it to a better name. The previous name `parentchange` might be seen as something active, a function that would directly change the parents, however this is just a context manager to frame the operation that will change the parents and adjust the dirstate content accordingly. In addition, the future sister method that will be about changes to tracking and files would have a hard time fitting in the same naming scheme in a clear way. The new naming uses a clear prefix will make it more distinct from other dirstate methods and easier to extend with other similar contexts.

dirstate: pass the repo to the `changeparent` method If we want the context to be responsible for writing (and we want it), we need to have access to a localrepository object. So we now requires a localrepository object as an argument to this context manager.

dirstate: warn about non-explicitly rolledback parent-change Now that the invalidation is dealt with by the context manager itself, we should no longer need this in the unlocking code. We start with issuing a warning if this is the case before actually dropping this code later.

dirstate: invalidate changes when parent-change fails When an error occurs during changing parents, we should invalidate all dirstate modifications and reload the dirstate. This is currently done by a `unlock` callback on the `wlock`. To fix this anomaly, we start dealing with the error directly in the context manager and its potential nesting. The "hard" part is to make sure that, when the parent-change context are nested, we and higher level nesting do not continue to use the invalidated dirstate. We introduce dedicated code to enforce that.

setup: Ensure target directory exists with building rust extension When the rust extension is the first to be build, the target directory may not exist so the copy fails.

filemerge: add union-other-first as internal merge tool See inline documentation for details.

rust: use `peek_mut` from the standard lib now that it's stable Just a little cleanup of a TODO found along the way.

clippy: actually fail if there are any warnings So far, only errors would trip the CI, this makes it so all warnings are elevated to errors, making it a CI fail if any warnings are present. This was the intended behavior, I just missed this when adding clippy.

rust-clippy: fix warning about nested ifs

hghave: refactor checks for pygments versions using checkvers()

hghave: make different has_pyoxidizer functions have different names There are 3 of them, and them all having the same name triggered code checkers.

tests: pygments 2.14+ highlight whitespace in python code

tests: make sure pygments can detect python script without extension This .t file was failing for me when running run-tests.py with python3.11. Then I tried to run it with python3.10 and it failed anyway, even though it's the default python3 interpreter. But with `python3 ./run-tests.py` it worked fine. And this is what I found while looking at the way pygments lexer checks if a file without extension is likely to be a python script: shebang_matches(text, r'pythonw?(3(\.\d)?)?') Take guess why it doesn't work for python >= 3.10. To work around this issue, we can simply hardcode an "easier" shebang for pygments. This path to python interpreter obviously doesn't need to be accurate, since we're not running this script.

convert: brz 3.3.0 moved NoSuchFile exception to breezy.transport

revset: the `random` sort should not depend on sys.maxsize (issue6770)

contrib: adjust heptapod CI flavor sizes Our CI currently runs on a number of different runners, some of which are run by Clever Cloud. By setting this environement variable, we're asking for a smaller size runner for our smaller jobs, which puts less load on the Clever Cloud infrastructure, while not affecting our overall pipeline times.

debugshell: allow commands to be specified as a CLI argument Add a `--command` option to `hg debugshell` that allows the user to pass in Python code to evaluate directly from the command line. This was inspired by the `--command` option present in Facebook's Sapling fork of Mercurial, which in turn was inspired by the `-c` option of the Python interpreter itself. It is particularly useful for writing tests, especially for getting visibility into things that otherwise aren't exposed via debug commands.

resourceutil: start using importlib.resources.files() when possible This avoids DeprecationWarnings related to our use of resources.open_binary() on Python 3.11.

heptapod-ci: add `clippy` to the CI This linter mostly makes our code more idiomatic, less surprising, has good suggestions and catches bugs. It's widely used in the Rust community and now part of the default toolchain when using `rustup`.

makefile: add `cargo clippy` to tests if cargo is available This linter mostly makes our code more idiomatic, less surprising, has good suggestions and catches bugs. It's widely used in the Rust community and now part of the default toolchain when using `rustup`.

rust-tests: run `cargo` tests inside the `rust` folder There is no reason to only run inside the `rust/hg-cpython` folder, even if the `cargo test --all` behavior ends up being the same.

tests: fix a typo in test-demandimport.t I guess we figured out why it wasn't lazy.

tests: os module is frozen in Python 3.11 (issue6786)

rust-clippy: merge "revlog" module definition and struct implementation Module inception is confusing when looking for code and almost never the right choice.

rust-clippy: merge "config" module definition and struct implementation Module inception is confusing when looking for code and almost never the right choice.

rust-clippy: upgrade `bytes-cast` dependency I released a new version of `bytes-cast` to get rid of the clippy warning, and bump to edition 2021, so let's use it.

rust-clippy: remove redundant suffix from enum Same as last time, this makes the code clearer in this instance.

rust-clippy: fix remaining warnings in `hg-cpython`

rust-clippy: disable some lints crate-wide for `hg-cpython` `rust-cpython` creates some pretty funky code that also needs to be compatible with pretty old Rust. This makes clippy quite useless in `hg-cpython` unless you disable the lints that are always triggered by `py_class!` and related. Maybe `clippy` will allow one day to exclude a dependency from its linting, but this seems quite unlikely, so this is the best we've got at the moment.

rust-clippy: fix remaining warnings in `rhg` All of these are simple changes that for the most part are clear improvements and the rest are at most equivalent.

rust-clippy: fix most warnings in `hg-core` All of these are simple changes that for the most part are clear improvements and the rest are at most equivalent. The remaining warnings have to be fixed either with a bigger refactor like for the nested "revlog" module, or in the dependency `bytes-cast`, which we own. This will be done sometime in the future.

rust-utils: strip redundant prefix from enum This was highlighted by `clippy`, I think this makes the code cleaner.