update: use long version of the flag in the test This helps readability and searchability. Differential Revision: https://phab.mercurial-scm.org/D11848

backout: backed out changeset 5d83a40cd3f8 This is the re-send of the backout in D11757 that I accidentally queued. I'll rewrite the original commit message: ...sigh We do not have Python 3 packaging for thg on Windows where the vast majority of of users use the thg installer for Mercurial. Until this is done (hopefully this cycle), we're keeping Python. (end of the old commit message) On top of this, we have a solid lead to have competent people take care of this packaging issue for us in time for the 6.1 release, which is really the main reason for us to wait. We're trying our best to make this work, so please bear with us. Differential Revision: https://phab.mercurial-scm.org/D11770

status: keep second-ambiguous mtimes during fixup Now that we support the feature, we can keep "second ambiguous" mtime during the fixup phase. These are the mtime that would be ambiguous if we did not had sub-second précions. See the v2 format documentation for details. Differential Revision: https://phab.mercurial-scm.org/D11847

dirstate-item: make sure we load `mtime-second-ambiguous` from disk Now that we support the associated logic, we can safely load it from it. It is no longer necessary to ignore the stored mtime when the flag is encountered. Differential Revision: https://phab.mercurial-scm.org/D11846

dirstate-item: make sure we set the mtime-second-ambiguous on v2 write We want to preserve the second-ambiguity alongside the ambiguous mtime. So we use the decimated flag for that. note: the C code was already doing so. No change was needed to it. Differential Revision: https://phab.mercurial-scm.org/D11845

dirstate-item: ignore mtime to write v1 when `mtime-second-ambiguous` is set We cannot preserve that information in the v1 format so that mtime is ambiguous. Differential Revision: https://phab.mercurial-scm.org/D11844

dirstate-item: implement the comparison logic for mtime-second-ambiguous If the flag is set we now process it properly. We "just" need to actually set it and persist it. Differential Revision: https://phab.mercurial-scm.org/D11843

dirstate-item: add a "second_ambiguous` flag in the mtime tuple This will be used to support the `mtime-second-ambiguous` flag from dirstate v2. See format documentation for details. For now, we only make it possible to store the information, no other logic have been added. Differential Revision: https://phab.mercurial-scm.org/D11842

dirstate: drop comparison primitive on the timestamp class All comparison are now managed without using operator : - the status mtime comparisons is handled by the DirstateItem, - the fixup reliability check, - the update "hack". So we no longer needs the operator and should discourage its usage. Differential Revision: https://phab.mercurial-scm.org/D11841

largefile: use the proper "mtime boundary" logic during fixup This will prevent ambiguous cache entry to be used in racy situation. This fix flakiness in test and some real live misbehavior. Differential Revision: https://phab.mercurial-scm.org/D11800

status: move the boundary comparison logic within the timestamp module Some extensions will need it too. So lets isolate the logic. It also makes things clearer. Differential Revision: https://phab.mercurial-scm.org/D11799

tests: remove potential mtime ambiguity in a dirstate test If the test was fast enough, some mtime where not stored. We now wait long enough to ensure the mtime is no longer ambiguous. Differential Revision: https://phab.mercurial-scm.org/D11798

dirstate: cleanup remaining of "now" during write Since the whole `need_delay` have been removed, we no longer need this. Differential Revision: https://phab.mercurial-scm.org/D11797

dirstate: remove need_delay logic Now that all¹ stored mtime are non ambiguous, we no longer need to apply the `need_delay` step. The need delay logic was not great are mtime gathered during longer operation could be ambiguous but younger than the `dirstate.write` call time. So, we don't need that logic anymore and can drop it This make the code much simpler. The code related to the test extension faking the dirstate write is now obsolete and associated test will be migrated as follow up. They currently do not break. [1] except the ones from `hg update`, but `need_delay` no longer help for them either. Differential Revision: https://phab.mercurial-scm.org/D11796

dirstate: remove `lastnormaltime` mechanism This is now redundant with the new, simpler `mtime_boundary` one. Differential Revision: https://phab.mercurial-scm.org/D11795

status: use filesystem time boundary to invalidate racy mtime We record the filesystem time at the start of the status walk and use that as a boundary to detect files that might be modified during (or right after) the status run without the mtime allowing that edition to be detected. We currently do this at a second precision. In a later patch, we will use nanosecond precision when available. To cope with "broken" time on the file system where file could be in the future, we also keep mtime for file over one day in the future. See inline comment for details. Large file tests get a bit more confused as we reduce the odds for race condition. As a "side effect", the win32text extension is happy again. Differential Revision: https://phab.mercurial-scm.org/D11794

test: use a different timestamp for the updated file In the test we want to trigger a write after the underlying dirstate changed. To do so, we need a write. And as we are about to make dirstate update smarter we need to meddle with the script a bit to make sure there will be a write. Differential Revision: https://phab.mercurial-scm.org/D11793

win32text: drop associated dirstate cache information on revert Otherwise the could get size from one version of the file while the on-disk version is still clean but with another size. This fix the previously introduced error. Differential Revision: https://phab.mercurial-scm.org/D11792

dirstate: stop gathering parentfiledata in update_file Gathering information here assume that they are valid cache information for a clean file. It is true most of the time, but not garanteed. Accurate data can still be explicitly provided. We drop the spontaneous and will let the next `hg status` call record actual information. Differential Revision: https://phab.mercurial-scm.org/D11791

dirstate: stop gathering parentfiledata in update_file_p1 Gathering information here assume that they are valid cache information for a clean file. It is true most of the time, but not garanteed. So we drop this and will let the next `hg status` call record actual information. Differential Revision: https://phab.mercurial-scm.org/D11790

dirstate: make it mandatory to provide parentfiledata in `set_clean` Gathering the mode, size and mtime, independently from determining that the file is clean is a race-machine. So we just make these information required arguments. (note that the data is still gathered in a racy way in practice, but at least the API is no longer encouraging it.) Differential Revision: https://phab.mercurial-scm.org/D11789

dirstate: do no use `set_clean` in revert The current `set_clean` usage is racy (the file might be modified between its restoration and the `set_clean` call). So we simply leave the file as ambiguous and the next status will fix that. We still have to make sure the copy information is dropped, so we teach dirstate how to do that. The win32txt extension is confused after this because current logic is broken in more location. However this series will ultimately fix that so we "ignore" it for now. Fixing it now is complicated without some extra fix landing later. Differential Revision: https://phab.mercurial-scm.org/D11788

status: adapt the "keyword" extensions to gather stats at lookup time See main core code for details. We don't factor the code in a common function yet, because we will have to adapt a bit more things in the keyword case at the end of the series. Differential Revision: https://phab.mercurial-scm.org/D11787

status: adapt largefile to gather stats at lookup time See the core code for details of why we are doing this. We don't factor the code in a common function yet, because we will have to adapt a bit more things in the largefile case at the end of the series. Differential Revision: https://phab.mercurial-scm.org/D11786

status: gather fixup info at comparison time This is still racy, but on a much small windows. In addition, the API now make it possible for it to not be racy. This also unlock other cleanups that we are about to do regarding mtime ambiguity at gathering time. Differential Revision: https://phab.mercurial-scm.org/D11785

update: filter the ambiguous mtime in update directly Right now, this filtering is done by `dirstate.write` using the time of `dirstate.write` method call. However that filtering is done "too late" It works "fine" as most command are "fast enough", and race rare enough. We are about to change the mtime filtering logic in the dirstate to be more accurate and reliable. However `hg update` will still need such filtering (mostly because it is actually quite racy, even with the existing filtering). So we explicitly implement a similar logic here. Before removing the older one later in the series. Differential Revision: https://phab.mercurial-scm.org/D11784

dirstate: move "get fs now" in the timestamp utility module We will need it during update. Differential Revision: https://phab.mercurial-scm.org/D11783

dirstate-item: allow mtime to be None in "parentdata" This will be useful to filter out unreliable mtime. Differential Revision: https://phab.mercurial-scm.org/D11782

dirstate: add a comment about a racy piece of code during updates This is a bit that is not really correct but works "fine" in practice. Let us write the details down so that people stop wondering how that logic might be correct… It is not. Differential Revision: https://phab.mercurial-scm.org/D11781

tests: add missing `head` for when things go wrong See comment above the changed line, you can get a millions of line of output in case of failure. When don't need to have them all. Differential Revision: https://phab.mercurial-scm.org/D11780

tests: ensure a status will have non ambiguous mtime in some race test For the test to work, we need some mtime to be recorded. For them to be recorded, they need to be "clearly in the past", otherwise edit with the same mtime would be possible. Strictly speaking there might be file system with a minimal mtime increment longer than one second. However it is unlikely that we will run the test on them for now. We can be smarter about this in the future if it becomes necessary. Differential Revision: https://phab.mercurial-scm.org/D11779

tests: make sure no ambiguities remains after the commit This will help to stabilize part of the test that are not relevant for what is actually tested. Differential Revision: https://phab.mercurial-scm.org/D11778

dirstate: clarify a `hg update` invocation in a test It is common for readers of that test to confuse the `hg co` call with a `hg commit`, while it actually means `hg checkout`, an alias for the more common `hg update. So let us use the clearer version. Differential Revision: https://phab.mercurial-scm.org/D11777

test: mark rhg output as flaky rhg is not updating the dirstate on status yet, which make this part of the test flaky. This will be fixed soon. Differential Revision: https://phab.mercurial-scm.org/D11833

chistedit: explain which order the commits are presented in It's not obvious which order the commits in chistedit (and text-based histedit), so let's add a note about it. Differential Revision: https://phab.mercurial-scm.org/D11832

errors: use detailed error for invalid commit-extras argument Differential Revision: https://phab.mercurial-scm.org/D11831

errors: use detailed exit code in pathauditor Differential Revision: https://phab.mercurial-scm.org/D11830

errors: use detailed exit code for RepoLookupError Differential Revision: https://phab.mercurial-scm.org/D11829

errors: use detailed exit code for detected case-collision Differential Revision: https://phab.mercurial-scm.org/D11828

errors: use detailed exit code when trying to merge file outside narrowspec Differential Revision: https://phab.mercurial-scm.org/D11827

errors: use detailed exit code for non-integer number of diff context lines Differential Revision: https://phab.mercurial-scm.org/D11826

patch: add hint about mangled whitespace on bad patch One of the most common reasons that a patch doesn't apply is because its whitespace has been mangled (e.g. by their mail client or though copy&paste). Let's provide a hint about that. Differential Revision: https://phab.mercurial-scm.org/D11825

errors: return more detailed errors when failing to parse or apply patch This patch adds subclasses of `PatchError` so we can distinguish between failure to parse a patch from failure to apply it. It updates the callers to raise either `InputError` or `StateError` depending on which type of error occurred. Differential Revision: https://phab.mercurial-scm.org/D11824

extensions: add a default "*" suboptions prefix This is similar to what we do in other section (e.g. `paths`) and allow to change the behavior for all extensions. Sub options on individual extensions overwrite the default settings. Differential Revision: https://phab.mercurial-scm.org/D11823

extension: add a `required` suboption to enforce the use of an extensions If `required` is set, failing to load an extensions will abort. See the test and documentation for details. Differential Revision: https://phab.mercurial-scm.org/D11822

extensions: highlight the name of the faulty extensions in the error message This make it easier to understand the message when the extensions name is common. Differential Revision: https://phab.mercurial-scm.org/D11821

extensions: refactor handling of loading error make it reusable We will need this in the next patch. Differential Revision: https://phab.mercurial-scm.org/D11820

extensions: ignore "sub-options" when looking for extensions config suboptions are separated by ":" (see the path one for example). So we dont want to confuse these config with actual extensions. We don't have extensions sub option yet, but I am about to introduce one for making sure an extensions can load. So lets level the floor first. Differential Revision: https://phab.mercurial-scm.org/D11819

automation: support Python 3.10 on Windows Python 3.10 is out and we should support it. This commit teaches the automation code to install and support building for Python 3.10. Differential Revision: https://phab.mercurial-scm.org/D11776

contrib: update Windows environment to Python 3.9.9 Let's keep the environment modern. Differential Revision: https://phab.mercurial-scm.org/D11775

rhg: implement the debugignorerhg subcommand This can be used to inspect the generated pattern, but also to benchmark the time it takes to parse hgignore. Differential Revision: https://phab.mercurial-scm.org/D11722

rhg: refactor to use IgnoreFnType alias more widely Differential Revision: https://phab.mercurial-scm.org/D11818

rhg: only complain about poorly configured fallback when falling back Differential Revision: https://phab.mercurial-scm.org/D11751

packaging: bump pygit2 to 1.7.1 Differential Revision: https://phab.mercurial-scm.org/D11805

packaging: bump windows_curses to 2.3.0 This is required for Python 3.10 support. Differential Revision: https://phab.mercurial-scm.org/D11804

packaging: regenerate the requirements files with pip-tools 6.4.0 Somewhere along the line, the formatting changed. There's no change in package content here- it's just some minor text changes. Py2 doesn't have any packages we'll be updating, so I'm not bothering there. Differential Revision: https://phab.mercurial-scm.org/D11803

rhg: Add support for `rhg status -n` The `RHG_STATUS=1` bit added here can be removed when `unset RHG_STATUS` near the top of the file is removed. Differential Revision: https://phab.mercurial-scm.org/D11815

filemerge: simplify slightly by using filectx.decodeddata() Differential Revision: https://phab.mercurial-scm.org/D11802

filemerge: rename _formatconflictmarker() since it formats a label `_formatconflictmarker()`'s name made me think that it would create something like `<<<<<<<` and maybe some more stuff after it, but it's actually just the label that comes after on the same line. So let's rename it to `_formatlabel()`. That's better aligned with the variables the result is assigned it. Differential Revision: https://phab.mercurial-scm.org/D11801

tests: fix test-fix on NetBSD Differential Revision: https://phab.mercurial-scm.org/D11816