dirstate: Use the Rust implementation of DirstateItem when Rust is enabled … instead of the C implementation, with C/Rust conversions at the FFI boundary Differential Revision: https://phab.mercurial-scm.org/D11486

rust: Add Python bindings for DirstateEntry as rustext.dirstate.DirstateItem Differential Revision: https://phab.mercurial-scm.org/D11485

rust: Align DirstateEntry internals with Python/C DirstateItem This propagate to this Rust struct the similar change that was made recently to the Python classe and C struct. Namely, instead of storing a four-valued `state` field we now store seven (bit-packed) booleans that give lower-level information. Additionally, the marker values -1 and -2 for mtime and size should not be used internally anymore. They are replaced by some combinations of booleans For now, all uses of of `DirstateEntry` still use the compatibility APIs with `state` and marker values. Later the Rust API for DirstateMap will be increasingly updated to the new style. Also change the expected result of the test_non_normal_other_parent_entries unit test. Only a `DirstateEntry` with `size == -2 && mtime != -1` is affected, but this case never occurs outside of unit tests. `size == -2` was the marker value for "from other parent" entries, where no meaningful mtime is stored. Differential Revision: https://phab.mercurial-scm.org/D11484

branching: merge stable into default

manifestlog: also monitor `00manifest.n` when applicable This let the locarepo's file cache detect outdated nodemap docket and reload the manifestlog after `localrepo.invalidate` when applicable. The same problem than issue6554 could affect the Manifest too. Differential Revision: https://phab.mercurial-scm.org/D11483

changelog: also monitor `00changelog.n` when applicable (issue6554) This let the locarepo's file cache detect outdated nodemap docket and reload the changelog after `localrepo.invalidate` when applicable. Differential Revision: https://phab.mercurial-scm.org/D11482

persistent-nodemap: introduce a test to highlight possible race Weakness in the current file caching of the changelog means that a writer can end up using an outdated docket. This might result in "committed" persistent-nodemap data from a previous writer to be overwritten by a later writer. This break the strong "append only" assumption of the persistent nodemap and can result in confused reader. The race windows are quite narrow. See the test documentation for details. The issues is fixed in the next changeset. Differential Revision: https://phab.mercurial-scm.org/D11481

test: enable share-safe in test-persistent-nodemap We will use some share in this test and I would rather have them safe. Differential Revision: https://phab.mercurial-scm.org/D11480

filecache: abstract the fetching of the list of tracked file We will need it for a coming fix that will requires to check a variable list of file for the changelog. Differential Revision: https://phab.mercurial-scm.org/D11479

persistent-nodemap: use quiet upgrade in tests This make them less verbose while keeping the important information in the checked output. Differential Revision: https://phab.mercurial-scm.org/D11478

persistent-nodemap: fix a typo in a test comment oops. Differential Revision: https://phab.mercurial-scm.org/D11477

bookmarks: add an option to make pull mirror remote bookmarks For backups for instance. Merging bookmarks is not a useful behavior in that case. Differential Revision: https://phab.mercurial-scm.org/D11490

errors: use InputError for bad path arguments to `hg annotate` Differential Revision: https://phab.mercurial-scm.org/D11498

errors: use InputError for bad --similarity value Differential Revision: https://phab.mercurial-scm.org/D11497

errors: use InputError for some invalid revsets and such Differential Revision: https://phab.mercurial-scm.org/D11496

merge: with stable

dirstate: fix leak of entry object in dirstate_item_from_v1_data()

dirstate: fix parse_dirstate() to error out if NULL entry created Since 83f0e93ec34b "dirstate-item: move the C implementation to the same logic", dirstate_item_from_v1_data() gets more likely to return NULL, and the fuzzer crashes because of that.

rust: Remove EntryState::Unknown This enum variant represented the `state == '?'` case, which was used to represent the absence of a dirstate entry/item (and therefore of that entry’s state). Now that previous refactors have removed this use in the Python/Rust FFI APIs, the remaining uses can be removed by replacing `EntryState` by `Option<EntryState>` where appropriate, using `None` to represent the absence of an entry. Differential Revision: https://phab.mercurial-scm.org/D11465

dirstate: Remove the `state == ' '` special case Previously this was used to create a `DirstateItem` representing dirstate tree nodes that semantically don’t have an associated `DirtateItem`. This isn’t used anymore now that `dirstatemap.debug_iter` yields plain tuples. Differential Revision: https://phab.mercurial-scm.org/D11464

debugsate: Change debug_iter() to yield tuples instead of DirstateItem This removes the need for `DirstateItem` to support the `state == ' '` special case which represents dirstate tree nodes without an item. Differential Revision: https://phab.mercurial-scm.org/D11463

debugstate: Always call dirstatemap.debug_iter() … passing it a new `all` argument for the `--all` CLI option, instead of conditionally calling `.debug_iter()` or `.items()` This prepares for the next commit. Differential Revision: https://phab.mercurial-scm.org/D11462

rust: Make the fields of DirstateEntry private This is a first step toward making its internal structure equivalent to Python’s DirstateItem. Differential Revision: https://phab.mercurial-scm.org/D11461

rust: Remove support for passing a dict to the Rust pathutil.dirs() That is only used by the Python dirstatemap, which not used when Rust is enabled. This allows removing the private `extract_dirstate` function which creates `DirstateEntry` values. This in turn will make easier upcoming changes to `DirstateEntry`. Differential Revision: https://phab.mercurial-scm.org/D11460

rust: Remove the `rustext.parsers` module It only exported Rust implementations of the parse_dirstate and pack_dirtate functions, which are only used (anymore) when Rust is not enabled. fakedirstatewritetime.py was detecting the presence of `rustext.parsers` but what it really wants to know is whether the Rust implementation of `dirstatemap` is used. This changes it to detect `rustext.dirstate` instead. Differential Revision: https://phab.mercurial-scm.org/D11459

dirstate: remove now-unused .directories() method It calls _rustmap.directories() which was already removed Differential Revision: https://phab.mercurial-scm.org/D11458

rust: Move DirstateEntry to its own module … and RawEntry to the dirstate::parsers module, the only one that uses it. Differential Revision: https://phab.mercurial-scm.org/D11457

patchbomb: fix traceback on outgoing when not specifying revisions A user reported that `hg email -n -o` raised with the following traceback: ``` ** unknown exception encountered, please report by visiting ** https://mercurial-scm.org/wiki/BugTracker ** Python 3.8.12 (default, Sep 13 2021, 22:12:54) [GCC 7.5.0] ** Mercurial Distributed SCM (version 5.9.1) ** Extensions loaded: patchbomb Traceback (most recent call last): File "/usr/pkg/bin/hg", line 61, in <module> dispatch.run() File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 144, in run status = dispatch(req) File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 250, in dispatch status = _rundispatch(req) File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 294, in _rundispatch ret = _runcatch(req) or 0 File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 470, in _runcatch return _callcatch(ui, _runcatchfunc) File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 480, in _callcatch return scmutil.callcatch(ui, func) File "/usr/pkg/lib/python3.8/site-packages/mercurial/scmutil.py", line 153, in callcatch return func() File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 460, in _runcatchfunc return _dispatch(req) File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 1273, in _dispatch return runcommand( File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 918, in runcommand ret = _runcommand(ui, options, cmd, d) File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 1285, in _runcommand return cmdfunc() File "/usr/pkg/lib/python3.8/site-packages/mercurial/dispatch.py", line 1271, in <lambda> d = lambda: util.checksignature(func)(ui, *args, **strcmdopt) File "/usr/pkg/lib/python3.8/site-packages/mercurial/util.py", line 1886, in check return func(*args, **kwargs) File "/usr/pkg/lib/python3.8/site-packages/hgext/patchbomb.py", line 817, in email revs = _getoutgoing(repo, dest, revs) File "/usr/pkg/lib/python3.8/site-packages/hgext/patchbomb.py", line 537, in _getoutgoing safe_paths = [urlutil.hidepassword(p.rawloc) for p in paths] File "/usr/pkg/lib/python3.8/site-packages/hgext/patchbomb.py", line 537, in <listcomp> safe_paths = [urlutil.hidepassword(p.rawloc) for p in paths] File "/usr/pkg/lib/python3.8/site-packages/mercurial/utils/urlutil.py", line 501, in get_push_paths msg %= dest TypeError: %b requires a bytes-like object, or an object that implements __bytes__, not 'NoneType' ``` This is due to patchbomb passing `[None]` in the `dests` parameter of the multi-paths outgoing function, we fix this in this change and add a non-regression test. Differential Revision: https://phab.mercurial-scm.org/D11456

extensions: prevent a crash on py3 with a `minimumhgversion` str value The expectation is that this field is bytes, but unported extensions are a thing and it shouldn't explode on a bad value. We already do this transformation in the version reporting mechanism. Differential Revision: https://phab.mercurial-scm.org/D11476

extensions: prevent a crash on py3 when testing a bad extension minimum A `None` placeholder is populated for each missing component by `util.versiontuple()`, which could safely be used with `>` on py2, but not py3. I guess there's another hole here where if the string is entirely bogus (i.e no numbers), it will be treated as 0.0, and always load. But that's always been the case. Differential Revision: https://phab.mercurial-scm.org/D11475

typing: drop annotations on `mercurial/logcmdutil.walkopts` attributes Pytype 2021-09-09 must have learned some things about `@attr`, because while these were presumably required to help it before (c1d0f83d62c4::9a26fea2b518 and c11099cc1de4::224c786f4fce), these instances now get flagged as an assignment mismatch: File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 717, in walkopts: Type annotation for pats does not match type of assignment [annotation-type-mismatch] Annotation: List[bytes] Assignment: mercurial.thirdparty.attr._make._CountingAttr File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 718, in walkopts: Type annotation for opts does not match type of assignment [annotation-type-mismatch] Annotation: Dict[bytes, Any] Assignment: mercurial.thirdparty.attr._make._CountingAttr File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 722, in walkopts: Type annotation for revspec does not match type of assignment [annotation-type-mismatch] Annotation: List[bytes] Assignment: mercurial.thirdparty.attr._make._CountingAttr File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 725, in walkopts: Type annotation for bookmarks does not match type of assignment [annotation-type-mismatch] Annotation: List[bytes] Assignment: mercurial.thirdparty.attr._make._CountingAttr File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 726, in walkopts: Type annotation for branches does not match type of assignment [annotation-type-mismatch] Annotation: List[bytes] Assignment: mercurial.thirdparty.attr._make._CountingAttr File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 727, in walkopts: Type annotation for date does not match type of assignment [annotation-type-mismatch] Annotation: Optional[bytes] Assignment: mercurial.thirdparty.attr._make._CountingAttr File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 728, in walkopts: Type annotation for keywords does not match type of assignment [annotation-type-mismatch] Annotation: List[bytes] Assignment: mercurial.thirdparty.attr._make._CountingAttr File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 729, in walkopts: Type annotation for no_merges does not match type of assignment [annotation-type-mismatch] Annotation: bool Assignment: mercurial.thirdparty.attr._make._CountingAttr File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 730, in walkopts: Type annotation for only_merges does not match type of assignment [annotation-type-mismatch] Annotation: bool Assignment: mercurial.thirdparty.attr._make._CountingAttr File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 731, in walkopts: Type annotation for prune_ancestors does not match type of assignment [annotation-type-mismatch] Annotation: List[bytes] Assignment: mercurial.thirdparty.attr._make._CountingAttr File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 732, in walkopts: Type annotation for users does not match type of assignment [annotation-type-mismatch] Annotation: List[bytes] Assignment: mercurial.thirdparty.attr._make._CountingAttr File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 735, in walkopts: Type annotation for include_pats does not match type of assignment [annotation-type-mismatch] Annotation: List[bytes] Assignment: mercurial.thirdparty.attr._make._CountingAttr File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 736, in walkopts: Type annotation for exclude_pats does not match type of assignment [annotation-type-mismatch] Annotation: List[bytes] Assignment: mercurial.thirdparty.attr._make._CountingAttr File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 739, in walkopts: Type annotation for follow does not match type of assignment [annotation-type-mismatch] Annotation: int Assignment: mercurial.thirdparty.attr._make._CountingAttr File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 743, in walkopts: Type annotation for force_changelog_traversal does not match type of assignment [annotation-type-mismatch] Annotation: bool Assignment: mercurial.thirdparty.attr._make._CountingAttr File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 747, in walkopts: Type annotation for filter_revisions_by_pats does not match type of assignment [annotation-type-mismatch] Annotation: bool Assignment: mercurial.thirdparty.attr._make._CountingAttr File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 750, in walkopts: Type annotation for sort_revisions does not match type of assignment [annotation-type-mismatch] Annotation: Optional[bytes] Assignment: mercurial.thirdparty.attr._make._CountingAttr File "/mnt/c/Users/Matt/hg/mercurial/logcmdutil.py", line 753, in walkopts: Type annotation for limit does not match type of assignment [annotation-type-mismatch] Annotation: Optional[int] Assignment: mercurial.thirdparty.attr._make._CountingAttr Differential Revision: https://phab.mercurial-scm.org/D11474

util: eliminate the possibility of returning None from `versiontuple()` This fixes the following error flagged by pytype: File "/mnt/c/Users/Matt/hg/mercurial/extensions.py", line 228, in load: unsupported operand types for > [unsupported-operands] Primitive types 'Optional[tuple]' and 'curver: Optional[tuple]' are not comparable. Differential Revision: https://phab.mercurial-scm.org/D11473

archival: force a `CompressionError` to bytes before passing to `error.Abort` I'm not sure what changed before pytype 09-09-2021 (from 04-15-2021), but this started getting flagged. This fixes: File "/mnt/c/Users/Matt/hg/mercurial/archival.py", line 199, in taropen: Function bytestr.__init__ was called with the wrong arguments [wrong-arg-types] Expected: (self, ints: Iterable[int]) Actually passed: (self, ints: tarfile.CompressionError) Attributes of protocol Iterable[int] are not implemented on tarfile.CompressionError: __iter__ Differential Revision: https://phab.mercurial-scm.org/D11472

parser: force a `ValueError` to bytes before passing to `error.ParseError` I'm not sure what changed before pytype 09-09-2021 (from 04-15-2021), but this started getting flagged. I think there's a pytype bug here, because I don't see how `.lower()` can be getting called on a `ValueError` after it is forced to a byte string. That's suppressed for now to make progress. This fixes: File "/mnt/c/Users/Matt/hg/mercurial/parser.py", line 219, in unescapestr: Function bytestr.__init__ was called with the wrong arguments [wrong-arg-types] Expected: (self, ints: Iterable[int]) Actually passed: (self, ints: ValueError) Attributes of protocol Iterable[int] are not implemented on ValueError: __iter__ File "/mnt/c/Users/Matt/hg/mercurial/parser.py", line 219, in unescapestr: No attribute 'lower' on ValueError [attribute-error] In Union[ValueError, mercurial.pycompat.bytestr] Differential Revision: https://phab.mercurial-scm.org/D11471

util: avoid a name-error warning in the `mmapread` exception handler Newly caught by pytype 2021-09-09. File "/mnt/c/Users/Matt/hg/mercurial/util.py", line 458, in mmapread: Name 'fd' is not defined [name-error] Differential Revision: https://phab.mercurial-scm.org/D11470

typing: suppress an name-error warning in `mercurial/windows.py` Newly caught by pytype 2021-09-09. Differential Revision: https://phab.mercurial-scm.org/D11469

typing: suppress an import-error warning in `mercurial/utils/resourceutil.py` Newly caught by pytype 2021-09-09. Differential Revision: https://phab.mercurial-scm.org/D11468

encoding: force a few Errors to bytes before passing to `error.Abort` I'm not sure what changed before pytype 09-09-2021 (from 04-15-2021), but these started getting flagged. PyCharm also flagged these. This fixes the following: File "/mnt/c/Users/Matt/hg/mercurial/encoding.py", line 243, in fromlocal: Function Abort.__init__ was called with the wrong arguments [wrong-arg-types] Expected: (self, message: Union[bytearray, bytes, memoryview], ...) Actually passed: (self, message: LookupError, ...) File "/mnt/c/Users/Matt/hg/mercurial/encoding.py", line 309, in lower: Function Abort.__init__ was called with the wrong arguments [wrong-arg-types] Expected: (self, message: Union[bytearray, bytes, memoryview], ...) Actually passed: (self, message: LookupError, ...) File "/mnt/c/Users/Matt/hg/mercurial/encoding.py", line 336, in upperfallback: Function Abort.__init__ was called with the wrong arguments [wrong-arg-types] Expected: (self, message: Union[bytearray, bytes, memoryview], ...) Actually passed: (self, message: LookupError, ...) Called from (traceback): line 391, in current file line 348, in get line 318, in upper Differential Revision: https://phab.mercurial-scm.org/D11467

typing: add a fake `__init__()` to bytestr to distract pytype I'm not sure what changed before pytype 09-09-2021 (from 04-15-2021), but these started getting flagged. This wrapping an exception in a `bytestr` pattern has been flagged before, and I've fixed it then with `stringutil.forcebytestr()`. But that doesn't work here, because it would create a circular import. I suspect the issue is `bytes.__new__()` wants `Iterable[int]`, so it just assumes the subclass will also take that. The referenced pytype bug isn't an exact match, but seems related and the suggested workaround helps. The specific warnings fixed are: File "/mnt/c/Users/Matt/hg/mercurial/encoding.py", line 212, in tolocal: Function bytestr.__init__ was called with the wrong arguments [wrong-arg-types] Expected: (self, ints: Iterable[int]) Actually passed: (self, ints: LookupError) Attributes of protocol Iterable[int] are not implemented on LookupError: __iter__ Called from (traceback): line 353, in current file File "/mnt/c/Users/Matt/hg/mercurial/encoding.py", line 240, in fromlocal: Function bytestr.__init__ was called with the wrong arguments [wrong-arg-types] Expected: (self, ints: Iterable[int]) Actually passed: (self, ints: UnicodeDecodeError) Attributes of protocol Iterable[int] are not implemented on UnicodeDecodeError: __iter__ Differential Revision: https://phab.mercurial-scm.org/D11466

merge: with stable

pep-517: remove the `build-backend` key to allow for users to build extensions PEP 517 changed how projects should define their build dependencies. The presence of a `pyproject.toml` file changes the behavior of `pip` to conform to PEP 517. Since we haven't updated the `setup.py` file yet (I'm not even sure we're not an edge case which would make it harder/impossible, I have not tried yet), this is a workaround to issue6589: it allows users to pass `--no-use-pep517` to pip to revert to the legacy installer. The build backend is simply assumed to be the `build_meta:__legacy__` one. Differential Revision: https://phab.mercurial-scm.org/D11393

dirstate: drop the `dirstatemap.dropfile` method All use have been migrated. Differential Revision: https://phab.mercurial-scm.org/D11443

dirstate: use `reset_state` instead of `dropfile` in test-rebuildstate.t As `dirstatemap.dropfile` is on its way out. Differential Revision: https://phab.mercurial-scm.org/D11442

dirstate: use `reset_state` instead of `dropfile` in largefile As `dirstatemap.dropfile` is on its way out. Differential Revision: https://phab.mercurial-scm.org/D11441

dirstate: use `reset_state` in `rebuild` instead of `dropfile` As `dirstatemap.dropfile` is on its way out. Differential Revision: https://phab.mercurial-scm.org/D11440

dirstate: use `reset_state` to drop file in `update_file_p1` The `dropfile` function is on its way out. Differential Revision: https://phab.mercurial-scm.org/D11439

dirstate: support file tracked nowhere in `reset_state` This let the dirstatemap decide when to drop files. Differential Revision: https://phab.mercurial-scm.org/D11438

dirstate: drop dirstatemap.addfile All users have been migrated. Differential Revision: https://phab.mercurial-scm.org/D11437

dirstate: update the documentation of the dirstatemap API The API changed so should the documentation. Differential Revision: https://phab.mercurial-scm.org/D11436

dirstate: drop the `_addpath` method All user have been migrated. Differential Revision: https://phab.mercurial-scm.org/D11435

dirstate: remove the `normallookup` function All use have been migrated. Differential Revision: https://phab.mercurial-scm.org/D11434

dirstate: stop using `_normallookup` in the adddrop extension That extension is just trying to produce a "corrupted" dirstate. Differential Revision: https://phab.mercurial-scm.org/D11433

dirstate: replace the use of `_normallookup` in `rebuild` Normal lookup is a complicated function that we want to get rid of. Differential Revision: https://phab.mercurial-scm.org/D11432

dirstate: replace the use of _normallookup in `setparents` normal lookup is a complicated function that we want to get rid of. Differential Revision: https://phab.mercurial-scm.org/D11431

dirstate: introduce a set_tracked method on "map" and "item" This move more implementation details withing the DirstateItem itself, which is what we have been doing for a while. Differential Revision: https://phab.mercurial-scm.org/D11430

dirstate: fix restoration of "merged" state after a remove Before this change, "merged" file that get removed and re-added later were recorded as "from_p2" instead. This came from 8fe74328f700, a 2014 changeset that start explicitly doing so for reason I have not been able to fully grasp. The graft test mentioned in the description are still happy after this changeset. So this changeset restore what seems to be the intended behavior. Restoring information as it was before the removal. Differential Revision: https://phab.mercurial-scm.org/D11429

dirstate: inline the last two `_drop` usage The function is small and having the associated code directly inline help use to cleanup the dirstate API. Differential Revision: https://phab.mercurial-scm.org/D11428

dirstate: make a conditionnal easier to read in `setparents` By grabing `nullid` in the local space, we get the conditionnal to fit in one line. Differential Revision: https://phab.mercurial-scm.org/D11427

dirstate: removed the now unused `_add` method Differential Revision: https://phab.mercurial-scm.org/D11426

dirstate: replace `_add` call in `setparent` Same logic as the previous changeset, it get us closer to removed older API on the dirstate. Differential Revision: https://phab.mercurial-scm.org/D11425

dirstate: directly call the right function in `set_tracked` This get use closer to removing some older API on the dirstate. Differential Revision: https://phab.mercurial-scm.org/D11424

dirstate: same logic as what we did for `_drop` This is part of the dirstatemap so let the dirstatemap deal with it. Differential Revision: https://phab.mercurial-scm.org/D11423

dirstate: use `tracked` property in `_addpath` It is semantically better. Differential Revision: https://phab.mercurial-scm.org/D11422

rust: Update the memmap2 crate to version 0.4.0 This version: * Optionally implements the `StableDeref` trait, removing the need for a wrapper type doing that * Adds support for mapping empty files, which can routinely happen for dirstate-v2 data files. This was a cause of some failing tests when running with `run-tests.py --rhg --extra-config-opt rhg.status=1` Differential Revision: https://phab.mercurial-scm.org/D11446

tests: make removeemptydirs more portable The behavior of pwd change from one system (i.e. default shell/sh) from one system to another so we use fuzzy matching for the output we do not care and we use narrower command call to highlight what we are trying to test here. Differential Revision: https://phab.mercurial-scm.org/D11449

check-code: make it possible to ignore the PWD check in some situation The check-code script does not motivate the banning of $PWD so I am not sure what it is about. And I will needs to use the env variable in the next tests. So I am adding a way to make an exception. Differential Revision: https://phab.mercurial-scm.org/D11450

relnotes: update next Differential Revision: https://phab.mercurial-scm.org/D11448

relnotes: add release notes for 5.9 Differential Revision: https://phab.mercurial-scm.org/D11447

dirstate: fix compilation warnings in `dirstate_item_set_possibly_dirty()` Since https://phab.mercurial-scm.org/D11387 (i.e. the same patch as mentioned in my previous patch), Clang has also started warning about `dirstate_item_set_possibly_dirty()` missing an explicit return, and about its use of the result of an assignment as a condition without using parentheses. This patch fixes that. Differential Revision: https://phab.mercurial-scm.org/D11445

dirstate: make dirstate flags char be unsigned Since https://phab.mercurial-scm.org/D11387, `CC='clang -Werror' make local` has started failing like this: ``` mercurial/cext/util.h:41:50: error: implicit conversion from 'int' to 'char' changes value from 128 to -128 [-Werror,-Wconstant-conversion] static const char dirstate_flag_rust_special = 1 << 7; ~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~^~~~ ``` This patch fixes that by making the flags be an unsigned char. That also matches the `bool` typedef we have in `util.h`, which seems good since many of the `dirstate_item_c_*()` functions return a `bool`. Differential Revision: https://phab.mercurial-scm.org/D11444

dirstate: introduce a `set_clean` method on dirstate's map and items This method is the "reverse" of "set possibly dirty", and can be used to more accurately other call that the dirstate was making. It is currently heavily influenced by its origin. Differential Revision: https://phab.mercurial-scm.org/D11421

dirstate: extract the logic to check file/dirname collision when adding a file Differential Revision: https://phab.mercurial-scm.org/D11420

dirstate: make dirstatemap.set_untracked deal with added file This merge dropfile in set_untracked. Differential Revision: https://phab.mercurial-scm.org/D11419

dirstate: remove some usage of `_drop` This is a step toward being able to remove the `_drop` method on `dirstate`. Differential Revision: https://phab.mercurial-scm.org/D11418

dirstate: move the copymap drop inside dropfile Since the copymap is part of the dirstatemap it make more sense for the dirstatemap to manage it directly. This is part of a generic effort to move unified logic at lower level and to clean up higher level API. Differential Revision: https://phab.mercurial-scm.org/D11417

rust: Rename get_node methods to data_for_node, get_rev to data_for_rev These are respective methods of Changelog, Manifestlog, and Filelog; three Rust structs that that wrap a Revlog struct. This rename clarifies that node IDs or revision numbers are parameters, not return values. Also reword doc-comments in Manifestlog and Filelog to separate node IDs and revision numbers that are local to a given (non-changelog) revlog from those of a changeset. Differential Revision: https://phab.mercurial-scm.org/D11416

rust: Rename the `Revlog::get_node_rev` method to `rev_from_node` This better describes the input and outputs of this method. Also rewrite the doc-comment, which seemed to have been left from copy-paste of another method. Differential Revision: https://phab.mercurial-scm.org/D11415

rust: Make private the `index` field of the `Revlog` struct To replace the previous use of this field from another module, add a `node_from_rev` method. This is the same method that already existed on `Changelog`. Differential Revision: https://phab.mercurial-scm.org/D11414

dirstate-v2: Remove the `.d` suffix in data file names It could cause confusion since `.d` is already used for revlogs. This suffix is not necessary since there is already a `dirstate.` prefix. Differential Revision: https://phab.mercurial-scm.org/D11413

rhg: Don’t compare ambiguous files one byte at a time Even though the use of `BufReader` reduces the number of syscalls to read the file from disk, `.bytes()` yields a separate `Result` for every byte. Creating those results and dispatching on them is most likely costly. Instead, this commit opts for simplicity by reading the entire file into memory and comparing a single pair of byte strings. Note that memory already needs to contain the entire previous contents of the file, as read from the filelog. So with an extremely large file this doubles memory use but does not make it grow by orders of magnitude. At first I wrote code that still avoids reading the entire file into memory and compares one buffer at a time with `BufReader`. Find this code below for posterity. However its correctness is subtle. I ended up preferring the simplicity of the obviously-correct single comparison. ```rust let mut reader = BufReader::new(fobj); let mut expected = &contents_in_p1[..]; loop { let buf = reader.fill_buf().when_reading_file(&fs_path)?; if buf.is_empty() { // Found EOF return Ok(expected.is_empty()); } else if let Some(rest) = expected.drop_prefix(buf) { // What we read so far matches the expected content, continue reading let buf_len = buf.len(); reader.consume(buf_len); expected = rest } else { // Found different content return Ok(false); } } ``` Differential Revision: https://phab.mercurial-scm.org/D11412

rhg: Reuse manifest when checking status of multiple ambiguous files When `rhg status` cannot determine whether a file is clean based on mtime and size alone, it needs to compare its contents with those found in the parent commit. Previously, rhg would find the (same) manifest of that commit again for every such file. This is lifted out of the loop and reused. Differential Revision: https://phab.mercurial-scm.org/D11411

rust: Return HgError instead of RevlogError in revlog constructors This leaves fewer cases for callers to handle, as RevlogError is more general Differential Revision: https://phab.mercurial-scm.org/D11410

rhg: Align with Python on some revset parsing corner cases In particular: * A string of ASCII digits can be either an integer on a hex prefix * The NULL node ID should convert to the NULL revision number Differential Revision: https://phab.mercurial-scm.org/D11409

rust: Add a Filelog struct that wraps Revlog Some filelog-specific logic is moved from code `rhg cat` into this struct where it can better be reused. Additionally, a missing end delimiter for metadata causes an error to be returned instead of being silently ignored. Differential Revision: https://phab.mercurial-scm.org/D11408

rust: Add Repo::manifest(revision) This deduplicates some common code. Differential Revision: https://phab.mercurial-scm.org/D11407

rust: Keep lazily-initialized Changelog and Manifest log on the Repo object That way if one of them is accessed multiple times it won’t be reopened from the filesystem. Differential Revision: https://phab.mercurial-scm.org/D11406

rust: Move lazy initialization of `Repo::dirstate_map` into a generic struct More components of `Repo` will be added following the same pattern. Differential Revision: https://phab.mercurial-scm.org/D11405

rust: Rename Manifest to Manifestlog, ManifestEntry to Manifest This appears to match the terminology used in Python code and on https://www.mercurial-scm.org/wiki/Manifest Differential Revision: https://phab.mercurial-scm.org/D11404

rust: Add Repo::dirstate_map and use it in `rhg status` This moves low-level dirstate wrangling out of the status command and into a more reusable location. The open dirstate map is lazily initialized and kept on the Repo object, for reuse by sub-sequent calls. Differential Revision: https://phab.mercurial-scm.org/D11398

rust: Switch to the memmap2-rs crate https://github.com/RazrFalcon/memmap2-rs This is a fork of the original memmap crate which appears to be unmaintained: https://github.com/danburkert/memmap-rs/issues/90 This fork is the most popular according to https://crates.io/keywords/mmap Differential Revision: https://phab.mercurial-scm.org/D11397

rust: Make OwningDirstateMap generic and move it into hg-core This will enable using it in rhg too. The `OwningDirstateMap::new_empty` constructor is generic and accepts a value of any type that gives acces to a bytes buffer. That buffer must stay valid as long as the value hasn’t been dropped, and must keep its memory address even if the value is moved. The `StableDeref` marker trait encodes those constraints. Previously no trait was needed because the value was always of type `PyBytes` which we know satisfies those constraints. The buffer type is ereased in the struct itself through boxing and dynamic dispatch, in order to simplify other signatures that mention `OwningDirstateMap`. Differential Revision: https://phab.mercurial-scm.org/D11396

rust: Move PyBytesWithData out of copy-tracing code So we can use it in other places to. Replace its `.data()` method with the `Deref<Target = [u8]>` trait, allowing this type to be used in generic contexts. Rename the type accordingly. Differential Revision: https://phab.mercurial-scm.org/D11395

rust: Move VFS code to its own module It was previously in the hg::repo module, but both repo code and vfs will likely grow in the future. Differential Revision: https://phab.mercurial-scm.org/D11394

rhg: Switch rhg.ignored-extensions config to Python-compatible list syntax This includes the ability to have an extension name/path that contains a comma, by double-quoting it. Differential Revision: https://phab.mercurial-scm.org/D11390

rhg: Port Python’s `ui.configlist` as `Config::get_list` This new method is not used yet outside of its own unit tests, so this changeset should make no observable change. The Rust parser implementation attempts to exactly replicate the behavior of the Python one, even in edge cases where that behavior is… surprising. New unit tests capture some of these edge cases. This started as a line-by-line port. The main changes are: * Pass around a parser mode enum instead of parser functions * Inline the whole parser into one function * Use `[u8]::get` which returns an `Option`, instead of indexing after explicitly checking the length. Differential Revision: https://phab.mercurial-scm.org/D11389

rust: Generalize the `trim_end_newlines` utility of byte strings … into `trim_end_matches` that takes a callack. Also add `trim_start_matches`. Differential Revision: https://phab.mercurial-scm.org/D11388

dirstate-item: move the C implementation to the same logic Now that we know were we are going, we can update the C implementation. Differential Revision: https://phab.mercurial-scm.org/D11387

dirstate-item: introduce low level C function Since a lot of logic is about to change, we need to isolate it into bits that are easy reusable in the rest of the C code. Especially we need to split the part that do the pure logic from the part that do the Python wrapping. Differential Revision: https://phab.mercurial-scm.org/D11386

dirstate-item: factor some code in the C implementation We have both `dirstate_item_from_v1_meth`, a class method, and `dirstate_item_from_v1_data`, a function taking low level C argument. Lets implement the former with the later. Differential Revision: https://phab.mercurial-scm.org/D11385

dirstate-item: `dirstate_item_from_v1_data` replaces make_dirstate_item For different format, we will need multiple capsule anyways. Differential Revision: https://phab.mercurial-scm.org/D11384

pathutil: replace the `skip` argument of `dirs` with a boolean It is ever only used for `r` file. So we make it a boolean this will give use more versatility later as we will stop storing the state explicitly. Differential Revision: https://phab.mercurial-scm.org/D11383

dirstate: clarify the message in nonnormal checking The previous message was quite verbose and hard to understand. The new one should be friendlier. Differential Revision: https://phab.mercurial-scm.org/D11382

merge: with stable

narrow: fix commits of empty files The problem is that when committing a new file with empty contents (or in general empty file with filelog p1 = -1), hg commit with narrow doesn't create a filelog revision at all, which causes failures in further commands. The problem seems to be that: - hg thinks that instead of creating a new filelog revision, it can use the filelog's p1 (the nullrev) - because it thinks the file contents is the same in that revision and in p1 - because `narrowfilelog.cmp(nullrev, b'')` is True (unlike with `filelog.cmp`) It's not clear to me which `cmp` behaves better. But I think it makes sense to change the commit code to not to "reuse" the null rev when adding an empty file with filelog p1 == filelog p2 == -1. This is consistent with never writing the null rev in the manifest, which `hg verify` claims is an invariant: ``` inside/c@4: manifest refers to unknown revision 000000000000 ``` Differential Revision: https://phab.mercurial-scm.org/D11400

narrow: show repo corruption when commiting empty files Differential Revision: https://phab.mercurial-scm.org/D11399

filemerge: be more strict when detecting conflict markers, add `|` markers I received a user complaint about detecting a line that contained 78 `=` followed by `*/` as a conflict marker. We'll never generate that, we generate 7 identical characters and either the end of the line, or a space. Let's be explicit about detecting exactly what we produce to reduce the chances of a false positive. While we're here, add `|||||||` as a detected conflict marker (generated with the `keep-merge3` style conflicts). Differential Revision: https://phab.mercurial-scm.org/D11391

dirstate-item: implement `v1_mtime` with higher level block This is much easier to read and maintain. Differential Revision: https://phab.mercurial-scm.org/D11375

dirstate-item: implement `v1_size` with higher level block This is much easier to read and maintain. Differential Revision: https://phab.mercurial-scm.org/D11374

dirstate-item: implement v1_state with higher level block This is much easier to read and maintain. Differential Revision: https://phab.mercurial-scm.org/D11373

dirstate-item: add more logic to `from_p2` This seem semantically better even if this tracking case seems to always comes with clean_p2 for now. Differential Revision: https://phab.mercurial-scm.org/D11372

dirstate-item: implement `merged_removed` in a simpler way We can simply use the underlying attribute. Differential Revision: https://phab.mercurial-scm.org/D11371

dirstate-item: implement `removed` in a simpler way We can simply use the underlying attribute. Differential Revision: https://phab.mercurial-scm.org/D11370

dirstate-item: implement `from_p2_removed` in a simpler way We can simply use the underlying attribute. Differential Revision: https://phab.mercurial-scm.org/D11369

dirstate-item: implement `from_p2` in a simpler way We can simply use the underlying attribute. Differential Revision: https://phab.mercurial-scm.org/D11368

dirstate-item: implement `merged` in a simpler way We can simply use the underlying attribute. Differential Revision: https://phab.mercurial-scm.org/D11367

dirstate-item: implement `added` in a simpler way We can simply use the underlying attribute. Differential Revision: https://phab.mercurial-scm.org/D11366

dirstate-item: implement `tracked` in a simpler way We can simply use the underlying attribute. Differential Revision: https://phab.mercurial-scm.org/D11365

dirstate-item: keep the full information in memory (for pure form) This changeset is finally reaching the area where we have been headed this whole time. Since all implementation details are contained inside de DirstateItem logic, we can change the way it is implemented. So we store the information that are passed to the object and use them to dynamically compute the "legacy" value. For now we only do this for the Pure implementation, as this is a good demonstration for the logic is fully insulated. The next step will be the update the C code too, to implement the various method with the new attribute (instead of the legacy one) and then start seriously thinking about the parameters we feed into DirstateItem. Differential Revision: https://phab.mercurial-scm.org/D11364

dirstate-item: have all the logic go through the v1_ accessors We are about to change the internal attribute. Having all the logic using the old "legacy" accessors will help to have a smooth transition. Differential Revision: https://phab.mercurial-scm.org/D11363

dirstate-item: drop the deprecated __getitem__ variante This is deprecated since 5.9. Differential Revision: https://phab.mercurial-scm.org/D11362