Augie Fackler <augie@google.com> [Wed, 29 May 2019 09:56:27 -0400] rev 42389
tests: sort some imports that were previously missed
I'm a little unclear why the import checker didn't catch this before,
but when I fixed it to work in Python 3 this failure started showing
up. Sigh.
Differential Revision: https://phab.mercurial-scm.org/D6454
Augie Fackler <augie@google.com> [Wed, 29 May 2019 09:55:35 -0400] rev 42388
contrib: fix import-checker to operate on str instead of bytes
I believe this is fallout from other Python 3 cleanups, and our code
linting tools are now leaning towards operating on str and not
bytes. I don't feel strongly, so I've just restored this tool to
working on Python 3.
Differential Revision: https://phab.mercurial-scm.org/D6453
Kyle Lippincott <spectral@google.com> [Tue, 28 May 2019 16:12:11 -0700] rev 42387
verify: use self._err not self.err, it changed in 7eaf4b1ac2a3
Differential Revision: https://phab.mercurial-scm.org/D6451
Kyle Lippincott <spectral@google.com> [Tue, 28 May 2019 23:22:46 -0700] rev 42386
tests: make run-tests exit non-zero if there are "errors"
Previously, if there was an error such as a broken .t file that caused
run-tests.py to encounter an exception during parsing, the test would be
considered in an "errored" state, which is separate from "failed".
The check for whether to exit non-zero or not was based entirely on whether
there were any tests in a "failed" state, so if there was only an error,
run-tests would exit with 0. Our test infrastructure would then consider the
test as passing, causing us to have some tests with false negatives that have
gone undetected for a few weeks now.
Differential Revision: https://phab.mercurial-scm.org/D6452
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 23 May 2019 18:15:08 +0200] rev 42385
perf: add a `perfhelper-mergecopies` command
This command gather data that are useful to pick argument for `perfmergecopies`.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 23 May 2019 14:48:02 +0200] rev 42384
perf: add a new `perfmergecopies` command
This command benchmark calls to `mercurial.copies.mergecopies`
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 23 May 2019 14:02:01 +0200] rev 42383
perf: factor selection of revisions involved in the merge out
We will introduce more performance command around merge. As a first step we
factor out pieces of `perfmergecalculate` that can be reused.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 23 May 2019 13:49:31 +0200] rev 42382
perf: allow to specify the base of the merge in perfmergecalculate
We can now test the rebase case.
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 23 May 2019 11:19:48 +0200] rev 42381
perf: add a --from flag to perfmergecalculate
Before this change, `perfmergecalculate` was always benchmarking the merge of
the working copy with another revision. We can now benchmark the
`mergecalculate` call for any arbitrary pair of revision.
Augie Fackler <augie@google.com> [Tue, 28 May 2019 09:57:53 -0400] rev 42380
merge with stable
Pulkit Goyal <7895pulkit@gmail.com> [Sat, 25 May 2019 19:49:44 +0300] rev 42379
py3: fix test-narrow* which started failing because of recent changes
#skip-blame because just r'' prefix
Differential Revision: https://phab.mercurial-scm.org/D6447
Matt Harbison <matt_harbison@yahoo.com> [Thu, 23 May 2019 22:50:11 -0400] rev 42378
manifest: add some documentation to _lazymanifest python code
It was not particularly easy figuring out the design of this class and keeping
track of how the pieces work. So might as well write some of it down for the
next person.
Matt Harbison <matt_harbison@yahoo.com> [Thu, 23 May 2019 21:54:24 -0400] rev 42377
manifest: avoid corruption by dropping removed files with pure (issue5801)
Previously, removed files would simply be marked by overwriting the first byte
with NUL and dropping their entry in `self.position`. But no effort was made to
ignore them when compacting the dictionary into text form. This allowed them to
slip into the manifest revision, since the code seems to be trying to minimize
the string operations by copying as large a chunk as possible. As part of this,
compact() walks the existing text based on entries in the `positions` list, and
consumed everything up to the next position entry. This typically resulted in
a ValueError complaining about unsorted manifest entries.
Sometimes it seems that files do get dropped in large repos- it seems to
correspond to there being a new entry that would take the same slot. A much
more trivial problem is that if the only changes were removals, `_compact()`
didn't even run because `__delitem__` doesn't add anything to `self.extradata`.
Now there's an explicit variable to flag this, both to allow `_compact()` to
run, and to avoid searching the manifest in cases where there are no removals.
In practice, this behavior was mostly obscured by the check in fastdelta() which
takes a different path that explicitly drops removed files if there are fewer
than 1000 changes. However, timeless has a repo where after rebasing tens of
commits, a totally different path[1] is taken that bypasses the change count
check and hits this problem.
[1] https://www.mercurial-scm.org/repo/hg/file/2338bdea4474/mercurial/manifest.py#l1511
Matt Harbison <matt_harbison@yahoo.com> [Thu, 23 May 2019 21:39:19 -0400] rev 42376
tests: demonstrate broken manifest generation with the pure module
This will be fixed next. But I don't fully understand how 'b.txt' is actually
removed properly in the second test, given what's broken. Also, I'm not sure
why 'bb.txt' is flagged as not being in the manifest, when it clearly appears
to be.
Martin von Zweigbergk <martinvonz@google.com> [Sat, 11 May 2019 00:06:06 -0700] rev 42375
tests: add test for {file_mods}, {file_adds}, {file_dels} on merge commit
Differential Revision: https://phab.mercurial-scm.org/D6368
Martin von Zweigbergk <martinvonz@google.com> [Thu, 18 Apr 2019 13:34:20 -0700] rev 42374
context: add ctx.files{modified,added,removed}() methods
Changeset-centric copy tracing is currently very slow because it often
reads manifests. One place it needs the manifest is in _chain(), where
it removes a copy X->Y if Y has subsequently gotten removed. I want to
speed that up by keeping track directly in the changeset of which
files are removed in the changeset. These methods will be similar to
ctx.p[12]copies() in that way: they will either read from the
changeset or calculate the information from the manifests otherwise.
Note that these are different from ctx.{modified,added,removed}() on
merge commits. Those functions always compare to p1, but the new ones
compare to both parents. filesadded() means "file does not exist in
either parent but exists now", filesremoved() means "file existed in
either parent but does not exist now", and filesmodified() means "file
existed in either parent and still exists". The set of files in
ctx.files() is the union of the files from the three new functions
(and the three new ones are all disjoint sets).
Also note that uncommitted merges are weird as usual. The invariant
mentioned above still holds, but the functions compare to p1 (and are
thus identical to the existing methods).
Differential Revision: https://phab.mercurial-scm.org/D6367
Martin von Zweigbergk <martinvonz@google.com> [Thu, 09 May 2019 15:09:07 -0700] rev 42373
copies: split up _chain() in naive chaining and filtering steps
The function now has two clearly defined steps. The first step is the
actual chaining. This step is very cheap. The second step is filtering
out invalid copies. This step is expensive. For changeset-centric copy
tracing, I want to do the filtering step only at the end. This patch
prepares for that.
Differential Revision: https://phab.mercurial-scm.org/D6418
Martin von Zweigbergk <martinvonz@google.com> [Fri, 24 May 2019 09:24:47 -0700] rev 42372
relnotes: document changed behavior of ui.origbackuppath pointing to file
Differential Revision: https://phab.mercurial-scm.org/D6446
Martin von Zweigbergk <martinvonz@google.com> [Sat, 11 May 2019 00:17:42 -0700] rev 42371
templatekw: move showfileadds() close to showfile{mods,dels}()
Differential Revision: https://phab.mercurial-scm.org/D6370
Pulkit Goyal <7895pulkit@gmail.com> [Fri, 24 May 2019 15:38:50 +0300] rev 42370
py3: use range() instead of xrange()
The latter does not exist on Python 3. This makes test-contrib-perf.t pass on
Python 3 again.
Differential Revision: https://phab.mercurial-scm.org/D6443
Pulkit Goyal <pulkit@yandex-team.ru> [Fri, 24 May 2019 15:59:59 +0300] rev 42369
narrow: move heads close to common as they are closely related
Differential Revision: https://phab.mercurial-scm.org/D6445
Pulkit Goyal <pulkit@yandex-team.ru> [Fri, 24 May 2019 15:57:00 +0300] rev 42368
narrow: pass binary nodeids to generateellipsesbundle2()
We generally work with binary nodeids and it's should be expected that new
function gets the nodeids in binary form already.
Differential Revision: https://phab.mercurial-scm.org/D6444
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 24 May 2019 12:33:46 +0200] rev 42367
match: stabilize _rootsdirsandparents doctest
Changeset c4b8f8637d7a tried to stabilize some matcher test by using a set. This
did not work because the set order is not stable. To fix it, we post process the
result to display a sorted version of the set.
Pulkit Goyal <7895pulkit@gmail.com> [Tue, 21 May 2019 05:32:14 +0530] rev 42366
narrow: factor out logic to build ellipses related b2parts in separate fn
This will help us switch more cleanly to using wireprotocol commands instead of
using exchange.pull() which exchanges more things then required.
Differential Revision: https://phab.mercurial-scm.org/D6435
Pulkit Goyal <7895pulkit@gmail.com> [Tue, 21 May 2019 04:49:18 +0530] rev 42365
narrow: remove unrequired compat code for old versions of hg
As the comment says, that if is only required for servers having hg version 3.1
and 3.2. Any client connecting having hg 3.1 or 3.2 locally and trying to use
narrow should already be broken taking in account the changes which have been
done since narrow moved to core.
Differential Revision: https://phab.mercurial-scm.org/D6434
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 23 May 2019 19:05:39 +0200] rev 42364
perf: make sure to explicitly disable any profiler after the first iteration
The current code work, because of some edge behavior of the `profile` class. We
make it explicit that the profiler is not in effect more than once.
Danny Hooper <hooper@google.com> [Wed, 22 May 2019 16:20:34 -0700] rev 42363
test: add missing 'cd ..' to test case
Differential Revision: https://phab.mercurial-scm.org/D6439
Martin von Zweigbergk <martinvonz@google.com> [Wed, 22 May 2019 14:16:44 -0700] rev 42362
match: remove an obsolete comment about util.finddirs()
Obsolete since 8e55c0c642c (util: make util.dirs() and util.finddirs()
include root directory (API), 2017-05-16).
Differential Revision: https://phab.mercurial-scm.org/D6433
Martin von Zweigbergk <martinvonz@google.com> [Wed, 22 May 2019 13:58:05 -0700] rev 42361
match: de-flake test-doctest.py by not depending on util.dirs() order
util.dirs() yields directories in arbitrary order, which has made
test-doctest.py flaky. I think they have been flaky since d8e55c0c642c
(util: make util.dirs() and util.finddirs() include root directory
(API), 2017-05-16). Before that commit, I think util.dirs() would
return at most one entry, so there was only one iteration order. This
patch fixes the problem by making _rootsdirsandparents() return a set
(whose __str__() is defined to be in sorted order, I believe). The
only caller wanted a set anyway.
Differential Revision: https://phab.mercurial-scm.org/D6432
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 21 May 2019 15:26:48 +0200] rev 42360
perf: add an option to profile the benchmark section
Running a perf command with --profile gather data for the whole command
execution, including setup and cleanup. This can significantly alter the data.
To work around this we introduce a new option, it trigger the profiling of only one
iteration of the benchmarked section.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 21 May 2019 15:08:06 +0200] rev 42359
perf: add a `pre-run` option
sometimes, the initial run is necessary to warm some cache that are not relevant
for the current measurement. We add a new `perf.pre-run` option to specify a
number of run of the benchmark logic that will happens before measurement are
taken.
Danny Hooper <hooper@google.com> [Mon, 20 May 2019 18:09:41 -0700] rev 42358
narrow: consider empty commits to be "inside the narrow spec" for templates
It doesn't seem useful to exclude them, or harmful to include them. Users
writing log templates using outsidenarrow as a predicate might consider it
unexpected if their locally created empty drafts are treated as if they
contained something outside the clone.
Differential Revision: https://phab.mercurial-scm.org/D6414
Georges Racinet <georges.racinet@octobus.net> [Tue, 21 May 2019 20:07:20 +0200] rev 42357
rust-python3: useless python2 specific import
This python27_sys import prevents building with python3,
it had been previously removed in a5fa9140ce4c, but that
has been since pruned
Differential Revision: https://phab.mercurial-scm.org/D6415
Georges Racinet <georges.racinet@octobus.net> [Thu, 16 May 2019 21:22:29 +0200] rev 42356
rust-python3: compatibility fix for incoming PyLong
On Python3, PyInt is PyLong and it doesn't have the
`value()` method.
Re upcasting to PythonObj as done here works, but we
might prefer taking a PythonObj from the onset
(would require more testing)
Differential Revision: https://phab.mercurial-scm.org/D6397
Pulkit Goyal <7895pulkit@gmail.com> [Tue, 21 May 2019 04:30:56 +0530] rev 42355
py3: add one new passing test found by buildbot
Differential Revision: https://phab.mercurial-scm.org/D6412
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 21 May 2019 13:08:22 +0200] rev 42354
discovery: slowly increase sampling size
Some pathological discovery runs can requires many roundtrip. When this happens
things can get very slow.
To make the algorithm more resilience again such pathological case. We slowly
increase the sample size with each roundtrip (+5%). This will have a negligible
impact on "normal" discovery with few roundtrips, but a large positive impact of
case with many roundtrips. Asking more question per roundtrip helps to reduce
the undecided set faster. Instead of reducing the undecided set a linear speed
(in the worst case), we reduce it as a guaranteed (small) exponential rate. The
data below show this slow ramp up in sample size:
round trip | 1 | 5 | 10 | 20 | 50 | 100 | 130 |
sample size | 200 | 254 | 321 | 517 | 2 199 | 25 123 | 108 549 |
covered nodes | 200 | 1 357 | 2 821 | 7 031 | 42 658 | 524 530 | 2 276 755 |
To be a bit more concrete, lets take a very pathological case as an example. We
are doing discovery from a copy of Mozilla-try to a more recent version of
mozilla-unified. Mozilla-unified heads are unknown to the mozilla-try repo and
there are over 1 million "missing" changesets. (the discovery is "local" to
avoid network interference)
Without this change, the discovery:
- last 1858 seconds (31 minutes),
- does 1700 round trip,
- asking about 340 000 nodes.
With this change, the discovery:
- last 218 seconds (3 minutes, 38 seconds a -88% improvement),
- does 94 round trip (-94%),
- asking about 344 211 nodes (+1%).
Of course, this is an extreme case (and 3 minutes is still slow). However this
give a good example of how this sample size increase act as a safety net
catching any bad situations.
We could image a steeper increase than 5%. For example 10% would give the
following number:
round trip | 1 | 5 | 10 | 20 | 50 | 75 | 100 |
sample size | 200 | 321 | 514 | 1 326 | 23 060 | 249 812 | 2 706 594 |
covered nodes | 200 | 1 541 | 3 690 | 12 671 | 251 871 | 2 746 254 | 29 770 966 |
In parallel, it is useful to understand these pathological cases and improve
them. However the current change provides a general purpose safety net to smooth
the impact of pathological cases.
To avoid issue with older http server, the increase in sample size only occurs
if the protocol has not limit on command argument size.
Juan Francisco Cantero Hurtado <iam@juanfra.info> [Tue, 21 May 2019 19:23:14 +0200] rev 42353
tests: make the grep pattern in remotefilelog-gcrepack portable (issue6122)
test-remotefilelog-gcrepack was using "\" to escape "|" in the grep pattern.
The most of implementations ignore "\" when it is followed by "|", so the regex
works. However, OpenBSD doesn't ignore "\" and considers "|" part of the text
instead of create two branches. Neither of both behaviors violate POSIX.
This change removes the unnecessary escape character and changes grep to egrep,
so the extended regular expression works on every unix.
This is part of the bug 6122. Tested on OpenBSD, GNU, FreeBSD, NetBSD, Solaris
11 and BusyBox.
Credits to Todd C. Miller, Paul de Weerd and Ingo Schwarze for helping me with
it.
Martin von Zweigbergk <martinvonz@google.com> [Mon, 20 May 2019 16:12:27 -0700] rev 42352
help: document new "bookmarksinstore" requirement in internals.requirements
Differential Revision: https://phab.mercurial-scm.org/D6413
Augie Fackler <augie@google.com> [Mon, 20 May 2019 14:00:12 -0400] rev 42351
absorb: fix interactive mode I didn't know existed
While investigating a bug in `hg absorb -e` I unintentionally
discovered `hg absorb --interactive` and its brokenness. This adds a
test and restores the functionality.
Note that this interface is still marked experimental, so we can
change this to be more sophisticated in the future.
Differential Revision: https://phab.mercurial-scm.org/D6411
Augie Fackler <augie@google.com> [Fri, 17 May 2019 11:13:12 -0400] rev 42350
tests: work around libressl being different about error strings (issue6122)
As far as I can tell, this is the right behavior. Thanks to Alex
Gaynor for checking what the string means by looking at libressl
sources for me.
Differential Revision: https://phab.mercurial-scm.org/D6410
Augie Fackler <augie@google.com> [Mon, 20 May 2019 11:40:47 -0400] rev 42349
merge with stable
Yuya Nishihara <yuya@tcha.org> [Mon, 20 May 2019 08:40:54 +0900] rev 42348
templatekw: change default value of 'requires' to ()
Since we dropped support for the old-style template keywords, we no longer
have to distinguish None (old-style) and an empty requirement (new-style).
Martin von Zweigbergk <martinvonz@google.com> [Tue, 14 May 2019 16:30:38 -0700] rev 42347
commit: move sorting of added and removed files list to lower level
localrepo.commitctx() has lists of all changed files, as well as lists
of added and removed files. The list of all files is unsorted and
changelog.add() will sort it. Let's also sort the lists of added and
removed files at a lower level (manifestrevlog.add()) for
consistency. It also seems safer to do it there, just before we write
them to the store. That way other callers won't be able to create
invalid commits (or whatever the consequence is) by passing in
unsorted lists. Also, alternative storages may not care that the lists
are sorted. I don't think this will be a performance problem (someone
should have fixed the sorting in changelog.add() if it were).
Differential Revision: https://phab.mercurial-scm.org/D6390
Martin von Zweigbergk <martinvonz@google.com> [Wed, 24 Apr 2019 09:39:40 -0700] rev 42346
match: drop unnecessary adding of '' to set of dirs
This breaks some tests for "rootfilesin:" in a pattern matcher even
more, but that just shows how broken that case is.
Differential Revision: https://phab.mercurial-scm.org/D6406
Martin von Zweigbergk <martinvonz@google.com> [Mon, 22 Apr 2019 22:43:00 -0700] rev 42345
narrowcommands: drop unnecessary adding of '' for root directory
It's now included by util.dirs().
Differential Revision: https://phab.mercurial-scm.org/D6405
Martin von Zweigbergk <martinvonz@google.com> [Wed, 17 Apr 2019 21:39:18 -0700] rev 42344
copies: remove hack for adding root dir to util.dirs object
Differential Revision: https://phab.mercurial-scm.org/D6404
Martin von Zweigbergk <martinvonz@google.com> [Tue, 16 May 2017 11:00:38 -0700] rev 42343
util: make util.dirs() and util.finddirs() include root directory (API)
This changes the behavior of test-origbackup-conflict.t so it no
longer errors out when the backup path points to an existing
file. Instead, it replaces the file by a directory. That seems
reasonable to me.
Differential Revision: https://phab.mercurial-scm.org/D6403
Martin von Zweigbergk <martinvonz@google.com> [Thu, 13 Jul 2017 23:43:16 -0700] rev 42342
dirstate: drop workaround for '.' matching root directory
The check was added in 31abcae33b4f (dirstate: do not ignore current
directory '.' (issue 1078), 2008-04-05) to fix issue1078. Funnily
enough, comment #2 on that issue mentions using '' instead of '.' to
represent the root directory, just like my previous patch did.
test-hgignore.t fails with this patch without the previous patch.
Differential Revision: https://phab.mercurial-scm.org/D6402
Martin von Zweigbergk <martinvonz@google.com> [Mon, 15 May 2017 00:12:19 -0700] rev 42341
match: use '' instead of '.' for root directory (API)
I think '' is generally a better value for the root directory than '.'
is. For example, os.path.join('', 'foo') => 'foo', while
os.path.join('.', 'foo') => './foo'.
This patch mostly makes it so we use '' internally in
match.py. However, it also affects the API in visitdir(),
visitchildrenset() and files(). The two former now also accept '' as
input. I've updated the callers of these methods. I've also added a
deprecation warning for passing '.' (for external callers). The only
caller I could find that was affected by files() returning '' instead
of '.' was in dirstate.walk(). I've updated that.
The next few patches show some workarounds we can remove by using ''
instead of '.'.
Differential Revision: https://phab.mercurial-scm.org/D6401
Martin von Zweigbergk <martinvonz@google.com> [Wed, 24 Apr 2019 09:32:29 -0700] rev 42340
dirstate: move special handling of files==['.'] together
I think it makes it a little clearer to have the two conditions for
files==['.'] near each other.
Differential Revision: https://phab.mercurial-scm.org/D6400
Martin von Zweigbergk <martinvonz@google.com> [Fri, 17 May 2019 00:57:57 -0700] rev 42339
convert: don't include file in "files" list if it's added in p2
If the file is from p2, we should clearly compare the flags to what
they were in p2.
Also note that manifest.flags('non-existent') unfortunately returns ''
instead of erroring out.
Differential Revision: https://phab.mercurial-scm.org/D6409
Martin von Zweigbergk <martinvonz@google.com> [Fri, 17 May 2019 11:32:48 -0700] rev 42338
convert: demonstrate broken {files} list in merge commits with file flags
When there is a merge in which the flags for a file from p2 is
non-empty, `hg convert` will incorrectly include that in the
changeset's files list.
Differential Revision: https://phab.mercurial-scm.org/D6408
Matt Harbison <matt_harbison@yahoo.com> [Sat, 18 May 2019 19:56:06 -0400] rev 42337
templater: drop support for old style keywords (API)
These changes originated from several commits over a period of time, so I'm
slightly unsure if this is correct. But the tests pass.
Matt Harbison <matt_harbison@yahoo.com> [Sat, 18 May 2019 19:38:47 -0400] rev 42336
commands: drop support for legacy ^cmd registration (API)
Matt Harbison <matt_harbison@yahoo.com> [Sat, 18 May 2019 19:33:48 -0400] rev 42335
extensions: drop support for extsetup() without `ui` argument (API)
Martin von Zweigbergk <martinvonz@google.com> [Fri, 17 May 2019 11:11:40 -0700] rev 42334
relnotes: mention removed support for mixed log graph lines
This adds release notes for 264a2cbb25d0 (graphmod: remove support for
graph lines mixing parent/grandparent styles (BC), 2018-10-16).
Differential Revision: https://phab.mercurial-scm.org/D6407
Augie Fackler <augie@google.com> [Fri, 17 May 2019 11:03:47 -0400] rev 42333
tests: fix test-clonebundles on recent openbsd
I guess openbsd feels like it needs to stringify this errno in
lowercase and omit the "host" part of "hostname. Okay.
Reported in a big test diff talking about libressl, see 6122. I'm not
flagging this because most of that issue is about a libressl string
change, so this doesn't really make a big difference there.
Differential Revision: https://phab.mercurial-scm.org/D6399
Georges Racinet <georges.racinet@octobus.net> [Thu, 16 May 2019 21:17:14 +0200] rev 42332
rust-python3: compatibility fix for integer conversion
On python3, `to_py_object()` on the usize gives us a PyLong,
whereas it is the generic `PyObject` already on python2, which fits
the `py.None()` default value.
Upcasting to `PyObject` explicitely in all cases solves the issue.
Differential Revision: https://phab.mercurial-scm.org/D6396
Augie Fackler <augie@google.com> [Fri, 17 May 2019 09:42:02 -0400] rev 42331
rust: sort dependencies entries in Cargo.toml
I should probably write a test to enforce this...
Differential Revision: https://phab.mercurial-scm.org/D6398
Pulkit Goyal <7895pulkit@gmail.com> [Fri, 17 May 2019 00:04:29 +0530] rev 42330
py3: make contrib/testparseutil.py to work on str(unicodes)
contrib/check-code work on unicodes and call functions from testparseutil.py
which before this patch used to work on bytes.
This path removes that inconsistency and make testparseutil.py work on unicodes.
This makes test-check-code.t and test-contrib-check-code.t work on Python 3
again.
Differential Revision: https://phab.mercurial-scm.org/D6391
Raphaël Gomès <rgomes@octobus.net> [Fri, 17 May 2019 09:36:29 -0400] rev 42329
rust-filepatterns: call new Rust implementations from Python
This change adds the import to the `rust-cpython` bindings and uses
them when appropriate.
A wrapper function has been defined in the case of `_regex` to
keep this patch simple.
Differential Revision: https://phab.mercurial-scm.org/D6273
Raphaël Gomès <rgomes@octobus.net> [Fri, 17 May 2019 09:36:29 -0400] rev 42328
rust-filepatterns: add `rust-cpython` bindings for `filepatterns`
This change adds the `rust-cpython` interface for top-level functions and
exceptions in the filepatterns module.
Contrary to the Python implementation, this tries to have finer-grained
exceptions to allow for better readability and flow control down the line.
Differential Revision: https://phab.mercurial-scm.org/D6272
Raphaël Gomès <rgomes@octobus.net> [Wed, 24 Apr 2019 11:34:09 +0200] rev 42327
rust-filepatterns: add a Rust implementation of pattern-related utils
This change introduces Rust implementations of two functions related to
pattern handling, all located in `match.py`:
- `_regex`
- `readpatternfile`
These utils are useful in the long-term effort to improve `hg status`'s
performance using Rust. Experimental work done by Valentin Gatien-Baron
shows very promising improvements, but is too different from the current
Mercurial core code structure to be used "as-is".
This is the first - albeit very small - step towards the code revamp
needed down the line.
Two dependencies were added: `regex` and `lazy_static`. Both of them
will be useful for a majority of the Rust code that will be written,
are well known and maintained either by the Rust core team, or by
very frequent contributors.
Differential Revision: https://phab.mercurial-scm.org/D6271
Martin von Zweigbergk <martinvonz@google.com> [Wed, 15 May 2019 22:11:41 -0700] rev 42326
exchange: don't take wlock if bookmarks are stored in .hg/store/
If bookmarks are stored in .hg/store/, there is no need for the
wlock().
Differential Revision: https://phab.mercurial-scm.org/D6388
Martin von Zweigbergk <martinvonz@google.com> [Wed, 15 May 2019 22:09:02 -0700] rev 42325
bookmarks: keep bookmarks in .hg/store if new config set
Bookmarks storage consists of two parts: (1) the set of bookmarks and
their positions, and (2) the current bookmark. The former can get
updated by exchange, while the latter cannot. However, they are both
stored in directly .hg/ and protected by repo.wlock(). As a result,
ugly workarounds were needed. This patch introduces a new config
option to store the set of bookmarks and their positions in .hg/store/
but still storing the current bookmark directory in .hg/. The config
option only takes effect at repo creation time. It results in a new
requirement being set.
Differential Revision: https://phab.mercurial-scm.org/D6387
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 20 May 2019 10:08:28 +0200] rev 42324
bookmark: also make bookmark cache depends of the changelog
Since the changelog is also used during the parsing of bookmark data, it should
be listed as a file cache dependency. This fix the race condition we just
introduced a test for.
This is a simple fix that might lead bookmark data to be invalidated more often
than necessary. We could have more complicated code to deal with this race in a
more "optimal" way. I feel it would be unsuitable for stable.
In addition, the performance impact of this is probably minimal and I don't
foresee the more advanced fix to actually be necessary.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 20 May 2019 10:08:17 +0200] rev 42323
localrepo: grab mixedrepostorecache class from 526750cdd02d
On default, Martin von Zweigbergk <martinvonz@google.com> introduced a more
advance filecache decorator. I need this decorator to fix a bug on stable. So I
am grafting the relevant part of 526750cdd02d.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 20 May 2019 10:06:53 +0200] rev 42322
bookmark: add a test for a race condition on push
Bookmark pointing to unknown nodes are ignored. Later these ignored bookmarks
are dropped when writing the file back on disk. On paper, this behavior should
be fine, but with the current implementation, it can lead to unexpected
bookmark deletions.
In theory, to make sure writer as a consistent view, taking the lock also
invalidate bookmark data we already loaded into memory. However this
invalidation is incomplete. The data are stored in a `filecache` that preserve
them if the bookmark related file are untouched. In practice, the bookmark data
in memory also depends of the changelog content, because of the step checking
if the bookmarks refers to a node known to the changelog. So if the bookmark
data were loaded from an up to date bookmark file but filtered with an outdated
changelog file this go undetected.
This condition is fairly specific, but can occurs very often in practice. We
introduce a test recreating the situation. The test comes in an independant
changeset to show it actually reproduce the situation. The fix will come soon
after.
A large share of the initial investigation of this race condition was made by
Valentin Gatien-Baron <valentin.gatienbaron@gmail.com>.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 20 May 2019 07:11:16 +0200] rev 42321
test: properly gate a zstd section
This part of the test can't run if zstd is not available. This was caught by
--pure test (who don't support zstd).
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 20 May 2019 07:11:06 +0200] rev 42320
test: update test for expected test output
In 1fac9b931d46 as new test session was introduced. It did not take in account
some part that only ran for pure.
The test is now fixed.
Yuya Nishihara <yuya@tcha.org> [Thu, 16 May 2019 08:15:20 +0900] rev 42319
log: flag topo-sorted set as such
This isn't required right now, but revs.istopo() should return True.
Martin von Zweigbergk <martinvonz@google.com> [Wed, 09 Jan 2019 15:54:45 -0800] rev 42318
copies: fix duplicatecopies() with overlay context
The reasoning for this check is in 78d760aa3607 (duplicatecopies: do
not mark items not in the dirstate as copies, 2013-03-28). The check
was then moved to workingfilectx in 754b5117622f (context: add
workingfilectx.markcopied, 2017-10-15) and no corresponding check was
added later when overlayworkingfilectx was added. Rather than adding
the check there, this patch adds a more generic check on the callers
side and removes the check in workingfilectx.markcopied().
Differential Revision: https://phab.mercurial-scm.org/D6380
Martin von Zweigbergk <martinvonz@google.com> [Wed, 15 May 2019 16:10:52 -0700] rev 42317
tests: demonstrate crash when rebasing across copy with --collapse
As reported by timeless.
Differential Revision: https://phab.mercurial-scm.org/D6379
Augie Fackler <augie@google.com> [Wed, 15 May 2019 17:18:57 -0400] rev 42316
exthelper: add some semi-useful trace logs
It'd be nice to make the trace functions a little better-named in the output,
but I'm not sure how much better we can do without overhead. This at least
lets you see if a single reposetup function is eating all the time or if it's
spread over all of them. I needed this because Google's uber-extension has a
long load time and I wasn't sure where the problem was.
Differential Revision: https://phab.mercurial-scm.org/D6381
Martin von Zweigbergk <martinvonz@google.com> [Wed, 15 May 2019 23:26:05 -0700] rev 42315
help: add missing blank line, making "revlog-compression" show up
Differential Revision: https://phab.mercurial-scm.org/D6386
Martin von Zweigbergk <martinvonz@google.com> [Wed, 15 May 2019 11:53:22 -0700] rev 42314
tests: fix share test to actually share the repo
"repo2" is clearly meant to be a share from "repo1" but without
sharing bookmarks. However, `hg unshare` was called in the repo, so it
had become completely unrelated and thus not testing what it was
supposed to test.
Differential Revision: https://phab.mercurial-scm.org/D6385
Martin von Zweigbergk <martinvonz@google.com> [Wed, 15 May 2019 11:38:45 -0700] rev 42313
tests: separate out bookmarks tests from test-share.t
Differential Revision: https://phab.mercurial-scm.org/D6384
Martin von Zweigbergk <martinvonz@google.com> [Wed, 15 May 2019 10:19:36 -0700] rev 42312
bookmarks: use vfs.tryread() instead of reimplementing it
Differential Revision: https://phab.mercurial-scm.org/D6383
Martin von Zweigbergk <martinvonz@google.com> [Wed, 15 May 2019 10:13:29 -0700] rev 42311
bookmarks: use context manager when writing files
Differential Revision: https://phab.mercurial-scm.org/D6382
timeless <timeless@mozdev.org> [Wed, 15 May 2019 10:54:36 -0400] rev 42310
bisect: do not crash with rewritten commits
Martin von Zweigbergk <martinvonz@google.com> [Wed, 01 May 2019 09:34:47 -0700] rev 42309
log: add config for making `hg log -G` always topo-sorted
I (and everyone else at Google) have an log alias that adds graph mode
and templating. I have another one that builds on the first and also
restricts the set of revisions to only show those I'm most likely to
care about. This second alias also adds topological sorting. I still
sometimes use the first one. When I do, it very often bothers me that
it's not topologically sorted (branches are interleaved). This patch
adds a config option for always using topological sorting with graph
log.
The revision set is sorted eagerly, which seems like a bad idea, but
it doesn't seem to make a big difference in the hg repo (150ms). I
initially tried to instead wrap the user's revset in sort(...,topo),
but that seemed much harder.
Differential Revision: https://phab.mercurial-scm.org/D6331
Martin von Zweigbergk <martinvonz@google.com> [Tue, 14 May 2019 09:13:39 -0700] rev 42308
log: remove an unnecessary "and opts.get('rev')" condition
As Yuya pointed out, the condition is unnecessary since
revs.isdescending() would be true if --follow without --rev.
Differential Revision: https://phab.mercurial-scm.org/D6372
Kyle Lippincott <spectral@google.com> [Tue, 16 Oct 2018 04:59:36 -0700] rev 42307
graphmod: remove support for graph lines mixing parent/grandparent styles (BC)
Currently, if the configuration for a graph edge draw style has multiple bytes
(at least on python2), it is interpreted as "this is a request to draw the line
partially in the style of the parent, partially in the style of the
grandparent". This precludes the configuration handling unicode characters
(which trigger the `len > 1` check, at least on python2), and I believe was part
of the reason that beautifygraph was written the way it was.
Talking with the person who implemented this, it appears to have been to achieve
feature parity with the rendering of the smartlog extension. I suspect that this
isn't actually used outside of that situation, so I think that we can remove it
without much issue.
This will make it so that multi-character edges are possible, and render any
existing configuration that uses this feature with these multiple characters.
This is *not* going to adjust the width of everything to make it line up
correctly, please see the test that's being modified in this changeset for an
example of how the previous configuration now renders.
Note also that the previous configuration seems to have been broken, or at least
it was behaving in a really non-obvious way - it was avoiding the grandparent
character(s) when it should have been displaying them! This is why so many "!"
characters changed to "3."; I don't know if this was intentional.
Differential Revision: https://phab.mercurial-scm.org/D5112
Pulkit Goyal <7895pulkit@gmail.com> [Wed, 15 May 2019 21:02:32 +0300] rev 42306
py3: add 5 new passing tests
Differential Revision: https://phab.mercurial-scm.org/D6378
Pulkit Goyal <7895pulkit@gmail.com> [Wed, 15 May 2019 20:37:39 +0300] rev 42305
py3: add a r'' to prevent transformer adding b''
# skip-blame because just r'' prefix
Differential Revision: https://phab.mercurial-scm.org/D6377
Raphaël Gomès <rgomes@octobus.net> [Mon, 06 May 2019 22:51:10 +0200] rev 42304
rust-dirstate: call parse/pack bindings from Python
A future patch will need to address the issue of Rust module policy,
to avoid having ugly duplicate imports and conditionals all over the place.
As the rewrite of dirstate in Rust progresses, we will need fewer of those
"contact points".
Differential Revision: https://phab.mercurial-scm.org/D6350
Raphaël Gomès <rgomes@octobus.net> [Mon, 06 May 2019 22:50:34 +0200] rev 42303
rust-dirstate: add rust-cpython bindings to the new parse/pack functions
This allows for Python code to call `parse/pack_dirstate` transparently.
These bindings are heavy given the relatively simple task, as they are bound
to implementation details of both the C and Python code. They will be slimmed
down in future patches and eventually completely removed once more of the
dirstate code has been refactored/rewritten in Rust.
Both functions emulate the mutate-on-loop style of the Python and C
implementations by looping over changed items in the compatibility layer,
instead of at the core functions.
Differential Revision: https://phab.mercurial-scm.org/D6349
Raphaël Gomès <rgomes@octobus.net> [Mon, 06 May 2019 22:48:09 +0200] rev 42302
rust-dirstate: add rust implementation of `parse_dirstate` and `pack_dirstate`
Working towards the goal of having a complete Rust implementation of
`hg status`, these two utils are a first step of many to be taken
to improve performance and code maintainability.
Two dependencies have been added: `memchr` and `byteorder`.
Both of them have been written by reputable community members and are
very mature crates.
The Rust code will often need to use their byte-oriented functions.
A few unit tests have been added and may help future development and debugging.
In a future patch that uses `parse_dirstate` to stat the working tree in
parallel - which neither the Python nor the C implementations do - actual
performance improvements will be seen for larger repositories.
Differential Revision: https://phab.mercurial-scm.org/D6348
Martin von Zweigbergk <martinvonz@google.com> [Tue, 14 May 2019 22:56:58 -0700] rev 42301
changelog: define changelogrevision.p[12]copies for null revision
Looks like I missed these in 5382d8f8530b (changelog: parse copy
metadata if available in extras, 2017-12-27). `hg debugp[12]copies -r
null` fails before this patch.
Differential Revision: https://phab.mercurial-scm.org/D6376
Martin von Zweigbergk <martinvonz@google.com> [Tue, 23 Apr 2019 13:29:13 -0700] rev 42300
copies: write empty entries in changeset when also writing to filelog
When writing to both changeset and filelog (during transition), we
don't want the reader to waste time by falling back to reading from
the filelog when there is no copy metadata. Let's write out empty copy
metadata instead (the read path is already prepared for this
case). Thanks to Greg for pointing this out.
Differential Revision: https://phab.mercurial-scm.org/D6306
timeless <timeless@mozdev.org> [Mon, 13 May 2019 14:19:36 -0400] rev 42299
rebase: hide help for revisions.Predicates._destautoorphanrebase
timeless <timeless@mozdev.org> [Fri, 03 May 2019 16:07:57 -0400] rev 42298
unshelve: add space to help
Martin von Zweigbergk <martinvonz@google.com> [Fri, 10 May 2019 22:24:47 -0700] rev 42297
context: default to using branch from dirstate only in workingctx
Same reasoning as previous commits: only the workingctx should know
about the dirstate.
committablectx now seems free of dirstate references.
Differential Revision: https://phab.mercurial-scm.org/D6374
Martin von Zweigbergk <martinvonz@google.com> [Fri, 10 May 2019 22:51:33 -0700] rev 42296
context: let caller pass in branch to committablectx.__init__()
committablectx.__init__() currently looks up the branch from the
dirstate unless it's passed in the extras. memctx.__init__() has a
branch argument, but since committablectx.__init__() doesn't accept
it, it lets that constructor look up the branch from the dirstate
before it overwrites it, which seems awkward.
Differential Revision: https://phab.mercurial-scm.org/D6366
Martin von Zweigbergk <martinvonz@google.com> [Fri, 10 May 2019 21:55:59 -0700] rev 42295
context: move contents of committablectx.markcommitted() to workingctx
Same reasoning as previous commits: this function updates the
dirstate. By not updating the dirstate here, we also fix the
close-head test.
Differential Revision: https://phab.mercurial-scm.org/D6365
Martin von Zweigbergk <martinvonz@google.com> [Fri, 10 May 2019 22:18:11 -0700] rev 42294
tests: demonstrate that close-head command updates working copy
The help text for the command says "...it doesn't change the working
directory", so I don't think this is intentional.
Differential Revision: https://phab.mercurial-scm.org/D6364
Martin von Zweigbergk <martinvonz@google.com> [Fri, 10 May 2019 21:53:41 -0700] rev 42293
context: move walk() and match() overrides from committablectx to workingctx
Same reasoning as previous commit: these functions update the dirstate.
Differential Revision: https://phab.mercurial-scm.org/D6363
Martin von Zweigbergk <martinvonz@google.com> [Fri, 10 May 2019 21:35:30 -0700] rev 42292
context: move flags overrides from committablectx to workingctx
These read from the dirstate, so they shouldn't be used in other
subclasses.
Differential Revision: https://phab.mercurial-scm.org/D6362
Martin von Zweigbergk <martinvonz@google.com> [Fri, 10 May 2019 13:41:42 -0700] rev 42291
context: reuse changectx._copies() in all but workingctx
This moves the dirstate-specific _copies() implementation from
committablectx into workingctx where it should be (I think all
dirstate-specific stuff should be moved into workingctx). The part of
changectx._copies() that is for producing changeset-wide copy dicts
from the filectxs is moved into basectx so it's reused by the other
subclasses. The part of changectx._copies() that's about reading copy
information from the changeset remains there. This fixes in-memory
rebase (and makes `hg convert` able to write copies to changesets).
Differential Revision: https://phab.mercurial-scm.org/D6219
Martin von Zweigbergk <martinvonz@google.com> [Fri, 10 May 2019 14:27:22 -0700] rev 42290
overlayworkingctx: don't include added-then-deleted files in memctx
If a file (such as a .orig file) is temporarily added to the
overlayworkingctx and then deleted, it's still going to be in the
_cache dict. In tomemctx(), we created the list of files from
_cache.keys(), so the memctx.files() would include the temporary
file. That was fine because the list of files was only used in
localrepo.commitctx() (I think), where there's an extra filtering of
incorrectly removed files (annotated with an inaccurate "update
manifest" comment). I'd like to call memctx.files() in another case,
but first we need to make it accurate.
Differential Revision: https://phab.mercurial-scm.org/D6361
Martin von Zweigbergk <martinvonz@google.com> [Fri, 10 May 2019 10:23:46 -0700] rev 42289
tests: demonstrate loss of changeset copy metadata on rebase
Differential Revision: https://phab.mercurial-scm.org/D6360
Martin von Zweigbergk <martinvonz@google.com> [Fri, 10 May 2019 11:03:54 -0700] rev 42288
overlaycontext: allow calling copydata() on clean context
We should just report no copy if the context is clean.
Differential Revision: https://phab.mercurial-scm.org/D6358
Martin von Zweigbergk <martinvonz@google.com> [Fri, 10 May 2019 10:23:08 -0700] rev 42287
tests: demonstrate another failure with in-memory rebase and copies
This is a similar to dd1ab72be983 (test: demonstrate crash with
in-memory rebase and copies, 2019-03-14). The new failure started with
57203e0210f8 (copies: calculate mergecopies() based on pathcopies(),
2019-04-11). It happens in the call to mergemod.update() on
rebase.py:1268 where we call mergemod.update() to graft a node. Since
the mergecopies() rewrite, that calls _related() with the filectx from
the overlaywctx instead of a filectx from the changectx where the file
was last modified. Either should be fine, so I don't think that's
a bug.
Differential Revision: https://phab.mercurial-scm.org/D6357
Martin von Zweigbergk <martinvonz@google.com> [Tue, 14 May 2019 16:40:49 -0700] rev 42286
commit: fix a typo ("form p1" -> "from p1")
Differential Revision: https://phab.mercurial-scm.org/D6375
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 27 Apr 2019 11:48:26 -0700] rev 42285
automation: initial support for running Linux tests
Building on top of our Windows automation support, this commit
implements support for performing automated tasks on remote Linux
machines. Specifically, we implement support for running tests
on ephemeral EC2 instances. This seems to be a worthwhile place
to start, as building packages on Linux is more or less a solved
problem because we already have facilities for building in Docker
containers, which provide "good enough" reproducibility guarantees.
The new `run-tests-linux` command works similarly to
`run-tests-windows`: it ensures an AMI with hg dependencies is
available, provisions a temporary EC2 instance with this AMI, pushes
local changes to that instance via SSH, then invokes `run-tests.py`.
Using this new command, I am able to run the entire test harness
substantially faster then I am on my local machine courtesy of
access to massive core EC2 instances:
wall: 16:20 ./run-tests.py -l (i7-6700K)
wall: 14:00 automation.py run-tests-linux --ec2-instance c5.2xlarge
wall: 8:30 automation.py run-tests-linux --ec2-instance m5.4xlarge
wall: 8:04 automation.py run-tests-linux --ec2-instance c5.4xlarge
wall: 4:30 automation.py run-tests-linux --ec2-instance c5.9xlarge
wall: 3:57 automation.py run-tests-linux --ec2-instance m5.12xlarge
wall: 3:05 automation.py run-tests-linux --ec2-instance m5.24xlarge
wall: 3:02 automation.py run-tests-linux --ec2-instance c5.18xlarge
~3 minute wall time to run pretty much the entire test harness is
not too bad!
The AMIs install multiple versions of Python. And the run-tests-linux
command specifies which one to use:
automation.py run-tests-linux --python system3
automation.py run-tests-linux --python 3.5
automation.py run-tests-linux --python pypy2.7
By default, the system Python 2.7 is used. Using this functionality,
I was able to identity some unexpected test failures on PyPy!
Included in the feature is support for running with alternate
filesystems. You can simply pass --filesystem to the command to
specify the type of filesystem to run tests on. When the ephemeral
instance is started, a new filesystem will be created and tests
will run from it:
wall: 4:30 automation.py run-tests-linux --ec2-instance c5.9xlarge
wall: 4:20 automation.py run-tests-linux --ec2-instance c5d.9xlarge --filesystem xfs
wall: 4:24 automation.py run-tests-linux --ec2-instance c5d.9xlarge --filesystem tmpfs
wall: 4:26 automation.py run-tests-linux --ec2-instance c5d.9xlarge --filesystem ext4
We also support multiple Linux distributions:
$ automation.py run-tests-linux --distro debian9
total time: 298.1s; setup: 60.7s; tests: 237.5s; setup overhead: 20.4%
$ automation.py run-tests-linux --distro ubuntu18.04
total time: 286.1s; setup: 61.3s; tests: 224.7s; setup overhead: 21.4%
$ automation.py run-tests-linux --distro ubuntu18.10
total time: 278.5s; setup: 58.2s; tests: 220.3s; setup overhead: 20.9%
$ automation.py run-tests-linux --distro ubuntu19.04
total time: 265.8s; setup: 42.5s; tests: 223.3s; setup overhead: 16.0%
Debian and Ubuntu are supported because those are what I use and am
most familiar with. It should be easy enough to add support for other
distros.
Unlike the Windows AMIs, Linux EC2 instances bill per second. So
the cost to instantiating an ephemeral instance isn't as severe.
That being said, there is some overhead, as it takes several dozen
seconds for the instance to boot, push local changes, and build
Mercurial. During this time, the instance is largely CPU idle and
wasting money. Even with this inefficiency, running tests is
relatively cheap: $0.15-$0.25 per full test run. A machine running
tests as efficiently as these EC2 instances would cost say $6,000, so
you can run the test harness a >20,000 times for the cost of an
equivalent machine. Running tests in EC2 is almost certainly cheaper
than buying a beefy machine for developers to use :)
# no-check-commit because foo_bar function names
Differential Revision: https://phab.mercurial-scm.org/D6319
Gregory Szorc <gregory.szorc@gmail.com> [Tue, 23 Apr 2019 21:57:32 -0700] rev 42284
automation: move image operations to own functions
An upcoming commit will need this functionality with slightly different
values and it is enough code to not want to duplicate. Let's refactor
into standalone functions so it can be reused.
Differential Revision: https://phab.mercurial-scm.org/D6318
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 19 Apr 2019 09:18:23 -0700] rev 42283
automation: add --version argument to build-all-windows-packages
This lets us pass a version string through when building all
Windows packages, just like we can do with the individual commands
which produce installers.
Differential Revision: https://phab.mercurial-scm.org/D6317
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 19 Apr 2019 08:32:24 -0700] rev 42282
automation: do a force push to synchronize
We don't know what the state of the remote is. Force pushing will
be more resilient.
Differential Revision: https://phab.mercurial-scm.org/D6316
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 19 Apr 2019 08:21:02 -0700] rev 42281
automation: add check that hg source directory is a repo
Synchronizing from e.g. source distributions is not yet supported.
Let's add a check so we fail with an error message indicating
such.
Differential Revision: https://phab.mercurial-scm.org/D6315
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 19 Apr 2019 07:34:55 -0700] rev 42280
automation: shore up rebooting behavior
There was a race condition in the old code. Use
instance.stop()/instance.start() to eliminate it.
As part of debugging this, I also found another race condition
related to PowerShell permissions after the reboot. Unfortunately,
I'm not sure the best way to work around it. I've added a comment
for now.
Differential Revision: https://phab.mercurial-scm.org/D6288
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 19 Apr 2019 06:07:00 -0700] rev 42279
automation: wait longer for WinRM connection
I got a few timeouts waiting for only 120s for the WinRM connection
to become available. Increasing to 180s seems to fix. I guess
AWS isn't as consistent as I would like :(
Differential Revision: https://phab.mercurial-scm.org/D6287
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 27 Apr 2019 11:38:58 -0700] rev 42278
automation: wait for instance profiles and roles
Otherwise there is a race condition between creating the resources
and us attempting to use them / them becoming available.
The role waiter API was recently introduced, so we had to upgrade
the boto3 package to get it. Other packages were also updated
to latest versions just because.
Even with this change, I still run into issues with the IAM instance
profile not being available when we attempt to create an EC2 instance
using a just-created profile. I'm not sure what's going on. Possibly
a bug on Amazon's end. But the new behavior is "more correct."
Differential Revision: https://phab.mercurial-scm.org/D6286
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 19 Apr 2019 05:20:33 -0700] rev 42277
automation: don't create resources when deleting things
Otherwise running these commands can result in resources being
created. In the case of `purge-ec2-resources`, we will create
resources only to delete them immediately afterwards!
With this change, `purge-ec2-resources` now no-ops if no
resources exist.
# no-check-commit because foo_bar function name
Differential Revision: https://phab.mercurial-scm.org/D6285
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 19 Apr 2019 05:15:43 -0700] rev 42276
automation: detach policies before deleting role
You can't delete an IAM role that has attached policies.
With this change, the purge-ec2-resources command now works.
Differential Revision: https://phab.mercurial-scm.org/D6284
Gregory Szorc <gregory.szorc@gmail.com> [Fri, 19 Apr 2019 05:07:44 -0700] rev 42275
automation: only iterate over our AMIs
We can't delete AMIs that we don't own. Iterating over other
AMIs won't work and slows down execution.
Differential Revision: https://phab.mercurial-scm.org/D6283
Martin von Zweigbergk <martinvonz@google.com> [Wed, 01 May 2019 15:34:03 -0700] rev 42274
remotefilelog: move most setup from onetimesetup() to uisetup()
All the wrappers moved in this patch check if remotefilelog is enabled
before they change behavior, so it's safe to always wrap.
Differential Revision: https://phab.mercurial-scm.org/D6334
Martin von Zweigbergk <martinvonz@google.com> [Wed, 01 May 2019 15:24:16 -0700] rev 42273
remotefilelog: move most functions in onetimeclientsetup() to top level
This is how most extensions seem to do it. It makes sure we don't
accidentally depend on the captured ui instance.
Differential Revision: https://phab.mercurial-scm.org/D6333
Martin von Zweigbergk <martinvonz@google.com> [Tue, 14 May 2019 09:46:38 -0700] rev 42272
tests: avoid the word "dirty" to mean "not a descendant of merge base"
The term "dirty" is no longer used in the code since 57203e0210f8
(copies: calculate mergecopies() based on pathcopies(), 2019-04-11).
Differential Revision: https://phab.mercurial-scm.org/D6373
Martin von Zweigbergk <martinvonz@google.com> [Wed, 01 May 2019 20:54:27 -0700] rev 42271
releasenotes: add a file in which to record release notes
I've just spent a few very boring hours going through the changelog
for the 5.0 release (829 commits). We only had 5 commits that used the
syntax that the release notes extension expects. This commit adds a
file in which we can record important changes. The file should
preferably be edited in the patch that makes the important change, but
it can also be edited after (I think this is an important benefit
compared to the release notes extension).
I'm thinking that we can rename the file from "next" to "5.1" or
something when it's time, and then we'd create a new "next" file on
the default branch.
I've used the syntax that we use on the our wiki in the template, but
I don't care much that we use any valid syntax at all. The idea is
mostly to record important changes when they happen. I expect that
some copy editing will be needed at release time anyway.
Differential Revision: https://phab.mercurial-scm.org/D6332
Matt Harbison <matt_harbison@yahoo.com> [Sat, 11 May 2019 22:08:57 -0400] rev 42270
record: avoid modifying the matcher passed as a method parameter
No problem observed, but I remember the previous pattern causing problems with
largefiles and/or subrepos. This special matcher was added in 419ac63fe29c, so
directly modifying the `fail` callback was probably an oversight in
44611ad4fbd9.
Differential Revision: https://phab.mercurial-scm.org/D6371
Augie Fackler <augie@google.com> [Sat, 04 May 2019 23:31:42 -0400] rev 42269
sslutil: add support for SSLKEYLOGFILE to wrapsocket
I recently learned of a Firefox/Chrome feature that allows
wiresharking otherwise-TLS'd network connections. Gloriously, there's
a pypi module that enables this same feature on Python, so let's add
support for it to Mercurial in case we need to wireshark some HTTPs
connections.
Differential Revision: https://phab.mercurial-scm.org/D6343
Ian Moody <moz-ian@perix.co.uk> [Sun, 05 May 2019 17:04:48 +0100] rev 42268
phabricator: add custom vcr matcher to match request bodies
Currently when the phabricator extension's conduit output changes the tests
don't notice since the default vcr matcher only matches on 'method' and 'uri',
not the body.
Add a custom matcher that checks the same params are in the body (ignoring
ordering).
vcr's in-built body matcher can't be used since it fails under py3 with a
"UnicodeEncodeError" on the "€ in commit message" tests.
The DREV ids have decreased since the recordings were generated against a
different phabricator instance to avoid spamming mercurial-devel.
Differential Revision: https://phab.mercurial-scm.org/D6347
Augie Fackler <augie@google.com> [Thu, 09 May 2019 18:37:37 -0400] rev 42267
merge with stable
Martin von Zweigbergk <martinvonz@google.com> [Wed, 08 May 2019 21:25:23 -0700] rev 42266
absorb: be more specific when erroring out on merge commit
When you have a merge commit checked out and run `hg absorb`, it would
tell you
abort: no mutable changeset to change
That makes it sound like the problem is public commits when isn't
really. Let's be more specific in this case.
There was already a test case that test this, so that now prints the
new message. I added a new test case that shows the old message (when
a public commit is checked out).
Differential Revision: https://phab.mercurial-scm.org/D6354
Augie Fackler <augie@google.com> [Wed, 08 May 2019 18:11:33 -0400] rev 42265
remotefilelog: log when we're about to fetch files
I'm debugging a slow client situation and knowing how many files are
in the batch request would be a nice thing.
Differential Revision: https://phab.mercurial-scm.org/D6353
Yuya Nishihara <yuya@tcha.org> [Tue, 30 Apr 2019 15:15:57 +0900] rev 42264
revset: populate wdir() by its hash or revision number
It belongs to the same category as the null hash/revision, and we do handle
these virtual identifiers in id()/rev() predicates. Let's do that more
consistently.
Augie Fackler <augie@google.com> [Wed, 08 May 2019 16:09:50 -0400] rev 42263
sslutil: fsencode path returned by certifi (issue6132)
By inspection, this is the only codepath that could be returning a
string instead of a bytes on Python 3.
Yuya Nishihara <yuya@tcha.org> [Tue, 30 Apr 2019 15:10:07 +0900] rev 42262
revset: extract private constant of {nullrev, wdirrev} set
I'll add a few more users of this constant to get around wdir identifiers.
Yuya Nishihara <yuya@tcha.org> [Tue, 30 Apr 2019 15:22:03 +0900] rev 42261
help: suggest merge() revset instead of -m/--only-merges
Suggested by Dr Rainer Woitok.
Martin von Zweigbergk <martinvonz@google.com> [Mon, 06 May 2019 22:06:23 -0700] rev 42260
tests: update annotate tests to work around simplemerge bug
test-annotate.t and test-fastannotate.hg were failing with --pure
since 57203e0210f8 (copies: calculate mergecopies() based on
pathcopies(), 2019-04-11). It turned out to be because the pure file
merge code behaved differently. I'm guessing it's the
mdiff.get_matching_blocks() that behaves differently, but I haven't
confirmed that.
With this content in the base:
a
a
a
And this on the local side:
a
z
a
And this on the other side:
a
a
a
b4
c
b6
It produced this conflict:
a
z
a
<<<<<<< working copy: b80e3e32f75a - test: c
||||||| base
a
=======
a
b4
c
b5
>>>>>>> merge rev: 64afcdf8e29e - test: mergeb
I don't care enough about the pure Python code to fix it, so this
patch just updates the tests to manually resolve the conflict.
Differential Revision: https://phab.mercurial-scm.org/D6351
Martin von Zweigbergk <martinvonz@google.com> [Tue, 07 May 2019 14:42:15 -0700] rev 42259
copies: delete misplaced comment
The comment was added in 78d760aa3607 (duplicatecopies: do not mark
items not in the dirstate as copies, 2013-03-28). It became misplaced
in 3666331164bb (cmdutil: add copy-filtering support to
duplicatecopies, 2014-06-07). Then the relevant code was moved far
away in 754b5117622f (context: add workingfilectx.markcopied,
2017-10-15).
Differential Revision: https://phab.mercurial-scm.org/D6352
Ian Moody <moz-ian@perix.co.uk> [Mon, 22 Apr 2019 18:55:27 +0100] rev 42258
phabricator: include branch in the phabread output
Depends on D6301
Differential Revision: https://phab.mercurial-scm.org/D6302
Ian Moody <moz-ian@perix.co.uk> [Mon, 22 Apr 2019 18:55:26 +0100] rev 42257
phabricator: fallback to reading metadata from diff for phabread
Differential Revision: https://phab.mercurial-scm.org/D6301
Ian Moody <moz-ian@perix.co.uk> [Sat, 20 Apr 2019 16:11:23 +0100] rev 42256
phabricator: include commit (node) and parent in the local:commits metadata
Differential Revision: https://phab.mercurial-scm.org/D6298
Martin von Zweigbergk <martinvonz@google.com> [Thu, 18 Apr 2019 00:34:45 -0700] rev 42255
copies: remove redundant filtering of ping-pong renames in _chain()
We already handle the ping-pong rename case in the filtering step, so
there's very little point in doing it in the chaining loop (ping-pong
renames are very rare, so I'm not worried about the cost of adding it
and then removing it again).
Differential Revision: https://phab.mercurial-scm.org/D6344
Augie Fackler <augie@google.com> [Fri, 03 May 2019 15:43:44 -0400] rev 42254
repair: reword comments that I noticed while working on source formatting
I think this is clearer, and one will also keep us from upsetting
check-code when other formatting cleanups happen.
Differential Revision: https://phab.mercurial-scm.org/D6339
Matt Harbison <matt_harbison@yahoo.com> [Mon, 06 May 2019 22:10:34 -0400] rev 42253
commit: allow --interactive to work again when naming a directory (issue6131)
Sietse Brouwer <sbbrouwer@gmail.com> [Fri, 26 Apr 2019 12:41:48 +0200] rev 42252
gendoc: nest command headers under category headers
Differential Revision: https://phab.mercurial-scm.org/D6329
Sietse Brouwer <sbbrouwer@gmail.com> [Fri, 26 Apr 2019 12:40:26 +0200] rev 42251
minirst: support subsubsubsubsections (header level 5) with marker ''''
Differential Revision: https://phab.mercurial-scm.org/D6328
Sietse Brouwer <sbbrouwer@gmail.com> [Fri, 03 May 2019 15:37:08 +0200] rev 42250
gendoc: guarantee that all commands were processed
The new logic renders the commands belonging to each category in turn.
Commands with an unregistered category are at risk of getting skipped
because their category is not in the list. By comparing the list of all
commands to a log of processed commands, we can detect commands with
unregistered categories and fail with an error message.
Differential Revision: https://phab.mercurial-scm.org/D6327
Sietse Brouwer <sbbrouwer@gmail.com> [Fri, 26 Apr 2019 17:53:01 +0200] rev 42249
gendoc: group commands by category in man page and HTML help
Make Mercurial's man page and HTML help group commands by category, and
present the categories in a helpful order. `hg help` already does this;
this patch uses the same metadata.
This patch uses the same header level for command categories and for
commands. A subsequent patch will push the command headers down one
level.
Differential Revision: https://phab.mercurial-scm.org/D6326
Sietse Brouwer <sbbrouwer@gmail.com> [Thu, 25 Apr 2019 19:15:17 +0200] rev 42248
gendoc: indent loop to make next patch more legible
Differential Revision: https://phab.mercurial-scm.org/D6325
Augie Fackler <augie@google.com> [Fri, 03 May 2019 15:53:56 -0400] rev 42247
contrib: have byteify-strings explode if run in Python 2
Differential Revision: https://phab.mercurial-scm.org/D6341
Augie Fackler <augie@google.com> [Fri, 03 May 2019 15:46:09 -0400] rev 42246
repair: reword comment about bookmarks logic
Again, this will help auto-formatting shortly.
Differential Revision: https://phab.mercurial-scm.org/D6340
Augie Fackler <augie@google.com> [Fri, 03 May 2019 15:42:13 -0400] rev 42245
monotone: fix a bogus _() wrapper that was caught when formatting code
There was a spurious space after `debug`, which hid the _() inside
ui.debug() from check-code. Sigh.
While here, wrap things more concisely.
Differential Revision: https://phab.mercurial-scm.org/D6338
Anton Shestakov <av6@dwimlabs.net> [Fri, 03 May 2019 14:11:16 +0800] rev 42244
commit: add ability to print file status after each successful invocation
When commands.commit.post-status is enabled, `hg commit` will effectively run
`hg status -mardu` after committing. It can help catch mistakes like not
committing all needed files or not adding unknown files that should've been
part of the just created commit.
Anton Shestakov <av6@dwimlabs.net> [Fri, 03 May 2019 14:07:14 +0800] rev 42243
tests: flatten repo structure in test-commit.t
Let's move to parent directory before `hg init` repos, since they don't need to
be nested. It makes amend/strip messages that include full path to the backup
bundle shorter, for instance.
Matt Harbison <matt_harbison@yahoo.com> [Sat, 04 May 2019 01:16:42 -0400] rev 42242
lfs: add a TODO file
This is a cleaned up and reorganized list of items I sent out about a year ago.
But tracking this in the repo (like the narrow extension) gives more visibility
in case anyone wants to help out.
Martin von Zweigbergk <martinvonz@google.com> [Sat, 27 Apr 2019 22:08:45 -0700] rev 42241
copies: make "limit" argument to _tracefile() mandatory
We always pass a limit. I think the fact that it was optional was also
the reason we checked ">=limit" before we used it. So now we can
remove that condition too.
Differential Revision: https://phab.mercurial-scm.org/D6335
Martin von Zweigbergk <martinvonz@google.com> [Fri, 03 May 2019 08:37:10 -0700] rev 42240
localrepo: don't use defaults arguments that will never be overridden
The commithook() callback will be called when the lock is
released. lock.release() calls the callback without arguments, so it
was quite confusing to me that this function declared extra
arguments. We can just close on the variables in the outer scope
instead.
Differential Revision: https://phab.mercurial-scm.org/D6336
Martin von Zweigbergk <martinvonz@google.com> [Fri, 03 May 2019 12:32:00 -0700] rev 42239
tags: avoid double-reversing a list
Differential Revision: https://phab.mercurial-scm.org/D6337
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 11 Mar 2019 02:35:18 +0100] rev 42238
updatecaches: also warm hgtagsfnodescache
Now that a full update of this cache run in a reasonable amount of time, we can
warm everything when during a full update.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 11 Mar 2019 01:10:20 +0100] rev 42237
hgtagsfnodescache: inherit fnode from parent when possible
If a changeset does not update the content of `.hgtags`, it means it will use
the same file-node (for `.hgtags`) as its parents. In this case we can
directly reuse the parent's file-node.
We use this property when updating the `hgtagsfnodescache` taking a faster path
if we already have a cached value for the parents of the node we are looking
at.
Doing so provides a large performance boost when looking at a lot of fnodes,
especially on repository with very large manifest:
timing for `tagsmod.fnoderevs(ui, repo, repo.changelog.revs())`
mercurial: (41907 revisions, 1923 files)
before: 6.9 seconds
after: 2.7 seconds (-54%)
pypy: (96266 revisions, 5198 files)
before: 80 seconds
after: 20 seconds (-75%)
mozilla-central: (463411 revisions, 272080 files)
before: 7166.4 seconds
after: 47.8 seconds (-99%, x150 speedup)
On a copy of mozilla-try with about 35K heads ans 1.7M changesets, this moves
the computation from many hours to a couple of minutes, making it more
interesting to do a full warm up of this cache before computing tags (from a
cold cache).
There seems to be other performance low hanging fruits, like avoiding the use of
changectx or a more revision centric logic. However, the new code is fast enough
for my needs right now.
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 11 Mar 2019 01:09:38 +0100] rev 42236
hgtagsfnodescache: handle nullid lookup
The null revision is empty, so it `.hgtags` content is `nullid` in regards with
the `hgtagsfnodescache`. Dealing with `nullid` will help with the next
changeset. Before this change, feeding `nullid` to `hgtagsfnodescache.getfnode` would
return a wrong result (fnode for tip).
Sietse Brouwer <sbbrouwer@gmail.com> [Fri, 26 Apr 2019 17:39:07 +0200] rev 42235
help: register the 'gpg' command category and give it a description
help.py expects extensions to register their command category in the
CATEGORY_ORDER and CATEGORY_NAMES variables. Once gendoc.py orders
commands by category, in the next patch, it'll assume this registration
(and raise an exception on encountering any unregistered categories).
Luckily, gpg is the only bundled extension with an unregistered custom
category, so let's fix it.
Differential Revision: https://phab.mercurial-scm.org/D6324
feyu@google.com [Thu, 25 Apr 2019 15:30:40 -0700] rev 42234
histedit: Speed up scrolling in patch view mode
Store patchcontents into the mode state, avoiding the expensive
call to ui for computing the patchcontents.
Before this change in large repos histedit patch view mode can
be very irresponsive.
Yu Feng <rainwoodman@gmail.com> [Thu, 02 May 2019 16:43:34 -0700] rev 42233
histedit: Show file names in multiple line format
Yuya Nishihara <yuya@tcha.org> [Fri, 03 May 2019 20:06:03 +0900] rev 42232
parser: fix crash by parsing "()" in keyword argument position
A tree node can be either None or a tuple because x=("group", None) is
reduced to x[1].
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 06 Apr 2019 17:46:19 +0200] rev 42231
repoview: introduce a `experimental.extra-filter-revs` config
The option define revisions to additionally filter out of all repository "view".
The end goal is to provide and easy to way to serve multiple subset of the same
repository using multiple "shares".
The simplest use case of this feature is to have one view serving the public
changesets and one view also serving the draft. This is currently achievable
using the new `server.view` option introduced recently by Joerg Sonnenberger.
However, more advanced use cases need more advanced definitions. For example
some needs a view dedicated to some release branches, or view that hides
security fixes to be released. Joerg Sonnenberger and I discussed this topic at
the recent mini-sprint and the both of us have seen real life use cases for
this. (This series got written during the same mini-sprint).
The feature is fully functional, and use similar cache-fallback mechanism to
ensure decent performance. However,there remaining room to ensure each share
caches and hooks collaborate with each others. This will come at a later time
once users start to actually test this feature on real usecase.
Martin von Zweigbergk <martinvonz@google.com> [Wed, 17 Apr 2019 23:10:29 -0700] rev 42230
copies: filter out copies from non-existent source later in _chain()
_changesetforwardcopies() repeatedly calls _chain(). That is very
expensive because _chain() does lookups in the manifest. I hope to
split up the function in two parts: 1) simple chaining, not
considering end points, and 2) filter out files that don't exist in
the end points (and ping-pong copies/renames).
This patches gets us closer to that by moving the check for
non-existent source later in the function. Now there are no more
checks for "src" and "dst" in the first loop; all the filtering of
invalid copies is done in the second loop. The code also looks much
more consistent now.
No measureable impact on `hg debugpathcopies 4.0 4.8`. That shouldn't
be surprising since the only case we're doing more checks now is in
case of chained copies/renames, which are quire rare in practice.
Differential Revision: https://phab.mercurial-scm.org/D6277
Martin von Zweigbergk <martinvonz@google.com> [Thu, 18 Apr 2019 00:12:56 -0700] rev 42229
copies: clarify mutually exclusive cases in _chain() with a s/if/elif/
If the 'b' dict has a rename from 'x' to 'y', it shouldn't be possible
for 'x' to be both (a key) in 'a' and in 'src'. That would mean that
'x' is a file in the source commit and also a rename destination in
the intermediate commit. But we currently don't allow renaming files
onto existing files, so that shouldn't happen. So let's clarify that
by using an "elif" instead of an "if". And if we did allow renaming
files onto existing files, we should prefer to use the rename
destination in the intermediate commit as source anyway.
Differential Revision: https://phab.mercurial-scm.org/D6276
Martin von Zweigbergk <martinvonz@google.com> [Thu, 18 Apr 2019 00:05:05 -0700] rev 42228
copies: delete a redundant cleanup step in _chain()
The check is redundant since d5edb5d3a337 (copies: filter out copies
when target is not in destination manifest, 2019-02-14). To test that
hypothesis, I made this change in the commit that commit, but all
tests still passed. I think the case was necessary before then, we
just didn't have tests for it.
Differential Revision: https://phab.mercurial-scm.org/D6275
Martin von Zweigbergk <martinvonz@google.com> [Wed, 17 Apr 2019 23:10:14 -0700] rev 42227
copies: document cases in _chain()
Differential Revision: https://phab.mercurial-scm.org/D6274
Martin von Zweigbergk <martinvonz@google.com> [Wed, 17 Apr 2019 14:44:18 -0700] rev 42226
copies: ignore heuristics copytracing when using changeset-centric algos
Differential Revision: https://phab.mercurial-scm.org/D6269
Martin von Zweigbergk <martinvonz@google.com> [Wed, 17 Apr 2019 14:42:23 -0700] rev 42225
copies: move check for experimental.copytrace==<falsy> earlier
I'm going to ignore experimental.copytrace when changeset-centric
algorithms are required. This little refactoring makes that easier to
add.
Differential Revision: https://phab.mercurial-scm.org/D6268
Martin von Zweigbergk <martinvonz@google.com> [Wed, 17 Apr 2019 14:11:54 -0700] rev 42224
copies: replace .items() by .values() where appropriate
As pointed out by Pierre-Yves.
Differential Revision: https://phab.mercurial-scm.org/D6266
Martin von Zweigbergk <martinvonz@google.com> [Fri, 12 Apr 2019 10:44:37 -0700] rev 42223
copies: inline _computenonoverlap() in mergecopies()
We now call pathcopies() from the base to each of the commits, and
that calls _computeforwardmissing(), which does file prefetching (in
the remotefilelog override). So the call to _computenonoverlap() is
now pointless (the sets of files from _computenonoverlap() are subsets
of the sets of files from _computeforwardmissing()).
This somehow also fixes a broken remotefilelog test.
Differential Revision: https://phab.mercurial-scm.org/D6256
Martin von Zweigbergk <martinvonz@google.com> [Thu, 11 Apr 2019 23:22:54 -0700] rev 42222
copies: calculate mergecopies() based on pathcopies()
When copies are stored in changesets, we need a changeset-centric
version of mergecopies() just like we have a changeset-centric version
of pathcopies(). I think the natural way of thinking about
mergecopies() is in terms of pathcopies() from the base to each of the
commits. So if we can rewrite mergecopies() based on two such
pathcopies() calls, we'll get the changeset-centric version for
free. That's what this patch does.
A nice bonus is that it ends up being a lot simpler. mergecopies() has
accumulated a lot of technical debt over time. One good example is the
code for dealing with grafts (the "partial/incomplete/dirty"
stuff). Since pathcopies() already deals with backwards renames and
ping-pong renames, we get that for free.
I've run tests with hard-coded debug logging for "fullcopy" and while
I haven't looked at every difference it produces, all the ones I have
looked at seemed reasonable to me. I'm a little surprised that no more
tests fail when run with '--extra-config-opt
experimental.copies.read-from=compatibility' compared to before this
patch. This patch also fixes the broken cases in test-annotate.t and
test-fastannotate.t. It also enables the part of test-copies.t that
was previously disabled exactly because mergecopies() needed to get a
changeset-centric version.
One drawback of the rewritten code is that we may now make
remotefilelog prefetch more files. We used to prefetch files that were
unique to either side of the merge compared to the other. We now
prefetch files that are unique to either side of the merge compared to
the base. This means that if you added the same file to each side, we
would not prefetch it before, but we would now. Such cases are
probably quite rare, but one likely scenario where they happen is when
moving from a commit to its successor (or the other way around). The
user will probably already have the files in the cache in such cases,
so it's probably not a big deal.
Some timings for calculating mergecopies between two revisions
(revisions shown on each line, all using the common ancestor as base):
In the hg repo:
4.8 4.9: 0.21s -> 0.21s
4.0 4.8: 0.35s -> 0.63s
In and old copy of the mozilla-unified repo:
FIREFOX_BETA_60_BASE^ FIREFOX_BETA_60_BASE: 0.82s -> 0.82s
FIREFOX_NIGHTLY_59_END FIREFOX_BETA_60_BASE: 2.5s -> 2.6s
FIREFOX_BETA_59_END FIREFOX_BETA_60_BASE: 3.9s -> 4.1s
FIREFOX_AURORA_50_BASE FIREFOX_BETA_60_BASE: 31s -> 33s
So it's measurably slower in most cases. The most significant
difference is in the hg repo between revisions 4.0 and 4.8. In that
case it seems to come from the fact that pathcopies() uses
fctx.isintroducedafter() (in _tracefile), while the old mergecopies()
used fctx.linkrev() (in _checkcopies()). That results in a single call
to filectx._adjustlinkrev(), which is responsible for the entire
difference in time (in my repo). So we pay a performance penalty but
we get more correct code (see change in
test-mv-cp-st-diff.t). Deleting the "== f.filenode()" in _tracefile()
recovers the lost performance in the hg repo.
There were are few other optimizations in _checkcopies() that I could
not measure any impact from. One was from the "seen" set. Another was
from a "continue" when the file was not in the destination manifest
(corresponding to "am" in _tracefile).
Also note that merge copies are not calculated when updating with a
clean working copy, which is probably the most common case. I
therefore think the much simpler code is worth the slowdown.
Differential Revision: https://phab.mercurial-scm.org/D6255
Martin von Zweigbergk <martinvonz@google.com> [Mon, 29 Apr 2019 14:38:54 -0700] rev 42221
tests: add test where copy source is deleted and added back
This shows another difference between pathcopies() and mergecopies():
mergecopies() considers files that have been deleted and then added
back as different files, but pathcopies() does not.
Differential Revision: https://phab.mercurial-scm.org/D6330
Augie Fackler <augie@google.com> [Wed, 01 May 2019 14:30:25 -0400] rev 42220
merge with stable
Augie Fackler <raf@durin42.com> [Wed, 01 May 2019 14:27:19 -0400] rev 42219
Added signature for changeset 07e479ef7c96
Augie Fackler <raf@durin42.com> [Wed, 01 May 2019 14:27:17 -0400] rev 42218
Added tag 5.0 for changeset 07e479ef7c96
Matt Harbison <matt_harbison@yahoo.com> [Mon, 29 Apr 2019 23:00:42 -0400] rev 42217
obsolete: drop the legacy `_enabled` variable
Evolve 8.5.0 stopped setting this, and it would have been easier to figure out
why TortoiseHg stopped allowing amends if it would have crashed on the missing
variable.
Pulkit Goyal <pulkit@yandex-team.ru> [Sat, 27 Apr 2019 14:43:43 +0300] rev 42216
discovery: only calculate closed branches if required
The number of new closed branches is required for printing in error message. So
let's only calculate them if we need to print error about new branches.
Differential Revision: https://phab.mercurial-scm.org/D6314
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 25 Apr 2019 19:17:02 +0200] rev 42215
hghave: deal with "rc" release
Without this change, 5.0rc0 is not recognised as 5.0
Pulkit Goyal <pulkit@yandex-team.ru> [Sat, 27 Apr 2019 02:13:43 +0300] rev 42214
branchcache: store the maximum tip in a variable inside for loop
Instead of assigning self.tiprev multiple times in the for loop, and calling
cl.node() on it, let's store that in a temporary variable and assign it in the
end of loop.
Differential Revision: https://phab.mercurial-scm.org/D6311
Martin von Zweigbergk <martinvonz@google.com> [Sat, 27 Apr 2019 23:30:19 -0700] rev 42213
tests: demonstrate that rename is followed to wrong parent from merge
This test case shows another way that copies are handled differently
between `hg st` (pathcopies()) and `hg co -m` (mergecopies). The
reason is that pathcopies() calls _tracefiles(), which checks that the
file nodeid of an ancestor matches the file nodeid in the base
commit. mergecopies() should probably be doing the same.
Differential Revision: https://phab.mercurial-scm.org/D6323
Martin von Zweigbergk <martinvonz@google.com> [Sat, 27 Apr 2019 23:14:49 -0700] rev 42212
test: demonstrate failure to follow rename with shadowed linkrev
This shows a difference in handling of copies between `hg st`
(pathcopies()) and `hg co -m`. The issue here is that mergecopies()
uses the unadjusted linkrev() for determining when to stop walking
ancestors.
Differential Revision: https://phab.mercurial-scm.org/D6322
Martin von Zweigbergk <martinvonz@google.com> [Sat, 27 Apr 2019 22:57:15 -0700] rev 42211
tests: slightly modify a linkrev test to prepare for expanding it
The test case checks that the copy tracing code doesn't get confused
by linkrevs when walking a file's ancestors. This patch chnages the
test slightly so a second commit is grafted, thus producing a second
"bad" linkrev. I'll use this in the next patch to demonstrate a bug.
Differential Revision: https://phab.mercurial-scm.org/D6321
Martin von Zweigbergk <martinvonz@google.com> [Sat, 27 Apr 2019 22:55:54 -0700] rev 42210
copies: process files in deterministic order for stable tests
I also fixed a typo while at it.
Differential Revision: https://phab.mercurial-scm.org/D6320
Pulkit Goyal <pulkit@yandex-team.ru> [Wed, 17 Apr 2019 15:06:41 +0300] rev 42209
narrow: send specs as bundle2 data instead of param (issue5952) (issue6019)
Before this patch, when ACL is involved, narrowspecs are send as bundle2
parameter for narrow:spec bundle2 part. The limitation of bundle2 parts are they
cannot send data larger than 255 bytes. Includes and excludes in narrow are not
limited by size and they can grow over 255 bytes.
This patch introduces a new mandatory bundle2 part and send narrowspecs as data
of that. The new bundle2 part is introduced to keep things cleaner and easy to
distinguish related to backward compatibility.
The part is mandatory because without server's narrowspec, the local ACL narrow
repo won't work.
This patch makes clients compatible with servers which have older versions.
However I left a comment that we should drop the other bundle2 part soon as
that's broken and people should not rely on that.
I named the new bundle2 part 'Narrow:responsespec' because:
1) Capital 'N' to make it mandatory
2) 'Narrow:spec' cannot be used because bundle2 enforces that there should not
be two different parts which resolve to same name when lowercased.
3) reponsespec clears that they are specs which are send as reponse by the
server
While I was here, I renamed `narrowhgacl` section to `narrowacl` as suggested by
idlsoft@ and martinvonz@.
Differential Revision: https://phab.mercurial-scm.org/D6310
Ludovic Chabant <ludovic@chabant.com> [Fri, 19 Apr 2019 14:26:32 +0000] rev 42208
py3: properly reject non-encoded strings given to hgweb
Ludovic Chabant <ludovic@chabant.com> [Fri, 19 Apr 2019 14:25:18 +0000] rev 42207
py3: handle meta-path finders that only use pre-python3.4 API
Danny Hooper <hooper@google.com> [Fri, 26 Apr 2019 17:41:22 -0700] rev 42206
remotefilelog: add missing argument to hg.verify wrapper
Differential Revision: https://phab.mercurial-scm.org/D6313
Boris Feld <boris.feld@octobus.net> [Thu, 24 Jan 2019 09:03:15 -0500] rev 42205
revsetbenchmark: track some simple use of "only"
The only revset is quite useful and has various possible optimisation. tracking
its timing seems useful.
Matt Harbison <matt_harbison@yahoo.com> [Fri, 26 Apr 2019 23:52:49 -0400] rev 42204
inno: bump keyring to 18.0.1 to avoid AttributeError (issue6043)
The error seems to be harmless, because it happens after closing the connection.
For whatever reason, this isn't bundled with the Wix installer.
https://github.com/jaraco/keyring/issues/386
https://bitbucket.org/Mekk/mercurial_keyring/issues/63/attributeerror-during-process-finish-with
Taapas Agrawal <taapas2897@gmail.com> [Fri, 01 Mar 2019 05:56:18 +0530] rev 42203
push: added clear warning message when pushing closed branches(issue6080)
Differential Revision: https://phab.mercurial-scm.org/D6038
Sushil khanchi <sushilkhanchi97@gmail.com> [Tue, 16 Apr 2019 02:06:20 +0530] rev 42202
branch: abort if closing branch from a non-branchhead cset
This patch make sure that we abort if the user is trying to
close a branch from a cset which is not a branch head.
Changes in test file reflect the fixed behaviour.
Differential Revision: https://phab.mercurial-scm.org/D6282
Sushil khanchi <sushilkhanchi97@gmail.com> [Tue, 16 Apr 2019 01:19:58 +0530] rev 42201
branch: add tests which shows branch can be closed from a non-branchhead cset
This patch shows that we can close a branch even from a cset which is not
a branch head. It was supposed to abort this operation.
Next patch will be fixing the issue.
Differential Revision: https://phab.mercurial-scm.org/D6281
Ian Moody <moz-ian@perix.co.uk> [Sat, 20 Apr 2019 17:27:24 +0100] rev 42200
phabricator: read more metadata from local:commits
local:commits metadata can contain branch info, and 'rev' has been superseded
by 'commit', see:
https://github.com/phacility/arcanist/blob/83661809e532c3fe444a8bf7c7d6936e6377691b/src/repository/api/ArcanistMercurialAPI.php#L281
Differential Revision: https://phab.mercurial-scm.org/D6300
Ian Moody <moz-ian@perix.co.uk> [Sat, 20 Apr 2019 17:22:35 +0100] rev 42199
phabricator: don't assume the existence of properties of local:commits
Not all the properties are guaranteed to be there, so if we don't check first
we could die with a KeyError.
Differential Revision: https://phab.mercurial-scm.org/D6299
Ian Moody <moz-ian@perix.co.uk> [Sat, 20 Apr 2019 16:01:47 +0100] rev 42198
phabricator: include branch in the diffproperty metadata
This does not make Phabricator display the branch in web UI anywhere as that
still need us to use creatediff API for that. However a future patch will make
phabread use this to include the branch in its `hg import`-able output.
Differential Revision: https://phab.mercurial-scm.org/D6297
Martin von Zweigbergk <martinvonz@google.com> [Wed, 24 Apr 2019 10:47:40 -0700] rev 42197
tests: demonstrate `hg log -r . <file>` linkrev bug
Differential Revision: https://phab.mercurial-scm.org/D6309
Joerg Sonnenberger <joerg@bec.de> [Fri, 19 Apr 2019 20:06:37 +0200] rev 42196
unionrepo: sync with repository API
Differential Revision: https://phab.mercurial-scm.org/D6289
Martin von Zweigbergk <martinvonz@google.com> [Tue, 23 Apr 2019 08:39:26 -0700] rev 42195
match: remove unused match.__iter__ implementation (API)
Differential Revision: https://phab.mercurial-scm.org/D6305
Danny Hooper <hooper@google.com> [Thu, 21 Mar 2019 18:32:45 -0700] rev 42194
fix: allow fixer tools to return metadata in addition to the file content
With this change, fixer tools can be configured to output a JSON object that
will be parsed and passed to hooks that can be used to print summaries of what
code was formatted or perform other post-fixing work.
The motivation for this change is to allow parallel executions of a
"meta-formatter" tool to report back statistics, which are then aggregated and
processed after all formatting has completed. Providing an extensible mechanism
inside fix.py is far simpler, and more portable, than trying to make a tool
like this communicate through some other channel.
Differential Revision: https://phab.mercurial-scm.org/D6167
Pulkit Goyal <pulkit@yandex-team.ru> [Wed, 24 Apr 2019 19:42:43 +0300] rev 42193
context: check file exists before getting data from _wrappedctx
overlayworkingctx class is used to do in-memory merging. The data() function of
that class has logic to look for data() in the wrappedctx if the file data in
cache is empty and if the file is dirty. This assumes that if a file is dirty
and cache has empty data for it, it will exists in the _wrappedctx.
However this assumption can be False in case when we are merging a file which is
empty in destination. In these cases, the backup file 'foo.orig' created by our
internal merge algorithms will be empty, however it won't be present in
_wrappedctx. This case will lead us to error like the one this patch is fixing.
Let's only fallback to getting data from wrappedctx if cache has 'None' as data.
Differential Revision: https://phab.mercurial-scm.org/D6308
Pulkit Goyal <pulkit@yandex-team.ru> [Wed, 24 Apr 2019 19:28:46 +0300] rev 42192
tests: show IMM is broken when merging file empty in destination
When we are doing in-memory merging, and we are merging a file which is empty in
merge destination, it leads to error 'abort: xxx not found in manifest'.
Next patch will fix this error.
Differential Revision: https://phab.mercurial-scm.org/D6307
Antonio Muci <a.mux@inwind.it> [Fri, 19 Apr 2019 02:24:25 +0200] rev 42191
buildrpm: bump bundled Python version to 2.7.16 when building for centos{5,6}
When building rpm packages for centos 5 and 6, we bundle a mercurial-specific
version of Python 2.7 in /opt/python-hg.
This change is analogous to 5e947367606c, and bumps the embedded Python version
from 2.7.14 (released in 2017) to 2.7.16 (latest as of today).
Augie Fackler <augie@google.com> [Tue, 23 Apr 2019 15:49:17 -0400] rev 42190
merge with stable
Ian Moody <moz-ian@perix.co.uk> [Mon, 22 Apr 2019 17:46:57 +0100] rev 42189
phabricator: set local:commits time metadata as an int, not a string
Same as arcanist does
Differential Revision: https://phab.mercurial-scm.org/D6296
Ian Moody <moz-ian@perix.co.uk> [Mon, 22 Apr 2019 17:46:01 +0100] rev 42188
phabricator: use templatefilters.json in writediffproperties
Instead of json.dumps, since it makes the code simpler and more readable.
This would have been the better option for 8fd19a7b4ed6 but I wasn't aware of
it at the time.
Differential Revision: https://phab.mercurial-scm.org/D6295
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 21 Apr 2019 09:34:16 -0700] rev 42187
commands: use byteskwargs() in verify()
Otherwise Python 3 complains about the missing key.
Differential Revision: https://phab.mercurial-scm.org/D6294
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 21 Apr 2019 09:29:55 -0700] rev 42186
match: use raw strings to avoid illegal baskslash escape
Python 3.8 was complaining about the invalid escape
sequences. Let's use raw strings to avoid the warning and
double baskslashes.
Differential Revision: https://phab.mercurial-scm.org/D6293
Pulkit Goyal <pulkit@yandex-team.ru> [Sat, 20 Apr 2019 00:48:16 +0300] rev 42185
revbranchcache: use context manager in _writerevs() to write to file
The other _writenames() is a bit complicated to use context manager.
Differential Revision: https://phab.mercurial-scm.org/D6292
Pulkit Goyal <pulkit@yandex-team.ru> [Sat, 20 Apr 2019 00:44:18 +0300] rev 42184
revbranchcache: factor logic to write names and revs in separate functions
Before this patch, the write function was so populated with upto 4 level of
indentation, it was hard to understand what's going on.
Differential Revision: https://phab.mercurial-scm.org/D6291
Martin von Zweigbergk <martinvonz@google.com> [Thu, 18 Apr 2019 22:16:33 -0700] rev 42183
tests: make log style a little easier to read in test-copytrace-heuristics.t
Revision numbers are much shorter and easier to read (especially
compared to the full nodeids that were used here), so I switched to
that. That's also what almost all the commands used (e.g. `hg rebase
-s . -d 1`). I updated the two instances that used nodeids. I also
made some other little cleanups to the log templates.
Differential Revision: https://phab.mercurial-scm.org/D6279
Martin von Zweigbergk <martinvonz@google.com> [Thu, 18 Apr 2019 22:23:26 -0700] rev 42182
tests: avoid cryptic nodeids in tests/test-rename-merge1.t
These two nodeids had not been part of any output before, so one can't
know which revision they refer to without adding something like `hg
log` before them. It turned out that '.^' was equivalent for both of
them, so that's what I replaced them with.
Differential Revision: https://phab.mercurial-scm.org/D6280
Martin von Zweigbergk <martinvonz@google.com> [Thu, 18 Apr 2019 22:08:58 -0700] rev 42181
tests: defines aliases for `hg log` calls in test-copytrace-heuristics.t
This also makes the test cases more consistent since a few had missed
the ":" in "changeset:" that the others used.
Differential Revision: https://phab.mercurial-scm.org/D6278
Georges Racinet <georges.racinet@octobus.net> [Thu, 14 Mar 2019 17:57:31 +0000] rev 42180
rust-discovery: implementing and exposing stats()
This time, it's simple enough that we can do it in all layers in
one shot.
Differential Revision: https://phab.mercurial-scm.org/D6233
Georges Racinet <georges.racinet@octobus.net> [Wed, 20 Feb 2019 09:04:39 +0100] rev 42179
rust-discovery: cpython bindings for the core logic
As previously done with the ancestors submodule, testing for
the bindings is provided from Python on a trivial case.
Differential Revision: https://phab.mercurial-scm.org/D6232
Georges Racinet <georges.racinet@octobus.net> [Tue, 19 Feb 2019 23:42:31 +0100] rev 42178
rust-discovery: starting core implementation
Once exposed to the Python side, this core object will avoid
costly roundtrips with potentially big sets of revisions.
This changeset implements the core logic of the object only, i.e.,
manipulation of the missing, common and undefined set-like revision
attributes.
Differential Revision: https://phab.mercurial-scm.org/D6231
Georges Racinet <georges.racinet@octobus.net> [Wed, 20 Feb 2019 18:33:53 +0100] rev 42177
rust-dagops: roots
Unsuprisingly, the algorithm is much easier than for heads, provided
we work on a set in the first place.
To improve the signature, a trait for set-likes object would be useful,
but that's not an immediate concern.
Differential Revision: https://phab.mercurial-scm.org/D6230
Georges Racinet <georges.racinet@octobus.net> [Tue, 19 Feb 2019 23:41:57 +0100] rev 42176
rust-dagops: range of revisions
This is a Rust implementation for what reachableroots2() does if
includepath is True.
The algorithmic details and performance notes are included in the
documentation comment.
Our main use case for now is a Rust counterpart of the partialdiscovery
object, so we don't really need bindings yet.
Differential Revision: https://phab.mercurial-scm.org/D6229
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 21 Apr 2019 08:57:01 -0700] rev 42175
setup: tweak error message for Python 3
We now have beta support for Python 3. In my opinion, it isn't
yet stable enough to allow `pip install Mercurial` to work with
Python 3 out of the box: we don't want people accidentally using
Mercurial with Python 3 just yet.
But I do think we should be more friendly about informing people
of their options.
This commit tweaks the error message that users see when running
setup.py with Python 3. We instruct them about the current level
of Python 3 support, point them at the wiki for more info, and
give them instructions on how to bypass the check.
As part of this, I also changed which version value is printed,
as we were printing a named tuple before.
Gregory Szorc <gregory.szorc@gmail.com> [Sun, 21 Apr 2019 07:21:08 -0700] rev 42174
setup: remove set and dict comprehensions
Yuya observed in a recent review that it is worthwhile to keep
setup.py parseable with Python 2.6 so a useful error message is
seen when attempting to run with Python 2.6.
This commit removes a set and dict comprehension so setup.py
is parseable with Python 2.6.
Pulkit Goyal <pulkit@yandex-team.ru> [Fri, 19 Apr 2019 23:13:28 +0300] rev 42173
branchcache: don't verify all nodes while writing
nodes are verified either when they are added or used. In case of commits. we
will load the whole branchmap, only verify nodes for the branch on which we are
committing and then we write.
However before this patch, writing the branchmap was validating all the nodes
whereas it should not. This patch fixes that.
Differential Revision: https://phab.mercurial-scm.org/D6290
Gregory Szorc <gregory.szorc@gmail.com> [Sat, 20 Apr 2019 07:29:07 -0700] rev 42172
setup: properly package distutils in py2exe virtualenv builds
Our in-repo py2exe packaging code uses virtualenvs for managing
dependencies. An advantage of this is that packaging is more
deterministic and reproducible. Without virtualenvs, we need to
install packages in the system Python install. Packages installed
by other consumers of the system Python could leak into the Mercurial
package.
A regression from this change was that py2exe packages contained
the virtualenv's hacked distutils modules instead of the original
distutils modules. (virtualenv installs a hacked distutils module
because distutils uses relative path lookups that fail when running
from a virtualenv.)
This commit introduces a workaround so py2exe packaging uses the
original distutils modules when running from a virtualenv.
With this change, `import distutils` no longer fails from py2exe
builds produced from a virtualenv. This fixes the regression.
Furthermore, we now include all distutils modules. Before, py2exe's
module finding would only find modules there were explicitly
referenced in code. So, we now package a complete copy of distutils
instead of a partial one. This is even better than before.
# no-check-commit foo_bar function name
Martin von Zweigbergk <martinvonz@google.com> [Wed, 17 Apr 2019 10:49:11 -0700] rev 42171
narrow: also warn when not deleting untracked or ignored files
Differential Revision: https://phab.mercurial-scm.org/D6265
Joerg Sonnenberger <joerg@bec.de> [Wed, 17 Apr 2019 14:37:06 +0200] rev 42170
setdiscovery: fix a few typos
Differential Revision: https://phab.mercurial-scm.org/D6263
Martin von Zweigbergk <martinvonz@google.com> [Mon, 15 Apr 2019 14:09:18 -0700] rev 42169
copies: delete debug message about "unmatched files new in both"
Same reasoning as previous patch.
Differential Revision: https://phab.mercurial-scm.org/D6251
Martin von Zweigbergk <martinvonz@google.com> [Fri, 12 Apr 2019 21:41:51 -0700] rev 42168
copies: delete debug message about changes since common ancestor
Same reasoning as previous patch.
Differential Revision: https://phab.mercurial-scm.org/D6250
Martin von Zweigbergk <martinvonz@google.com> [Thu, 11 Apr 2019 23:28:38 -0700] rev 42167
copies: delete debug message about search limit
I'm about to rewrite mergecopies() and this message will no longer be
emitted then. Let's remove the message now to remove a distraction
from that patch.
Differential Revision: https://phab.mercurial-scm.org/D6249
Martin von Zweigbergk <martinvonz@google.com> [Mon, 15 Apr 2019 22:58:10 -0700] rev 42166
copies: move early return for "no copies" case a little earlier
We can return before the block that prints debug messages. That block
will not be run anyway when there are no copies.
Differential Revision: https://phab.mercurial-scm.org/D6248
Martin von Zweigbergk <martinvonz@google.com> [Mon, 15 Apr 2019 16:46:41 -0700] rev 42165
copies: fix up "fullcopy" with missing entries from "diverge"
Similar to the previous patch, but this doesn't even affect tests. It
does affect tests if you change them to turn on debug logging. I'm
fixing it here so reviewers of the later rewrite patch can hard-code
debug logging to be on and more easily compare test results.
Differential Revision: https://phab.mercurial-scm.org/D6247
Martin von Zweigbergk <martinvonz@google.com> [Mon, 15 Apr 2019 16:41:43 -0700] rev 42164
copies: fix up "fullcopy" with missing entries from "copy"
This is just a workaround similar to the previous one. It will make it
easier to follow later patches.
Differential Revision: https://phab.mercurial-scm.org/D6246
Martin von Zweigbergk <martinvonz@google.com> [Sun, 14 Apr 2019 00:46:25 -0700] rev 42163
merge: remove workaround for issue5020
As I explained in the previous commit, I think the filtering added
there is a better fix for the issue, so the workaround from
41f6af50c0d8 (merge: fix crash on criss cross merge with dir move and
delete (issue5020), 2017-01-31) should no longer be needed.
Differential Revision: https://phab.mercurial-scm.org/D6245
Martin von Zweigbergk <martinvonz@google.com> [Fri, 12 Apr 2019 22:03:04 -0700] rev 42162
copies: don't include copies that are not in source in directory move
I've been working on a rewrite of mergecopies(). I compared the output
of the rewritten version with the current version. I noticed that
between FIREFOX_NIGHTLY_59_END and FIREFOX_BETA_60_BASE in the
mozilla-unified repo, there were many copies that the current version
detected that the rewritten version did not. One example was
js/src/gc/Iteration.h -> js/src/gc/PublicIterators.h. Then I realized
that js/src/gc/Iteration.h doesn't even exist in
FIREFOX_NIGHTLY_59_END.
This patch adds a filtering step for the "fullcopy" dict. It turns out
that that change also affects the test for issue5020 in
test-merge-criss-cross.t. The 'dm' action no longer happens there. At
first I thought that the test case change meant that this patch was
broken, but I think it's actually correct tha the 'dm' action should
not happen there. The result of the bid merge is still the same.
I suspect this filtering is a better solution for the issue than
41f6af50c0d8 (merge: fix crash on criss cross merge with dir move and
delete (issue5020), 2017-01-31). I also suspect that it was broken
just a few months earlier by a005c33d0bd7 (mergecopies: add logic to
process incomplete data, 2016-10-04). Note that bid merge had been
enabled for a few years at that point, since 19903277f035 (merge: use
bid merge by default (BC), 2014-10-01).
This patch is still just a workaround. It will be cleaned up soon
(with the rewrite of mergecopies()). But doing this in a separate
patch makes later patches easier to understand and gives a place to
explain why this is changing.
Differential Revision: https://phab.mercurial-scm.org/D6244
Martin von Zweigbergk <martinvonz@google.com> [Sat, 13 Apr 2019 00:24:17 -0700] rev 42161
tests: add test for issue5343 (grafting with copies)
It seems that issue5353 resulted in a lot of tests in test-graft.t,
but the bug actually reported in that issue didn't get a test
case. This patch adds one for the "move" and one for the "copy"
version of it. I also added a "copy+modify" case, to show what should
be a merge conflict. I didn't add one for the "backwards" version of
it since the comment says that that was already covered by previous
work.
The tests added by this patch show the broken behavior (the bug is
still open). I suspect the results returned from mergecopies() are not
expressive enough to fix this issue: it has a dict for copies to merge
with, but that can only give one more filename, but here we need two
(one for the path on the remote side and one for the path in the merge
base). I want to have it tested anyway since I'm about to refactor
mergecopies().
Differential Revision: https://phab.mercurial-scm.org/D6242
Jordi Gutiérrez Hermoso <jordigh@octave.org> [Tue, 16 Apr 2019 13:12:21 -0400] rev 42160
chistedit: use context manager to set verbose ui
I'm still not exactly sure why this is necessary -- perhaps setting it
unconditionally would leak this setting in chg invocations.
Regardless, this would have looked very out of place as compared to
how this setting is done everywhere else, so at least for the sake of
style, let's be consistent with the rest of the codebase.
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 16 Apr 2019 17:26:38 +0200] rev 42159
setdiscovery: stop limiting the number of local head we initially send
In our testing this limitation provides now real gain and instead triggers
pathological discovery timing for some repository with many heads.
See inline documentation for details.
Some timing below:
Mozilla try repository, (~1M revs, ~35K heads), discovery between 2 clones with
100 head missing on each side
before:
! wall 1.492111 comb 1.490000 user 1.450000 sys 0.040000 (best of 20)
! wall 1.813992 comb 1.820000 user 1.700000 sys 0.120000 (max of 20)
! wall 1.574326 comb 1.573500 user 1.522000 sys 0.051500 (avg of 20)
! wall 1.572583 comb 1.570000 user 1.520000 sys 0.050000 (median of 20)
after:
! wall 1.147834 comb 1.150000 user 1.090000 sys 0.060000 (best of 20)
! wall 1.449144 comb 1.450000 user 1.330000 sys 0.120000 (max of 20)
! wall 1.204618 comb 1.202500 user 1.146500 sys 0.056000 (avg of 20)
! wall 1.194407 comb 1.190000 user 1.140000 sys 0.050000 (median of 20)
pypy (~100 heads, 317 heads) discovery between clones with only 42 common heads
before:
! wall 0.031653 comb 0.030000 user 0.030000 sys 0.000000 (best of 25)
! wall 0.055719 comb 0.050000 user 0.040000 sys 0.010000 (max of 25)
! wall 0.038939 comb 0.039600 user 0.038400 sys 0.001200 (avg of 25)
! wall 0.038660 comb 0.050000 user 0.040000 sys 0.010000 (median of 25)
after:
! wall 0.018754 comb 0.020000 user 0.020000 sys 0.000000 (best of 49)
! wall 0.034505 comb 0.040000 user 0.030000 sys 0.010000 (max of 49)
! wall 0.019631 comb 0.019796 user 0.018367 sys 0.001429 (avg of 49)
! wall 0.019132 comb 0.020000 user 0.020000 sys 0.000000 (median of 49)
Private repository (~1M revs, ~3K heads), discovery from a strip subset, about
100 changesets to be pulled.
before:
! wall 1.837729 comb 1.840000 user 1.790000 sys 0.050000 (best of 20)
! wall 2.203468 comb 2.200000 user 2.100000 sys 0.100000 (max of 20)
! wall 2.049355 comb 2.048500 user 2.002500 sys 0.046000 (avg of 20)
! wall 2.035315 comb 2.040000 user 2.000000 sys 0.040000 (median of 20)
after:
! wall 0.136598 comb 0.130000 user 0.110000 sys 0.020000 (best of 20)
! wall 0.330519 comb 0.330000 user 0.260000 sys 0.070000 (max of 20)
! wall 0.157254 comb 0.155500 user 0.123000 sys 0.032500 (avg of 20)
! wall 0.149870 comb 0.140000 user 0.110000 sys 0.030000 (median of 20)
Same private repo, discovery between two clone with 500 different heads on each
side:
before:
! wall 2.372919 comb 2.370000 user 2.320000 sys 0.050000 (best of 20)
! wall 2.622422 comb 2.610000 user 2.510000 sys 0.100000 (max of 20)
! wall 2.450135 comb 2.450000 user 2.402000 sys 0.048000 (avg of 20)
! wall 2.443896 comb 2.450000 user 2.410000 sys 0.040000 (median of 20)
after:
! wall 0.625497 comb 0.620000 user 0.570000 sys 0.050000 (best of 20)
! wall 0.834723 comb 0.820000 user 0.730000 sys 0.090000 (max of 20)
! wall 0.675725 comb 0.675500 user 0.628000 sys 0.047500 (avg of 20)
! wall 0.671614 comb 0.680000 user 0.640000 sys 0.040000 (median of 20)
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 17 Apr 2019 17:56:30 +0200] rev 42158
peer: introduce a limitedarguments attributes
When set to True, it signal that the peer cannot receive too larges arguments
and that algorithm must adapt. This should only be True for http peer that does
not support argument passed as "post".
This will be useful to unlock better discovery performance in the next
changesets.
I am using a dedicated argument because this is not really a usual
"capabilities" things. An alternative approach would be to adds a
"large-arguments" to all peer, but the http peers. That seemed a bit too hacky
to me.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 06 Mar 2019 15:06:53 +0100] rev 42157
verify: also check full manifest validity during verify runs
Before this changes, `hg verify` only checked if a manifest revision existed and
referenced the proper files. However it never checked the manifest revision
content itself.
Mercurial is expecting manifest entries to be sorted and will crash otherwise.
Since `hg verify` did not attempted a full restoration of manifest entry, it
could ignore this kind of corruption.
This new check significantly increases the cost of a `hg verify` run. This
especially affects large repository not using `sparse-revlog`. For now, this is
hidden behind the `--full` experimental flag.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 17 Apr 2019 01:11:09 +0200] rev 42156
verify: introduce an experimental --full flag
The flag currently has no effect, see next changeset for details. We introduce
the flag as experimental to keep the freedom of changing our mind on the final
UI.
Note: this patch highlight a small but in `hg help`. An option section is
generated even if no option are visible.
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 17 Apr 2019 01:12:21 +0200] rev 42155
verify: introduce a notion of "level"
Some checks are slower than others, to help the user to run the checks he needs,
we are about to introduce new flag to select faster vs deeper runs. This put
the scaffolding in place to do this.
Martin von Zweigbergk <martinvonz@google.com> [Sat, 13 Apr 2019 23:18:56 -0700] rev 42154
tests: split out separate test for issue5020
The test was added to the existing setup in 41f6af50c0d8 (merge: fix
crash on criss cross merge with dir move and delete (issue5020),
2017-01-31). I'm about to make a change that affects that test and
it's much easier to follow if the test case for issue5020 is a
separate test case. The separate test case is based on what mpm
provided in comment 12 on the issue.
`hg diff -r 41f6af50c0d8^ tests/test-merge-criss-cross.t` after this
patch is pretty small (besides the added test). It's probably easier
for reviewers to look at that than to try to understand the diff
itself (I don't understand it).
Differential Revision: https://phab.mercurial-scm.org/D6243
Martin von Zweigbergk <martinvonz@google.com> [Mon, 15 Apr 2019 18:04:54 -0700] rev 42153
tests: avoid a rename/delete conflict when updating in test-narrow-update.t
After the upcoming rewrite of mergecopies(), this test would otherwise
(accurately) start warning about "inside/f1 was deleted and renamed".
Differential Revision: https://phab.mercurial-scm.org/D6254
Martin von Zweigbergk <martinvonz@google.com> [Mon, 15 Apr 2019 15:28:41 -0700] rev 42152
tests: delete unused function in test-rename-merge2.t
Differential Revision: https://phab.mercurial-scm.org/D6253
Martin von Zweigbergk <martinvonz@google.com> [Sun, 14 Apr 2019 13:46:40 -0700] rev 42151
tests: make merge conflicts explicit in `hg annotate` tests
We were using `true` as merge tool. I think it makes the test easier
to understand if we make the conflicts explcit. It also papered over a
conflict that shouldn't have been a conflict (just a bug in copy
tracing). I've marked that "BROKEN".
Differential Revision: https://phab.mercurial-scm.org/D6252
Martin von Zweigbergk <martinvonz@google.com> [Thu, 18 Apr 2019 03:05:42 +0530] rev 42150
narrow: make warning about possibly dirty files respect ui.relative-paths
Differential Revision: https://phab.mercurial-scm.org/D6264