Augie Fackler <augie@google.com> [Mon, 05 Nov 2018 17:48:23 -0500] rev 40526
remotefilelog: fix various minor py3 problems
# skip-blame b prefixes and pycompat.long, nothing remotely interesting
Differential Revision: https://phab.mercurial-scm.org/D5223
Augie Fackler <augie@google.com> [Mon, 05 Nov 2018 17:37:37 -0500] rev 40525
remotefilelog: rip out a missed mention of lz4
Differential Revision: https://phab.mercurial-scm.org/D5222
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Nov 2018 17:47:23 +0900] rev 40524
ui: wrap whole _write() block with timeblockedsection
I think the cost of color labeling is negligible compared to the I/O
syscalls. Let's simply wrap the whole write() function so that we can
eliminate _write() and _write_err() in later changeset.
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Nov 2018 17:43:57 +0900] rev 40523
ui: indent _writenobuf() to prepare moving bits from _write() functions
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Nov 2018 17:42:05 +0900] rev 40522
ui: simply concatenate messages before applying color labels
This should be cheaper in space than applying labels for each message.
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Nov 2018 17:36:10 +0900] rev 40521
ui: simplify interface of low-level write() functions
_write() and _write_err() will be replaced with fout.write() and ferr.write()
respectively. This is the first step.
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Nov 2018 17:32:35 +0900] rev 40520
ui: factor out function that writes data to fout/ferr with labeling
I'm thinking of adding an option to send status messages to stderr (or a
dedicated command-server channel) so that structured output (e.g. JSON)
would never be interleaved with non-formatter output. A unified write()
interface helps to do that.
Yuya Nishihara <yuya@tcha.org> [Sat, 03 Nov 2018 16:04:16 +0900] rev 40519
ui: consolidate places where _progclear() is called
The progress bar has to be cleared when we start writing some data to the
output stream. Let's make it always triggered immediately before switching
by _colormode, so that we can easily factor out helper functions.
Yuya Nishihara <yuya@tcha.org> [Sun, 04 Nov 2018 20:44:26 +0900] rev 40518
templater: compute revset lazily
This speeds up e.g. "{ifcontains(rev, revset('::.'), ...)}" in common cases
where 'rev' is near the working parent.
The templater API is ugly, but it helps here. 'f' can be either a generator
or a function returning a generator.
Pulkit Goyal <pulkit@yandex-team.ru> [Wed, 24 Oct 2018 18:48:43 +0300] rev 40517
remotefilelog: drop compat code for "getbundle_shallow" wireprotocol command
Doing some annotate on hgexperimental shows that getbundle_shallow used to exist
in 2013 or before. We don't have any pre-2013 remotefilelog users except Fb
themselves and I doubt they are going to use in-core remotefilelog. So it's safe
to remove this.
Differential Revision: https://phab.mercurial-scm.org/D5193
Pulkit Goyal <pulkit@yandex-team.ru> [Wed, 24 Oct 2018 17:54:34 +0300] rev 40516
tests: remove lz4 as dependency while running tests
One of the previous patch authored by Augie rips out the lz4 dependency and
things should work without it.
Now there are just 2-3 tests failing because of same change in emitrevisions()
API.
Differential Revision: https://phab.mercurial-scm.org/D5192
Pulkit Goyal <pulkit@yandex-team.ru> [Wed, 24 Oct 2018 17:51:49 +0300] rev 40515
remotefilelogserver: add a matcher argument to _walkstreamfiles()
Implementing narrow stream clones in core, I added an optional matcher argument.
The function in remotefilelogserver.py does not know about that argument and
does not accept that and hence some tests fails.
Differential Revision: https://phab.mercurial-scm.org/D5191
Pulkit Goyal <pulkit@yandex-team.ru> [Mon, 05 Nov 2018 11:52:10 -0800] rev 40514
shallowutil: introduce a helper function isenabled()
This patch introduces a inenabled() function which will check whether
remotefilelog is enabled or not. The function is then also used at all the
places where check whether remotefilelog is enabled or not. The new function
makes code easy to read without need to understand what is the constant involved
and why we are checking repo.requirements.
Differential Revision: https://phab.mercurial-scm.org/D5190
Pulkit Goyal <pulkit@yandex-team.ru> [Wed, 24 Oct 2018 16:59:05 +0300] rev 40513
remotefilelog: remove some BC code related to streamclone
Since remotefilelog is now moved to core and we have
streamclone._walkstreamfiles() in core, we don't need to have the backward
compatibility code.
People with old mercurial version should use remotefilelog from hg-experimental
repo as IMO remotefilelog will go under a good refactoring and old clients will
break.
# no-check-commit foo_bar function name
Differential Revision: https://phab.mercurial-scm.org/D5189
Augie Fackler <augie@google.com> [Wed, 17 Oct 2018 09:12:29 -0400] rev 40512
remotefilelog: add some docstring
Differential Revision: https://phab.mercurial-scm.org/D5131
Augie Fackler <augie@google.com> [Thu, 04 Oct 2018 10:03:10 -0400] rev 40511
remotefilelog: rename wireproto methods and remaining capabilities
This is a mess, in part because there should be more constants
throughout. I know we typically do exp- instead of the x_ business in
this change, but I also had to use this in some function names, so I
figured until I can break that coupling I'd go with this. If it's too
unpleasant during review, let me know and I can probably clean it up
some more.
# no-check-commit due to new foo_bar naming - too hard to avoid right now :(
Differential Revision: https://phab.mercurial-scm.org/D5129
Augie Fackler <augie@google.com> [Tue, 16 Oct 2018 17:02:48 -0400] rev 40510
remotefilelog: consolidate and rename on-disk store requirement
The value of this constant appeared in too many places. While we're
here, rename it to be more consistent with our naming conventions for
experimental functionality.
Differential Revision: https://phab.mercurial-scm.org/D5128
Augie Fackler <augie@google.com> [Tue, 16 Oct 2018 17:30:47 -0400] rev 40509
remotefilelog: consolidate and rename bundle2 capability
Bonus: we catch a spot where the shallowrepo requirement was being
misused as the bundle2 capability.
Differential Revision: https://phab.mercurial-scm.org/D5127
Augie Fackler <augie@google.com> [Tue, 16 Oct 2018 16:38:40 -0400] rev 40508
remotefilelog: rename capability for legacy ssh file fetching method
I think I want to delete this, but for now we'll just rename this so
it's easy to tell apart from other strings that have the same value.
Differential Revision: https://phab.mercurial-scm.org/D5126
Augie Fackler <augie@google.com> [Thu, 04 Oct 2018 00:11:37 -0400] rev 40507
remotefilelog: rip out lz4 support
Some methods are left teased out so it's easier to build a migration
extension from v1 packs to v2. I also anticipate those methods will be
of use if we get around to adding configurable compression engines
before we end up jettisoning packs.
I think it would make sense to register one-shot `compress` and
`decompress` methods on our compression engines in util.py, but
indygreg mentioned that might not be straightforward.
Differential Revision: https://phab.mercurial-scm.org/D4944
Augie Fackler <augie@google.com> [Wed, 03 Oct 2018 23:25:31 -0400] rev 40506
remotefilelog: bump pack file formats to version 2
Drop support for format 0 and 1 packs. Version 2 will be different in
that it'll use zlib compression instead of lz4, as the latter isn't
really suitable for core.
If I can make it work, I'll make compression engines pluggable, but
the logic for handling these files is spread out enough it's a little
tricky to get that right.
Test changes are only due to pack files being renamed. The contents of
the packs are the same.
Differential Revision: https://phab.mercurial-scm.org/D4943
Augie Fackler <augie@google.com> [Wed, 03 Oct 2018 23:47:43 -0400] rev 40505
remotefilelog: avoid hard-coding pack names in tests
I'm about to alter the pack format, which alters the content-addressed
name of the pack file. In order to reduce my own confusion I found it
helpful to use an ls | head pipeline since we always want to inspect
the newest pack.
Differential Revision: https://phab.mercurial-scm.org/D4942
Augie Fackler <augie@google.com> [Tue, 16 Oct 2018 14:04:41 -0400] rev 40504
remotefilelog: remove now-unused wirepack code
Differential Revision: https://phab.mercurial-scm.org/D5124
Augie Fackler <augie@google.com> [Tue, 16 Oct 2018 14:04:11 -0400] rev 40503
remotefilelogserver: remove pack-serving functionality
Differential Revision: https://phab.mercurial-scm.org/D5123
Augie Fackler <augie@google.com> [Tue, 16 Oct 2018 13:17:16 -0400] rev 40502
remotefilelog: remove support for fetching packs
We've never used this at Google, and it's strictly an optimization
that indygreg and I are confident we can handle more elegantly
later. Let's rip it out to keep the surface area of the imported
remotefilelog small.
Some tests have changes because the tests for regular packs and
repacking were mixed with tests of fetching packs. I'm pretty sure
I've done the right thing here.
Differential Revision: https://phab.mercurial-scm.org/D5122
Augie Fackler <augie@google.com> [Wed, 03 Oct 2018 17:20:44 -0400] rev 40501
remotefilelog: jettison example cache client
We can always reconstruct it from history if it becomes interesting.
Differential Revision: https://phab.mercurial-scm.org/D4941
Augie Fackler <augie@google.com> [Wed, 03 Oct 2018 17:18:20 -0400] rev 40500
remotefilelog: prune obsolete method
Differential Revision: https://phab.mercurial-scm.org/D4940
Augie Fackler <augie@google.com> [Wed, 03 Oct 2018 14:01:22 -0400] rev 40499
logtoprocess: use new runbgcommand from procutil
Differential Revision: https://phab.mercurial-scm.org/D4939
Augie Fackler <augie@google.com> [Tue, 16 Oct 2018 14:42:57 -0400] rev 40498
procutil: port over windows encoding fixes from logtoprocess
Differential Revision: https://phab.mercurial-scm.org/D5121
Augie Fackler <augie@google.com> [Wed, 03 Oct 2018 14:01:04 -0400] rev 40497
remotefilelog: transplant runbgcommand to procutil
While cleaning up the deprecated runshellcommand I noticed a
near-clone of this in logtoprocess, so I'm standardizing on what
appears to be the newer one by moving it to procutil.
Differential Revision: https://phab.mercurial-scm.org/D4938
Augie Fackler <augie@google.com> [Wed, 03 Oct 2018 13:54:45 -0400] rev 40496
remotefilelog: remove function that was described as deprecated
Differential Revision: https://phab.mercurial-scm.org/D4937
Augie Fackler <augie@google.com> [Thu, 27 Sep 2018 13:03:19 -0400] rev 40495
remotefilelog: import pruned-down remotefilelog extension from hg-experimental
This is remotefilelog as of my recent patches for compatibility with
current tip of hg, minus support for old versions of Mercurial and
some FB-specific features like their treemanifest extension and
fetching linkrev data from a patched phabricator. The file extutil.py
moved from hgext3rd to remotefilelog.
This is not yet ready to be landed, consider it a preview for
now. Planned changes include:
* replace lz4 with zstd
* rename some capabilities, requirements and wireproto commands to mark
them as experimental
* consolidate bits of shallowutil with related functions (eg readfile)
I'm certainly open to other (small) changes, but my rough mission is
to land this largely as-is so we can use it as a model of the
functionality we need going forward for lazy-fetching of file contents
from a server.
# no-check-commit because of a few foo_bar functions
Differential Revision: https://phab.mercurial-scm.org/D4782
Pulkit Goyal <pulkit@yandex-team.ru> [Wed, 17 Oct 2018 17:42:32 +0300] rev 40494
store: introduce _matchtrackedpath() and use it to filter store files
This patch introduces a function to filter store files on the basis of the path
which they are tracking.
The function assumes that the entries can be of two types, 'meta/*' and 'data/*'
which means it will just work on revlog based storage and not with another
storage ways.
For the 'data/*' entries, we remove the 'data/' part and '.i/.d' part from the
beginning and the end then pass that to matcher.
For the 'meta/*' entries, we remove the 'meta/' and '/00manifest.(i/d)' part from
beginning and end then call matcher.visitdir() with it to make sure all the
parent directories are also downloaded.
Since the storage filtering for narrow stream clones is implemented with this
patch, we remove the un-implemented error message, add some more tests and add
the treemanifest case to tests too.
The tests demonstrate that it works correctly.
After this patch, we have now narrow stream clones working. Narrow stream clones
are a very important feature for large repositories who have good internet
connection because they use streamclones for cloning and if they do normal
narrow clone, that takes more time then a full streamclone. Also narrow-stream
clone will drastically speed up clone timings.
Differential Revision: https://phab.mercurial-scm.org/D5139
Pulkit Goyal <pulkit@yandex-team.ru> [Wed, 17 Oct 2018 17:24:55 +0300] rev 40493
narrow: only send includepats and excludepats if they are not empty
If we send an empty includepats or excludepats argument to getbundle, it's
translated to `['']` on the server which causes problems because even though
it's empty, bool of that value if True and we end up creating differencematcher
with narrowspec.match() which results in unexpected behavior.
Differential Revision: https://phab.mercurial-scm.org/D5138
Pulkit Goyal <pulkit@yandex-team.ru> [Tue, 16 Oct 2018 17:53:26 +0300] rev 40492
exchange: pass includepats and excludepats as arguments to getbundle()
This will help in implementing narrow stream clones. Also narrow extension used
to add these arguments, now we add them by default if they are not empty.
Since reading includepats and excludepats on the server only works when narrow
is enabled, we check if narrow if enabled or not before passing them.
Differential Revision: https://phab.mercurial-scm.org/D5119
Kyle Lippincott <spectral@google.com> [Thu, 01 Nov 2018 16:51:21 -0700] rev 40491
catapult: add a bit more documentation on how to use catapult tracing
Differential Revision: https://phab.mercurial-scm.org/D5217
Kyle Lippincott <spectral@google.com> [Thu, 01 Nov 2018 16:44:02 -0700] rev 40490
catapult: introduce HGTESTCATAPULTSERVERPIPE to control run-tests' tracing
If unset, it will inherit its value from HGCATAPULTSERVERPIPE (which hg itself
also respects). By setting only HGTESTCATAPULTSERVERPIPE, we can get per-command
breakdowns of the test runtimes for the whole test suite without overloading the
trace file with the contents of the tracing from hg (such as demandimport, etc.)
Differential Revision: https://phab.mercurial-scm.org/D5216
Kyle Lippincott <spectral@google.com> [Thu, 01 Nov 2018 16:44:37 -0700] rev 40489
catapult: rename 'active' to 'activetrace'; this isn't storing a boolean state
(This is an attempt to make it more obvious that the indentation is correct, to
prevent a repeat of 97f52862b1bd)
Differential Revision: https://phab.mercurial-scm.org/D5215
Kyle Lippincott <spectral@google.com> [Thu, 01 Nov 2018 16:44:32 -0700] rev 40488
catapult: fix broken run-tests catapult tracing
Commit 97f52862b1bd thought this block was misindented, but it was not; the
indentation after that commit means we never log a catapult trace for individual
commands. Also, we've never emitted the "end" trace line when the test was
finished, so the trace viewer claims that these "did not finish", fixing that as
well.
Differential Revision: https://phab.mercurial-scm.org/D5214
Sangeet Kumar Mishra <mail2sangeetmishra@gmail.com> [Sat, 03 Nov 2018 23:24:15 +0530] rev 40487
test: fix self._testdir to use the right mercurial library during testing
Currently if you run tests from some other directory other than the
`../tests/`, you will get a warning stating
```
warning: Testing with unexpected mercurial lib: mercurial
(expected /tmp/hgtests.xxxxxx/install/lib/python/mercurial)
```
This is because the current directory being added to the 'PATH', if the
`self._testdir != runtestdir`, owing to this line
```
if self._testdir != runtestdir:
path = [self._testdir] + path
```
Also say you ran the tests from the hg base directory,
because directory is being added in the PATH (see the above snippet, at
that stage the `self._testdir` has the value as `cwd`, owing to a faulty
initialization). And since the current directory already has the 'hg',
that is used in place of the hg that is installed for the testing purposes
in `/tmp/hgtests.xxxxxx/...`.
Differential Revision: https://phab.mercurial-scm.org/D5199
Matt Harbison <matt_harbison@yahoo.com> [Sat, 03 Nov 2018 20:50:04 -0400] rev 40486
tests: conditonalize a difference in test-merge-tools.t on Windows
Augie Fackler <augie@google.com> [Thu, 01 Nov 2018 17:12:22 -0400] rev 40485
statprof: clean up unicode/bytes a little
I'm not really sure how this worked before, but something perturbed it
and what I've got in this change I believe is a little tidier. This
fixes test-profile.t on Python 3.
Differential Revision: https://phab.mercurial-scm.org/D5210
Augie Fackler <augie@google.com> [Thu, 01 Nov 2018 17:11:31 -0400] rev 40484
statprof: add a couple of asserts to avoid storing unicodes
I don't feel strongly about this change, so it'd be fine to drop it
during review, but it was _extremely_ helpful in drafting the next
change.
Differential Revision: https://phab.mercurial-scm.org/D5209
Augie Fackler <augie@google.com> [Thu, 01 Nov 2018 17:09:55 -0400] rev 40483
tests: add some helpful `|| cat` bits to test-profile.t
This way if the hg invocation crashes, you get to see a stacktrace
without having to edit the test.
Differential Revision: https://phab.mercurial-scm.org/D5208
Augie Fackler <augie@google.com> [Thu, 01 Nov 2018 16:13:01 -0400] rev 40482
tests: skip wireproto clientreactor tests on Python 3.6.0-3.6.3 inclusive
See comment next to the if statement for sad details.
Differential Revision: https://phab.mercurial-scm.org/D5207
Augie Fackler <augie@google.com> [Thu, 01 Nov 2018 15:58:01 -0400] rev 40481
contrib: tweak import-checker to always use bytes for module names
We could go the other route and make embedded() return this as a
bytes, but then it would be harder to portably doctest, so I prefer
this option.
Differential Revision: https://phab.mercurial-scm.org/D5206
Augie Fackler <augie@google.com> [Thu, 01 Nov 2018 15:43:43 -0400] rev 40480
contrib: fix import-checker to not b'' module names on Python 3
Caught by the doctests.
Differential Revision: https://phab.mercurial-scm.org/D5205
Augie Fackler <augie@google.com> [Thu, 01 Nov 2018 15:47:26 -0400] rev 40479
tests: add a critical flush() to run-tests.py to make output stable on py3
Differential Revision: https://phab.mercurial-scm.org/D5204
Kyle Lippincott <spectral@google.com> [Sat, 13 Oct 2018 07:49:20 -0700] rev 40478
merge-tools: when calling external merge tool, describe the resolve inputs
It is a common complaint that a user will be running some operation (histedit,
rebase, evolve, etc.), get into a merge-conflict situation, and not understand
what they are seeing - it is possible that the merge tool is configured to
display the hash, but it's difficult for most merge tools to display a good
snippet of the description.
In the worst case, configuring this template will lead to output that is
immediately covered by a terminal application, maybe the user can hit ctrl-z to
see it. In the common case, the output will be in a terminal window and a GUI
program will start, and it should be possible to view both the terminal and the
GUI program at the same time.
Differential Revision: https://phab.mercurial-scm.org/D5094
Yuya Nishihara <yuya@tcha.org> [Fri, 26 Oct 2018 21:46:37 +0900] rev 40477
templatekw: deprecate p1rev/p2rev/p1node/p2node in favor of p1/p2
Yuya Nishihara <yuya@tcha.org> [Fri, 26 Oct 2018 21:43:20 +0900] rev 40476
templatekw: add p1/p2 keywords which switches the current ctx
This is just an example of mappingdict.
I have no idea what should be displayed as {p1|json}. Currently it is an
empty dict since {'ctx'} isn't displayable, which is clearly useless and
should be changed later.
Yuya Nishihara <yuya@tcha.org> [Fri, 19 Oct 2018 21:11:30 +0900] rev 40475
templater: add wrapper for a single template mapping
This can be used to nest template mappings without inserting a sequence-like
layer. See the next patch for example.
Yuya Nishihara <yuya@tcha.org> [Fri, 26 Oct 2018 21:28:20 +0900] rev 40474
templatekw: extract internal "{rev}:{node|formatnode}" template to constant
This will be used later.
Kyle Lippincott <spectral@google.com> [Mon, 05 Nov 2018 15:01:45 -0800] rev 40473
tweakdefaults: remove commands.resolve.mark-check=abort, it is too broken
See issue6020 for the current case. I don't want to continue attempting to fix
this on the stable branch, so I'm removing from tweakdefaults and will send
fixes meant for the default branch and 4.9.
Differential Revision: https://phab.mercurial-scm.org/D5225
Kyle Lippincott <spectral@google.com> [Fri, 02 Nov 2018 11:57:45 -0700] rev 40472
resolve: when resolve.mark-check=abort, downgrade to warning if pats specified
Previously, with --config resolve.mark-check=abort, running `hg resolve -m foo`
would abort and emit a message saying to use --all. This command does not work,
though: `hg resolve -m foo --all`, and it's really weird for --all to be the
"--force" flag.
My original goal with the option was to make it so that `hg resolve -m` (no
filename arguments) was safer, which is why --all is used; in my mind, `hg
resolve -m foo` should always mark it as resolved, and `--all` is how you
specify "all the files", so that's why I chose `hg resolve -m --all` as the way
out of `hg resolve -m` aborting. This commit makes all of this work the way it
was meant to in my head :)
Differential Revision: https://phab.mercurial-scm.org/D5218
Boris Feld <boris.feld@octobus.net> [Thu, 01 Nov 2018 14:35:18 +0100] rev 40471
test: remove changes preparing the drop of `HGMERGE`
These changes are no longer necessary so we better drop them.
Boris Feld <boris.feld@octobus.net> [Thu, 01 Nov 2018 14:29:21 +0100] rev 40470
run-tests: define the default merge tool through configuration
Using the `HGMERGE` environment variable generates confusion as it overrides
any value set through configuration.
By setting the default value through the default HGRC, the usual overriding
rules apply and it should reduce confusion.
Boris Feld <boris.feld@octobus.net> [Fri, 19 Oct 2018 01:52:16 +0200] rev 40469
run-tests: explicitly declare the list of dropped environment variable
It will make the list clearer and more maintainable.
Boris Feld <boris.feld@octobus.net> [Thu, 01 Nov 2018 14:29:14 +0100] rev 40468
test: drop usage of `HGMERGE` in `test-lfconvert.t`
Using configuration instead of environment variable has a strange effect on
the output, but no impact on the end result.
Boris Feld <boris.feld@octobus.net> [Thu, 01 Nov 2018 14:26:30 +0100] rev 40467
test: drop usage of `HGMERGE` in `test-merge-types.t`
Using configuration instead of environment variable has a strange effect on
the output, but no impact on the end result.
Boris Feld <boris.feld@octobus.net> [Thu, 01 Nov 2018 14:24:34 +0100] rev 40466
test: use the "correct" merge tool in `test-rebase-detach.t`
The value of `HGMERGE` overrides the value passed through --config. We are
about to drop this `HGMERGE` environment variable so we update the value used
in config to match what is currently in use.
This is another example of confusion introduced by the `HGMERGE` environment
variable.
Boris Feld <boris.feld@octobus.net> [Wed, 31 Oct 2018 19:12:32 +0100] rev 40465
test: also deactivate `ui.merge` when testing merge tool
We are about to introduce a default config for `ui.merge`. Since
`test-merge-tools.t` is about testing merge configuration, we need to make sure
it won't interfere.
Boris Feld <boris.feld@octobus.net> [Wed, 31 Oct 2018 18:43:56 +0100] rev 40464
tests: adjust merge tool config in test-largefiles-update.t
Previously, the `ui.merge=internal:fail` configuration was shadowed by the the
`HGMERGE=internal:merge` environment variable. We would like to remove this
usage of `HGMERGE` variable because it is error-prone (eg: this very test). We
start by cleaning up test independently to avoid large churn when the change
happens.
Boris Feld <boris.feld@octobus.net> [Fri, 20 Apr 2018 18:08:54 +0200] rev 40463
extensions: include current version in "invalid version" message
It is "not so rare" for the mercurial version to be badly detected at build
time. In such case, version check for extensions gets confused.
To help pinpoint the error, we now include the Mercurial's version number in
the error message.
Boris Feld <boris.feld@octobus.net> [Fri, 19 Oct 2018 01:07:08 +0200] rev 40462
obsolete: prefetch the repo.obsstore used in phasedivergence loop
The speedup is probably quite negligible, but it cannot hurt.
Boris Feld <boris.feld@octobus.net> [Fri, 19 Oct 2018 00:53:18 +0200] rev 40461
obsutil: prefetch method in allpredecessors loop
We don't expect a massive speedup from this, but the change was laying around
in my repository and it cannot hurt.
Augie Fackler <augie@google.com> [Fri, 02 Nov 2018 14:24:29 -0400] rev 40460
merge with stable
Augie Fackler <raf@durin42.com> [Fri, 02 Nov 2018 14:18:29 -0400] rev 40459
Added signature for changeset a91a2837150b
Augie Fackler <raf@durin42.com> [Fri, 02 Nov 2018 14:18:26 -0400] rev 40458
Added tag 4.8 for changeset a91a2837150b
Danny Hooper <hooper@google.com> [Sat, 13 Oct 2018 01:55:40 -0700] rev 40457
beautifygraph: don't substitute anything for 'X' in rendered graphs
It looks like we never actually render graphs this way, although there's an
example in a comment next to the code that prevents it from being needed (see
graphmod.ascii()).
Differential Revision: https://phab.mercurial-scm.org/D5103
Yuya Nishihara <yuya@tcha.org> [Wed, 24 Oct 2018 21:29:04 +0900] rev 40456
py3: do not stringify integers in revlog stats by '%s'
Yuya Nishihara <yuya@tcha.org> [Wed, 24 Oct 2018 21:28:03 +0900] rev 40455
py3: convert revlog stats to a dict of (bytes, int) pairs
Py_DECREF(t) is replaced with Py_CLEAR(t) so that t is set to NULL once
decrefed. Otherwise, it would be excessively decrefed if a subsequent
PyBytes_FromString() failed.
Yuya Nishihara <yuya@tcha.org> [Sun, 28 Oct 2018 21:16:36 +0900] rev 40454
rust: fix signature of rustlazyancestors_init() function
Obviously, sizeof(int) != mem::size_of::<usize>() on amd64, though the
argument would be passed in 64-bit register anyway.
Yuya Nishihara <yuya@tcha.org> [Fri, 02 Nov 2018 21:25:35 +0900] rev 40453
tests: require SQLite 3.8.3+ as sqlitestore relies on "WITH" clause
The test fails on gcc112 because the SQLite is too old.
https://sqlite.org/changes.html#version_3_8_3
Anton Shestakov <av6@dwimlabs.net> [Fri, 19 Oct 2018 22:09:53 +0800] rev 40452
relnotes: various tweaks for release notes
Stop filtering out commits that are expected to be covered by releasenotes
extension: now we want two lists, one for WhatsNew and one for ReleaseX.Y.
Use `only(stoprev, startrev)` to make `relnotes -h` output be actually true
about what revisions are included.
More filter rules, mostly obvious.
More classifying rules to have less things in "unsorted".
Looks like nargs=1 was just making args.startrev and args.stoprev be lists for
no reason.
BC and API sections are renamed to what we're using on the WhatsNew page, and
also just skipped if empty.
Boris Feld <boris.feld@octobus.net> [Thu, 01 Nov 2018 12:52:16 +0100] rev 40451
delta: skip "empty delta" optimisation for non-general case (issue6006)
Non-general delta repository cannot delta against anything than prev. So even if
the delta to prev is empty we should use it.
This is similar to the change made in bafa1c4bb7a8.
Differential Revision: https://phab.mercurial-scm.org/D5201
Martin von Zweigbergk <martinvonz@google.com> [Thu, 01 Nov 2018 16:32:16 -0700] rev 40450
narrow: fix copies._fullcopytracing() narrowspec filtering in graft case
I broke this too in 707c3804e607 (narrow: move copies overrides to
core, 2018-09-28). Hopefully I'm done fixing things broken by that
commit now.
Differential Revision: https://phab.mercurial-scm.org/D5213
Martin von Zweigbergk <martinvonz@google.com> [Thu, 01 Nov 2018 16:28:11 -0700] rev 40449
tests: demonstrate broken copies._fullcopytracing()
Turns out copies._fullcopytracing() was also broken.
Differential Revision: https://phab.mercurial-scm.org/D5212
Martin von Zweigbergk <martinvonz@google.com> [Thu, 01 Nov 2018 13:20:12 -0700] rev 40448
narrow: make copies.pathcopies() filter with narrowspec again
I broke this in 707c3804e607 (narrow: move copies overrides to core,
2018-09-28).
Differential Revision: https://phab.mercurial-scm.org/D5203
Martin von Zweigbergk <martinvonz@google.com> [Thu, 01 Nov 2018 11:24:45 -0700] rev 40447
tests: demonstrate broken copies.pathcopies()
Differential Revision: https://phab.mercurial-scm.org/D5202
"Paul Morelle <paul.morelle@octobus.net" [Wed, 31 Oct 2018 20:32:42 +0100] rev 40446
setup: explain to distutils how we write rc versions
When we use a rc version number (e.g. 4.8rc0), bdist_msi is using
distutils.StrictVersion to parse it into a tuple of numbers.
By default, StrictVersion.version_re only recognizes [ab] for alpha/beta,
where mercurial may use '-rc' or 'rc'.
This change makes StrictVersion parse correctly our version numbers, so that
bdist_msi doesn't fail on rc versions.
Boris Feld <boris.feld@octobus.net> [Wed, 31 Oct 2018 12:08:37 -0700] rev 40445
changegroup: restore default node ordering (issue6001)
Changeset db5501d9 changed the default node ordering from "storage" to
"linearize".
While the new API is more explicit and cleaner, the "linearize" order is
problematic on certain repositories like netbeans where it makes bundling
slower the more nodes we bundle.
Pushing and pulling 100 changesets was ~20% slower and pushing and pulling
1000 changesets was ~600% slower.
A very quick analysis of profile traces showed that the pull operation was
taking more time creating the delta.
Putting back the old default order seems to be the safe option. With more time
during the next cycle, we can understand better the impact of sorting with the
DAG order by default, the source of the regression and how to mitigate it.
/!\ We are still waiting for the full performance impact but with this patch,
bundling and pulling locally (not on the performance workstation) 1000
changesets on the netbeans repository is as fast as before the regression.
Differential Revision: https://phab.mercurial-scm.org/D5196
Boris Feld <boris.feld@octobus.net> [Mon, 29 Oct 2018 17:26:25 +0100] rev 40444
changegroup: introduce an explicit linear sorting
We still need to linearize the revisions in some cases, introduce an explicit
`linear` sorting before changing back the default order.
Differential Revision: https://phab.mercurial-scm.org/D5195
Matt Harbison <matt_harbison@yahoo.com> [Thu, 25 Oct 2018 22:13:22 -0400] rev 40443
py3: roll up threading.Thread constructor args into **kwargs
The constructor doesn't have a `verbose` keyword argument in py3.
Matt Harbison <matt_harbison@yahoo.com> [Thu, 25 Oct 2018 21:04:47 -0400] rev 40442
tests: glob over a single quote vs double quote difference on Windows
Augie Fackler <augie@google.com> [Tue, 23 Oct 2018 11:31:33 -0400] rev 40441
py3: port test-log-exthook.t to Python 3
For once an easy one.
Differential Revision: https://phab.mercurial-scm.org/D5184
Augie Fackler <augie@google.com> [Fri, 12 Oct 2018 12:30:47 -0400] rev 40440
extensions: fix up many many debug logs that use %r
This gets us a bunch closer on Python 3, but I'll still have to use a
ton of sad globs. A previous version of this patch tried to preserve
the %r formatting, but upon review Yuya noted that special characters
in extension names is very unlikely, so we can just use %s and not
sweat the quoting.
Differential Revision: https://phab.mercurial-scm.org/D5002
Matt DeVore <matvore@google.com> [Wed, 17 Oct 2018 15:48:01 -0700] rev 40439
blackbox: add configitem for format of log timestamps
Sometimes blackbox logs are used to report performance problems, but the
timestamps are only at second granularity, so often the timings have to
stated separately by the reporter. This is inconvenient and error-prone,
so I would like to include %f in the date format. This patch makes that
possible.
Martin von Zweigbergk <martinvonz@google.com> [Tue, 23 Oct 2018 14:04:17 -0700] rev 40438
narrow: replace filtering in list comprehension by set operations
I didn't think of this while reviewing the patch.
Differential Revision: https://phab.mercurial-scm.org/D5188
Boris Feld <boris.feld@octobus.net> [Tue, 23 Oct 2018 19:20:22 +0200] rev 40437
storage: update sqlitestore to use the new `deltamode` parameter
While updating the filelog class, I forget to update the sqlitestore, tests
are now passing with this patch.
Differential Revision: https://phab.mercurial-scm.org/D5185
Pulkit Goyal <pulkit@yandex-team.ru> [Tue, 23 Oct 2018 16:24:04 +0300] rev 40436
narrow: rework logic to check whether we need to widen and narrow
This patch reworks logic which calculates whether we need to extend or narrow
our working copy or not.
We filter the addincludes, removeincludes, addexcludes and removeexcludes passed
from user to the actual added and removed includes and excludes. What that means
is a user can pass an already included path as addincludes, a path which is not
included as removeincludes etc. In such situations the old logic use to think we
need to do some work, whereas we don't need to do that work.
In old logic, even if we don't have anything new to include but it believes we
need to call widen, this adds some good amount of work on large repository. A
widen calls involves computing incomming csets, calling the narrow_widen() which
in non-ellipses cases goes through all the set of csets which are available
which can take ~2-3 mins on large repos. Those 2-3 minutes are spend on doing
nothing which a client can prevent by checking is there really anything which
needs to be included.
The tests changes shows that we don't go to the server anymore in such cases
which is nice.
Differential Revision: https://phab.mercurial-scm.org/D5183
Pulkit Goyal <pulkit@yandex-team.ru> [Tue, 23 Oct 2018 14:26:17 +0300] rev 40435
tests: show that adding an already included path still calls narrow_widen()
This patch adds tests demonstrating that we still go to the server in
non-ellipses widening when we have that path already on the client and there is
nothing new to download.
The next patch will try to make client side logic smart and not go to the server
if we don't need to download anything.
Differential Revision: https://phab.mercurial-scm.org/D5182
Mads Kiilerich <mads@kiilerich.com> [Sun, 14 Oct 2018 17:08:18 +0200] rev 40434
graft: introduce --base option for using custom base revision while merging
The graft command usually performs an internal merge of the current parent
revision with the graft revision, using p1 of the grafted revision as base for
the merge.
As a trivial extension of this, we introduce the --base option to allow for
using another base revision.
This can be used as a building block for grafting and collapsing multiple
changesets at once, or for grafting the resulting change from a merge as a
single simple change. (This is kind of similar to backout --parent ... only
different: this graft base must be an ancestor, but is usually *not* a parent.)
This is probably an advanced use case, and we do thus not show it in the
non-verbose help.
Boris Feld <boris.feld@octobus.net> [Thu, 18 Oct 2018 12:31:06 +0200] rev 40433
changegroup: add a option to create bundle with full snapshot only
This is easy to implement now and can be useful for benchmarking.
Boris Feld <boris.feld@octobus.net> [Wed, 10 Oct 2018 00:21:02 +0200] rev 40432
changegroup: allow to force delta to be against p1
This new developer option is useful to general more "generic" bundle. Without
this option, a bundle generated from the repository use deltas similar to the
one stored in the specific repository it was generated from. This makes
performance testing a bit tricky.
Using deltas similar to the final result means all delta stored in the bundle
can be applied to the target repository without any further processing (except
for the rare case of a full snapshot). The application of such bundles
(almost) never exercises the (slower) path of searching for a new valid delta.
This result in unrealistic and too favorable timing and profile.
Instead, we introduce an option to make sure all revisions are stored as a
delta against p1. It might not be the best generation option, but it
guarantees that the content will be "generic", not favoring a specific target.
Yuya Nishihara <yuya@tcha.org> [Wed, 31 Oct 2018 21:16:54 +0900] rev 40431
fix: disable use of thread-based worker
getfixes() accesses to repo, changectx, filectx, etc., so I believe there
are code paths triggering data race. Mercurial API isn't thread safe in
general.
Boris Feld <boris.feld@octobus.net> [Tue, 09 Oct 2018 23:26:35 +0200] rev 40430
storage: also use `deltamode argument` for ifiledata
Now that lower level uses such argument, we can propagate the change to higher
layers.
Pulkit Goyal <pulkit@yandex-team.ru> [Wed, 31 Oct 2018 15:27:06 +0300] rev 40429
configitems: rename the config to prevent adding an alias in future
Right now the config option looks like:
[experimental.server]
stream-narrow-clones=
which does not match how config options are generally defined in core. So let's
rename this to:
[experimental]
server.stream-narrow-clones=
before the new release so that we don't have to add an alias in future for this.
Differential Revision: https://phab.mercurial-scm.org/D5198
Boris Feld <boris.feld@octobus.net> [Wed, 31 Oct 2018 11:02:08 +0100] rev 40428
sparse-revlog: only refine delta candidates in the sparse case (issue6006)
Starting with 5aef5afa8654, a valid delta parent might be "refined". This
allows repository using sparse-revlog to produce better delta chain by using
better intermediate snapshot base.
However, this refining step was performed in all cases, including for
repository not using sparse-revlog. This could produce a strange chain in the
general delta case and corrupted repository in the non-general delta case.
We now skip this step unless sparse-revlog is in use.
In issue 6006, Yuya Nishihara provided a test case using an external
repository, so we did not include it. Finding "laboratory" condition to
reproduce this case and implementing an efficient test reproducing it is a bit
tricky. We do not foresee to have the time to provide one by the release date.
Differential Revision: https://phab.mercurial-scm.org/D5197
Boris Feld <boris.feld@octobus.net> [Tue, 09 Oct 2018 22:02:01 +0200] rev 40427
changegroup: refactor emitrevision to use a `deltamode` argument
This new argument gathers the semantic of `sendfulltext` and `deltaprevious`
in a single value. We are about to introduce a new type of constraints.
Avoiding yet another argument sounds like a plus.
Augie Fackler <augie@google.com> [Mon, 29 Oct 2018 16:23:42 -0400] rev 40426
http: work around custom http client classes that refuse extra attrs
I have no idea what is going on with our custom http client code at Google,
but it chokes on these extra attributes we're tucking on http clients. Since
it feels more than a little wrong to just stuff extra data on a client, let's
degrade gracefully when the client class refuses the attributes.
Yuya Nishihara <yuya@tcha.org> [Tue, 23 Oct 2018 21:11:13 +0900] rev 40425
branchmap: do not specify changelog as an argument
Since (unfiltered)repo.changelog lookup gets as fast as __dict__ lookup,
there's no point to pass in changelog instance.
$ hg perfbranchmap --clear-revbranch -R mozilla-central
! base
(orig) wall 20.593091 comb 20.600000 user 20.520000 sys 0.080000 (best of 3)
(this) wall 20.129126 comb 20.130000 user 20.020000 sys 0.110000 (best of 3)
This backs out most of the changes in 76d4272bd57b and 47c03042cd1d.
Yuya Nishihara <yuya@tcha.org> [Sat, 20 Oct 2018 17:56:00 +0900] rev 40424
filecache: unimplement __set__() and __delete__() (API)
Implementing __set__() implies that the descriptor can't be overridden by
obj.__dict__, which means any property access involves slow function call.
"Data descriptors with __set__() and __get__() defined always override
a redefinition in an instance dictionary. In contrast, non-data descriptors
can be overridden by instances."
https://docs.python.org/2.7/reference/datamodel.html#invoking-descriptors
This patch basically backs out 236bb604dc39, "scmutil: update cached copy
when filecached attribute is assigned (issue3263)." The problem described
in issue3263 (which is #3264 in Bugzilla) should no longer happen since
repo._bookmarkcurrent has been moved to repo._bookmarks.active. We still
have a risk of introducing similar bugs, but I think that's the cost we
have to pay.
$ hg perfrevset 'branch(tip)' -R mercurial
(orig) wall 0.139511 comb 0.140000 user 0.140000 sys 0.000000 (best of 66)
(prev) wall 0.114195 comb 0.110000 user 0.110000 sys 0.000000 (best of 81)
(this) wall 0.099038 comb 0.110000 user 0.100000 sys 0.010000 (best of 93)
Yuya Nishihara <yuya@tcha.org> [Sat, 20 Oct 2018 19:13:05 +0900] rev 40423
filecache: use try-except for faster __dict__ lookup
Python function call is slow, and the cost could be significant here.
$ hg perfrevset 'branch(tip)' -R mercurial
(orig) wall 0.139511 comb 0.140000 user 0.140000 sys 0.000000 (best of 66)
(this) wall 0.114195 comb 0.110000 user 0.110000 sys 0.000000 (best of 81)
Anton Shestakov <av6@dwimlabs.net> [Thu, 25 Oct 2018 21:33:43 +0800] rev 40422
crecord: make nextsametype() check that parent item exists (issue6009)
Items that represent files in curses interface don't have parents.
Valentin Gatien-Baron <vgatien-baron@janestreet.com> [Wed, 24 Oct 2018 10:05:13 -0400] rev 40421
help: describe what ui.tweakdefaults changes, concretely
Currently, one has to look at the code.
A couple things are suboptimal:
- probably not translatable
- lines don't get wrapped (a couple are a bit too long)
but it seems to better this way than without help at all.
Differential Revision: https://phab.mercurial-scm.org/D5187
Matt Harbison <matt_harbison@yahoo.com> [Thu, 25 Oct 2018 00:22:42 -0400] rev 40420
logexchange: convert paths to unix when detecting the active path
This fixes the problem in the tests[1] where Windows was showing the whole path
as the remotename for local repositories.
Somebody with a better understanding of this extension should probably take a
deeper look. There may be other cases that need to be converted- specifically
the `elif not instance` and the missing `else` cases in activepath(). I also
noticed when adding debug prints that the absolute path is stored in the file,
probably not normalized. (It's wrapped up in $TESTTMP.)
[1] https://buildbot.mercurial-scm.org/builders/Win7%20x86_64%20hg%20tests/builds/1042/steps/run-tests.py%20%28python%202.7.13%29/logs/stdio
Matt Harbison <matt_harbison@yahoo.com> [Wed, 24 Oct 2018 22:40:48 -0400] rev 40419
help: update the default value specified for `profiling.time-track`
I tried conditionalizing this in a `.. container::` block, but that seemed to
add an extra blank line between the main text and the parenthetical.
Matt Harbison <matt_harbison@yahoo.com> [Wed, 24 Oct 2018 22:24:10 -0400] rev 40418
profiling: revert the default mode back to 'cpu' on Windows
On Windows, os.times() only returns user and system times. Real elapsed time is
0. That results in no actual times reported, an end wall time of 0.000000, and
seemingly randomly sorted stack frames. This at least provides test stability
in test-profile.t.
I kind of think that `default=pycompat.iswindows and 'cpu' or 'real'` would be a
better way to set the default in configitems, but I didn't see any other
examples of this, and thought maybe there's a reason for that. That might allow
plugging the value into the help text automatically- the documented default
wasn't updated in db0dba2d157d.
Boris Feld <boris.feld@octobus.net> [Wed, 17 Oct 2018 14:47:01 +0200] rev 40417
phase: add an archived phase
This phase allows for hidden changesets in the "user space". It differs from
the "internal" phase which is intended for internal by-product only. There
have been discussions at the 4.8 sprint to use such phase to speedup cleanup
after history rewriting operation.
Shipping it in the same release as the 'internal-phase' groups the associated
`requires` entry. The important bit is to have support for this phase in the
earliest version of mercurial possible. Adding the UI to manipulate this new
phase later seems fine.
The current plan for archived usage and user interface are as follow. On a
repository with internal-phase on and evolution off:
* history rewriting command set rewritten changeset in the archived phase.
(This mean updating the cleanupnodes method).
* keep `hg unbundle .hg/strip-backup/X.hg` as a way to restore changeset for
now
(backup bundle need to contains phase data)
* [maybe] add a `hg strip --soft` advance flag
(a light way to expose the feature without getting in the way of a better
UI)
Mercurial 4.8 freeze is too close to get the above in by then.
We don't introduce a new repository `requirement` as we reuse the one
introduced with the 'archived' phase during the 4.8 cycle.
Yuya Nishihara <yuya@tcha.org> [Tue, 23 Oct 2018 20:46:21 +0900] rev 40416
exewrapper: apply clang-format to silence test-check-clang-format.t
rdamazio@google.com [Thu, 18 Oct 2018 19:57:30 -0700] rev 40415
help: displaying extension commands by default
Differential Revision: https://phab.mercurial-scm.org/D5156
rdamazio@google.com [Thu, 18 Oct 2018 19:57:05 -0700] rev 40414
help: displaying documented aliases by default
This makes aliases be displayed in "hg help" when they have a :doc config
entry, and also allows them to be assigned to a category with :category.
Differential Revision: https://phab.mercurial-scm.org/D5087
rdamazio@google.com [Sat, 13 Oct 2018 05:43:39 -0700] rev 40413
help: allow hiding of help topics
Differential Revision: https://phab.mercurial-scm.org/D5077
rdamazio@google.com [Sat, 13 Oct 2018 05:02:55 -0700] rev 40412
help: allow commands to be hidden
This is useful in enterprise environments where some workflows are
discouraged.
Differential Revision: https://phab.mercurial-scm.org/D5076
Pulkit Goyal <pulkit@yandex-team.ru> [Sat, 20 Oct 2018 00:12:20 +0300] rev 40411
py3: add one more passing test to whitelist
Caught by python 3 builder.
Differential Revision: https://phab.mercurial-scm.org/D5175
Pulkit Goyal <pulkit@yandex-team.ru> [Sat, 20 Oct 2018 00:05:50 +0300] rev 40410
py3: make sure we pass sysstr in sqlite3.connect()
Differential Revision: https://phab.mercurial-scm.org/D5174
Martin von Zweigbergk <martinvonz@google.com> [Tue, 05 Sep 2017 15:24:25 -0700] rev 40409
archive: use manifest.matches() to simplify and speed up matching
manifest.matches() can avoid walking paths the user did not want to
archive.
Differential Revision: https://phab.mercurial-scm.org/D5178
Martin von Zweigbergk <martinvonz@google.com> [Tue, 05 Sep 2017 15:24:22 -0700] rev 40408
archive: create alwaysmatcher when no matcher provided
Differential Revision: https://phab.mercurial-scm.org/D5177
Martin von Zweigbergk <martinvonz@google.com> [Tue, 05 Sep 2017 15:21:21 -0700] rev 40407
archive: change "matcnfn" argument to a real matcher
All callers seem to be passing a real matcher, not just a function. We
were also passing it into match.subdirmatcher(), which assumes it is a
matcher.
Differential Revision: https://phab.mercurial-scm.org/D5176