Sun, 10 Mar 2024 03:29:12 +0100 branchcache: use update_disk to refresh 'served' and 'served.hidden'
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 10 Mar 2024 03:29:12 +0100] rev 51496
branchcache: use update_disk to refresh 'served' and 'served.hidden' The `update_disk` method is dedicated to this kind of usecase. Now that the writting patterns are more consistent, we can use it to warm these two important cache. I am dropping the first comment about "refreshing all the others" because it is false. If a branchmap already exist for "served", none of the subset will be updated.
Sun, 10 Mar 2024 03:25:04 +0100 branchcache: explictly update disk state only if no transaction exist
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 10 Mar 2024 03:25:04 +0100] rev 51495
branchcache: explictly update disk state only if no transaction exist If a transaction exist the `write_dirty` call will eventually be done and the state will be synched on disk. It is better to no interfer with that.
Sun, 10 Mar 2024 03:32:50 +0100 branchcache: do not use `__getitem__` in updatecache
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 10 Mar 2024 03:32:50 +0100] rev 51494
branchcache: do not use `__getitem__` in updatecache The `update_disk` method uses `updatecache` and the point of `update_disk` is to be able to do alternative processing to the one we do in `__getitem__`. So we calling `__getitem__` in `updatecache` defeat this purpose. Instead we do the equivalent explicitly to preserve the spirit of `update_disk` (that we will actually put to use soon, I promise)
Sun, 10 Mar 2024 05:10:00 +0100 branchcache: explicitly track inheritence "state"
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 10 Mar 2024 05:10:00 +0100] rev 51493
branchcache: explicitly track inheritence "state" We move from a binary "dirty" flag to a three value "state": "clean", "inherited", "dirty". The "inherited" means that the branch cache is not only "clean", but it is a duplicate of its parent filter. If a branch cache is "inherited", we can non only skip writing its value on disk, but it is a good idea to delete any stale value on disk, as those will just waste time (and possibly induce bug) in the future. We only do this in the update related to transaction or explicit cache update (e.g `hg debugupdatecache`). Deleting the file when we simply detected a stall cache during a read only operation seems more dangerous. We rename `copy` to `inherit_for` to clarify we associate a stronger semantic to the operation.
Sun, 10 Mar 2024 04:53:17 +0100 branchcache: stop writing more branchcache file on disk than needed
Pierre-Yves David <pierre-yves.david@octobus.net> [Sun, 10 Mar 2024 04:53:17 +0100] rev 51492
branchcache: stop writing more branchcache file on disk than needed Before this change, we were unconditionally writing a branchmap file for the filter level passed to `update_disk`. This is actually counter productive if no update were needed for this filter level. In many case, the branch cache for a filter level is identical to its parent "subset" and it is better to simply keep the subset update and reuse it every time instead of having to do identical work for similar subset. So we change the `update_disk` method to only write a file when that filter level differ from its parent. This removes many cases where identical files were written, requiring multiple boring update in the test suite. The only notable changes is the change to `test-strip-branch-cache.t`, this case was checking a scenario that no longer reproduce the bug as writing less branchmap file result in less stalled cache on disk. Strictly speaking, we could create a more convoluted scenario that create a similar issue. However the next changeset would also cover that scenario so we directly updated that test case to a "no longer buggy" state.
Fri, 08 Mar 2024 16:49:06 +0100 branchcache: do not copy the `_dirty` flag
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 08 Mar 2024 16:49:06 +0100] rev 51491
branchcache: do not copy the `_dirty` flag If the inherited branch cache is dirty, it will be written on disk, and the super-set did not need to modify it, the on disk value for the subset will be re-useable as is. So the super set does not needs to write the very same content itself.
Fri, 08 Mar 2024 16:52:08 +0100 branchcache: explicitly assert that copy is always about inheritance
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 08 Mar 2024 16:52:08 +0100] rev 51490
branchcache: explicitly assert that copy is always about inheritance This would catch cases where copy is used for something else if any existed.
Sat, 09 Mar 2024 02:07:15 +0100 branchcache: stop using `copy(…)` in `replace(…)`
Pierre-Yves David <pierre-yves.david@octobus.net> [Sat, 09 Mar 2024 02:07:15 +0100] rev 51489
branchcache: stop using `copy(…)` in `replace(…)` The `copy` method is mostly used for a filter level to inherit the branchmap from a subset. So we stop using (abusing) it in "replace" to ensure `copy` is used only for inheritance purposes. Since `replace` is a method of the BranchMapCache, it seems fine to do lower level operation there.
Fri, 08 Mar 2024 16:47:32 +0100 branchcache: change the _delayed flag to an explicit `_dirty` flag
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 08 Mar 2024 16:47:32 +0100] rev 51488
branchcache: change the _delayed flag to an explicit `_dirty` flag This is more consistent with the logic we use for other object and it open the way to a clearer management of the cache state. Now, cache are created clean, cache update mark them dirty, writing them on disk mark them clean again.
Fri, 08 Mar 2024 15:50:15 +0100 branchcache: write branchmap in subset inheritance order
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 08 Mar 2024 15:50:15 +0100] rev 51487
branchcache: write branchmap in subset inheritance order This way, we can guarantee a valid subset has been written before touching the branchmap of another filter. This is especially useful as we are bout to start deleting outdated branchmap file.
Fri, 08 Mar 2024 15:06:54 +0100 branchcache: do not accept "empty update"
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 08 Mar 2024 15:06:54 +0100] rev 51486
branchcache: do not accept "empty update" This currently does not happens and it will be simpler that is remains that way. If all update do something, we will be able to simply declare, in a later changesets, that all update to result in a dirty branchcache.
Thu, 07 Mar 2024 11:04:34 +0100 branchcache: avoid created a `None` filter repoview when writing
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 07 Mar 2024 11:04:34 +0100] rev 51485
branchcache: avoid created a `None` filter repoview when writing The repoview class is not intended to be used for unfiltered repository.
Wed, 28 Feb 2024 22:49:55 +0100 stream-clone-tests: stop filtering non existent warning
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 28 Feb 2024 22:49:55 +0100] rev 51484
stream-clone-tests: stop filtering non existent warning This filtering was introduced in 74c004a515bc, however there is already no warning in that changeset. So I guess the warnings existed when we the patch was created but the problem was solved in another changeset that 74c004a515bc, rebased on.
Wed, 28 Feb 2024 22:46:12 +0100 stream-clone-test: simplify case testing obsolescence
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 28 Feb 2024 22:46:12 +0100] rev 51483
stream-clone-test: simplify case testing obsolescence There is only two important things in this test: - the number of file we send, to show we picked the obsstore. - the resulting state, to show we did alter things in the process. The rest are of the number are very fragile and consume a lot of time for little value when adjusting formats, caches, and protocol.
Wed, 28 Feb 2024 22:43:07 +0100 stream-clone-test: simplify the case testing phases
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 28 Feb 2024 22:43:07 +0100] rev 51482
stream-clone-test: simplify the case testing phases There is only two important things in this test: - the number of file we send, to show we picked the phase roots. - the resulting phases, to show we did not modified them. The rest are of the number are very fragile and consume a lot of time for little value when adjusting formats, caches, and protocol.
Wed, 28 Feb 2024 22:39:10 +0100 stream-clone-test: simplify bookmark clone
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 28 Feb 2024 22:39:10 +0100] rev 51481
stream-clone-test: simplify bookmark clone The important things to test here is the number of file included (to catch that the bookmark file was sent). So we keep that part non glob'ed but glob the rest. The glob'ed numbers are very fragile and consume a lot of time for little value when adjusting formats, caches, and protocol.
Wed, 28 Feb 2024 22:31:42 +0100 stream-clone-test: add a verify call to the "clone while changing" case
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 28 Feb 2024 22:31:42 +0100] rev 51480
stream-clone-test: add a verify call to the "clone while changing" case It seems useful to very that the clone did not result in a corrupted copy.
Fri, 08 Mar 2024 10:59:51 +0100 stream-clone-test: add title to various test cases
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 08 Mar 2024 10:59:51 +0100] rev 51479
stream-clone-test: add title to various test cases These case are fine as is, but as we are adding title to all the other as we simplify them, lets add title for all cases.
Wed, 28 Feb 2024 22:28:07 +0100 stream-clone-test: simplify testing of secret cloning restriction
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 28 Feb 2024 22:28:07 +0100] rev 51478
stream-clone-test: simplify testing of secret cloning restriction Here, we just want to check if the streaming clone is allowed and used or not. We do not care about the details of the clone itself.
Wed, 28 Feb 2024 22:26:27 +0100 stream-clone-test: simplify the background file closing test
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 28 Feb 2024 22:26:27 +0100] rev 51477
stream-clone-test: simplify the background file closing test Here we just care about the fact the background file closing logic actually ran. We don't need to check the details of the cloning. The details of the output is very fragile and consume a lot of time for little value when adjusting formats, caches, and protocol. So we filter it out.
Fri, 08 Mar 2024 10:51:01 +0100 stream-clone-test: simplify the --uncompressed alias check
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 08 Mar 2024 10:51:01 +0100] rev 51476
stream-clone-test: simplify the --uncompressed alias check To check that --uncompressed is an alias we just need to check it trigger a stream clone, we don't need to check anything else.
Fri, 08 Mar 2024 10:50:42 +0100 stream-clone-test: drop an automatic pattern replacement
Pierre-Yves David <pierre-yves.david@octobus.net> [Fri, 08 Mar 2024 10:50:42 +0100] rev 51475
stream-clone-test: drop an automatic pattern replacement That pattern is nice, but it prevent us to glob the number of bytes when we don't care about them. We don't care about them more often that what we currently checks so dropping this pattern will help use to simplify various tests.
Wed, 28 Feb 2024 22:15:33 +0100 stream-clone-test: simplify the test for getbundle with stream=1
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 28 Feb 2024 22:15:33 +0100] rev 51474
stream-clone-test: simplify the test for getbundle with stream=1 The core of this tests is about checking we receive a stream bundle with such request. We don't need to look at too much of the details of the stream itself. Since the content of the stream if shifting overtime, Such check is very fragile and consume a lot of time for little value when adjusting formats, caches, and protocol. So we reduce the size of what we check to focus on "is this a stream clone" question.
Wed, 28 Feb 2024 22:05:28 +0100 stream-clone-test: factor some piece of basic clone test out
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 28 Feb 2024 22:05:28 +0100] rev 51473
stream-clone-test: factor some piece of basic clone test out Multiple parts of this case (listing cache, checking error) are common to all cases and don't need to be in the conditionnal block. This simplify the test update.
Wed, 28 Feb 2024 22:01:09 +0100 stream-clone-test: simplify the case where server disabled it
Pierre-Yves David <pierre-yves.david@octobus.net> [Wed, 28 Feb 2024 22:01:09 +0100] rev 51472
stream-clone-test: simplify the case where server disabled it We have an option to disable it, we don't need to test it with all protocol variants. In addition there is little value in looking at the bytes to bytes details of the reply. Such check is very fragile and consume a lot of time for little value when adjusting formats, caches, and protocol.
Mon, 11 Mar 2024 13:36:25 +0100 rust-matchers: raw regular expression builder stable
Georges Racinet <georges.racinet@octobus.net> [Mon, 11 Mar 2024 13:36:25 +0100] rev 51471
rust-matchers: raw regular expression builder Extracting this `re_builder()` from `re_matcher()` makes it reusable in more general cases than matching `HgPath` instances and would help reducing code duplication in RHGitaly.
Mon, 11 Mar 2024 13:23:18 +0100 rust-filepatterns: export glob_to_re function stable
Georges Racinet <georges.racinet@octobus.net> [Mon, 11 Mar 2024 13:23:18 +0100] rev 51470
rust-filepatterns: export glob_to_re function Making this function public should not risk freezing the internal API, and it can be useful for all downstream code that needs to perform glob matching against byte strings, such as RHGitaly where it will be useful to match on branches and tags.
Mon, 11 Mar 2024 01:20:12 +0100 repoview: prevent `None` to be passed as the filtername
Pierre-Yves David <pierre-yves.david@octobus.net> [Mon, 11 Mar 2024 01:20:12 +0100] rev 51469
repoview: prevent `None` to be passed as the filtername We let such instantiation slip in a previous commit, so we add an explicit check to prevent it to happen in the future.
Thu, 07 Mar 2024 11:04:34 +0100 branchcache: avoid created a `None` filter repoview when writing
Pierre-Yves David <pierre-yves.david@octobus.net> [Thu, 07 Mar 2024 11:04:34 +0100] rev 51468
branchcache: avoid created a `None` filter repoview when writing The repoview class is not intended to be used for unfiltered repository.
Tue, 05 Mar 2024 15:07:47 +0100 rust-index: don't use mutable borrow to computed filtered heads stable
Pierre-Yves David <pierre-yves.david@octobus.net> [Tue, 05 Mar 2024 15:07:47 +0100] rev 51467
rust-index: don't use mutable borrow to computed filtered heads This does not need to mutate the index. This is the prime suspect for some RuntimeError raised during some pushes.
(0) -30000 -10000 -3000 -1000 -300 -100 -50 -30 +30 +50 +100 tip