commands: make commit acquire locks before processing (issue4368) Before this patch, "hg commit" (process A) executes steps below: 1. get current branch heads via 'repo.branchheads()' - cache 'repo.changelog' 2. invoke 'repo.commit()' 3. acquire wlock - invalidate 'repo.dirstate' 4. access 'repo.dirstate' - re-read '.hg/dirstate' - check validity of parent revisions with 'repo.changelog' 5. invoke 'repo.commitctx()' 6. acquire store lock (slock) - invalidate 'repo.changelog' 7. do committing 8. release slock 9. release wlock 10. check new branch head (via 'cmdutil.commitstatus()') If acquisition of wlock at (3) above waits for another "hg commit" (process B) or so running parallelly to release wlock, process A causes creating orphan revision, because: - '.hg/dirstate' refers the revision, which is newly added by process B, as its parent - but already cached 'repo.changelog' doesn't contain such revision - therefore, validating parents of '.hg/dirstate' at (4) above replaces such revision with 'nullid' Then, process A creates "orphan" revision, of which parent is "null" revision. In addition to it, "created new head" may be shown at the end of process A unintentionally, if store is updated parallelly, because both getting branch heads (1) and checking new branch head (10) are executed outside slock scope. To avoid this issue, this patch makes "hg commit" acquire wlock and slock before processing. This patch resolves the issue between "hg commit" processes, but not one between "hg commit" and other commands. Subsequent patches resolve the latter. Even after this patch, there are still corner case problems below: - filecache may overlook changes of '.hg/dirstate', and it causes similar issue (see below for detail) https://bz.mercurial-scm.org/show_bug.cgi?id=4368#c10 - 3rd party extension may cause similar issue, if it directly uses 'repo.commit()' without acquisition of wlock and slock This can be fixed by acquisition of slock at the beginning of 'repo.commit()', but it seems suitable for "default" branch In fact, acquisition of slock itself is already introduced at "default" branch by 4414d500604f, but acquisition is not at the beginning of 'repo.commit()'. This patch also changes some tests: - test-fncache.t needs this tricky wrapping, to release (= forced failure of) wlock certainly - order of "hg commit" output is changed by widening scope of locks, because some hooks are fired after releasing wlock

     1

Test how largefiles abort in case the disk runs full

     2

     3

  $ cat > criple.py <<EOF

     4

  > import os, errno, shutil

     5

  > from mercurial import util

     6

  > #

     7

  > # this makes the original largefiles code abort:

     8

  > def copyfileobj(fsrc, fdst, length=16*1024):

     9

  >     fdst.write(fsrc.read(4))

    10

  >     raise IOError(errno.ENOSPC, os.strerror(errno.ENOSPC))

    11

  > shutil.copyfileobj = copyfileobj

    12

  > #

    13

  > # this makes the rewritten code abort:

    14

  > def filechunkiter(f, size=65536, limit=None):

    15

  >     yield f.read(4)

    16

  >     raise IOError(errno.ENOSPC, os.strerror(errno.ENOSPC))

    17

  > util.filechunkiter = filechunkiter

    18

  > #

    19

  > def oslink(src, dest):

    20

  >     raise OSError("no hardlinks, try copying instead")

    21

  > util.oslink = oslink

    22

  > EOF

    23

    24

  $ echo "[extensions]" >> $HGRCPATH

    25

  $ echo "largefiles =" >> $HGRCPATH

    26

    27

  $ hg init alice

    28

  $ cd alice

    29

  $ echo "this is a very big file" > big

    30

  $ hg add --large big

    31

  $ hg commit --config extensions.criple=$TESTTMP/criple.py -m big

    32

  abort: No space left on device

    33

  [255]

    34

    35

The largefile is not created in .hg/largefiles:

    36

    37

  $ ls .hg/largefiles

    38

  dirstate

    39

    40

The user cache is not even created:

    41

    42

  >>> import os; os.path.exists("$HOME/.cache/largefiles/")

    43

  False

    44

    45

Make the commit with space on the device:

    46

    47

  $ hg commit -m big

    48

    49

Now make a clone with a full disk, and make sure lfutil.link function

    50

makes copies instead of hardlinks:

    51

    52

  $ cd ..

    53

  $ hg --config extensions.criple=$TESTTMP/criple.py clone --pull alice bob

    54

  requesting all changes

    55

  adding changesets

    56

  adding manifests

    57

  adding file changes

    58

  added 1 changesets with 1 changes to 1 files

    59

  updating to branch default

    60

  getting changed largefiles

    61

  abort: No space left on device

    62

  [255]

    63

    64

The largefile is not created in .hg/largefiles:

    65

    66

  $ ls bob/.hg/largefiles

    67

  dirstate