tests/test-largefiles-small-disk.t
author Manuel Jacob <me@manueljacob.de>
Mon, 11 Jul 2022 01:51:20 +0200
branchstable
changeset 49378 094a5fa3cf52
parent 48876 42d2b31cee0b
child 50195 11e6eee4b063
permissions -rw-r--r--
procutil: make stream detection in make_line_buffered more correct and strict In make_line_buffered(), we don’t want to wrap the stream if we know that lines get flushed to the underlying raw stream already. Previously, the heuristic was too optimistic. It assumed that any stream which is not an instance of io.BufferedIOBase doesn’t need wrapping. However, there are buffered streams that aren’t instances of io.BufferedIOBase, like Mercurial’s own winstdout. The new logic is different in two ways: First, only for the check, if unwraps any combination of WriteAllWrapper and winstdout. Second, it skips wrapping the stream only if it is an instance of io.RawIOBase (or already wrapped). If it is an instance of io.BufferedIOBase, it gets wrapped. In any other case, the function raises an exception. This ensures that, if an unknown stream is passed or we add another wrapper in the future, we don’t wrap the stream if it’s already line buffered or not wrap the stream if it’s not line buffered. In fact, this was already helpful during development of this change. Without it, I possibly would have forgot that WriteAllWrapper needs to be ignored for the check, leading to unnecessary wrapping if stdout is unbuffered. The alternative would have been to always wrap unknown streams. However, I don’t think that anyone would benefit from being less strict. We can expect streams from the standard library to be subclassing either io.RawIOBase or io.BufferedIOBase, so running Mercurial in the standard way should not regress by this change. Py2exe might replace sys.stdout and sys.stderr, but that currently breaks Mercurial anyway and also these streams don’t claim to be interactive, so this function is not called for them.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
15571
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
     1
Test how largefiles abort in case the disk runs full
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
     2
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
     3
  $ cat > criple.py <<EOF
33962
991134261057 tests: update test-largefiles-small-disk to pass our import checker
Augie Fackler <raf@durin42.com>
parents: 31207
diff changeset
     4
  > import errno
991134261057 tests: update test-largefiles-small-disk to pass our import checker
Augie Fackler <raf@durin42.com>
parents: 31207
diff changeset
     5
  > import os
991134261057 tests: update test-largefiles-small-disk to pass our import checker
Augie Fackler <raf@durin42.com>
parents: 31207
diff changeset
     6
  > import shutil
15571
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
     7
  > from mercurial import util
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
     8
  > #
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
     9
  > # this makes the original largefiles code abort:
31207
1ef37b16b8e8 dirstate: try to use hardlink to backup dirstate
Jun Wu <quark@fb.com>
parents: 30181
diff changeset
    10
  > _origcopyfileobj = shutil.copyfileobj
41738
c70bdd222dcd tests: bulk changes to avoid whitespace errors of check-code.py
FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
parents: 37439
diff changeset
    11
  > def copyfileobj(fsrc, fdst, length=16 * 1024):
31207
1ef37b16b8e8 dirstate: try to use hardlink to backup dirstate
Jun Wu <quark@fb.com>
parents: 30181
diff changeset
    12
  >     # allow journal files (used by transaction) to be written
36113
a42817fede27 py3: use b'' in inline extension
Gregory Szorc <gregory.szorc@gmail.com>
parents: 34661
diff changeset
    13
  >     if b'journal.' in fdst.name:
31207
1ef37b16b8e8 dirstate: try to use hardlink to backup dirstate
Jun Wu <quark@fb.com>
parents: 30181
diff changeset
    14
  >         return _origcopyfileobj(fsrc, fdst, length)
15571
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    15
  >     fdst.write(fsrc.read(4))
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    16
  >     raise IOError(errno.ENOSPC, os.strerror(errno.ENOSPC))
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    17
  > shutil.copyfileobj = copyfileobj
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    18
  > #
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    19
  > # this makes the rewritten code abort:
30181
7356e6b1f5b8 util: increase filechunkiter size to 128k
Mads Kiilerich <madski@unity3d.com>
parents: 20063
diff changeset
    20
  > def filechunkiter(f, size=131072, limit=None):
15571
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    21
  >     yield f.read(4)
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    22
  >     raise IOError(errno.ENOSPC, os.strerror(errno.ENOSPC))
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    23
  > util.filechunkiter = filechunkiter
15572
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    24
  > #
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    25
  > def oslink(src, dest):
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    26
  >     raise OSError("no hardlinks, try copying instead")
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    27
  > util.oslink = oslink
15571
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    28
  > EOF
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    29
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    30
  $ echo "[extensions]" >> $HGRCPATH
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    31
  $ echo "largefiles =" >> $HGRCPATH
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    32
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    33
  $ hg init alice
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    34
  $ cd alice
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    35
  $ echo "this is a very big file" > big
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    36
  $ hg add --large big
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    37
  $ hg commit --config extensions.criple=$TESTTMP/criple.py -m big
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    38
  abort: No space left on device
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    39
  [255]
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    40
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    41
The largefile is not created in .hg/largefiles:
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    42
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    43
  $ ls .hg/largefiles
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    44
  dirstate
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    45
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    46
The user cache is not even created:
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    47
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    48
  >>> import os; os.path.exists("$HOME/.cache/largefiles/")
809788118aa2 largefiles: write .hg/largefiles/ files atomically
Martin Geisler <mg@aragost.com>
parents:
diff changeset
    49
  False
15572
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    50
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    51
Make the commit with space on the device:
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    52
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    53
  $ hg commit -m big
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    54
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    55
Now make a clone with a full disk, and make sure lfutil.link function
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    56
makes copies instead of hardlinks:
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    57
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    58
  $ cd ..
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    59
  $ hg --config extensions.criple=$TESTTMP/criple.py clone --pull alice bob
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    60
  requesting all changes
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    61
  adding changesets
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    62
  adding manifests
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    63
  adding file changes
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    64
  added 1 changesets with 1 changes to 1 files
34661
eb586ed5d8ce transaction-summary: show the range of new revisions upon pull/unbundle (BC)
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 33962
diff changeset
    65
  new changesets 390cf214e9ac
15572
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    66
  updating to branch default
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    67
  getting changed largefiles
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    68
  abort: No space left on device
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    69
  [255]
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    70
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    71
The largefile is not created in .hg/largefiles:
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    72
926bc23d0b6a largefiles: copy files into .hg/largefiles atomically
Martin Geisler <mg@aragost.com>
parents: 15571
diff changeset
    73
  $ ls bob/.hg/largefiles
20063
8a021cd38719 largefiles: update in two steps, handle interrupted updates better
Mads Kiilerich <madski@unity3d.com>
parents: 18459
diff changeset
    74
  dirstate