tests/test-remotefilelog-gc.t
author Augie Fackler <augie@google.com>
Thu, 04 Oct 2018 00:11:37 -0400
changeset 40507 e2a1584e9e3f
parent 40506 10c10da14c5d
child 40571 95a079ea1e19
permissions -rw-r--r--
remotefilelog: rip out lz4 support Some methods are left teased out so it's easier to build a migration extension from v1 packs to v2. I also anticipate those methods will be of use if we get around to adding configurable compression engines before we end up jettisoning packs. I think it would make sense to register one-shot `compress` and `decompress` methods on our compression engines in util.py, but indygreg mentioned that might not be straightforward. Differential Revision: https://phab.mercurial-scm.org/D4944
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
     1
  $ PYTHONPATH=$TESTDIR/..:$PYTHONPATH
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
     2
  $ export PYTHONPATH
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
     3
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
     4
  $ . "$TESTDIR/remotefilelog-library.sh"
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
     5
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
     6
  $ hginit master
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
     7
  $ cd master
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
     8
  $ cat >> .hg/hgrc <<EOF
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
     9
  > [remotefilelog]
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    10
  > server=True
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    11
  > serverexpiration=-1
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    12
  > EOF
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    13
  $ echo x > x
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    14
  $ hg commit -qAm x
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    15
  $ cd ..
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    16
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    17
  $ hgcloneshallow ssh://user@dummy/master shallow -q
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    18
  1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over *s (glob)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    19
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    20
# Set the prefetchdays config to zero so that all commits are prefetched
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    21
# no matter what their creation date is.
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    22
  $ cd shallow
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    23
  $ cat >> .hg/hgrc <<EOF
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    24
  > [remotefilelog]
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    25
  > prefetchdays=0
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    26
  > EOF
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    27
  $ cd ..
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    28
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    29
# commit a new version of x so we can gc the old one
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    30
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    31
  $ cd master
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    32
  $ echo y > x
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    33
  $ hg commit -qAm y
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    34
  $ cd ..
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    35
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    36
  $ cd shallow
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    37
  $ hg pull -q
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    38
  $ hg update -q
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    39
  1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over *s (glob)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    40
  $ cd ..
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    41
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    42
# gc client cache
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    43
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    44
  $ lastweek=`$PYTHON -c 'import datetime,time; print(datetime.datetime.fromtimestamp(time.time() - (86400 * 7)).strftime("%y%m%d%H%M"))'`
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    45
  $ find $CACHEDIR -type f -exec touch -t $lastweek {} \;
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    46
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    47
  $ find $CACHEDIR -type f | sort
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    48
  $TESTTMP/hgcache/master/11/f6ad8ec52a2984abaafd7c3b516503785c2072/1406e74118627694268417491f018a4a883152f0 (glob)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    49
  $TESTTMP/hgcache/master/11/f6ad8ec52a2984abaafd7c3b516503785c2072/48023ec064c1d522f0d792a5a912bb1bf7859a4a (glob)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    50
  $TESTTMP/hgcache/repos (glob)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    51
  $ hg gc
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    52
  finished: removed 1 of 2 files (0.00 GB to 0.00 GB)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    53
  $ find $CACHEDIR -type f | sort
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    54
  $TESTTMP/hgcache/master/11/f6ad8ec52a2984abaafd7c3b516503785c2072/48023ec064c1d522f0d792a5a912bb1bf7859a4a (glob)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    55
  $TESTTMP/hgcache/repos
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    56
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    57
# gc server cache
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    58
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    59
  $ find master/.hg/remotefilelogcache -type f | sort
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    60
  master/.hg/remotefilelogcache/x/1406e74118627694268417491f018a4a883152f0 (glob)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    61
  master/.hg/remotefilelogcache/x/48023ec064c1d522f0d792a5a912bb1bf7859a4a (glob)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    62
  $ hg gc master
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    63
  finished: removed 0 of 1 files (0.00 GB to 0.00 GB)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    64
  $ find master/.hg/remotefilelogcache -type f | sort
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    65
  master/.hg/remotefilelogcache/x/48023ec064c1d522f0d792a5a912bb1bf7859a4a (glob)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    66
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    67
# Test that GC keepset includes pullprefetch revset if it is configured
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    68
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    69
  $ cd shallow
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    70
  $ cat >> .hg/hgrc <<EOF
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    71
  > [remotefilelog]
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    72
  > pullprefetch=all()
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    73
  > EOF
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    74
  $ hg prefetch
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    75
  1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over *s (glob)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    76
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    77
  $ cd ..
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    78
  $ hg gc
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    79
  finished: removed 0 of 2 files (0.00 GB to 0.00 GB)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    80
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    81
# Ensure that there are 2 versions of the file in cache
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    82
  $ find $CACHEDIR -type f | sort
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    83
  $TESTTMP/hgcache/master/11/f6ad8ec52a2984abaafd7c3b516503785c2072/1406e74118627694268417491f018a4a883152f0 (glob)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    84
  $TESTTMP/hgcache/master/11/f6ad8ec52a2984abaafd7c3b516503785c2072/48023ec064c1d522f0d792a5a912bb1bf7859a4a (glob)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    85
  $TESTTMP/hgcache/repos (glob)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    86
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    87
# Test that if garbage collection on repack and repack on hg gc flags are set then incremental repack with garbage collector is run
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    88
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    89
  $ hg gc --config remotefilelog.gcrepack=True --config remotefilelog.repackonhggc=True
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    90
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    91
# Ensure that loose files are repacked
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    92
  $ find $CACHEDIR -type f | sort
40507
e2a1584e9e3f remotefilelog: rip out lz4 support
Augie Fackler <augie@google.com>
parents: 40506
diff changeset
    93
  $TESTTMP/hgcache/master/packs/320dab99b7e3f60512b97f347689625263d22cf5.dataidx
e2a1584e9e3f remotefilelog: rip out lz4 support
Augie Fackler <augie@google.com>
parents: 40506
diff changeset
    94
  $TESTTMP/hgcache/master/packs/320dab99b7e3f60512b97f347689625263d22cf5.datapack
40506
10c10da14c5d remotefilelog: bump pack file formats to version 2
Augie Fackler <augie@google.com>
parents: 40495
diff changeset
    95
  $TESTTMP/hgcache/master/packs/837b83c1ef6485a336eb4421ac5973c0ec130fbb.histidx
10c10da14c5d remotefilelog: bump pack file formats to version 2
Augie Fackler <augie@google.com>
parents: 40495
diff changeset
    96
  $TESTTMP/hgcache/master/packs/837b83c1ef6485a336eb4421ac5973c0ec130fbb.histpack
40495
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    97
  $TESTTMP/hgcache/master/packs/repacklock
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    98
  $TESTTMP/hgcache/repos
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
    99
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
   100
# Test that warning is displayed when there are no valid repos in repofile
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
   101
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
   102
  $ cp $CACHEDIR/repos $CACHEDIR/repos.bak
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
   103
  $ echo " " > $CACHEDIR/repos
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
   104
  $ hg gc
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
   105
  warning: no valid repos in repofile
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
   106
  $ mv $CACHEDIR/repos.bak $CACHEDIR/repos
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
   107
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
   108
# Test that warning is displayed when the repo path is malformed
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
   109
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
   110
  $ printf "asdas\0das" >> $CACHEDIR/repos
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
   111
  $ hg gc 2>&1 | head -n2
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
   112
  warning: malformed path: * (glob)
3a333a582d7b remotefilelog: import pruned-down remotefilelog extension from hg-experimental
Augie Fackler <augie@google.com>
parents:
diff changeset
   113
  Traceback (most recent call last):