rust/hg-core/src/dirstate.rs
author Raphaël Gomès <rgomes@octobus.net>
Fri, 11 Oct 2019 13:39:57 +0200
changeset 43271 99394e6c5d12
parent 42957 7a01778bc7b7
child 43605 8210c3f46912
permissions -rw-r--r--
rust-dirstate-status: add first Rust implementation of `dirstate.status` Note: This patch also added the rayon crate as a Cargo dependency. It will help us immensely in making Rust code parallel and easy to maintain. It is a stable, well-known, and supported crate maintained by people on the Rust team. The current `dirstate.status` method has grown over the years through bug reports and new features to the point where it got too big and too complex. This series does not yet improve the logic, but adds a Rust fast-path to speed up certain cases. Tested on mozilla-try-2019-02-18 with zstd compression: - `hg diff` on an empty working copy: - c: 1.64(+-)0.04s - rust+c before this change: 2.84(+-)0.1s - rust+c: 849(+-)40ms - `hg commit` when creating a file: - c: 5.960s - rust+c before this change: 5.828s - rust+c: 4.668s - `hg commit` when updating a file: - c: 4.866s - rust+c before this change: 4.371s - rust+c: 3.855s - `hg status -mard` - c: 1.82(+-)0.04s - rust+c before this change: 2.64(+-)0.1s - rust+c: 896(+-)30ms The numbers are clear: the current Rust `dirstatemap` implementation is super slow, its performance needs to be addressed. This will be done in a future series, immediately after this one, with the goal of getting Rust to be at least to the speed of the Python + C implementation in all cases before the 5.2 freeze. At worse, we gate dirstatemap to only be used in those cases. Cases where the fast-path is not executed: - for commands that need ignore support (`status`, for example) - if subrepos are found (should not be hard to add, but winter is coming) - any other matcher than an `alwaysmatcher`, like patterns, etc. - with extensions like `sparse` and `fsmonitor` The next step after this is to rethink the logic to be closer to Jane Street's Valentin Gatien-Baron's Rust fast-path which does a lot less work when possible. Differential Revision: https://phab.mercurial-scm.org/D7058
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
42748
7cae6bc29ff9 rust-parsers: switch to parse/pack_dirstate to mutate-on-loop
Raphaël Gomès <rgomes@octobus.net>
parents: 42609
diff changeset
     1
// dirstate module
7cae6bc29ff9 rust-parsers: switch to parse/pack_dirstate to mutate-on-loop
Raphaël Gomès <rgomes@octobus.net>
parents: 42609
diff changeset
     2
//
7cae6bc29ff9 rust-parsers: switch to parse/pack_dirstate to mutate-on-loop
Raphaël Gomès <rgomes@octobus.net>
parents: 42609
diff changeset
     3
// Copyright 2019 Raphaël Gomès <rgomes@octobus.net>
7cae6bc29ff9 rust-parsers: switch to parse/pack_dirstate to mutate-on-loop
Raphaël Gomès <rgomes@octobus.net>
parents: 42609
diff changeset
     4
//
7cae6bc29ff9 rust-parsers: switch to parse/pack_dirstate to mutate-on-loop
Raphaël Gomès <rgomes@octobus.net>
parents: 42609
diff changeset
     5
// This software may be used and distributed according to the terms of the
7cae6bc29ff9 rust-parsers: switch to parse/pack_dirstate to mutate-on-loop
Raphaël Gomès <rgomes@octobus.net>
parents: 42609
diff changeset
     6
// GNU General Public License version 2 or any later version.
7cae6bc29ff9 rust-parsers: switch to parse/pack_dirstate to mutate-on-loop
Raphaël Gomès <rgomes@octobus.net>
parents: 42609
diff changeset
     7
42957
7a01778bc7b7 rust-hgpath: replace all paths and filenames with HgPath/HgPathBuf
Raphaël Gomès <rgomes@octobus.net>
parents: 42886
diff changeset
     8
use crate::{utils::hg_path::HgPathBuf, DirstateParseError};
42886
7083ac37314f rust-dirstate: provide CopyMapIter and StateMapIter types
Yuya Nishihara <yuya@tcha.org>
parents: 42802
diff changeset
     9
use std::collections::hash_map;
42748
7cae6bc29ff9 rust-parsers: switch to parse/pack_dirstate to mutate-on-loop
Raphaël Gomès <rgomes@octobus.net>
parents: 42609
diff changeset
    10
use std::collections::HashMap;
42749
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    11
use std::convert::TryFrom;
42748
7cae6bc29ff9 rust-parsers: switch to parse/pack_dirstate to mutate-on-loop
Raphaël Gomès <rgomes@octobus.net>
parents: 42609
diff changeset
    12
42536
2dcee6497b0b rust-dirstate: add "dirs" Rust implementation
Raphaël Gomès <rgomes@octobus.net>
parents: 42424
diff changeset
    13
pub mod dirs_multiset;
42753
fce6dc93a510 rust-dirstate: rust implementation of dirstatemap
Raphaël Gomès <rgomes@octobus.net>
parents: 42749
diff changeset
    14
pub mod dirstate_map;
42424
d3b5cbe311d9 rust-dirstate: create dirstate submodule
Raphaël Gomès <rgomes@octobus.net>
parents:
diff changeset
    15
pub mod parsers;
43271
99394e6c5d12 rust-dirstate-status: add first Rust implementation of `dirstate.status`
Raphaël Gomès <rgomes@octobus.net>
parents: 42957
diff changeset
    16
pub mod status;
42424
d3b5cbe311d9 rust-dirstate: create dirstate submodule
Raphaël Gomès <rgomes@octobus.net>
parents:
diff changeset
    17
42748
7cae6bc29ff9 rust-parsers: switch to parse/pack_dirstate to mutate-on-loop
Raphaël Gomès <rgomes@octobus.net>
parents: 42609
diff changeset
    18
#[derive(Debug, PartialEq, Clone)]
7cae6bc29ff9 rust-parsers: switch to parse/pack_dirstate to mutate-on-loop
Raphaël Gomès <rgomes@octobus.net>
parents: 42609
diff changeset
    19
pub struct DirstateParents {
7cae6bc29ff9 rust-parsers: switch to parse/pack_dirstate to mutate-on-loop
Raphaël Gomès <rgomes@octobus.net>
parents: 42609
diff changeset
    20
    pub p1: [u8; 20],
7cae6bc29ff9 rust-parsers: switch to parse/pack_dirstate to mutate-on-loop
Raphaël Gomès <rgomes@octobus.net>
parents: 42609
diff changeset
    21
    pub p2: [u8; 20],
42424
d3b5cbe311d9 rust-dirstate: create dirstate submodule
Raphaël Gomès <rgomes@octobus.net>
parents:
diff changeset
    22
}
d3b5cbe311d9 rust-dirstate: create dirstate submodule
Raphaël Gomès <rgomes@octobus.net>
parents:
diff changeset
    23
d3b5cbe311d9 rust-dirstate: create dirstate submodule
Raphaël Gomès <rgomes@octobus.net>
parents:
diff changeset
    24
/// The C implementation uses all signed types. This will be an issue
d3b5cbe311d9 rust-dirstate: create dirstate submodule
Raphaël Gomès <rgomes@octobus.net>
parents:
diff changeset
    25
/// either when 4GB+ source files are commonplace or in 2038, whichever
d3b5cbe311d9 rust-dirstate: create dirstate submodule
Raphaël Gomès <rgomes@octobus.net>
parents:
diff changeset
    26
/// comes first.
42748
7cae6bc29ff9 rust-parsers: switch to parse/pack_dirstate to mutate-on-loop
Raphaël Gomès <rgomes@octobus.net>
parents: 42609
diff changeset
    27
#[derive(Debug, PartialEq, Copy, Clone)]
42424
d3b5cbe311d9 rust-dirstate: create dirstate submodule
Raphaël Gomès <rgomes@octobus.net>
parents:
diff changeset
    28
pub struct DirstateEntry {
42749
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    29
    pub state: EntryState,
42424
d3b5cbe311d9 rust-dirstate: create dirstate submodule
Raphaël Gomès <rgomes@octobus.net>
parents:
diff changeset
    30
    pub mode: i32,
d3b5cbe311d9 rust-dirstate: create dirstate submodule
Raphaël Gomès <rgomes@octobus.net>
parents:
diff changeset
    31
    pub mtime: i32,
d3b5cbe311d9 rust-dirstate: create dirstate submodule
Raphaël Gomès <rgomes@octobus.net>
parents:
diff changeset
    32
    pub size: i32,
d3b5cbe311d9 rust-dirstate: create dirstate submodule
Raphaël Gomès <rgomes@octobus.net>
parents:
diff changeset
    33
}
d3b5cbe311d9 rust-dirstate: create dirstate submodule
Raphaël Gomès <rgomes@octobus.net>
parents:
diff changeset
    34
42957
7a01778bc7b7 rust-hgpath: replace all paths and filenames with HgPath/HgPathBuf
Raphaël Gomès <rgomes@octobus.net>
parents: 42886
diff changeset
    35
pub type StateMap = HashMap<HgPathBuf, DirstateEntry>;
7a01778bc7b7 rust-hgpath: replace all paths and filenames with HgPath/HgPathBuf
Raphaël Gomès <rgomes@octobus.net>
parents: 42886
diff changeset
    36
pub type StateMapIter<'a> = hash_map::Iter<'a, HgPathBuf, DirstateEntry>;
7a01778bc7b7 rust-hgpath: replace all paths and filenames with HgPath/HgPathBuf
Raphaël Gomès <rgomes@octobus.net>
parents: 42886
diff changeset
    37
pub type CopyMap = HashMap<HgPathBuf, HgPathBuf>;
7a01778bc7b7 rust-hgpath: replace all paths and filenames with HgPath/HgPathBuf
Raphaël Gomès <rgomes@octobus.net>
parents: 42886
diff changeset
    38
pub type CopyMapIter<'a> = hash_map::Iter<'a, HgPathBuf, HgPathBuf>;
42536
2dcee6497b0b rust-dirstate: add "dirs" Rust implementation
Raphaël Gomès <rgomes@octobus.net>
parents: 42424
diff changeset
    39
42749
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    40
#[derive(Copy, Clone, Debug, Eq, PartialEq)]
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    41
pub enum EntryState {
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    42
    Normal,
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    43
    Added,
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    44
    Removed,
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    45
    Merged,
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    46
    Unknown,
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    47
}
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    48
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    49
impl TryFrom<u8> for EntryState {
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    50
    type Error = DirstateParseError;
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    51
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    52
    fn try_from(value: u8) -> Result<Self, Self::Error> {
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    53
        match value {
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    54
            b'n' => Ok(EntryState::Normal),
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    55
            b'a' => Ok(EntryState::Added),
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    56
            b'r' => Ok(EntryState::Removed),
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    57
            b'm' => Ok(EntryState::Merged),
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    58
            b'?' => Ok(EntryState::Unknown),
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    59
            _ => Err(DirstateParseError::CorruptedEntry(format!(
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    60
                "Incorrect entry state {}",
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    61
                value
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    62
            ))),
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    63
        }
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    64
    }
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    65
}
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    66
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    67
impl Into<u8> for EntryState {
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    68
    fn into(self) -> u8 {
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    69
        match self {
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    70
            EntryState::Normal => b'n',
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    71
            EntryState::Added => b'a',
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    72
            EntryState::Removed => b'r',
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    73
            EntryState::Merged => b'm',
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    74
            EntryState::Unknown => b'?',
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    75
        }
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    76
    }
7ceded4419a3 rust-dirstate: use EntryState enum instead of literals
Raphaël Gomès <rgomes@octobus.net>
parents: 42748
diff changeset
    77
}