contrib/automation/hgautomation/windows.py
author Augie Fackler <augie@google.com>
Sun, 06 Oct 2019 09:45:02 -0400
changeset 43076 2372284d9457
parent 42907 92593d72e10b
child 43513 081a77df7bc6
permissions -rw-r--r--
formatting: blacken the codebase This is using my patch to black (https://github.com/psf/black/pull/826) so we don't un-wrap collection literals. Done with: hg files 'set:**.py - mercurial/thirdparty/** - "contrib/python-zstandard/**"' | xargs black -S # skip-blame mass-reformatting only # no-check-commit reformats foo_bar functions Differential Revision: https://phab.mercurial-scm.org/D6971
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     1
# windows.py - Automation specific to Windows
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     2
#
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     3
# Copyright 2019 Gregory Szorc <gregory.szorc@gmail.com>
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     4
#
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     5
# This software may be used and distributed according to the terms of the
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     6
# GNU General Public License version 2 or any later version.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     7
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     8
# no-check-code because Python 3 native.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     9
42907
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
    10
import datetime
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    11
import os
42907
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
    12
import paramiko
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    13
import pathlib
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    14
import re
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    15
import subprocess
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    16
import tempfile
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    17
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
    18
from .pypi import upload as pypi_upload
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
    19
from .winrm import run_powershell
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    20
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    21
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    22
# PowerShell commands to activate a Visual Studio 2008 environment.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    23
# This is essentially a port of vcvarsall.bat to PowerShell.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    24
ACTIVATE_VC9_AMD64 = r'''
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    25
Write-Output "activating Visual Studio 2008 environment for AMD64"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    26
$root = "$env:LOCALAPPDATA\Programs\Common\Microsoft\Visual C++ for Python\9.0"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    27
$Env:VCINSTALLDIR = "${root}\VC\"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    28
$Env:WindowsSdkDir = "${root}\WinSDK\"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    29
$Env:PATH = "${root}\VC\Bin\amd64;${root}\WinSDK\Bin\x64;${root}\WinSDK\Bin;$Env:PATH"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    30
$Env:INCLUDE = "${root}\VC\Include;${root}\WinSDK\Include;$Env:PATH"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    31
$Env:LIB = "${root}\VC\Lib\amd64;${root}\WinSDK\Lib\x64;$Env:LIB"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    32
$Env:LIBPATH = "${root}\VC\Lib\amd64;${root}\WinSDK\Lib\x64;$Env:LIBPATH"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    33
'''.lstrip()
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    34
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    35
ACTIVATE_VC9_X86 = r'''
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    36
Write-Output "activating Visual Studio 2008 environment for x86"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    37
$root = "$env:LOCALAPPDATA\Programs\Common\Microsoft\Visual C++ for Python\9.0"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    38
$Env:VCINSTALLDIR = "${root}\VC\"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    39
$Env:WindowsSdkDir = "${root}\WinSDK\"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    40
$Env:PATH = "${root}\VC\Bin;${root}\WinSDK\Bin;$Env:PATH"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    41
$Env:INCLUDE = "${root}\VC\Include;${root}\WinSDK\Include;$Env:INCLUDE"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    42
$Env:LIB = "${root}\VC\Lib;${root}\WinSDK\Lib;$Env:LIB"
42601
862f6bddacce automation: correct the path separator in LIBPATH on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 42282
diff changeset
    43
$Env:LIBPATH = "${root}\VC\lib;${root}\WinSDK\Lib;$Env:LIBPATH"
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    44
'''.lstrip()
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    45
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    46
HG_PURGE = r'''
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    47
$Env:PATH = "C:\hgdev\venv-bootstrap\Scripts;$Env:PATH"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    48
Set-Location C:\hgdev\src
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    49
hg.exe --config extensions.purge= purge --all
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    50
if ($LASTEXITCODE -ne 0) {
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    51
    throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    52
}
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    53
Write-Output "purged Mercurial repo"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    54
'''
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    55
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    56
HG_UPDATE_CLEAN = r'''
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    57
$Env:PATH = "C:\hgdev\venv-bootstrap\Scripts;$Env:PATH"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    58
Set-Location C:\hgdev\src
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    59
hg.exe --config extensions.purge= purge --all
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    60
if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    61
    throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    62
}}
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    63
hg.exe update -C {revision}
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    64
if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    65
    throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    66
}}
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    67
hg.exe log -r .
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    68
Write-Output "updated Mercurial working directory to {revision}"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    69
'''.lstrip()
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    70
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    71
BUILD_INNO = r'''
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    72
Set-Location C:\hgdev\src
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    73
$python = "C:\hgdev\python27-{arch}\python.exe"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    74
C:\hgdev\python37-x64\python.exe contrib\packaging\inno\build.py --python $python
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    75
if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    76
    throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    77
}}
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    78
'''.lstrip()
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    79
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    80
BUILD_WHEEL = r'''
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    81
Set-Location C:\hgdev\src
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    82
C:\hgdev\python27-{arch}\Scripts\pip.exe wheel --wheel-dir dist .
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    83
if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    84
    throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    85
}}
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    86
'''
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    87
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    88
BUILD_WIX = r'''
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    89
Set-Location C:\hgdev\src
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    90
$python = "C:\hgdev\python27-{arch}\python.exe"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    91
C:\hgdev\python37-x64\python.exe contrib\packaging\wix\build.py --python $python {extra_args}
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    92
if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    93
    throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    94
}}
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    95
'''
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    96
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    97
RUN_TESTS = r'''
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    98
C:\hgdev\MinGW\msys\1.0\bin\sh.exe --login -c "cd /c/hgdev/src/tests && /c/hgdev/{python_path}/python.exe run-tests.py {test_flags}"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    99
if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   100
    throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   101
}}
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   102
'''
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   103
42907
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   104
X86_WHEEL_FILENAME = 'mercurial-{version}-cp27-cp27m-win32.whl'
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   105
X64_WHEEL_FILENAME = 'mercurial-{version}-cp27-cp27m-win_amd64.whl'
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   106
X86_EXE_FILENAME = 'Mercurial-{version}.exe'
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   107
X64_EXE_FILENAME = 'Mercurial-{version}-x64.exe'
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   108
X86_MSI_FILENAME = 'mercurial-{version}-x86.msi'
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   109
X64_MSI_FILENAME = 'mercurial-{version}-x64.msi'
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   110
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   111
MERCURIAL_SCM_BASE_URL = 'https://mercurial-scm.org/release/windows'
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   112
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   113
X86_USER_AGENT_PATTERN = '.*Windows.*'
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   114
X64_USER_AGENT_PATTERN = '.*Windows.*(WOW|x)64.*'
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   115
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   116
X86_EXE_DESCRIPTION = (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   117
    'Mercurial {version} Inno Setup installer - x86 Windows '
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   118
    '- does not require admin rights'
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   119
)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   120
X64_EXE_DESCRIPTION = (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   121
    'Mercurial {version} Inno Setup installer - x64 Windows '
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   122
    '- does not require admin rights'
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   123
)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   124
X86_MSI_DESCRIPTION = (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   125
    'Mercurial {version} MSI installer - x86 Windows ' '- requires admin rights'
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   126
)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   127
X64_MSI_DESCRIPTION = (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   128
    'Mercurial {version} MSI installer - x64 Windows ' '- requires admin rights'
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   129
)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   130
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   131
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   132
def get_vc_prefix(arch):
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   133
    if arch == 'x86':
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   134
        return ACTIVATE_VC9_X86
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   135
    elif arch == 'x64':
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   136
        return ACTIVATE_VC9_AMD64
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   137
    else:
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   138
        raise ValueError('illegal arch: %s; must be x86 or x64' % arch)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   139
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   140
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   141
def fix_authorized_keys_permissions(winrm_client, path):
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   142
    commands = [
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   143
        '$ErrorActionPreference = "Stop"',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   144
        'Repair-AuthorizedKeyPermission -FilePath %s -Confirm:$false' % path,
42064
0e9066db5e44 automation: use raw strings when there are backslashes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42024
diff changeset
   145
        r'icacls %s /remove:g "NT Service\sshd"' % path,
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   146
    ]
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   147
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   148
    run_powershell(winrm_client, '\n'.join(commands))
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   149
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   150
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   151
def synchronize_hg(hg_repo: pathlib.Path, revision: str, ec2_instance):
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   152
    """Synchronize local Mercurial repo to remote EC2 instance."""
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   153
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   154
    winrm_client = ec2_instance.winrm_client
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   155
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   156
    with tempfile.TemporaryDirectory() as temp_dir:
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   157
        temp_dir = pathlib.Path(temp_dir)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   158
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   159
        ssh_dir = temp_dir / '.ssh'
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   160
        ssh_dir.mkdir()
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   161
        ssh_dir.chmod(0o0700)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   162
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   163
        # Generate SSH key to use for communication.
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   164
        subprocess.run(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   165
            [
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   166
                'ssh-keygen',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   167
                '-t',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   168
                'rsa',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   169
                '-b',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   170
                '4096',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   171
                '-N',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   172
                '',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   173
                '-f',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   174
                str(ssh_dir / 'id_rsa'),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   175
            ],
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   176
            check=True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   177
            capture_output=True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   178
        )
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   179
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   180
        # Add it to ~/.ssh/authorized_keys on remote.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   181
        # This assumes the file doesn't already exist.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   182
        authorized_keys = r'c:\Users\Administrator\.ssh\authorized_keys'
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   183
        winrm_client.execute_cmd(r'mkdir c:\Users\Administrator\.ssh')
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   184
        winrm_client.copy(str(ssh_dir / 'id_rsa.pub'), authorized_keys)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   185
        fix_authorized_keys_permissions(winrm_client, authorized_keys)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   186
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   187
        public_ip = ec2_instance.public_ip_address
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   188
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   189
        ssh_config = temp_dir / '.ssh' / 'config'
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   190
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   191
        with open(ssh_config, 'w', encoding='utf-8') as fh:
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   192
            fh.write('Host %s\n' % public_ip)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   193
            fh.write('  User Administrator\n')
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   194
            fh.write('  StrictHostKeyChecking no\n')
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   195
            fh.write('  UserKnownHostsFile %s\n' % (ssh_dir / 'known_hosts'))
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   196
            fh.write('  IdentityFile %s\n' % (ssh_dir / 'id_rsa'))
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   197
42281
4274b1369b75 automation: add check that hg source directory is a repo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42064
diff changeset
   198
        if not (hg_repo / '.hg').is_dir():
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   199
            raise Exception(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   200
                '%s is not a Mercurial repository; '
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   201
                'synchronization not yet supported' % hg_repo
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   202
            )
42281
4274b1369b75 automation: add check that hg source directory is a repo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42064
diff changeset
   203
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   204
        env = dict(os.environ)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   205
        env['HGPLAIN'] = '1'
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   206
        env['HGENCODING'] = 'utf-8'
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   207
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   208
        hg_bin = hg_repo / 'hg'
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   209
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   210
        res = subprocess.run(
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   211
            ['python2.7', str(hg_bin), 'log', '-r', revision, '-T', '{node}'],
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   212
            cwd=str(hg_repo),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   213
            env=env,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   214
            check=True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   215
            capture_output=True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   216
        )
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   217
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   218
        full_revision = res.stdout.decode('ascii')
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   219
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   220
        args = [
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   221
            'python2.7',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   222
            hg_bin,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   223
            '--config',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   224
            'ui.ssh=ssh -F %s' % ssh_config,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   225
            '--config',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   226
            'ui.remotecmd=c:/hgdev/venv-bootstrap/Scripts/hg.exe',
42684
9e0f1c80cddb automation: push changes affecting .hgtags
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42660
diff changeset
   227
            # Also ensure .hgtags changes are present so auto version
9e0f1c80cddb automation: push changes affecting .hgtags
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42660
diff changeset
   228
            # calculation works.
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   229
            'push',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   230
            '-f',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   231
            '-r',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   232
            full_revision,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   233
            '-r',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   234
            'file(.hgtags)',
42282
5c242c427897 automation: do a force push to synchronize
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42281
diff changeset
   235
            'ssh://%s/c:/hgdev/src' % public_ip,
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   236
        ]
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   237
42660
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42601
diff changeset
   238
        res = subprocess.run(args, cwd=str(hg_repo), env=env)
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42601
diff changeset
   239
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42601
diff changeset
   240
        # Allow 1 (no-op) to not trigger error.
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42601
diff changeset
   241
        if res.returncode not in (0, 1):
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42601
diff changeset
   242
            res.check_returncode()
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   243
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   244
        run_powershell(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   245
            winrm_client, HG_UPDATE_CLEAN.format(revision=full_revision)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   246
        )
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   247
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   248
        # TODO detect dirty local working directory and synchronize accordingly.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   249
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   250
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   251
def purge_hg(winrm_client):
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   252
    """Purge the Mercurial source repository on an EC2 instance."""
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   253
    run_powershell(winrm_client, HG_PURGE)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   254
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   255
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   256
def find_latest_dist(winrm_client, pattern):
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   257
    """Find path to newest file in dist/ directory matching a pattern."""
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   258
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   259
    res = winrm_client.execute_ps(
42064
0e9066db5e44 automation: use raw strings when there are backslashes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42024
diff changeset
   260
        r'$v = Get-ChildItem -Path C:\hgdev\src\dist -Filter "%s" '
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   261
        '| Sort-Object LastWriteTime -Descending '
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   262
        '| Select-Object -First 1\n'
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   263
        '$v.name' % pattern
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   264
    )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   265
    return res[0]
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   266
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   267
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   268
def copy_latest_dist(winrm_client, pattern, dest_path):
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   269
    """Copy latest file matching pattern in dist/ directory.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   270
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   271
    Given a WinRM client and a file pattern, find the latest file on the remote
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   272
    matching that pattern and copy it to the ``dest_path`` directory on the
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   273
    local machine.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   274
    """
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   275
    latest = find_latest_dist(winrm_client, pattern)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   276
    source = r'C:\hgdev\src\dist\%s' % latest
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   277
    dest = dest_path / latest
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   278
    print('copying %s to %s' % (source, dest))
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   279
    winrm_client.fetch(source, str(dest))
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   280
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   281
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   282
def build_inno_installer(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   283
    winrm_client, arch: str, dest_path: pathlib.Path, version=None
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   284
):
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   285
    """Build the Inno Setup installer on a remote machine.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   286
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   287
    Using a WinRM client, remote commands are executed to build
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   288
    a Mercurial Inno Setup installer.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   289
    """
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   290
    print('building Inno Setup installer for %s' % arch)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   291
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   292
    extra_args = []
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   293
    if version:
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   294
        extra_args.extend(['--version', version])
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   295
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   296
    ps = get_vc_prefix(arch) + BUILD_INNO.format(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   297
        arch=arch, extra_args=' '.join(extra_args)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   298
    )
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   299
    run_powershell(winrm_client, ps)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   300
    copy_latest_dist(winrm_client, '*.exe', dest_path)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   301
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   302
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   303
def build_wheel(winrm_client, arch: str, dest_path: pathlib.Path):
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   304
    """Build Python wheels on a remote machine.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   305
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   306
    Using a WinRM client, remote commands are executed to build a Python wheel
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   307
    for Mercurial.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   308
    """
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   309
    print('Building Windows wheel for %s' % arch)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   310
    ps = get_vc_prefix(arch) + BUILD_WHEEL.format(arch=arch)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   311
    run_powershell(winrm_client, ps)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   312
    copy_latest_dist(winrm_client, '*.whl', dest_path)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   313
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   314
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   315
def build_wix_installer(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   316
    winrm_client, arch: str, dest_path: pathlib.Path, version=None
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   317
):
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   318
    """Build the WiX installer on a remote machine.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   319
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   320
    Using a WinRM client, remote commands are executed to build a WiX installer.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   321
    """
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   322
    print('Building WiX installer for %s' % arch)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   323
    extra_args = []
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   324
    if version:
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   325
        extra_args.extend(['--version', version])
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   326
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   327
    ps = get_vc_prefix(arch) + BUILD_WIX.format(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   328
        arch=arch, extra_args=' '.join(extra_args)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   329
    )
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   330
    run_powershell(winrm_client, ps)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   331
    copy_latest_dist(winrm_client, '*.msi', dest_path)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   332
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   333
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   334
def run_tests(winrm_client, python_version, arch, test_flags=''):
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   335
    """Run tests on a remote Windows machine.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   336
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   337
    ``python_version`` is a ``X.Y`` string like ``2.7`` or ``3.7``.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   338
    ``arch`` is ``x86`` or ``x64``.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   339
    ``test_flags`` is a str representing extra arguments to pass to
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   340
    ``run-tests.py``.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   341
    """
42064
0e9066db5e44 automation: use raw strings when there are backslashes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42024
diff changeset
   342
    if not re.match(r'\d\.\d', python_version):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   343
        raise ValueError(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   344
            r'python_version must be \d.\d; got %s' % python_version
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   345
        )
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   346
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   347
    if arch not in ('x86', 'x64'):
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   348
        raise ValueError('arch must be x86 or x64; got %s' % arch)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   349
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   350
    python_path = 'python%s-%s' % (python_version.replace('.', ''), arch)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   351
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   352
    ps = RUN_TESTS.format(python_path=python_path, test_flags=test_flags or '',)
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   353
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   354
    run_powershell(winrm_client, ps)
42907
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   355
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   356
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   357
def resolve_wheel_artifacts(dist_path: pathlib.Path, version: str):
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   358
    return (
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   359
        dist_path / X86_WHEEL_FILENAME.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   360
        dist_path / X64_WHEEL_FILENAME.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   361
    )
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   362
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   363
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   364
def resolve_all_artifacts(dist_path: pathlib.Path, version: str):
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   365
    return (
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   366
        dist_path / X86_WHEEL_FILENAME.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   367
        dist_path / X64_WHEEL_FILENAME.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   368
        dist_path / X86_EXE_FILENAME.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   369
        dist_path / X64_EXE_FILENAME.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   370
        dist_path / X86_MSI_FILENAME.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   371
        dist_path / X64_MSI_FILENAME.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   372
    )
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   373
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   374
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   375
def generate_latest_dat(version: str):
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   376
    x86_exe_filename = X86_EXE_FILENAME.format(version=version)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   377
    x64_exe_filename = X64_EXE_FILENAME.format(version=version)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   378
    x86_msi_filename = X86_MSI_FILENAME.format(version=version)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   379
    x64_msi_filename = X64_MSI_FILENAME.format(version=version)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   380
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   381
    entries = (
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   382
        (
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   383
            '10',
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   384
            version,
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   385
            X86_USER_AGENT_PATTERN,
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   386
            '%s/%s' % (MERCURIAL_SCM_BASE_URL, x86_exe_filename),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   387
            X86_EXE_DESCRIPTION.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   388
        ),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   389
        (
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   390
            '10',
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   391
            version,
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   392
            X64_USER_AGENT_PATTERN,
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   393
            '%s/%s' % (MERCURIAL_SCM_BASE_URL, x64_exe_filename),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   394
            X64_EXE_DESCRIPTION.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   395
        ),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   396
        (
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   397
            '10',
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   398
            version,
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   399
            X86_USER_AGENT_PATTERN,
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   400
            '%s/%s' % (MERCURIAL_SCM_BASE_URL, x86_msi_filename),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   401
            X86_MSI_DESCRIPTION.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   402
        ),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   403
        (
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   404
            '10',
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   405
            version,
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   406
            X64_USER_AGENT_PATTERN,
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   407
            '%s/%s' % (MERCURIAL_SCM_BASE_URL, x64_msi_filename),
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   408
            X64_MSI_DESCRIPTION.format(version=version),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   409
        ),
42907
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   410
    )
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   411
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   412
    lines = ['\t'.join(e) for e in entries]
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   413
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   414
    return '\n'.join(lines) + '\n'
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   415
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   416
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   417
def publish_artifacts_pypi(dist_path: pathlib.Path, version: str):
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   418
    """Publish Windows release artifacts to PyPI."""
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   419
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   420
    wheel_paths = resolve_wheel_artifacts(dist_path, version)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   421
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   422
    for p in wheel_paths:
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   423
        if not p.exists():
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   424
            raise Exception('%s not found' % p)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   425
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   426
    print('uploading wheels to PyPI (you may be prompted for credentials)')
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   427
    pypi_upload(wheel_paths)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   428
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   429
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   430
def publish_artifacts_mercurial_scm_org(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   431
    dist_path: pathlib.Path, version: str, ssh_username=None
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   432
):
42907
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   433
    """Publish Windows release artifacts to mercurial-scm.org."""
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   434
    all_paths = resolve_all_artifacts(dist_path, version)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   435
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   436
    for p in all_paths:
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   437
        if not p.exists():
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   438
            raise Exception('%s not found' % p)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   439
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   440
    client = paramiko.SSHClient()
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   441
    client.load_system_host_keys()
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   442
    # We assume the system SSH configuration knows how to connect.
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   443
    print('connecting to mercurial-scm.org via ssh...')
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   444
    try:
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   445
        client.connect('mercurial-scm.org', username=ssh_username)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   446
    except paramiko.AuthenticationException:
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   447
        print('error authenticating; is an SSH key available in an SSH agent?')
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   448
        raise
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   449
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   450
    print('SSH connection established')
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   451
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   452
    print('opening SFTP client...')
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   453
    sftp = client.open_sftp()
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   454
    print('SFTP client obtained')
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   455
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   456
    for p in all_paths:
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   457
        dest_path = '/var/www/release/windows/%s' % p.name
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   458
        print('uploading %s to %s' % (p, dest_path))
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   459
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   460
        with p.open('rb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   461
            data = fh.read()
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   462
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   463
        with sftp.open(dest_path, 'wb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   464
            fh.write(data)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   465
            fh.chmod(0o0664)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   466
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   467
    latest_dat_path = '/var/www/release/windows/latest.dat'
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   468
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   469
    now = datetime.datetime.utcnow()
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   470
    backup_path = dist_path / (
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   471
        'latest-windows-%s.dat' % now.strftime('%Y%m%dT%H%M%S')
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   472
    )
42907
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   473
    print('backing up %s to %s' % (latest_dat_path, backup_path))
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   474
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   475
    with sftp.open(latest_dat_path, 'rb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   476
        latest_dat_old = fh.read()
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   477
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   478
    with backup_path.open('wb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   479
        fh.write(latest_dat_old)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   480
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   481
    print('writing %s with content:' % latest_dat_path)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   482
    latest_dat_content = generate_latest_dat(version)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   483
    print(latest_dat_content)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   484
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   485
    with sftp.open(latest_dat_path, 'wb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   486
        fh.write(latest_dat_content.encode('ascii'))
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   487
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   488
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   489
def publish_artifacts(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   490
    dist_path: pathlib.Path,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   491
    version: str,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   492
    pypi=True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   493
    mercurial_scm_org=True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   494
    ssh_username=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   495
):
42907
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   496
    """Publish Windows release artifacts.
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   497
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   498
    Files are found in `dist_path`. We will look for files with version string
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   499
    `version`.
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   500
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   501
    `pypi` controls whether we upload to PyPI.
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   502
    `mercurial_scm_org` controls whether we upload to mercurial-scm.org.
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   503
    """
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   504
    if pypi:
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   505
        publish_artifacts_pypi(dist_path, version)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   506
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   507
    if mercurial_scm_org:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   508
        publish_artifacts_mercurial_scm_org(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   509
            dist_path, version, ssh_username=ssh_username
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   510
        )