contrib/automation/hgautomation/windows.py
author Gregory Szorc <gregory.szorc@gmail.com>
Fri, 24 Apr 2020 12:11:08 -0700
branchstable
changeset 44771 802ee93c205d
parent 44770 47609da15379
child 44772 5e788dc7fb5d
permissions -rw-r--r--
automation: support building Python 3 Inno installers The core packaging code now supports building Python 3 installers using PyOxidizer. Let's teach the automation code to invoke it so that we produce both Python 2 and Python 3 based exe installers. When publishing the artifacts, the Python 3 versions are preferred over the Python 2 versions given their higher weight (10 versus 9). This may be a controversial change. But I think making Python 3 the default is warranted, as it is the future. The Python 2 installers are still fully supported and can be installed should issues with Python 3 arise. Differential Revision: https://phab.mercurial-scm.org/D8483
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     1
# windows.py - Automation specific to Windows
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     2
#
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     3
# Copyright 2019 Gregory Szorc <gregory.szorc@gmail.com>
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     4
#
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     5
# This software may be used and distributed according to the terms of the
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     6
# GNU General Public License version 2 or any later version.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     7
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     8
# no-check-code because Python 3 native.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
     9
42907
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
    10
import datetime
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    11
import os
42907
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
    12
import paramiko
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    13
import pathlib
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    14
import re
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    15
import subprocess
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    16
import tempfile
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    17
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
    18
from .pypi import upload as pypi_upload
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
    19
from .winrm import run_powershell
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    20
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    21
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    22
# PowerShell commands to activate a Visual Studio 2008 environment.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    23
# This is essentially a port of vcvarsall.bat to PowerShell.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    24
ACTIVATE_VC9_AMD64 = r'''
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    25
Write-Output "activating Visual Studio 2008 environment for AMD64"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    26
$root = "$env:LOCALAPPDATA\Programs\Common\Microsoft\Visual C++ for Python\9.0"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    27
$Env:VCINSTALLDIR = "${root}\VC\"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    28
$Env:WindowsSdkDir = "${root}\WinSDK\"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    29
$Env:PATH = "${root}\VC\Bin\amd64;${root}\WinSDK\Bin\x64;${root}\WinSDK\Bin;$Env:PATH"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    30
$Env:INCLUDE = "${root}\VC\Include;${root}\WinSDK\Include;$Env:PATH"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    31
$Env:LIB = "${root}\VC\Lib\amd64;${root}\WinSDK\Lib\x64;$Env:LIB"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    32
$Env:LIBPATH = "${root}\VC\Lib\amd64;${root}\WinSDK\Lib\x64;$Env:LIBPATH"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    33
'''.lstrip()
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    34
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    35
ACTIVATE_VC9_X86 = r'''
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    36
Write-Output "activating Visual Studio 2008 environment for x86"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    37
$root = "$env:LOCALAPPDATA\Programs\Common\Microsoft\Visual C++ for Python\9.0"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    38
$Env:VCINSTALLDIR = "${root}\VC\"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    39
$Env:WindowsSdkDir = "${root}\WinSDK\"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    40
$Env:PATH = "${root}\VC\Bin;${root}\WinSDK\Bin;$Env:PATH"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    41
$Env:INCLUDE = "${root}\VC\Include;${root}\WinSDK\Include;$Env:INCLUDE"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    42
$Env:LIB = "${root}\VC\Lib;${root}\WinSDK\Lib;$Env:LIB"
42601
862f6bddacce automation: correct the path separator in LIBPATH on Windows
Matt Harbison <matt_harbison@yahoo.com>
parents: 42282
diff changeset
    43
$Env:LIBPATH = "${root}\VC\lib;${root}\WinSDK\Lib;$Env:LIBPATH"
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    44
'''.lstrip()
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    45
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    46
HG_PURGE = r'''
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    47
$Env:PATH = "C:\hgdev\venv-bootstrap\Scripts;$Env:PATH"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    48
Set-Location C:\hgdev\src
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    49
hg.exe --config extensions.purge= purge --all
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    50
if ($LASTEXITCODE -ne 0) {
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    51
    throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    52
}
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    53
Write-Output "purged Mercurial repo"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    54
'''
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    55
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    56
HG_UPDATE_CLEAN = r'''
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    57
$Env:PATH = "C:\hgdev\venv-bootstrap\Scripts;$Env:PATH"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    58
Set-Location C:\hgdev\src
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    59
hg.exe --config extensions.purge= purge --all
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    60
if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    61
    throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    62
}}
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    63
hg.exe update -C {revision}
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    64
if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    65
    throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    66
}}
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    67
hg.exe log -r .
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    68
Write-Output "updated Mercurial working directory to {revision}"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    69
'''.lstrip()
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    70
44771
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
    71
BUILD_INNO_PYTHON3 = r'''
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
    72
$Env:RUSTUP_HOME = "C:\hgdev\rustup"
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
    73
$Env:CARGO_HOME = "C:\hgdev\cargo"
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
    74
Set-Location C:\hgdev\src
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
    75
C:\hgdev\python37-x64\python.exe contrib\packaging\packaging.py inno --pyoxidizer-target {pyoxidizer_target} --version {version}
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
    76
if ($LASTEXITCODE -ne 0) {{
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
    77
    throw "process exited non-0: $LASTEXITCODE"
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
    78
}}
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
    79
'''
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
    80
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
    81
BUILD_INNO_PYTHON2 = r'''
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    82
Set-Location C:\hgdev\src
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    83
$python = "C:\hgdev\python27-{arch}\python.exe"
44770
47609da15379 automation: add extra arguments when building Inno
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44769
diff changeset
    84
C:\hgdev\python37-x64\python.exe contrib\packaging\packaging.py inno --python $python {extra_args}
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    85
if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    86
    throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    87
}}
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    88
'''.lstrip()
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    89
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    90
BUILD_WHEEL = r'''
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    91
Set-Location C:\hgdev\src
44768
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43513
diff changeset
    92
C:\hgdev\python{python_version}-{arch}\python.exe -m pip wheel --wheel-dir dist .
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    93
if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    94
    throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    95
}}
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    96
'''
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    97
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    98
BUILD_WIX = r'''
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
    99
Set-Location C:\hgdev\src
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   100
$python = "C:\hgdev\python27-{arch}\python.exe"
43513
081a77df7bc6 packaging: consolidate CLI functionality into packaging.py
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43076
diff changeset
   101
C:\hgdev\python37-x64\python.exe contrib\packaging\packaging.py wix --python $python {extra_args}
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   102
if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   103
    throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   104
}}
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   105
'''
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   106
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   107
RUN_TESTS = r'''
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   108
C:\hgdev\MinGW\msys\1.0\bin\sh.exe --login -c "cd /c/hgdev/src/tests && /c/hgdev/{python_path}/python.exe run-tests.py {test_flags}"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   109
if ($LASTEXITCODE -ne 0) {{
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   110
    throw "process exited non-0: $LASTEXITCODE"
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   111
}}
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   112
'''
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   113
44768
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43513
diff changeset
   114
WHEEL_FILENAME_PYTHON27_X86 = 'mercurial-{version}-cp27-cp27m-win32.whl'
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43513
diff changeset
   115
WHEEL_FILENAME_PYTHON27_X64 = 'mercurial-{version}-cp27-cp27m-win_amd64.whl'
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43513
diff changeset
   116
WHEEL_FILENAME_PYTHON37_X86 = 'mercurial-{version}-cp37-cp37m-win32.whl'
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43513
diff changeset
   117
WHEEL_FILENAME_PYTHON37_X64 = 'mercurial-{version}-cp37-cp37m-win_amd64.whl'
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43513
diff changeset
   118
WHEEL_FILENAME_PYTHON38_X86 = 'mercurial-{version}-cp38-cp38-win32.whl'
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43513
diff changeset
   119
WHEEL_FILENAME_PYTHON38_X64 = 'mercurial-{version}-cp38-cp38-win_amd64.whl'
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43513
diff changeset
   120
44771
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   121
EXE_FILENAME_PYTHON2_X86 = 'Mercurial-{version}-x86-python2.exe'
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   122
EXE_FILENAME_PYTHON2_X64 = 'Mercurial-{version}-x64-python2.exe'
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   123
EXE_FILENAME_PYTHON3_X86 = 'Mercurial-{version}-x86.exe'
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   124
EXE_FILENAME_PYTHON3_X64 = 'Mercurial-{version}-x64.exe'
44769
9ade217b550d packaging: add -python2 to Windows installer filenames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44768
diff changeset
   125
X86_MSI_FILENAME = 'mercurial-{version}-x86-python2.msi'
9ade217b550d packaging: add -python2 to Windows installer filenames
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44768
diff changeset
   126
X64_MSI_FILENAME = 'mercurial-{version}-x64-python2.msi'
42907
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   127
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   128
MERCURIAL_SCM_BASE_URL = 'https://mercurial-scm.org/release/windows'
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   129
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   130
X86_USER_AGENT_PATTERN = '.*Windows.*'
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   131
X64_USER_AGENT_PATTERN = '.*Windows.*(WOW|x)64.*'
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   132
44771
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   133
EXE_PYTHON2_X86_DESCRIPTION = (
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   134
    'Mercurial {version} Inno Setup installer - x86 Windows (Python 2) '
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   135
    '- does not require admin rights'
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   136
)
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   137
EXE_PYTHON2_X64_DESCRIPTION = (
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   138
    'Mercurial {version} Inno Setup installer - x64 Windows (Python 2) '
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   139
    '- does not require admin rights'
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   140
)
44771
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   141
# TODO remove Python version once Python 2 is dropped.
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   142
EXE_PYTHON3_X86_DESCRIPTION = (
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   143
    'Mercurial {version} Inno Setup installer - x86 Windows (Python 3) '
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   144
    '- does not require admin rights'
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   145
)
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   146
EXE_PYTHON3_X64_DESCRIPTION = (
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   147
    'Mercurial {version} Inno Setup installer - x64 Windows (Python 3) '
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   148
    '- does not require admin rights'
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   149
)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   150
X86_MSI_DESCRIPTION = (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   151
    'Mercurial {version} MSI installer - x86 Windows ' '- requires admin rights'
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   152
)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   153
X64_MSI_DESCRIPTION = (
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   154
    'Mercurial {version} MSI installer - x64 Windows ' '- requires admin rights'
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   155
)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   156
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   157
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   158
def get_vc_prefix(arch):
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   159
    if arch == 'x86':
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   160
        return ACTIVATE_VC9_X86
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   161
    elif arch == 'x64':
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   162
        return ACTIVATE_VC9_AMD64
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   163
    else:
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   164
        raise ValueError('illegal arch: %s; must be x86 or x64' % arch)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   165
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   166
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   167
def fix_authorized_keys_permissions(winrm_client, path):
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   168
    commands = [
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   169
        '$ErrorActionPreference = "Stop"',
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   170
        'Repair-AuthorizedKeyPermission -FilePath %s -Confirm:$false' % path,
42064
0e9066db5e44 automation: use raw strings when there are backslashes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42024
diff changeset
   171
        r'icacls %s /remove:g "NT Service\sshd"' % path,
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   172
    ]
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   173
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   174
    run_powershell(winrm_client, '\n'.join(commands))
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   175
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   176
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   177
def synchronize_hg(hg_repo: pathlib.Path, revision: str, ec2_instance):
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   178
    """Synchronize local Mercurial repo to remote EC2 instance."""
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   179
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   180
    winrm_client = ec2_instance.winrm_client
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   181
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   182
    with tempfile.TemporaryDirectory() as temp_dir:
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   183
        temp_dir = pathlib.Path(temp_dir)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   184
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   185
        ssh_dir = temp_dir / '.ssh'
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   186
        ssh_dir.mkdir()
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   187
        ssh_dir.chmod(0o0700)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   188
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   189
        # Generate SSH key to use for communication.
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   190
        subprocess.run(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   191
            [
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   192
                'ssh-keygen',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   193
                '-t',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   194
                'rsa',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   195
                '-b',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   196
                '4096',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   197
                '-N',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   198
                '',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   199
                '-f',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   200
                str(ssh_dir / 'id_rsa'),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   201
            ],
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   202
            check=True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   203
            capture_output=True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   204
        )
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   205
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   206
        # Add it to ~/.ssh/authorized_keys on remote.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   207
        # This assumes the file doesn't already exist.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   208
        authorized_keys = r'c:\Users\Administrator\.ssh\authorized_keys'
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   209
        winrm_client.execute_cmd(r'mkdir c:\Users\Administrator\.ssh')
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   210
        winrm_client.copy(str(ssh_dir / 'id_rsa.pub'), authorized_keys)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   211
        fix_authorized_keys_permissions(winrm_client, authorized_keys)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   212
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   213
        public_ip = ec2_instance.public_ip_address
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   214
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   215
        ssh_config = temp_dir / '.ssh' / 'config'
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   216
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   217
        with open(ssh_config, 'w', encoding='utf-8') as fh:
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   218
            fh.write('Host %s\n' % public_ip)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   219
            fh.write('  User Administrator\n')
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   220
            fh.write('  StrictHostKeyChecking no\n')
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   221
            fh.write('  UserKnownHostsFile %s\n' % (ssh_dir / 'known_hosts'))
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   222
            fh.write('  IdentityFile %s\n' % (ssh_dir / 'id_rsa'))
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   223
42281
4274b1369b75 automation: add check that hg source directory is a repo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42064
diff changeset
   224
        if not (hg_repo / '.hg').is_dir():
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   225
            raise Exception(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   226
                '%s is not a Mercurial repository; '
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   227
                'synchronization not yet supported' % hg_repo
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   228
            )
42281
4274b1369b75 automation: add check that hg source directory is a repo
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42064
diff changeset
   229
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   230
        env = dict(os.environ)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   231
        env['HGPLAIN'] = '1'
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   232
        env['HGENCODING'] = 'utf-8'
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   233
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   234
        hg_bin = hg_repo / 'hg'
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   235
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   236
        res = subprocess.run(
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   237
            ['python2.7', str(hg_bin), 'log', '-r', revision, '-T', '{node}'],
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   238
            cwd=str(hg_repo),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   239
            env=env,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   240
            check=True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   241
            capture_output=True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   242
        )
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   243
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   244
        full_revision = res.stdout.decode('ascii')
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   245
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   246
        args = [
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   247
            'python2.7',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   248
            hg_bin,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   249
            '--config',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   250
            'ui.ssh=ssh -F %s' % ssh_config,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   251
            '--config',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   252
            'ui.remotecmd=c:/hgdev/venv-bootstrap/Scripts/hg.exe',
42684
9e0f1c80cddb automation: push changes affecting .hgtags
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42660
diff changeset
   253
            # Also ensure .hgtags changes are present so auto version
9e0f1c80cddb automation: push changes affecting .hgtags
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42660
diff changeset
   254
            # calculation works.
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   255
            'push',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   256
            '-f',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   257
            '-r',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   258
            full_revision,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   259
            '-r',
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   260
            'file(.hgtags)',
42282
5c242c427897 automation: do a force push to synchronize
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42281
diff changeset
   261
            'ssh://%s/c:/hgdev/src' % public_ip,
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   262
        ]
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   263
42660
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42601
diff changeset
   264
        res = subprocess.run(args, cwd=str(hg_repo), env=env)
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42601
diff changeset
   265
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42601
diff changeset
   266
        # Allow 1 (no-op) to not trigger error.
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42601
diff changeset
   267
        if res.returncode not in (0, 1):
24cd5b0ba5b3 automation: allow exit code of 1 for `hg push`
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42601
diff changeset
   268
            res.check_returncode()
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   269
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   270
        run_powershell(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   271
            winrm_client, HG_UPDATE_CLEAN.format(revision=full_revision)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   272
        )
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   273
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   274
        # TODO detect dirty local working directory and synchronize accordingly.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   275
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   276
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   277
def purge_hg(winrm_client):
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   278
    """Purge the Mercurial source repository on an EC2 instance."""
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   279
    run_powershell(winrm_client, HG_PURGE)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   280
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   281
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   282
def find_latest_dist(winrm_client, pattern):
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   283
    """Find path to newest file in dist/ directory matching a pattern."""
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   284
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   285
    res = winrm_client.execute_ps(
42064
0e9066db5e44 automation: use raw strings when there are backslashes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42024
diff changeset
   286
        r'$v = Get-ChildItem -Path C:\hgdev\src\dist -Filter "%s" '
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   287
        '| Sort-Object LastWriteTime -Descending '
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   288
        '| Select-Object -First 1\n'
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   289
        '$v.name' % pattern
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   290
    )
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   291
    return res[0]
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   292
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   293
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   294
def copy_latest_dist(winrm_client, pattern, dest_path):
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   295
    """Copy latest file matching pattern in dist/ directory.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   296
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   297
    Given a WinRM client and a file pattern, find the latest file on the remote
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   298
    matching that pattern and copy it to the ``dest_path`` directory on the
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   299
    local machine.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   300
    """
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   301
    latest = find_latest_dist(winrm_client, pattern)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   302
    source = r'C:\hgdev\src\dist\%s' % latest
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   303
    dest = dest_path / latest
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   304
    print('copying %s to %s' % (source, dest))
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   305
    winrm_client.fetch(source, str(dest))
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   306
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   307
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   308
def build_inno_installer(
44771
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   309
    winrm_client,
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   310
    python_version: int,
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   311
    arch: str,
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   312
    dest_path: pathlib.Path,
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   313
    version=None,
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   314
):
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   315
    """Build the Inno Setup installer on a remote machine.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   316
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   317
    Using a WinRM client, remote commands are executed to build
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   318
    a Mercurial Inno Setup installer.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   319
    """
44771
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   320
    print(
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   321
        'building Inno Setup installer for Python %d %s'
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   322
        % (python_version, arch)
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   323
    )
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   324
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   325
    if python_version == 3:
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   326
        # TODO fix this limitation in packaging code
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   327
        if not version:
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   328
            raise Exception(
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   329
                "version string is required when building for Python 3"
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   330
            )
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   331
44771
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   332
        if arch == "x86":
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   333
            target_triple = "i686-pc-windows-msvc"
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   334
        elif arch == "x64":
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   335
            target_triple = "x86_64-pc-windows-msvc"
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   336
        else:
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   337
            raise Exception("unhandled arch: %s" % arch)
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   338
44771
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   339
        ps = BUILD_INNO_PYTHON3.format(
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   340
            pyoxidizer_target=target_triple, version=version,
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   341
        )
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   342
    else:
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   343
        extra_args = []
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   344
        if version:
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   345
            extra_args.extend(['--version', version])
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   346
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   347
        ps = get_vc_prefix(arch) + BUILD_INNO_PYTHON2.format(
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   348
            arch=arch, extra_args=' '.join(extra_args)
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   349
        )
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   350
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   351
    run_powershell(winrm_client, ps)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   352
    copy_latest_dist(winrm_client, '*.exe', dest_path)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   353
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   354
44768
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43513
diff changeset
   355
def build_wheel(
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43513
diff changeset
   356
    winrm_client, python_version: str, arch: str, dest_path: pathlib.Path
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43513
diff changeset
   357
):
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   358
    """Build Python wheels on a remote machine.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   359
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   360
    Using a WinRM client, remote commands are executed to build a Python wheel
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   361
    for Mercurial.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   362
    """
44768
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43513
diff changeset
   363
    print('Building Windows wheel for Python %s %s' % (python_version, arch))
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43513
diff changeset
   364
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43513
diff changeset
   365
    ps = BUILD_WHEEL.format(
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43513
diff changeset
   366
        python_version=python_version.replace(".", ""), arch=arch
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43513
diff changeset
   367
    )
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43513
diff changeset
   368
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43513
diff changeset
   369
    # Python 2.7 requires an activated environment.
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43513
diff changeset
   370
    if python_version == "2.7":
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43513
diff changeset
   371
        ps = get_vc_prefix(arch) + ps
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43513
diff changeset
   372
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   373
    run_powershell(winrm_client, ps)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   374
    copy_latest_dist(winrm_client, '*.whl', dest_path)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   375
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   376
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   377
def build_wix_installer(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   378
    winrm_client, arch: str, dest_path: pathlib.Path, version=None
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   379
):
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   380
    """Build the WiX installer on a remote machine.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   381
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   382
    Using a WinRM client, remote commands are executed to build a WiX installer.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   383
    """
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   384
    print('Building WiX installer for %s' % arch)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   385
    extra_args = []
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   386
    if version:
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   387
        extra_args.extend(['--version', version])
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   388
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   389
    ps = get_vc_prefix(arch) + BUILD_WIX.format(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   390
        arch=arch, extra_args=' '.join(extra_args)
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   391
    )
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   392
    run_powershell(winrm_client, ps)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   393
    copy_latest_dist(winrm_client, '*.msi', dest_path)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   394
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   395
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   396
def run_tests(winrm_client, python_version, arch, test_flags=''):
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   397
    """Run tests on a remote Windows machine.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   398
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   399
    ``python_version`` is a ``X.Y`` string like ``2.7`` or ``3.7``.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   400
    ``arch`` is ``x86`` or ``x64``.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   401
    ``test_flags`` is a str representing extra arguments to pass to
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   402
    ``run-tests.py``.
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   403
    """
42064
0e9066db5e44 automation: use raw strings when there are backslashes
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42024
diff changeset
   404
    if not re.match(r'\d\.\d', python_version):
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   405
        raise ValueError(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   406
            r'python_version must be \d.\d; got %s' % python_version
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   407
        )
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   408
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   409
    if arch not in ('x86', 'x64'):
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   410
        raise ValueError('arch must be x86 or x64; got %s' % arch)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   411
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   412
    python_path = 'python%s-%s' % (python_version.replace('.', ''), arch)
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   413
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   414
    ps = RUN_TESTS.format(python_path=python_path, test_flags=test_flags or '',)
42024
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   415
b05a3e28cf24 automation: perform tasks on remote machines
Gregory Szorc <gregory.szorc@gmail.com>
parents:
diff changeset
   416
    run_powershell(winrm_client, ps)
42907
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   417
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   418
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   419
def resolve_wheel_artifacts(dist_path: pathlib.Path, version: str):
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   420
    return (
44768
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43513
diff changeset
   421
        dist_path / WHEEL_FILENAME_PYTHON27_X86.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43513
diff changeset
   422
        dist_path / WHEEL_FILENAME_PYTHON27_X64.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43513
diff changeset
   423
        dist_path / WHEEL_FILENAME_PYTHON37_X86.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43513
diff changeset
   424
        dist_path / WHEEL_FILENAME_PYTHON37_X64.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43513
diff changeset
   425
        dist_path / WHEEL_FILENAME_PYTHON38_X86.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43513
diff changeset
   426
        dist_path / WHEEL_FILENAME_PYTHON38_X64.format(version=version),
42907
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   427
    )
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   428
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   429
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   430
def resolve_all_artifacts(dist_path: pathlib.Path, version: str):
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   431
    return (
44768
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43513
diff changeset
   432
        dist_path / WHEEL_FILENAME_PYTHON27_X86.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43513
diff changeset
   433
        dist_path / WHEEL_FILENAME_PYTHON27_X64.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43513
diff changeset
   434
        dist_path / WHEEL_FILENAME_PYTHON37_X86.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43513
diff changeset
   435
        dist_path / WHEEL_FILENAME_PYTHON37_X64.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43513
diff changeset
   436
        dist_path / WHEEL_FILENAME_PYTHON38_X86.format(version=version),
9d441f820c8b automation: support building Windows wheels for Python 3.7 and 3.8
Gregory Szorc <gregory.szorc@gmail.com>
parents: 43513
diff changeset
   437
        dist_path / WHEEL_FILENAME_PYTHON38_X64.format(version=version),
44771
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   438
        dist_path / EXE_FILENAME_PYTHON2_X86.format(version=version),
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   439
        dist_path / EXE_FILENAME_PYTHON2_X64.format(version=version),
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   440
        dist_path / EXE_FILENAME_PYTHON3_X86.format(version=version),
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   441
        dist_path / EXE_FILENAME_PYTHON3_X64.format(version=version),
42907
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   442
        dist_path / X86_MSI_FILENAME.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   443
        dist_path / X64_MSI_FILENAME.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   444
    )
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   445
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   446
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   447
def generate_latest_dat(version: str):
44771
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   448
    python2_x86_exe_filename = EXE_FILENAME_PYTHON2_X86.format(version=version)
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   449
    python2_x64_exe_filename = EXE_FILENAME_PYTHON2_X64.format(version=version)
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   450
    python3_x86_exe_filename = EXE_FILENAME_PYTHON3_X86.format(version=version)
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   451
    python3_x64_exe_filename = EXE_FILENAME_PYTHON3_X64.format(version=version)
42907
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   452
    x86_msi_filename = X86_MSI_FILENAME.format(version=version)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   453
    x64_msi_filename = X64_MSI_FILENAME.format(version=version)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   454
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   455
    entries = (
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   456
        (
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   457
            '10',
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   458
            version,
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   459
            X86_USER_AGENT_PATTERN,
44771
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   460
            '%s/%s' % (MERCURIAL_SCM_BASE_URL, python3_x86_exe_filename),
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   461
            EXE_PYTHON3_X86_DESCRIPTION.format(version=version),
42907
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   462
        ),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   463
        (
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   464
            '10',
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   465
            version,
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   466
            X64_USER_AGENT_PATTERN,
44771
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   467
            '%s/%s' % (MERCURIAL_SCM_BASE_URL, python3_x64_exe_filename),
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   468
            EXE_PYTHON3_X64_DESCRIPTION.format(version=version),
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   469
        ),
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   470
        (
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   471
            '9',
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   472
            version,
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   473
            X86_USER_AGENT_PATTERN,
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   474
            '%s/%s' % (MERCURIAL_SCM_BASE_URL, python2_x86_exe_filename),
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   475
            EXE_PYTHON2_X86_DESCRIPTION.format(version=version),
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   476
        ),
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   477
        (
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   478
            '9',
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   479
            version,
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   480
            X64_USER_AGENT_PATTERN,
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   481
            '%s/%s' % (MERCURIAL_SCM_BASE_URL, python2_x64_exe_filename),
802ee93c205d automation: support building Python 3 Inno installers
Gregory Szorc <gregory.szorc@gmail.com>
parents: 44770
diff changeset
   482
            EXE_PYTHON2_X64_DESCRIPTION.format(version=version),
42907
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   483
        ),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   484
        (
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   485
            '10',
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   486
            version,
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   487
            X86_USER_AGENT_PATTERN,
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   488
            '%s/%s' % (MERCURIAL_SCM_BASE_URL, x86_msi_filename),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   489
            X86_MSI_DESCRIPTION.format(version=version),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   490
        ),
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   491
        (
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   492
            '10',
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   493
            version,
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   494
            X64_USER_AGENT_PATTERN,
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   495
            '%s/%s' % (MERCURIAL_SCM_BASE_URL, x64_msi_filename),
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   496
            X64_MSI_DESCRIPTION.format(version=version),
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   497
        ),
42907
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   498
    )
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   499
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   500
    lines = ['\t'.join(e) for e in entries]
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   501
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   502
    return '\n'.join(lines) + '\n'
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   503
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   504
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   505
def publish_artifacts_pypi(dist_path: pathlib.Path, version: str):
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   506
    """Publish Windows release artifacts to PyPI."""
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   507
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   508
    wheel_paths = resolve_wheel_artifacts(dist_path, version)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   509
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   510
    for p in wheel_paths:
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   511
        if not p.exists():
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   512
            raise Exception('%s not found' % p)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   513
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   514
    print('uploading wheels to PyPI (you may be prompted for credentials)')
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   515
    pypi_upload(wheel_paths)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   516
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   517
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   518
def publish_artifacts_mercurial_scm_org(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   519
    dist_path: pathlib.Path, version: str, ssh_username=None
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   520
):
42907
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   521
    """Publish Windows release artifacts to mercurial-scm.org."""
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   522
    all_paths = resolve_all_artifacts(dist_path, version)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   523
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   524
    for p in all_paths:
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   525
        if not p.exists():
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   526
            raise Exception('%s not found' % p)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   527
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   528
    client = paramiko.SSHClient()
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   529
    client.load_system_host_keys()
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   530
    # We assume the system SSH configuration knows how to connect.
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   531
    print('connecting to mercurial-scm.org via ssh...')
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   532
    try:
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   533
        client.connect('mercurial-scm.org', username=ssh_username)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   534
    except paramiko.AuthenticationException:
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   535
        print('error authenticating; is an SSH key available in an SSH agent?')
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   536
        raise
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   537
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   538
    print('SSH connection established')
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   539
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   540
    print('opening SFTP client...')
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   541
    sftp = client.open_sftp()
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   542
    print('SFTP client obtained')
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   543
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   544
    for p in all_paths:
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   545
        dest_path = '/var/www/release/windows/%s' % p.name
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   546
        print('uploading %s to %s' % (p, dest_path))
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   547
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   548
        with p.open('rb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   549
            data = fh.read()
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   550
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   551
        with sftp.open(dest_path, 'wb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   552
            fh.write(data)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   553
            fh.chmod(0o0664)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   554
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   555
    latest_dat_path = '/var/www/release/windows/latest.dat'
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   556
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   557
    now = datetime.datetime.utcnow()
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   558
    backup_path = dist_path / (
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   559
        'latest-windows-%s.dat' % now.strftime('%Y%m%dT%H%M%S')
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   560
    )
42907
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   561
    print('backing up %s to %s' % (latest_dat_path, backup_path))
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   562
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   563
    with sftp.open(latest_dat_path, 'rb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   564
        latest_dat_old = fh.read()
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   565
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   566
    with backup_path.open('wb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   567
        fh.write(latest_dat_old)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   568
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   569
    print('writing %s with content:' % latest_dat_path)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   570
    latest_dat_content = generate_latest_dat(version)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   571
    print(latest_dat_content)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   572
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   573
    with sftp.open(latest_dat_path, 'wb') as fh:
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   574
        fh.write(latest_dat_content.encode('ascii'))
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   575
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   576
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   577
def publish_artifacts(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   578
    dist_path: pathlib.Path,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   579
    version: str,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   580
    pypi=True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   581
    mercurial_scm_org=True,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   582
    ssh_username=None,
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   583
):
42907
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   584
    """Publish Windows release artifacts.
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   585
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   586
    Files are found in `dist_path`. We will look for files with version string
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   587
    `version`.
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   588
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   589
    `pypi` controls whether we upload to PyPI.
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   590
    `mercurial_scm_org` controls whether we upload to mercurial-scm.org.
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   591
    """
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   592
    if pypi:
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   593
        publish_artifacts_pypi(dist_path, version)
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   594
92593d72e10b automation: implement "publish-windows-artifacts" command
Gregory Szorc <gregory.szorc@gmail.com>
parents: 42684
diff changeset
   595
    if mercurial_scm_org:
43076
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   596
        publish_artifacts_mercurial_scm_org(
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   597
            dist_path, version, ssh_username=ssh_username
2372284d9457 formatting: blacken the codebase
Augie Fackler <augie@google.com>
parents: 42907
diff changeset
   598
        )