You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
depot_tools/bootstrap/win/README.md

59 lines
2.3 KiB
Markdown

# Windows binary tool bootstrap
This directory has the 'magic' for the `depot_tools` windows binary update
mechanisms.
## Software bootstrapped
* Python (https://www.python.org/)
* Git for Windows (https://git-for-windows.github.io/)
## Mechanism
Any time a user runs `gclient` on windows, it will invoke the `depot_tools`
autoupdate script [depot_tools.bat](../../update_depot_tools.bat). This, in
turn, will run [win_tools.bat](./win_tools.bat), which does the bulk of the
work.
`win_tools.bat` will successively look to see if the local version of the binary
package is present, and if so, if it's the expected version. If either of those
cases is not true, it will download and unpack the respective binary.
Revert "[win_tools] Use bundled Python CIPD packages." This reverts commit bf1446791e4b1c5b102559de9fb090ed5ba50cf5. Reason for revert: broke win bots http://o/410053 Original change's description: > [win_tools] Use bundled Python CIPD packages. > > Enable bundled Python CIPD packages in bleeding-edge mode. This > replaces the ZIP unpacking approach used before, and introduces > validation and management through the CIPD tool. The bleeding edge > version will only install if a sentinel file is present in the > "depot_tools" root; otherwise, default behavior will continue. > > This method adds a upgrade and downgrade path to/from ZIP and > CIPD installations. This is done by rewriting the "win_tools.bat" > process: > > 1) Ensure that a bootstrap Python is present. > 2) Use it to run "win_tools.py", which has the functionality of > "git_bootstrap.py" plus Python installation. > 3) Run "win_tools.py" with appropriate flags. > > Some tricks were employed to handle cases where there is an > already-running Python instance that uses the current Python > installation and executable. This happens on bots because the > system uses the same "depot_tools" checkout at multiple launch > layers. To this end, we use the "python.bat" as the "current Python" > authority and refrain from cleaning up old Python directories if their > "python.exe" binaries are currently in use. > > We change the Git bleeding edge file to share the same > sentinel file as Python, ".bleeding_edge". > > The new Python should have the same facilities as the original Python > bundle. > > BUG=chromium:740171 > TEST=local > > Change-Id: I1b3b7d31d47d1a37a9dba9114d31681bec558736 > Reviewed-on: https://chromium-review.googlesource.com/563036 > Commit-Queue: Daniel Jacques <dnj@chromium.org> > Reviewed-by: Robbie Iannucci <iannucci@chromium.org> TBR=iannucci@chromium.org,dnj@chromium.org Change-Id: I84574a01bbad6596912e4aaa34f019d24720b638 No-Presubmit: true No-Tree-Checks: true No-Try: true Bug: chromium:740171 Reviewed-on: https://chromium-review.googlesource.com/567009 Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org> Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
8 years ago
Downloading is done with [get_file.js](./get_file.js), which is a windows script
host javascript utility to vaguely impersonate `wget`.
Revert "[win_tools] Use bundled Python CIPD packages." This reverts commit bf1446791e4b1c5b102559de9fb090ed5ba50cf5. Reason for revert: broke win bots http://o/410053 Original change's description: > [win_tools] Use bundled Python CIPD packages. > > Enable bundled Python CIPD packages in bleeding-edge mode. This > replaces the ZIP unpacking approach used before, and introduces > validation and management through the CIPD tool. The bleeding edge > version will only install if a sentinel file is present in the > "depot_tools" root; otherwise, default behavior will continue. > > This method adds a upgrade and downgrade path to/from ZIP and > CIPD installations. This is done by rewriting the "win_tools.bat" > process: > > 1) Ensure that a bootstrap Python is present. > 2) Use it to run "win_tools.py", which has the functionality of > "git_bootstrap.py" plus Python installation. > 3) Run "win_tools.py" with appropriate flags. > > Some tricks were employed to handle cases where there is an > already-running Python instance that uses the current Python > installation and executable. This happens on bots because the > system uses the same "depot_tools" checkout at multiple launch > layers. To this end, we use the "python.bat" as the "current Python" > authority and refrain from cleaning up old Python directories if their > "python.exe" binaries are currently in use. > > We change the Git bleeding edge file to share the same > sentinel file as Python, ".bleeding_edge". > > The new Python should have the same facilities as the original Python > bundle. > > BUG=chromium:740171 > TEST=local > > Change-Id: I1b3b7d31d47d1a37a9dba9114d31681bec558736 > Reviewed-on: https://chromium-review.googlesource.com/563036 > Commit-Queue: Daniel Jacques <dnj@chromium.org> > Reviewed-by: Robbie Iannucci <iannucci@chromium.org> TBR=iannucci@chromium.org,dnj@chromium.org Change-Id: I84574a01bbad6596912e4aaa34f019d24720b638 No-Presubmit: true No-Tree-Checks: true No-Try: true Bug: chromium:740171 Reviewed-on: https://chromium-review.googlesource.com/567009 Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org> Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
8 years ago
Through a comedy of history, each binary is stored and retrieved differently.
Revert "[win_tools] Use bundled Python CIPD packages." This reverts commit bf1446791e4b1c5b102559de9fb090ed5ba50cf5. Reason for revert: broke win bots http://o/410053 Original change's description: > [win_tools] Use bundled Python CIPD packages. > > Enable bundled Python CIPD packages in bleeding-edge mode. This > replaces the ZIP unpacking approach used before, and introduces > validation and management through the CIPD tool. The bleeding edge > version will only install if a sentinel file is present in the > "depot_tools" root; otherwise, default behavior will continue. > > This method adds a upgrade and downgrade path to/from ZIP and > CIPD installations. This is done by rewriting the "win_tools.bat" > process: > > 1) Ensure that a bootstrap Python is present. > 2) Use it to run "win_tools.py", which has the functionality of > "git_bootstrap.py" plus Python installation. > 3) Run "win_tools.py" with appropriate flags. > > Some tricks were employed to handle cases where there is an > already-running Python instance that uses the current Python > installation and executable. This happens on bots because the > system uses the same "depot_tools" checkout at multiple launch > layers. To this end, we use the "python.bat" as the "current Python" > authority and refrain from cleaning up old Python directories if their > "python.exe" binaries are currently in use. > > We change the Git bleeding edge file to share the same > sentinel file as Python, ".bleeding_edge". > > The new Python should have the same facilities as the original Python > bundle. > > BUG=chromium:740171 > TEST=local > > Change-Id: I1b3b7d31d47d1a37a9dba9114d31681bec558736 > Reviewed-on: https://chromium-review.googlesource.com/563036 > Commit-Queue: Daniel Jacques <dnj@chromium.org> > Reviewed-by: Robbie Iannucci <iannucci@chromium.org> TBR=iannucci@chromium.org,dnj@chromium.org Change-Id: I84574a01bbad6596912e4aaa34f019d24720b638 No-Presubmit: true No-Tree-Checks: true No-Try: true Bug: chromium:740171 Reviewed-on: https://chromium-review.googlesource.com/567009 Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org> Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
8 years ago
### Git
Revert "[win_tools] Use bundled Python CIPD packages." This reverts commit bf1446791e4b1c5b102559de9fb090ed5ba50cf5. Reason for revert: broke win bots http://o/410053 Original change's description: > [win_tools] Use bundled Python CIPD packages. > > Enable bundled Python CIPD packages in bleeding-edge mode. This > replaces the ZIP unpacking approach used before, and introduces > validation and management through the CIPD tool. The bleeding edge > version will only install if a sentinel file is present in the > "depot_tools" root; otherwise, default behavior will continue. > > This method adds a upgrade and downgrade path to/from ZIP and > CIPD installations. This is done by rewriting the "win_tools.bat" > process: > > 1) Ensure that a bootstrap Python is present. > 2) Use it to run "win_tools.py", which has the functionality of > "git_bootstrap.py" plus Python installation. > 3) Run "win_tools.py" with appropriate flags. > > Some tricks were employed to handle cases where there is an > already-running Python instance that uses the current Python > installation and executable. This happens on bots because the > system uses the same "depot_tools" checkout at multiple launch > layers. To this end, we use the "python.bat" as the "current Python" > authority and refrain from cleaning up old Python directories if their > "python.exe" binaries are currently in use. > > We change the Git bleeding edge file to share the same > sentinel file as Python, ".bleeding_edge". > > The new Python should have the same facilities as the original Python > bundle. > > BUG=chromium:740171 > TEST=local > > Change-Id: I1b3b7d31d47d1a37a9dba9114d31681bec558736 > Reviewed-on: https://chromium-review.googlesource.com/563036 > Commit-Queue: Daniel Jacques <dnj@chromium.org> > Reviewed-by: Robbie Iannucci <iannucci@chromium.org> TBR=iannucci@chromium.org,dnj@chromium.org Change-Id: I84574a01bbad6596912e4aaa34f019d24720b638 No-Presubmit: true No-Tree-Checks: true No-Try: true Bug: chromium:740171 Reviewed-on: https://chromium-review.googlesource.com/567009 Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org> Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
8 years ago
Git installs are mirrored versions of the official Git-for-Windows Portable
releases.
* Original: `https://github.com/git-for-windows/git/releases`
* Mirror: `gs://chrome-infra/PortableGit*.7z.exe`
Revert "[win_tools] Use bundled Python CIPD packages." This reverts commit bf1446791e4b1c5b102559de9fb090ed5ba50cf5. Reason for revert: broke win bots http://o/410053 Original change's description: > [win_tools] Use bundled Python CIPD packages. > > Enable bundled Python CIPD packages in bleeding-edge mode. This > replaces the ZIP unpacking approach used before, and introduces > validation and management through the CIPD tool. The bleeding edge > version will only install if a sentinel file is present in the > "depot_tools" root; otherwise, default behavior will continue. > > This method adds a upgrade and downgrade path to/from ZIP and > CIPD installations. This is done by rewriting the "win_tools.bat" > process: > > 1) Ensure that a bootstrap Python is present. > 2) Use it to run "win_tools.py", which has the functionality of > "git_bootstrap.py" plus Python installation. > 3) Run "win_tools.py" with appropriate flags. > > Some tricks were employed to handle cases where there is an > already-running Python instance that uses the current Python > installation and executable. This happens on bots because the > system uses the same "depot_tools" checkout at multiple launch > layers. To this end, we use the "python.bat" as the "current Python" > authority and refrain from cleaning up old Python directories if their > "python.exe" binaries are currently in use. > > We change the Git bleeding edge file to share the same > sentinel file as Python, ".bleeding_edge". > > The new Python should have the same facilities as the original Python > bundle. > > BUG=chromium:740171 > TEST=local > > Change-Id: I1b3b7d31d47d1a37a9dba9114d31681bec558736 > Reviewed-on: https://chromium-review.googlesource.com/563036 > Commit-Queue: Daniel Jacques <dnj@chromium.org> > Reviewed-by: Robbie Iannucci <iannucci@chromium.org> TBR=iannucci@chromium.org,dnj@chromium.org Change-Id: I84574a01bbad6596912e4aaa34f019d24720b638 No-Presubmit: true No-Tree-Checks: true No-Try: true Bug: chromium:740171 Reviewed-on: https://chromium-review.googlesource.com/567009 Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org> Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
8 years ago
#### Updating git version
1. Download the new `PortableGit-X.Y.Z-{32,64}.7z.exe` from the
git-for-windows release page.
1. From either console.developers.google.com or the CLI, do:
1. Upload those to the gs://chrome-infra Google Storage bucket.
1. Set the `allUsers Reader` permission (click the "Public link" checkbox
next to the binaries).
1. Edit the `git_version.txt` or `git_version_bleeding_edge.txt` file to
be the new version.
1. You can use the bleeding edge version to get early feedback/stage a
rollout/etc. Users can select this version by 'touch'ing the
`.git_bleeding_edge` file in the root depot_tools directory.
1. Commit the CL.
Note that in order for the update to take effect, `gclient` currently needs to
run twice. The first time it will update the `depot_tools` repo, and the second
time it will see the new git version and update to it. This is a bug that should
be fixed, in case you're reading this and this paragraph infuriates you more
than the rest of this README.
Revert "[win_tools] Use bundled Python CIPD packages." This reverts commit bf1446791e4b1c5b102559de9fb090ed5ba50cf5. Reason for revert: broke win bots http://o/410053 Original change's description: > [win_tools] Use bundled Python CIPD packages. > > Enable bundled Python CIPD packages in bleeding-edge mode. This > replaces the ZIP unpacking approach used before, and introduces > validation and management through the CIPD tool. The bleeding edge > version will only install if a sentinel file is present in the > "depot_tools" root; otherwise, default behavior will continue. > > This method adds a upgrade and downgrade path to/from ZIP and > CIPD installations. This is done by rewriting the "win_tools.bat" > process: > > 1) Ensure that a bootstrap Python is present. > 2) Use it to run "win_tools.py", which has the functionality of > "git_bootstrap.py" plus Python installation. > 3) Run "win_tools.py" with appropriate flags. > > Some tricks were employed to handle cases where there is an > already-running Python instance that uses the current Python > installation and executable. This happens on bots because the > system uses the same "depot_tools" checkout at multiple launch > layers. To this end, we use the "python.bat" as the "current Python" > authority and refrain from cleaning up old Python directories if their > "python.exe" binaries are currently in use. > > We change the Git bleeding edge file to share the same > sentinel file as Python, ".bleeding_edge". > > The new Python should have the same facilities as the original Python > bundle. > > BUG=chromium:740171 > TEST=local > > Change-Id: I1b3b7d31d47d1a37a9dba9114d31681bec558736 > Reviewed-on: https://chromium-review.googlesource.com/563036 > Commit-Queue: Daniel Jacques <dnj@chromium.org> > Reviewed-by: Robbie Iannucci <iannucci@chromium.org> TBR=iannucci@chromium.org,dnj@chromium.org Change-Id: I84574a01bbad6596912e4aaa34f019d24720b638 No-Presubmit: true No-Tree-Checks: true No-Try: true Bug: chromium:740171 Reviewed-on: https://chromium-review.googlesource.com/567009 Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org> Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
8 years ago
### Python
Revert "[win_tools] Use bundled Python CIPD packages." This reverts commit bf1446791e4b1c5b102559de9fb090ed5ba50cf5. Reason for revert: broke win bots http://o/410053 Original change's description: > [win_tools] Use bundled Python CIPD packages. > > Enable bundled Python CIPD packages in bleeding-edge mode. This > replaces the ZIP unpacking approach used before, and introduces > validation and management through the CIPD tool. The bleeding edge > version will only install if a sentinel file is present in the > "depot_tools" root; otherwise, default behavior will continue. > > This method adds a upgrade and downgrade path to/from ZIP and > CIPD installations. This is done by rewriting the "win_tools.bat" > process: > > 1) Ensure that a bootstrap Python is present. > 2) Use it to run "win_tools.py", which has the functionality of > "git_bootstrap.py" plus Python installation. > 3) Run "win_tools.py" with appropriate flags. > > Some tricks were employed to handle cases where there is an > already-running Python instance that uses the current Python > installation and executable. This happens on bots because the > system uses the same "depot_tools" checkout at multiple launch > layers. To this end, we use the "python.bat" as the "current Python" > authority and refrain from cleaning up old Python directories if their > "python.exe" binaries are currently in use. > > We change the Git bleeding edge file to share the same > sentinel file as Python, ".bleeding_edge". > > The new Python should have the same facilities as the original Python > bundle. > > BUG=chromium:740171 > TEST=local > > Change-Id: I1b3b7d31d47d1a37a9dba9114d31681bec558736 > Reviewed-on: https://chromium-review.googlesource.com/563036 > Commit-Queue: Daniel Jacques <dnj@chromium.org> > Reviewed-by: Robbie Iannucci <iannucci@chromium.org> TBR=iannucci@chromium.org,dnj@chromium.org Change-Id: I84574a01bbad6596912e4aaa34f019d24720b638 No-Presubmit: true No-Tree-Checks: true No-Try: true Bug: chromium:740171 Reviewed-on: https://chromium-review.googlesource.com/567009 Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org> Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
8 years ago
Python installs are sourced from gs://chrome-infra/python276_bin.zip .
Revert "[win_tools] Use bundled Python CIPD packages." This reverts commit bf1446791e4b1c5b102559de9fb090ed5ba50cf5. Reason for revert: broke win bots http://o/410053 Original change's description: > [win_tools] Use bundled Python CIPD packages. > > Enable bundled Python CIPD packages in bleeding-edge mode. This > replaces the ZIP unpacking approach used before, and introduces > validation and management through the CIPD tool. The bleeding edge > version will only install if a sentinel file is present in the > "depot_tools" root; otherwise, default behavior will continue. > > This method adds a upgrade and downgrade path to/from ZIP and > CIPD installations. This is done by rewriting the "win_tools.bat" > process: > > 1) Ensure that a bootstrap Python is present. > 2) Use it to run "win_tools.py", which has the functionality of > "git_bootstrap.py" plus Python installation. > 3) Run "win_tools.py" with appropriate flags. > > Some tricks were employed to handle cases where there is an > already-running Python instance that uses the current Python > installation and executable. This happens on bots because the > system uses the same "depot_tools" checkout at multiple launch > layers. To this end, we use the "python.bat" as the "current Python" > authority and refrain from cleaning up old Python directories if their > "python.exe" binaries are currently in use. > > We change the Git bleeding edge file to share the same > sentinel file as Python, ".bleeding_edge". > > The new Python should have the same facilities as the original Python > bundle. > > BUG=chromium:740171 > TEST=local > > Change-Id: I1b3b7d31d47d1a37a9dba9114d31681bec558736 > Reviewed-on: https://chromium-review.googlesource.com/563036 > Commit-Queue: Daniel Jacques <dnj@chromium.org> > Reviewed-by: Robbie Iannucci <iannucci@chromium.org> TBR=iannucci@chromium.org,dnj@chromium.org Change-Id: I84574a01bbad6596912e4aaa34f019d24720b638 No-Presubmit: true No-Tree-Checks: true No-Try: true Bug: chromium:740171 Reviewed-on: https://chromium-review.googlesource.com/567009 Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org> Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
8 years ago
The process to create them is sort-of-documented in the README of the python
zip file.