This reverts commit e121d14b12.
Reason for revert: git for mac arm is widely available now so
this hack is no longer needed. See: b/228226478
Original change's description:
> [cipd] Detect Mac ARM running under Rosetta
>
> If a user has git for x64, `git cl` results in downloading x64 cipd
> packages. Other operations, such as gclient or git-cl will result in
> downloading arm64. This flip-flopping between architectures causes
> unexpected behavior:
> (1) slows down commands since packages are wiped and new ones are
> downloaded.
> (2) long running processes, such as goma may crash.
>
> This change adds additional check for all macs running on x86 to check
> if is running under Rosetta and sets ARCH=arm64 if true.
>
> R=bryner@google.com, masonf@chromium.org
>
> Bug: 1311733
> Change-Id: I65b127467d5c47d5bf07952d2ecdcb93630c7c87
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3573068
> Reviewed-by: Brian Ryner <bryner@google.com>
> Reviewed-by: Mason Freed <masonf@chromium.org>
> Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Bug: 1311733
Bug: b/228226478
Change-Id: I3888769944d7fe200a0bff8cf0a1c04407d50bdf
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4763444
Reviewed-by: Jonathan Nieder <jrn@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
This info is only used when bootstrapping for the first time,
so move these vars to that function to avoid wasted effort.
Bug: 1469696
Change-Id: I21b97b2429cf2dac6242e2e86aa7668b09ed47c6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4752396
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Mike Frysinger <vapier@chromium.org>
Avoid recomputing this value multiple times between vpython & cipd.
Bug: 1469696
Change-Id: If82d58180f94cb2833a5a177f7b1e431e08f2892
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4750040
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Mike Frysinger <vapier@chromium.org>
A lot of scripts like to compute the dirname of $0 to find itself to
load more files. Have some scripts export & resuse this value so we
avoid recalculating when not needed.
Bug: 1469696
Change-Id: I3aac2e6119dbfa3c68715208df2786f67cdce92e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4750430
Auto-Submit: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
This reverts commit c962e86f7b.
Reason for revert: broke MacOS:
realpath: command not found
Original change's description:
> Resolve symlinks to cipd, vpython3
>
> If we point a symlink at vpython3 (or various other tools), it will try
> to resolve the depot_tools path to wherever that symlink is stored. We
> should instead follow the symlink before determining the path.
>
> Prior art:
>
> * in python, we're doing Path.resolve() in many cases
> * in shell, there's a "$(readlink "$(dirname "$0"))" (man/push_to_gs.sh)
>
> BUG=b:270994985
> TEST=symlink to vpython3 works
>
> Change-Id: Ifee014d2571ccf58d830a2702c8a0ea225dd4821
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4295094
> Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
> Commit-Queue: Brian Norris <briannorris@chromium.org>
Bug: b:270994985
Change-Id: I69342f3a52153b8cb1ed4366ede5883dfe5ec0b9
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4310530
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Auto-Submit: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
If we point a symlink at vpython3 (or various other tools), it will try
to resolve the depot_tools path to wherever that symlink is stored. We
should instead follow the symlink before determining the path.
Prior art:
* in python, we're doing Path.resolve() in many cases
* in shell, there's a "$(readlink "$(dirname "$0"))" (man/push_to_gs.sh)
BUG=b:270994985
TEST=symlink to vpython3 works
Change-Id: Ifee014d2571ccf58d830a2702c8a0ea225dd4821
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4295094
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Brian Norris <briannorris@chromium.org>
Bash has a builtin for reading files, so use that instead of running
the cat program to save on time & overhead.
Change-Id: Ia90fc22a47f2ead4ef056020a79ebdebd7bd70ca
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4111082
Reviewed-by: Brian Ryner <bryner@google.com>
Commit-Queue: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
If a user has git for x64, `git cl` results in downloading x64 cipd
packages. Other operations, such as gclient or git-cl will result in
downloading arm64. This flip-flopping between architectures causes
unexpected behavior:
(1) slows down commands since packages are wiped and new ones are
downloaded.
(2) long running processes, such as goma may crash.
This change adds additional check for all macs running on x86 to check
if is running under Rosetta and sets ARCH=arm64 if true.
R=bryner@google.com, masonf@chromium.org
Bug: 1311733
Change-Id: I65b127467d5c47d5bf07952d2ecdcb93630c7c87
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3573068
Reviewed-by: Brian Ryner <bryner@google.com>
Reviewed-by: Mason Freed <masonf@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Enough works in that config that it seems like a better default by now.
Checking out, building, and uploading patches works (but uploading still
requires `--bypass-hooks` for now until crbug.com/1205263 is fixed).
(In case something important I'm not aware of breaks, we can revert this.)
Bug: 1103236,1205263
Change-Id: Id432ffd4ab19b15c019614953a2cefa787b46225
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2896468
Reviewed-by: Dirk Pranke <dpranke@google.com>
Commit-Queue: Nico Weber <thakis@chromium.org>
To switch a depot_tools installation to use mac-arm64 binaries,
run:
$ cd depot_tools
$ echo "mac-arm64" > .cipd_client_platform
$ ./cipd help
$ <observe "infra/tools/cipd/mac-arm64" in the version string>
Deleting .cipd_client_platform file restores the original logic
(which is currently to use mac-amd64 binaries even on arm64 host).
R=iannucci@chromium.org, tandrii@chromium.orgCC=thakis@chromium.org
BUG=1102967
Change-Id: I0248fd913eb3d71f2a228f74a601dc4b12fc8824
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2699692
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Transient network conditions can cause the CIPD client download to fail,
so add a retry to the download.
When using `wget` we additionally retry usually fatal errors like
"connection refused" via `--retry-connrefused`. `curl` also has that
parameter, but it was added more recently in version 7.52.0, which isn't
available on supported platforms like Ubuntu 16.04. Once we raise the
minimum supported version of curl, we should add that parameter as well.
In either case, we retry 3 times and wait 5 seconds between each try,
which matches the retries recently added for Windows.
Bug: 1145741
Change-Id: Ib73bb3413ba4d5251c2b027a87dabd1378ab6c44
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2532896
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Commit-Queue: Matthew Kotsenas <mattkot@microsoft.com>
There's no mac-arm64 yet, since Go doesn't support this. Presumably
mac-amd64 should work fine on ARM Mac too, see the bug.
Note that this will transitively affect all binaries fetched by
CIPD, they all will be mac-amd64.
BUG=1102967
R=iannucci@chromium.org, thakis@chromium.org
Change-Id: Ie3c5b5776c3d2f83b5dfa411016862dbbc53a9e2
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2282940
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
on non-linux systems bash is not the default shell (by default) and
lives in a completely different path
Change-Id: I6580c776dda1b60b3d5c5f5f1ce4d6694ac45fe9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1605902
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Linux and OSX only for now. This also rolls CIPD client to a version that
supports pinned hashes (v2.2.5).
CIPD_CLIENT_VER and CIPD_CLIENT_SRV are no longer supported as env vars, since
it makes no sense when pinning hashes of the binaries at specific version on
the specific backend.
Also somewhat cleanup 'cipd' script to use "${VAR}", stderr and colored output
consistently.
R=iannucci@chromium.org, nodir@chromium.org
BUG=870166, 874586
Change-Id: Iac67fbb6b5d07dcd81d44536737b03b146f1ad14
Reviewed-on: https://chromium-review.googlesource.com/1176727
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
This reverts commit eebc3d8232.
Reason for revert: crbug.com/874586
Original change's description:
> [cipd] Check CIPD client hash against pinned SHA256 during updates.
>
> Linux and OSX only for now. This also rolls CIPD client to a version that
> supports pinned hashes (v2.2.5).
>
> CIPD_CLIENT_VER and CIPD_CLIENT_SRV are no longer supported as env vars, since
> it makes no sense when pinning hashes of the binaries at specific version on
> the specific backend.
>
> Also somewhat cleanup 'cipd' script to use "${VAR}", stderr and colored output
> consistently.
>
> R=iannucci@chromium.org, nodir@chromium.org
> BUG=870166
>
> Change-Id: I9e61f9f8fbdcf10985c52828b2bfbec64b4234f0
> Reviewed-on: https://chromium-review.googlesource.com/1171957
> Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
> Reviewed-by: Nodir Turakulov <nodir@chromium.org>
TBR=iannucci@chromium.org,vadimsh@chromium.org,nodir@chromium.org
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: 870166
Change-Id: I9aa8e7a7f07520aa69d366c76e4dbccae345bc00
Reviewed-on: https://chromium-review.googlesource.com/1175294
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Linux and OSX only for now. This also rolls CIPD client to a version that
supports pinned hashes (v2.2.5).
CIPD_CLIENT_VER and CIPD_CLIENT_SRV are no longer supported as env vars, since
it makes no sense when pinning hashes of the binaries at specific version on
the specific backend.
Also somewhat cleanup 'cipd' script to use "${VAR}", stderr and colored output
consistently.
R=iannucci@chromium.org, nodir@chromium.org
BUG=870166
Change-Id: I9e61f9f8fbdcf10985c52828b2bfbec64b4234f0
Reviewed-on: https://chromium-review.googlesource.com/1171957
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
When updating the CIPD client to be identified by SHA256 hash, old clients (that
have no idea about SHA256) fail during 'selfupdate'.
We'll roll our SHA256 support in two stages:
1. Deploy new client that understand SHA256 using its SHA1 name, so
self-update from old clients works.
2. Deploy same (or newer) client using its SHA256 name. This would work since
the client doing the self-update already understands SHA256 at this point.
But we can't guarantee that ALL depot_tools deployments will update through
stages (1) and (2) sequentially. Some of them may skip (1) and end-up directly
in (2), failing on 'selfupdate'.
This CL makes sure they can recover from this state by rebootstraping the client
from scratch (this works with SHA256 hashes).
R=nodir@chromium.org, iannucci@chromium.org
BUG=821194
Change-Id: I27dece19e0305b5b2d6f8b0130631c1bf5f6499c
Reviewed-on: https://chromium-review.googlesource.com/1149454
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
We only compile and tag cipd packages with armv6, so armv7 hosts get
404s from cipd. This will use armv6 packages on armv7 bots, which should
work.
Bug: 776703
Change-Id: I6e54c2f9e1b220ce038a99af1df28cb6c0a9e579
Reviewed-on: https://chromium-review.googlesource.com/730850
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Benjamin Pastene <bpastene@chromium.org>
The uname was reported incorrectly, and should have read "ppc64le".
Update the script to reference this.
TBR=iannucci@chromium.org
BUG=chromium:773857
TEST=None
Change-Id: Ice5ffe1257333b2bc1bfef7da60dcea4ead4d4c8
Reviewed-on: https://chromium-review.googlesource.com/723162
Reviewed-by: Daniel Jacques <dnj@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>
This is best-effort only; s390x is still not a supported platform.
CIPD pin bump contains no client changes; just points to a version
which is built for s390x as well our normal platforms.
R=dnj@chromium.org, vadimsh@chromium.org
Bug: 764087
Change-Id: I777246712eea1b39036f3b14b88cbd41c7cf0507
Reviewed-on: https://chromium-review.googlesource.com/661787
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
On Debian's default install curl is not available, but wget is.
Bug: 762568
Change-Id: Ibc8c52676fd5e37ef1b9f1ffc061b2be27ec80c4
Reviewed-on: https://chromium-review.googlesource.com/653517
Reviewed-by: Daniel Jacques <dnj@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Bump the CIPD client version to
b482bfa19d3fa79b4b40dd5ac7a34479052e3df1. This version inculdes support
for the "-log-level" flag.
Update CIPD bootstrap scripts and tool installation scripts to log at
warning level, so that standard operation doesn't produce any logs.
This version differs from the previous CL in that the "selfupdate"
command no longer uses "-log-level". This fixes an issue where the
current CIPD, which doesn't support "-log-level", is given the flag
prior to actually updating to a version that does.
BUG=chromium:748651
TEST=canary
- Tested this CIPD version on canary waterfall successfully.
Change-Id: I69dfe522d890eb307214f18d0467ce921a766446
Reviewed-on: https://chromium-review.googlesource.com/589568
Commit-Queue: Daniel Jacques <dnj@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
This reverts commit 18aa2d93ed.
Reason for revert: <INSERT REASONING HERE>
Original change's description:
> [cipd] Bump version, log at warning.
>
> Bump the CIPD client version to
> e2a98b41adb3403b5b7771c4101b42a55dc2dd9e. This version inculdes support
> for the "-log-level" flag.
>
> Update CIPD bootstrap scripts and tool installation scripts to log at
> warning level, so that standard operation doesn't produce any logs.
>
> BUG=chromium:748651
> TEST=canary
> - Tested this CIPD version on canary waterfall successfully.
> - Tested "gclient" bootstrap and "vpython" / "led" on Windows system,
> no output and works.
> - Tested "vpython" / "led" on OSX system, no output and works.
>
> Change-Id: Iae316908283c43c504f293aafdf897df83dccf8d
> Reviewed-on: https://chromium-review.googlesource.com/588889
> Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> Commit-Queue: Daniel Jacques <dnj@chromium.org>
TBR=iannucci@chromium.org,dnj@chromium.org
Change-Id: Ia611c1aeb204c609ff3469076b8256c54e702230
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:748651
Reviewed-on: https://chromium-review.googlesource.com/589527
Reviewed-by: Daniel Jacques <dnj@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>
Bump the CIPD client version to
e2a98b41adb3403b5b7771c4101b42a55dc2dd9e. This version inculdes support
for the "-log-level" flag.
Update CIPD bootstrap scripts and tool installation scripts to log at
warning level, so that standard operation doesn't produce any logs.
BUG=chromium:748651
TEST=canary
- Tested this CIPD version on canary waterfall successfully.
- Tested "gclient" bootstrap and "vpython" / "led" on Windows system,
no output and works.
- Tested "vpython" / "led" on OSX system, no output and works.
Change-Id: Iae316908283c43c504f293aafdf897df83dccf8d
Reviewed-on: https://chromium-review.googlesource.com/588889
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>
It's currently possible for CIPD bootstraps that provision concurrently to:
1) On Linux, step on each other during download, and
2) On Windows, fail.
Fix these respective scripts so that bootstraps are safe to use
concurrently.
On Linux and Mac, we download to a temporary file and use "mv" (atomic)
to write it to the final destination. Concurrent initializations will
perform parallel downloads, execute the "mv", and copy their downloaded
file to the destination path.
On Windows, we use filesystem locking to lock the operation and ensure
that only one download can happen.
BUG=chromium:739195
TEST=local
- Ran in parallel on Windows, Linux, and Max.
Change-Id: Ie050d37598da67389f21728e781bd58904ef9c17
Reviewed-on: https://chromium-review.googlesource.com/560521
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>
Make Linux/Mac CIPD wrapper fail if any step returns an error.
Include the "-f" curl flag, which will cause it to error on non-200
status.
BUG=chromium:739195
TEST=local
R=iannucci@chromium.org
Change-Id: I3efea3f38c1ee322cb9d63f29819472564107ddc
Reviewed-on: https://chromium-review.googlesource.com/560723
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>
This takes advantage of powershell on windows for a cleanish duplicate of the
posix version.
R=dnj@chromium.org, vadimsh@chromium.org
BUG=663843
Change-Id: Ib23a044ff912e3239b58848a26143eb6575826d5
Reviewed-on: https://chromium-review.googlesource.com/414228
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Yiwei Zhang
Gaby Baghdadi
Josip Sokcevic
Mike Frysinger
Robbie Iannucci
Wang Qing
Brian Norris
Theodore Siu
Rebecca Chang Swee Fun
Josip Sokcevic
Nico Weber
Vadim Shtayura
Matt Kotsenas
Robert Nagy
Vasili Skurydzin
Brian White
Benjamin Pastene
Dan Jacques
William Hesse
Pawel Hajdan, Jr
Dan Jacques