Normally, if dfgs is started with multiple threads, the first
time gsutil.py is called, it may be called by many different
threads at the same time and hit a race condition where
multiple threads are all trying to download gsutil at the same
time. This works around the problem by calling it for the first
time in the main thread before anything else happens.
Bug: 773671
Change-Id: Ie75d90d6d20d11f46a601500b75a198fa5e5f111
Reviewed-on: https://chromium-review.googlesource.com/713750
Reviewed-by: Jao-ke Chin-Lee <jchinlee@chromium.org>
Commit-Queue: Ryan Tseng <hinoka@chromium.org>
Sometimes it is necessary to recreate a VS toolchain package with a tiny
change such as editing or removing a single file. Recreating the VS
environment can be time consuming or even impossible (the VS installer
does not allow going to arbitrary versions) so this switch allows
repackaging an existing unpacked directory. The typical usage would be
to make the modifications to one of the toolchain directories in
depot_tools\win_toolchain\vs_files and then repackage it using:
python package_from_installed.py --repackage third_party\depot_tools\win_toolchain\vs_files\9bc7ccbf9f4bd50d4a3bd185e8ca94ff1618de0b
Bug: 772123
Change-Id: I77b928f695e5f07e33f68dd37711c8761a3c7a22
Reviewed-on: https://chromium-review.googlesource.com/713562
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
This reverts commit c3eb3fa335.
Reason for revert: Broke everything, probably due to version pin mismatch somewhere (gclient/git_cache likely getting called across repo boundary)
Original change's description:
> git_cache: Remove locks
>
> These aren't in use, and the original problem they were
> meant to solve has been solved at the gclient.py layer
> using resource locking:
> https://codereview.chromium.org/2049583003
>
> Bug: 773008
> Change-Id: I6609f39d7f15604e0bb3d742a41c4f9fec87a57a
> Reviewed-on: https://chromium-review.googlesource.com/707728
> Reviewed-by: Aaron Gable <agable@chromium.org>
> Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> Commit-Queue: Ryan Tseng <hinoka@chromium.org>
TBR=iannucci@chromium.org,hinoka@chromium.org,agable@chromium.org,phajdan.jr@chromium.org
Change-Id: Ia9061f0ac827599d81d7638f535f97e15efa75fe
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 773008
Reviewed-on: https://chromium-review.googlesource.com/711055
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Commit-Queue: Ryan Tseng <hinoka@chromium.org>
These aren't in use, and the original problem they were
meant to solve has been solved at the gclient.py layer
using resource locking:
https://codereview.chromium.org/2049583003
Bug: 773008
Change-Id: I6609f39d7f15604e0bb3d742a41c4f9fec87a57a
Reviewed-on: https://chromium-review.googlesource.com/707728
Reviewed-by: Aaron Gable <agable@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Ryan Tseng <hinoka@chromium.org>
Sometimes I want to get try bot results without checking out the
relevant branch.
R=agable@chromium.org
Bug:
Change-Id: I89d118962ee37b10d95cf4044b61a0f2234b7e60
Reviewed-on: https://chromium-review.googlesource.com/710500
Commit-Queue: Stefan Zager <szager@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
clang-cl relies on this env var being set for 2017 to find the linker.
Bug: 772123
Change-Id: Id03896504d38dc706b2bd96a2c3834c6cb9db9fe
TBR=brucedawson
Reviewed-on: https://chromium-review.googlesource.com/709697
Commit-Queue: Nico Weber <thakis@chromium.org>
Reviewed-by: Nico Weber <thakis@chromium.org>
Bug: 772843
Change-Id: I994f829cba7577e6b628d36d894f4c5d1f3c429c
Reviewed-on: https://chromium-review.googlesource.com/708656
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Apparently several scripts in other repros call this function directly.
Bug: 772741
Change-Id: I486483e44a072af6cf8c75373a2da6ef5469fc2c
TBR=dpranke
Reviewed-on: https://chromium-review.googlesource.com/707937
Commit-Queue: Nico Weber <thakis@chromium.org>
Reviewed-by: Nico Weber <thakis@chromium.org>
If -m option value starts with "luci.", most probably the user intends
to schedule a build on a LUCI bucket. Print an error message asking
the user to use -B option.
Bug: 749696
Change-Id: If7a038a1f629ee808e3261ae1c753e388bfea893
Reviewed-on: https://chromium-review.googlesource.com/707534
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
The script prints a bunch of stuff behind `if verbose` checks, but then
"verbose" was turned on by default in
https://bugs.chromium.org/p/chromium/issues/detail?id=504452
I think the wrong lesson was learned from that bug – it sounds like the
problem was that an error message was printed only if verbose was set.
After this change, the script is silent when it does nothing, and prints
something if something happens. (Arguably, it still prints too much in the
case where it successfully downloads something.)
This is part of a few changes to make `gclient runhooks` less noisy.
Bug: 772741,504452
Change-Id: I5823c296abe97611ba4411bab2743673b10dca4b
Reviewed-on: https://chromium-review.googlesource.com/706915
Commit-Queue: Nico Weber <thakis@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
The newline is probably here to protect against a hook that doesn't print
a trailing newline. I've never seen a hook that did that, and if we found one,
we could make the logic look like "print a trailing newline if it's not there"
-- or just fix the hook.
The newline has been around since depot_tools was created
(https://codereview.chromium.org/92087, gclient.py), back then this was in
a general "run stuff" function, not in hooks-specific code. Maybe it made
more sense back then.
This is part of a few changes to make `gclient runhooks` less noisy.
Bug: 772741
Change-Id: I285f76dc3f01c5acf5bbaa0be4db9f6edb9c0366
Reviewed-on: https://chromium-review.googlesource.com/706914
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Nico Weber <thakis@chromium.org>
Update "google-compute-engine" and "boto" package versions.
Newer "gsutil" versions expect a newer "google-compute-engine" package
version, and fail with an ImportError on "constants.py".
BUG=None
TEST=prod
- ssh into failing bot, ran with current "gsutil.vpython", reproduced.
- ssh into failing bot, ran with new "gsutil.vpython", fixed.
- ssh into old GCE image, ran with new "gsutil.vpython", works.
Change-Id: Ied277b6dc1cda4bc46cc9d58a633620c055e9d7c
Reviewed-on: https://chromium-review.googlesource.com/706807
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>
Roll CIPD version to:
bbb30ff0932ace4c6bed7ca1ba14e8a1ccc819cb
TBR=iannucci@chromium.org
BUG=chromium:771277
TEST=canary
Change-Id: I4fb064dcad7072a6a4f728f6c6384a8a65ad56a3
Reviewed-on: https://chromium-review.googlesource.com/701434
Reviewed-by: Daniel Jacques <dnj@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>
This is supported by gclient and has the effect of treating the solution
as a purely local, standalone DEPS (i.e. not as a source repo to
checkout or process in any way). It also prevents the solution from
being added to the output of 'gclient flatten'.
R=agable@chromium.org, phajdan.jr@chromium.org
Change-Id: Ie11246ec7a74097b96ea6a26983a63b3fb3123a5
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build
Reviewed-on: https://chromium-review.googlesource.com/698271
Commit-Queue: Michael Moss <mmoss@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
Previously, git-cl-diff went through a dance where it would create
a new branch, download the uploaded patch onto that branch, and
then diff against that. This had all sorts of problems: if you
aborted the command, it might leave you on that branch; if you have
local changes, they might get clobbered or the command would refuse
to run.
Now that we're in a Gerrit-only world, and patchsets are by definition
equivalent to commits, we can simply diff against whatever local commit
was last uploaded or, in a pinch, fetch the uploaded commit and diff
against that.
Bug: 759893
Change-Id: Ia4b93dcfb9b8aba85817e62731f68d6450026e75
Reviewed-on: https://chromium-review.googlesource.com/639915
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Aaron Gable <agable@chromium.org>
Bug: 661382
Change-Id: Id50b2541132002452bc5d86bb013758e8be0f4b0
Reviewed-on: https://chromium-review.googlesource.com/697813
Reviewed-by: Michael Moss <mmoss@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
This code appears unused, and if it is used, it isn't useful anymore.
R=tandrii@chromium.org
Bug: 770408
Change-Id: I8b96a119be8ed20132e0b7439e466ad90f3757c2
Reviewed-on: https://chromium-review.googlesource.com/693337
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Aaron Gable <agable@chromium.org>
This is the second attempt at fixing autoninja to allow passing '^^' to
it to specify that ninja should build the outputs of the specified file
instead of building that file itself. The problem is that '^' is a
special character and when extra layers of indirection are added the
number of '^' characters needed grows exponentially in some poorly
understood way. The first fix attempt just quoted the arguments that
autoninja.bat passed to autoninja.py, but that meant they came in as
one argument. This fix expands on that by modifying autoninja.py to
understand how to deal with the monolithic argument. With this change
this once again works:
autoninja -C out\debug_component ..\..\base\win\enum_variant.cc^^
It can be convenient to have a ninja.bat file which starts goma and lets
users keep typing the same build commands. However even with this fix
the previously recommended ninja.bat file must be invoked with four
'^' characters. If that is too much then the new recommended ninja.bat
is to copy autoninja.bat and modify as needed, perhaps like this:
@echo off
call python c:\goma\goma-win64\goma_ctl.py ensure_start >nul
FOR /f "usebackq tokens=*" %%a in (`python c:\src\depot_tools\autoninja.py "%*"`) do echo %%a & %%a
BUG: 758725
Change-Id: Ieee9cf343ee5f22e9988a1969cb7a7a90687666b
Reviewed-on: https://chromium-review.googlesource.com/656478
Reviewed-by: Sébastien Marchand <sebmarchand@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Add a "ensure_bootstrap" script, which syncs the bootstrapped state of
"depot_tools" so that remaining operations become read-only.
BUG=chromium:769369
TEST=local
Change-Id: I2dab36112b378a926d8201d2ab31a7daa9f5da29
Reviewed-on: https://chromium-review.googlesource.com/691318
Commit-Queue: Daniel Jacques <dnj@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
This is a step away from patch_project, which was a holdover
from Rietveld. Instead of mapping "projects" (which are
poorly-defined and not guaranteed to be unique) to subpaths,
instead map repository urls (which are at least unique).
Eventually we may be able to compute this directly from DEPS
instead of hardcoding the mapping here, but at least this is
a step in the right direction.
Bug: 765633
Change-Id: Idd65984fc6edefcbedb0438d38c2338b10b7e8e5
Reviewed-on: https://chromium-review.googlesource.com/690776
Commit-Queue: Aaron Gable <agable@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Roll "vpython" to git_revision:0577e51184b2da24260cbb01763eba51915b9755
TBR=iannucci@chromium.org
BUG=chromium:764087
TEST=None
Change-Id: I117a0c2d368303805abb6426e3b9e4cbb8997b91
Reviewed-on: https://chromium-review.googlesource.com/688584
Reviewed-by: Daniel Jacques <dnj@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>
Adding inherit-review-settings-ok to a subrepository causes the
PRESUBMIT.py of the parent repository to be run. In this case, the
CheckPatchFormatted() check needs to run "git cl format" with
CWD=repository root.
Bug: 768962
Change-Id: I44c642185efa78ba3a1bd752f94c3c09feff9cf1
Reviewed-on: https://chromium-review.googlesource.com/685355
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Andrew Grieve <agrieve@chromium.org>
This arg used to be used in a 'sync' call that was removed in
crrev.com/7090f652d180eeff2663e3e3091d5db14123284c, but it was never
applied to the remaining 'sync' call. This removes the arg and leaves
"--with_branch_heads" (and adds "--with_tags") as default sync flags.
Change-Id: Ice5a592d95a0b09dafddab24769f5edd8835a1c9
Recipe-Manual-Change: build_limited_scripts_slave
Recipe-Manual-Change: build
Reviewed-on: https://chromium-review.googlesource.com/677754
Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Commit-Queue: Michael Moss <mmoss@chromium.org>
Since git-cl TBR always sets +1, it does not allow auto-submit where
using +2 score as approval. Let me make git-cl automatically adjust
to proper score.
Bug: 762425
Change-Id: I71fe1af1b8bf5e68d2509c60e8bf05024b6bdbb7
Reviewed-on: https://chromium-review.googlesource.com/680717
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Yoshisato Yanagisawa <yyanagisawa@chromium.org>
Ryan Tseng
Michael Achenbach
Andrii Shyshkalov
recipe-roller
Stefan Zager
Bruce Dawson
Ryan Tseng
Vadim Shtayura
Nico Weber
William Hesse
Dan Jacques
Paweł Hajdan, Jr
Nodir Turakulov
Dirk Pranke
Xiaoyin Liu
Michael Moss
Aaron Gable
Sigurdur Asgeirsson
Eric Boren
Andrew Grieve
Yoshisato Yanagisawa