Currently the output_json method generates revisions for projects that
are in the revision_mapping and only those projects appear in the
manifest and source_manifest, using the generated revision. This method
is used to construct the default json output for bot_update in tests.
This has 2 problems:
1. If a project has a fixed revision, its manifest entries and the
property used the generated revision, meaning the gitiles input revision
will not be used in the json output.
2. If a project has a fixed revision and isn't present in
revision_mapping then manifest entries aren't generated for it.
This changes the method to use the revisions in the fixed_revisions dict
and to create manifest entries for the projects that appear in the
fixed_revisions dict which should make fewer situations where
bot_update's output_json needs to be explicitly set.
Change-Id: I99de52edc9248c28e9d3aa3a04cc77bf9fccbe37
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: chrome_release
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2304644
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Garrett Beaty <gbeaty@chromium.org>
We no longer need it to bridge path differences between buildbot and
LUCI, and it shouldn't be in use downstream now either.
R=machenbach@chromium.org, martiniss@chromium.org, tandrii@chromium.org, yiwzhang@google.com
Bug: 1015181
Change-Id: Iaba2100653ae9f06799d1d510bef993f7f710dc4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2220171
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
This is a reland of 72a245e4c6
The "git checkout master --detach" operation has been modified to run in the solution
directory.
Original change's description:
> Fetch the input commit ref when both ref and id are specified.
>
> This causes --refs refs/head/master to be added to bot_update calls when
> the input build message contains a commit id and the refs/head/master
> ref. This is an effective no-op.
>
> When bot_update clones a git repository, it results in HEAD referring to
> master in contrast to the non-clone case which results in a detached
> HEAD. This prevents fetching refs/heads/master, so detach the HEAD after
> cloning.
>
> Bug: 922150
> Change-Id: I357a78305020218e9f5d350832a706b8f2b27967
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Recipe-Nontrivial-Roll: infra
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1900735
> Commit-Queue: Garrett Beaty <gbeaty@chromium.org>
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Bug: 922150
Change-Id: I7dc6ebed2dcbd880ff4d515f7e9fc6428ff24099
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: infra
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1910260
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Garrett Beaty <gbeaty@chromium.org>
This reverts commit 72a245e4c6.
Reason for revert: crbug.com/1022617
Original change's description:
> Fetch the input commit ref when both ref and id are specified.
>
> This causes --refs refs/head/master to be added to bot_update calls when
> the input build message contains a commit id and the refs/head/master
> ref. This is an effective no-op.
>
> When bot_update clones a git repository, it results in HEAD referring to
> master in contrast to the non-clone case which results in a detached
> HEAD. This prevents fetching refs/heads/master, so detach the HEAD after
> cloning.
>
> Bug: 922150
> Change-Id: I357a78305020218e9f5d350832a706b8f2b27967
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Recipe-Nontrivial-Roll: infra
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1900735
> Commit-Queue: Garrett Beaty <gbeaty@chromium.org>
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
TBR=ehmaldonado@chromium.org,gbeaty@chromium.org
Change-Id: Ibd6c6ed272b0c98f82be50cefa521aaee0444f11
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 922150, 1022617
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1904099
Reviewed-by: Stephen Martinis <martiniss@chromium.org>
Commit-Queue: Stephen Martinis <martiniss@chromium.org>
This causes --refs refs/head/master to be added to bot_update calls when
the input build message contains a commit id and the refs/head/master
ref. This is an effective no-op.
When bot_update clones a git repository, it results in HEAD referring to
master in contrast to the non-clone case which results in a detached
HEAD. This prevents fetching refs/heads/master, so detach the HEAD after
cloning.
Bug: 922150
Change-Id: I357a78305020218e9f5d350832a706b8f2b27967
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: infra
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1900735
Commit-Queue: Garrett Beaty <gbeaty@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
This reverts commit 512f92b731.
Reason for revert: bot_update is having infra failure on ci bots.
For example https://ci.chromium.org/p/chromium/builders/ci/Closure%20Compilation%20Linux/249684.
Original change's description:
> Fetch the input commit ref when both ref and id are specified.
>
> This causes --refs refs/head/master to be added to bot_update
> calls when the input build message contains a commit id and the
> refs/head/master ref. This is an effective no-op.
>
> Bug: 922150
> Change-Id: I119d9b3cede727713a0b66bff83b5ee4e1848790
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: infra
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1884457
> Commit-Queue: Garrett Beaty <gbeaty@chromium.org>
> Auto-Submit: Garrett Beaty <gbeaty@chromium.org>
> Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
TBR=tandrii@google.com,gbeaty@chromium.org
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: 922150
Change-Id: Ie9275c21827688f59449f052c768452a8652e139
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1891000
Commit-Queue: Haiyang Pan <hypan@google.com>
Reviewed-by: Haiyang Pan <hypan@google.com>
This causes --refs refs/head/master to be added to bot_update
calls when the input build message contains a commit id and the
refs/head/master ref. This is an effective no-op.
Bug: 922150
Change-Id: I119d9b3cede727713a0b66bff83b5ee4e1848790
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1884457
Commit-Queue: Garrett Beaty <gbeaty@chromium.org>
Auto-Submit: Garrett Beaty <gbeaty@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).
Please review the expectation changes, and LGTM+CQ.
More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
https://crrev.com/e7fec5ccb9e70bf17b7878e63052a874afa06856 Change is_luci default to True. (mmoss@google.com)
R=recipe-roller@chromium.org, tandrii@chromium.org
Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I9eb8913a8f65b8f107e5856c46aceda10e14c00c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1752476
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Michael Moss <mmoss@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
got_revision and got_revision_cp point to the commit that
Build.output.gitiles_commit should point to.
Therefore, derive output commit from these properties.
Derive output ref from got_revision_cp if it available.
Otherwise, try to read it from input commit
Bug: 940214
Change-Id: I108d782ad93c70114581830d7427b95d3efd9d98
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1588506
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Use get_gerrit_patch_root in bot_update.ensure_checkout.
This fixes some tests expectations that expected CI builds to pass --patch_root
flag.
Also unify "root" and "patch_root" variables in ensure_checkout. Simpler.
This CL requires nontrivial rolls in all downstream packages because all
packages have incorrect expectations and/or tests.
In particular, this removes --patch_root flag from non-tryjob test
cases. Some tests are supposed to test tryjob scenario, but they use
properties.generic instead of properties.tryserver (or buildbucket.try_build)
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: release_scripts
Recipe-Nontrivial-Roll: skiabuildbot
Bug: 694348
Change-Id: Ibc594fa42e3b4d2823672b41dda2d97154b10d38
Reviewed-on: https://chromium-review.googlesource.com/1231280
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
This reverts commit 355b11a31f.
Reason for revert:
Easier to do this another way that doesn't cause expectation changes,
so it doesn't require non-trivial rolls.
Original change's description:
> bot_update: Patch on gclient by default.
>
> See [4] for the design doc for this change.
>
> We're moving the patching logic from bot_update to gclient.
> This way we can test patches of dependencies that modify their DEPS
> files as well, for example when ANGLE uploads a patch that modifies
> DEPS files to be tested by a Chromium trybot.
>
> Chromium Linux CQ and Angle trybots are already running this way
> (see [1] and [2]) and a PSA has already been sent [3]
>
> [1] https://chromium.googlesource.com/chromium/src/+/master/infra/config/global/cr-buildbucket.cfg#214
> [2] https://chromium.googlesource.com/angle/angle/+/infra/config/cr-buildbucket.cfg#105
> [3] https://groups.google.com/a/google.com/forum/#!topic/chrome-infra/FVHN2kdUX4I
> [4] https://docs.google.com/document/d/1GSTw4ysm5AYjNR6vMA7NJUxLK2VN9IpOEp_v0u-wyOE/edit?ts=5ad78165#
>
> Bug: 643346
> Change-Id: Ic9a9dcfae86aca730d35fbb26c26b21135e05bf3
> Recipe-Nontrivial-Roll: skia
> Recipe-Nontrivial-Roll: infra
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Recipe-Nontrivial-Roll: skiabuildbot
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: release_scripts
> Reviewed-on: https://chromium-review.googlesource.com/1014356
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Aaron Gable <agable@chromium.org>
TBR=agable@chromium.org,ehmaldonado@chromium.org
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: 643346
Change-Id: I0e694b0e398770b8bd2e5a3170e158a7e54c5a61
Reviewed-on: https://chromium-review.googlesource.com/1029310
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
The test output was incorrect, this matches what it actually should look like.
Bug: 772529
Change-Id: I1ca9ceeb6d6b6f55b2ac7c90667c6a9904d2a43b
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: skia
Recipe-Nontrivial-Roll: release_scripts
Change-Id: I1ca9ceeb6d6b6f55b2ac7c90667c6a9904d2a43b
Reviewed-on: https://chromium-review.googlesource.com/751694
Commit-Queue: Ryan Tseng <hinoka@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
This reverts commit 47b67c426b.
Reason for revert: Bug in bot_update.py resource, breaks recipe roller.
Original change's description:
> Reland "bot_update recipe: Upload source manifest"
>
> This reverts commit c3d1208d5c.
>
> Also:
> * Instead of replacing "manifest", just add a new "source_manifest"
> to the output JSON. This allow transition without breakage.
> * Change the test api so test for recipe output changes.
>
> The plan is to land this first, switch all downstream to "source_manifest",
> and then remove the original "manifest" key.
>
> Bug: 772529,776299
> Change-Id: Iffb75f18046f8e4c058afe077872d4257b9dd754
> Recipe-Nontrivial-Roll: infra
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Recipe-Nontrivial-Roll: skiabuildbot
> Recipe-Nontrivial-Roll: release_scripts
> Recipe-Nontrivial-Roll: skia
> Recipe-Nontrivial-Roll: skiabuildbot
> Recipe-Nontrivial-Roll: build
> Reviewed-on: https://chromium-review.googlesource.com/731378
> Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> Commit-Queue: Ryan Tseng <hinoka@chromium.org>
TBR=iannucci@chromium.org,hinoka@chromium.org
Change-Id: I7a4ee904075e8b75b8a47f9ef0cd8a633af85a9c
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 772529, 776299
Reviewed-on: https://chromium-review.googlesource.com/748312
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Commit-Queue: Ryan Tseng <hinoka@chromium.org>
This reverts commit c3d1208d5c.
Also:
* Instead of replacing "manifest", just add a new "source_manifest"
to the output JSON. This allow transition without breakage.
* Change the test api so test for recipe output changes.
The plan is to land this first, switch all downstream to "source_manifest",
and then remove the original "manifest" key.
Bug: 772529,776299
Change-Id: Iffb75f18046f8e4c058afe077872d4257b9dd754
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: skiabuildbot
Recipe-Nontrivial-Roll: release_scripts
Recipe-Nontrivial-Roll: skia
Recipe-Nontrivial-Roll: skiabuildbot
Recipe-Nontrivial-Roll: build
Reviewed-on: https://chromium-review.googlesource.com/731378
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Ryan Tseng <hinoka@chromium.org>
This adds angle to the got_revision_mapping to take account of the checked
out revision and use it correctly in the without-patch application.
This just adds the got_revision property to all test cases in the output
and fixes the second bot_update run of two angle test cases.
Bug: 740456
Change-Id: I6ae6bf627653280bc2a94230ec77949411dc4331
Reviewed-on: https://chromium-review.googlesource.com/573741
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
This makes bot_update query Gerrit when syncing for Gerrit tryjobs.
The query will establish the actual destination branch of the CL,
which can be different from master (e.g. a feature branch).
Bot_update will ensure to use this destination branch for the repo
that corresponds to the CL's project. Both the main project or a
deps'ed project work.
Initially, this lives behind a flag that can be controlled in
downstream recipes. Eventually we'll set this to default after a
gradual roll-out.
Branches in branch-heads are not supported yet.
Bug: 740456
Change-Id: I4a0d50e2ca8fe90f8d29964a3ffab17291f7be60
Reviewed-on: https://chromium-review.googlesource.com/566824
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
"bot_update" currently passes a full "gclient" specification as a
command-line argument to "bot_update.py". This file contains all sorts
of junk that clutters the command-line, and revealed a batch file
escaping bug that seems to be present in the recipe engine. Change this
so that "bot_update" passes the spec as file content instead of a
command-line flag.
Also, "bot_update" apparently called "--spec", which optparse kindly
recognized as the real flag, "--specs". This has always(?) been a bug,
and is no longer a problem.
BUG=chromium:738348
TEST=expectations
Change-Id: I5324b94dd5ef78503e7cfdc28b5fde8f434e4359
Reviewed-on: https://chromium-review.googlesource.com/558207
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>
Prefix "depot_tools" to PATH instead of appending. This will ensure that
"depot_tools" pulls resources from the recipe checkout as a preference,
not as a last resort.
Use "env_prefixes" instead of "env" to prefix PATH.
Remove Git speed limit settings, which are taken care of by the Git
wrapper now.
BUG=chromium:738176
TEST=expectations
R=hinoka@chromium.org, iannucci@chromium.org
Change-Id: I2e509a331266e867be323bc70e3bc0f08f85246f
Reviewed-on: https://chromium-review.googlesource.com/556287
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>
Currently, "bot_update" relies on a BuildBot cleanup mechanism and, to a
lesser extent, the standard BuildBot directory layout. Both of these are
problematic when projecting it into other circumstances, notably
"remote_run" and LUCI.
Have "bot_update" handle its own cleanup. It will now choose a cleanup
directory within the hierarchy of its checkout, and explicitly purge it
prior to execution if it exists. This enforces its expected behavior in
all circumstances and removes its expectations of the greater checkout
layout.
Export "cleanup_dir" via "infra_paths" to point to "build.dead" when
running on BuildBot builds. Otherwise, it is a default directory which,
on Kitchen, is ephemeral by design.
BUG=chromium:725631
TEST=expectations
Change-Id: I664434c542a25aaa7ff3eac216208a2425730fde
Reviewed-on: https://chromium-review.googlesource.com/528057
Commit-Queue: Daniel Jacques <dnj@chromium.org>
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Garrett Beaty
Arthur Wang
Robert Iannucci
Josip Sokcevic
Stephen Martinis
Haiyang Pan
recipe-roller
Nodir Turakulov
Stephen Martinis
Edward Lesmes
Ryan Tseng
Ryan Tseng
Michael Achenbach
Dan Jacques
Dan Jacques