Commit Graph

27 Commits (b495bf501777c639131f54328d83b38a3163035e)

Author SHA1 Message Date
Michael Achenbach c14b4f1518 Fix bot_update default value for empty revision
Bug: 753297
Change-Id: I31c031ce1efa3a1a7d30fd7eeb71d3e4b3baa720
Reviewed-on: https://chromium-review.googlesource.com/605609
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
8 years ago
Sergey Berezin 8038dfe3be bot_update recipe: parameterize step name in apply_gerrit_ref
Needed for parity with Rietveld two-sided patch application.

BUG=628259
R=agable@chromium.org

Change-Id: I79221f54e89a9291257a71b70cc52acc51167854
Reviewed-on: https://chromium-review.googlesource.com/591047
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Sergey Berezin <sergeyberezin@chromium.org>
8 years ago
Sergey Berezin 5aa5cd76f0 depot_tools: add gerrit repo and ref to apply_gerrit_ref()
The only user of this method needs to apply two patches from two
different repos in a try job. Extending the method to handle this.

BUG=628259
R=agable@chromium.org

Change-Id: Ia119780d1323add0094631e5c291b0e7a573e6e5
Reviewed-on: https://chromium-review.googlesource.com/588008
Commit-Queue: Sergey Berezin <sergeyberezin@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
8 years ago
Michael Achenbach 98681d113f Remove obsolete destination-branch-check flag
This was removed in downstream recipes after it was turned on by default.

Bug: 740456
Change-Id: Idae1905af97333970c8b31508634f8b71550c6f9
Reviewed-on: https://chromium-review.googlesource.com/574030
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
8 years ago
Michael Achenbach f3d5a1147f Enable Gerrit-destination-branch check by default
Bug: 740456
Change-Id: Ib078d6aef7286bbe945513a5818e185d34f52058
Reviewed-on: https://chromium-review.googlesource.com/571802
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
8 years ago
Michael Achenbach 02e59414c1 Fix bot_update's usage of the destination branch
The previously passed branch:HEAD notation is understood by bot_update
for the main project, but not by gclient for deps'ed projects.

We don't need the colon notation at all as passing a branch implies
using the HEAD of that branch.

Bug: 740456
Change-Id: I95eb88f0de2e06bee8a3e7db24c4ad85cdb76d3e
Reviewed-on: https://chromium-review.googlesource.com/566866
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
8 years ago
Michael Achenbach 7dadf05dad Query Gerrit for destination branch in bot_update
This makes bot_update query Gerrit when syncing for Gerrit tryjobs.

The query will establish the actual destination branch of the CL,
which can be different from master (e.g. a feature branch).

Bot_update will ensure to use this destination branch for the repo
that corresponds to the CL's project. Both the main project or a
deps'ed project work.

Initially, this lives behind a flag that can be controlled in
downstream recipes. Eventually we'll set this to default after a
gradual roll-out.

Branches in branch-heads are not supported yet.

Bug: 740456
Change-Id: I4a0d50e2ca8fe90f8d29964a3ffab17291f7be60
Reviewed-on: https://chromium-review.googlesource.com/566824
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
8 years ago
Dan Jacques 70b3f3aae5 [bot_update] Pass spec as file.
"bot_update" currently passes a full "gclient" specification as a
command-line argument to "bot_update.py". This file contains all sorts
of junk that clutters the command-line, and revealed a batch file
escaping bug that seems to be present in the recipe engine. Change this
so that "bot_update" passes the spec as file content instead of a
command-line flag.

Also, "bot_update" apparently called "--spec", which optparse kindly
recognized as the real flag, "--specs". This has always(?) been a bug,
and is no longer a problem.

BUG=chromium:738348
TEST=expectations

Change-Id: I5324b94dd5ef78503e7cfdc28b5fde8f434e4359
Reviewed-on: https://chromium-review.googlesource.com/558207
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>
8 years ago
Dan Jacques dc598e8223 [bot_update] Prefix PATH, cleanup.
Prefix "depot_tools" to PATH instead of appending. This will ensure that
"depot_tools" pulls resources from the recipe checkout as a preference,
not as a last resort.

Use "env_prefixes" instead of "env" to prefix PATH.

Remove Git speed limit settings, which are taken care of by the Git
wrapper now.

BUG=chromium:738176
TEST=expectations
R=hinoka@chromium.org, iannucci@chromium.org

Change-Id: I2e509a331266e867be323bc70e3bc0f08f85246f
Reviewed-on: https://chromium-review.googlesource.com/556287
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>
8 years ago
Dan Jacques 59a74ed558 Add cleanup dir, use with bot_update.
Currently, "bot_update" relies on a BuildBot cleanup mechanism and, to a
lesser extent, the standard BuildBot directory layout. Both of these are
problematic when projecting it into other circumstances, notably
"remote_run" and LUCI.

Have "bot_update" handle its own cleanup. It will now choose a cleanup
directory within the hierarchy of its checkout, and explicitly purge it
prior to execution if it exists. This enforces its expected behavior in
all circumstances and removes its expectations of the greater checkout
layout.

Export "cleanup_dir" via "infra_paths" to point to "build.dead" when
running on BuildBot builds. Otherwise, it is a default directory which,
on Kitchen, is ephemeral by design.

BUG=chromium:725631
TEST=expectations

Change-Id: I664434c542a25aaa7ff3eac216208a2425730fde
Reviewed-on: https://chromium-review.googlesource.com/528057
Commit-Queue: Daniel Jacques <dnj@chromium.org>
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
8 years ago
Paweł Hajdan, Jr 6cfe8d4301 Add disable_syntax_validation gclient config to easily whitelist builders
Bug: 570091
Change-Id: Icb15bcd354c9b524674c6b4496f430c26cf7b89e
Reviewed-on: https://chromium-review.googlesource.com/521482
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
8 years ago
Paweł Hajdan, Jr c515a8bc46 bot_update: expose --disable-syntax-validation to recipes
Some recipes interacting with older revisions (e.g. bisect)
will need this.

Bug: 570091
Change-Id: I38e5ffa2db1a9bfae3667f015f00977c32ebe51e
Reviewed-on: https://chromium-review.googlesource.com/519407
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
8 years ago
Paweł Hajdan, Jr 16d2ae8cb0 bot_update: fix regression with swallowed exception
This was regressed in https://chromium-review.googlesource.com/475950

Downstream regression test added in https://chromium-review.googlesource.com/c/508697/

Bug: 723572
Change-Id: I44dbaf9e4996388b898359141db879bd0a1023d9
Reviewed-on: https://chromium-review.googlesource.com/508790
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Daniel Jacques <dnj@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
8 years ago
Robert Iannucci 5702f0baef [recipes] convert all recipe_modules to use new context module.
R=recipe-roller@chromium.org

Bug:
Change-Id: I84c19965b779f1010c00cd8c857b7533eec7413f
Reviewed-on: https://chromium-review.googlesource.com/502929
Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
8 years ago
Michael Achenbach 644a1d87e6 Stop passing deprecated manifest flag to bot_update
The flag has been deprecated in:
https://chromium-review.googlesource.com/c/486640

This is a separate CL for the expectation changes.

Bug: 398105
Change-Id: I129186323e5ecffe441febe7fa3c6c72bfc9c4cd
Reviewed-on: https://chromium-review.googlesource.com/486782
Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
8 years ago
Michael Achenbach 86989d7b0f Reland "Remove obsolete bot_update flag"
This reverts commit c91686c087.

Reason for revert: Reland a light version of the original CL that keeps a dummy flag around.

Original change's description:
> Revert "Remove obsolete bot_update flag"
> 
> This reverts commit 72048266d5.
> 
> Reason for revert: Skia depends on it. Need to clean that up first. Currently blocks roll:
> https://skia-review.googlesource.com/c/14145/
> 
> Original change's description:
> > Remove obsolete bot_update flag
> > 
> > Using the manifest is the default since a long time. The flag was only introduced to smoothly switch it on. Now the flag is not needed anymore.
> > 
> > Bug: 398105
> > Change-Id: I4eb1c98e26e025a69f8b0a7c48877d78c031702e
> > Reviewed-on: https://chromium-review.googlesource.com/485659
> > Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> > Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
> > Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> > 
> 
> TBR=phajdan.jr@chromium.org,machenbach@chromium.org,tandrii@chromium.org,sergiyb@chromium.org,chromium-reviews@chromium.org
> NOPRESUBMIT=true
> NOTREECHECKS=true
> NOTRY=true
> 
> Change-Id: Iffc4d8076abd3c92feebfd7f3ef88be7ce34b692
> Reviewed-on: https://chromium-review.googlesource.com/486639
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> 

R=phajdan.jr@chromium.org,machenbach@chromium.org,tandrii@chromium.org,sergiyb@chromium.org,chromium-reviews@chromium.org
Bug: 398105

Change-Id: I817d446f8a8eff84c5d7143ac2b9c527a40991f1
Reviewed-on: https://chromium-review.googlesource.com/486640
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
8 years ago
Michael Achenbach c91686c087 Revert "Remove obsolete bot_update flag"
This reverts commit 72048266d5.

Reason for revert: Skia depends on it. Need to clean that up first. Currently blocks roll:
https://skia-review.googlesource.com/c/14145/

Original change's description:
> Remove obsolete bot_update flag
> 
> Using the manifest is the default since a long time. The flag was only introduced to smoothly switch it on. Now the flag is not needed anymore.
> 
> Bug: 398105
> Change-Id: I4eb1c98e26e025a69f8b0a7c48877d78c031702e
> Reviewed-on: https://chromium-review.googlesource.com/485659
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> 

TBR=phajdan.jr@chromium.org,machenbach@chromium.org,tandrii@chromium.org,sergiyb@chromium.org,chromium-reviews@chromium.org
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true

Change-Id: Iffc4d8076abd3c92feebfd7f3ef88be7ce34b692
Reviewed-on: https://chromium-review.googlesource.com/486639
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
8 years ago
Michael Achenbach 72048266d5 Remove obsolete bot_update flag
Using the manifest is the default since a long time. The flag was only introduced to smoothly switch it on. Now the flag is not needed anymore.

Bug: 398105
Change-Id: I4eb1c98e26e025a69f8b0a7c48877d78c031702e
Reviewed-on: https://chromium-review.googlesource.com/485659
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
8 years ago
Michael Achenbach 521c9ed147 Support overriding gclient config in a bot_update method
Bug: 713356
TBR=tandrii@chromium.org

Change-Id: I15d73492712b66c7670793f3c47e4cbd89d4e1cd
Reviewed-on: https://chromium-review.googlesource.com/484440
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
8 years ago
Michael Achenbach 93f69da3b2 Support merging got_revision mappings during transition phase.
A few rare recipes use two or more solutions side-by-side.

This allows switching one solution in isolation to use the new mapping.

Bug: 713356
Change-Id: Id51f2f777681401843004577a4de65a6c5d1e4a6
Reviewed-on: https://chromium-review.googlesource.com/484341
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
8 years ago
Michael Achenbach 81b4d420bf Reverse got_revision_mapping in bot_update
This is the first step in reversing got_revision_mapping.

This CL:
- Adds a new configuration got_revision_reverse_mapping.
- Ensures only the old or the new configuration is used.
- Transforms the old to the new configuration within bot_update.

In follow up CLs we will migrate all clients to the new mapping, then delete the old and rename the new again.

Bug: 713356
Change-Id: Icdbd6a3748c6e5399b9c4867a3d8292973325160
Reviewed-on: https://chromium-review.googlesource.com/483479
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
8 years ago
Nodir Turakulov a7ba43a3a8 Revert "bot_update: pin depot_tools"
This reverts commit e8acd40a11.

Reason for revert: broke windows builds :(

Original change's description:
> bot_update: pin depot_tools
> 
> gclient runhooks often calls scripts that depend on scripts in depot_tools.
> Currently bot_update only ensures that depot_tools is present in $PATH, but
> otherwise uses whatever depot_tools already specified in $PATH. We don't pin
> depot_tools before running a recipe, so using whatever depot_tools in $PATH
> is not hermetic.
> 
> Insert the path to the pinned depot_tools to the beginning of $PATH
> 
> R=​iannucci@chromium.org
> 
> Bug:
> 
> Change-Id: I31d04ba2ec7c409f89d58a51502ce3565e3a02a1
> Reviewed-on: https://chromium-review.googlesource.com/481072
> Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> Commit-Queue: Nodir Turakulov <nodir@chromium.org>
> 

TBR=iannucci@chromium.org,nodir@chromium.org,phajdan.jr@chromium.org,chromium-reviews@chromium.org
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true

Change-Id: I1a70f0f54e9b9cea7fc2f9897e62f558204a4684
Reviewed-on: https://chromium-review.googlesource.com/481440
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
8 years ago
Nodir Turakulov e8acd40a11 bot_update: pin depot_tools
gclient runhooks often calls scripts that depend on scripts in depot_tools.
Currently bot_update only ensures that depot_tools is present in $PATH, but
otherwise uses whatever depot_tools already specified in $PATH. We don't pin
depot_tools before running a recipe, so using whatever depot_tools in $PATH
is not hermetic.

Insert the path to the pinned depot_tools to the beginning of $PATH

R=iannucci@chromium.org

Bug:

Change-Id: I31d04ba2ec7c409f89d58a51502ce3565e3a02a1
Reviewed-on: https://chromium-review.googlesource.com/481072
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
8 years ago
Dan Jacques 8dd4edae99 bot_update: Show failed patch in "Patch Failure".
Have "bot_update.py" export the failing patch body in its JSON. Have the
recipe actually emit the patch body in the "Patch Failure" step instead
of the "bot_update" step, and remove the "Patch Failure" step's
indirection to the "bot_update" step for patch failure.

This will provide a more straightforward user interface: the red step
that says "Patch Failure" now includes the details of the patch failure!

BUG=chromium:710477
TEST=expectations

Change-Id: I6ad305f0b972de89e4f0b5eb70edf41980447abd
Reviewed-on: https://chromium-review.googlesource.com/475950
Commit-Queue: Daniel Jacques <dnj@chromium.org>
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
8 years ago
Paweł Hajdan, Jr f4597a4f91 bot_update: add with_tags argument to ensure_checkout
Bug: 710817
Change-Id: Ifb2a9abe605d9e403c55ec1360d186af12dd2f34
Reviewed-on: https://chromium-review.googlesource.com/475775
Reviewed-by: Michael Moss <mmoss@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
8 years ago
Paweł Hajdan, Jr 36d66752a7 bot_update: add basic deapply_patch logic
This will make it reusable, e.g. for https://chromium-review.googlesource.com/c/465417/

Bug: none
Change-Id: I96f13c6ff3aff22c787e290979602e7a0d725213
Reviewed-on: https://chromium-review.googlesource.com/474139
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
8 years ago
Paweł Hajdan, Jr f14709556d Move recipes to their own subdirectory
This will help remove confusion when depot_tools' recipes.py
ends up in PATH.

BUG=699120

Change-Id: Id4c21b0cc6bb022ea2c21145abe76bebb0a8d9c1
Reviewed-on: https://chromium-review.googlesource.com/458430
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
8 years ago