Commit Graph

136 Commits (2b51c930e009ff0facd2b04af3975a826eef0a23)

Author SHA1 Message Date
Andrii Shyshkalov 5eab58ee90 Revert "bot_update: allow rebasing the patch onto an older revision."
This reverts commit fa573785df.

Reason for revert: broke CQ on branches.
Example: https://ci.chromium.org/p/chromium/builders/luci.chromium.try/chromium_presubmit/28220

Original change's description:
> bot_update: allow rebasing the patch onto an older revision.
>
> When applying a rebase, we normally go from an older base commit to a newer one.
>
> A ---- B ---- C ---- D -origin/master
>                \
>                 E -branch
>
> In this case, `git rebase D` would certainly work as expected. However,
> writing `git rebase B` would NOT get us to the following state:
>
> A ---- B ---- C ---- D -origin/master
>         \
>          E' -branch
>
> In fact, it would have no effect.
>
> This article http://matthew-brett.github.io/pydagogue/rebase_without_tears.html
> explains the general invocation as
> > `git rebase --onto <graft-point> <exclude-from> <include-from>`
> > If you don’t specify --onto, <graft-point> defaults to <exclude-from>
>
> So what's happening is, by writing `git rebase B` we're rebasing onto B,
> excluding commits that are in B. Commit C is not "in" B so it is kept and we're
> back to the starting point.
>
> So I suggest to change the invocation to `git rebase --onto B origin/master`,
> which rebases onto B, excluding commits that are in origin/master. This works
> more generally and allows rebasing "backwards".
>
> Bug: None
> Change-Id: I68e4d805811530b585550bc75099354fef4e9c15
> Reviewed-on: https://chromium-review.googlesource.com/904004
> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
> Commit-Queue: Oleh Prypin <oprypin@chromium.org>

TBR=iannucci@chromium.org,tandrii@chromium.org,oprypin@chromium.org

Change-Id: I7243641d84428bf4504ee798bf31ad0afbfd4865
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: None
Reviewed-on: https://chromium-review.googlesource.com/905862
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
8 years ago
Andrii Shyshkalov cd33775a7a bot_update: optimize checkout of solution's main repo.
Goal: reduce number of git fetches from remote to the minimum possible.
Minimum possible is either:
 * 0 if desired revision is pinned (!= HEAD) and already exists in cache
 * 1 if desired revision is HEAD
 * >= 2 if revision is pinned but we talk temporarily
   out-of-date git server not yet having it.

This CL achieves the above.

No fetch:
  https://ci.chromium.org/swarming/task/39dad20230c84110?server=chromium-swarm.appspot.com
  bot_update takes <1min, but it used to take at least 2. The steps are now like this:

  ...
  ===Running git cat-file -e 33bf8a94dcd5e0abbdf83e4afaed24b5180e3eb2===
  In directory: /b/swarming/w/ir/cache/git/chromium.googlesource.com-chromium-src
  ===Succeeded in 0.0 mins===
  ===Running /b/swarming/w/ir/cipd_bin_packages/bin/python -u /b/swarming/w/ir/recipe-checkout-dir/depot_tools/git_cache.py exists --quiet --cache-dir /b/swarming/w/ir/cache/git https://chromium.googlesource.com/chromium/src.git===
  In directory: /b/swarming/w/ir/cache/builder/linux
  /b/swarming/w/ir/cache/git/chromium.googlesource.com-chromium-src
  ===Succeeded in 0.0 mins===
  ===Running git remote set-url origin /b/swarming/w/ir/cache/git/chromium.googlesource.com-chromium-src===
  In directory: /b/swarming/w/ir/cache/builder/linux/src
  ===Succeeded in 0.0 mins===
  ===Running git fetch origin===
  In directory: /b/swarming/w/ir/cache/builder/linux/src
  From /b/swarming/w/ir/cache/git/chromium.googlesource.com-chromium-src
     003eedbfee47..5210107a20e8  master     -> origin/master
  ===Succeeded in 0.0 mins===
  ===Running git checkout --force 33bf8a94dcd5e0abbdf83e4afaed24b5180e3eb2===
  ...

Fetch & success:
  https://ci.chromium.org/swarming/task/39db022b874ffd10?server=chromium-swarm.appspot.com

Fetch & retry because missing rev:
  https://ci.chromium.org/swarming/task/39db04ce8b4b8910?server=chromium-swarm.appspot.com

Fetch because rev=HEAD:
  https://ci.chromium.org/swarming/task/39db0c0dd5ab5d10?server=chromium-swarm.appspot.com

This has already been adopted in gclient.

Bug:
Change-Id: Id99892b62719fdf3f7e6e59058986d1500384f8d
Reviewed-on: https://chromium-review.googlesource.com/771591
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Jao-ke Chin-Lee <jchinlee@chromium.org>
8 years ago
Oleh Prypin fa573785df bot_update: allow rebasing the patch onto an older revision.
When applying a rebase, we normally go from an older base commit to a newer one.

A ---- B ---- C ---- D -origin/master
               \
                E -branch

In this case, `git rebase D` would certainly work as expected. However,
writing `git rebase B` would NOT get us to the following state:

A ---- B ---- C ---- D -origin/master
        \
         E' -branch

In fact, it would have no effect.

This article http://matthew-brett.github.io/pydagogue/rebase_without_tears.html
explains the general invocation as
> `git rebase --onto <graft-point> <exclude-from> <include-from>`
> If you don’t specify --onto, <graft-point> defaults to <exclude-from>

So what's happening is, by writing `git rebase B` we're rebasing onto B,
excluding commits that are in B. Commit C is not "in" B so it is kept and we're
back to the starting point.

So I suggest to change the invocation to `git rebase --onto B origin/master`,
which rebases onto B, excluding commits that are in origin/master. This works
more generally and allows rebasing "backwards".

Bug: None
Change-Id: I68e4d805811530b585550bc75099354fef4e9c15
Reviewed-on: https://chromium-review.googlesource.com/904004
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Oleh Prypin <oprypin@chromium.org>
8 years ago
Lei Zhang f2b5774fef Add got_revision key to pdfium buildbot config.
This is needed for run_presubmit.py to work.

BUG=pdfium:971

Change-Id: Ib598fbfd6eab3c0b7628301eb3bd54c548a4885e
Reviewed-on: https://chromium-review.googlesource.com/898039
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Lei Zhang <thestig@chromium.org>
8 years ago
Michael Moss b13fba7efb Allow bools in gclient recipe custom_vars.
This has been supported in gclient, and used in DEPS, since
https://crrev.com/c/687496.

R=dpranke@google.com, iannucci@google.com

Bug: 570091,807318
Change-Id: I59935625e3bae6010a371ad7bb1413e177a09176
Reviewed-on: https://chromium-review.googlesource.com/896835
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Michael Moss <mmoss@chromium.org>
8 years ago
Takuto Ikuta 4fbf4bece2 Remove windows specific git pack limit
This is full version of
https://chromium-review.googlesource.com/823544

After the CL landing, we see no performance regression in bot_update step.
And regular 95%ile in 1h window improved from more than 300s to less than or around 200s.
See tandrii's query in https://bugs.chromium.org/p/chromium/issues/detail?id=749709#c91 for detail.

Let me reduce frequency of slow bot_update step more like below.
https://ci.chromium.org/buildbot/tryserver.chromium.win/win7_chromium_rel_ng/69844
https://ci.chromium.org/buildbot/tryserver.chromium.win/win7_chromium_rel_ng/69886
https://ci.chromium.org/buildbot/tryserver.chromium.win/win7_chromium_rel_ng/69892

Bug: 749709
Change-Id: I44474aa9ecdd733d68fd13fe8b791820fb80381f
Reviewed-on: https://chromium-review.googlesource.com/838766
Commit-Queue: Takuto Ikuta <tikuta@google.com>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
8 years ago
Aaron Gable de8ce79f79 Remove rietveld support from tryserver recipe module
Bug: 770408
Recipe-Nontrivial-Roll: build
Change-Id: Icafd4bd11009975c40e53d93901589c07ecd738e
Reviewed-on: https://chromium-review.googlesource.com/707723
Commit-Queue: Aaron Gable <agable@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
8 years ago
recipe-roller ac4e7911c0 Roll recipe dependencies (nontrivial).
This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).


Please review the expectation changes, and LGTM+CQ.


More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
  https://crrev.com/77667b5ba38c4d205a743d04eb625ac068bdd78e Always use Gerrit for tryserver recipe tests (agable@chromium.org)


R=dnj@chromium.org, martiniss@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I2df24323974ae0d3b3b0524240b64be5740977c6
Reviewed-on: https://chromium-review.googlesource.com/836760
Commit-Queue: Aaron Gable <agable@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
8 years ago
Aaron Gable aac382b3b6 Remove rietveld-based tests from tryserver recipe module
This introduces a pragma: no cover, which is unfortunate.
But this is a prerequisite for removing rietveld support
entirely from the recipe engine without having to do
manual recipe rolls. The entire block of uncovered code
will be removed shortly.

Bug: 770408
Change-Id: I37319f3ae70413adcda655c443325bccffe695d8
Reviewed-on: https://chromium-review.googlesource.com/834455
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Aaron Gable <agable@chromium.org>
8 years ago
Takuto Ikuta 9fce213bdb Partial revert of "git_cache: lower max num of .pack files before re-bootstrap on Win."
This partially reverts commit d51ed57edb.

Reason for revert:
New git client for windows was rolled including fix for slow `git fetch`.
I guess smaller pack limit causes frequent bootstrap taking 2~3 minutes longer than
the case it does not happen.
Let me see what happen if we increase pack limit 9 -> 30.
I will increase this to 50 if this won't cause regression again.

Original change's description:
> git_cache: lower max num of .pack files before re-bootstrap on Win.
>
> It used to be 50, I think ~9 gives best results for Chromium on Win:
>   on golo VM, it takes <4 minutes to re-boostrap + git fetch small
>   delta, assuming zipped git checkout for bootstrap is fresh (~1day).
>
> For other repos, which are significantly smaller, this change should
> have minor effect if at all.
>
> Test: I tested this using `led` tool on Win7 machines running LUCI
> stack extensively. For example,
>
>   * https://ci.chromium.org/swarming/task/3a0102e8c8657410
>     shows case with few .pack files, hence just 1 fetch
>
>   * https://ci.chromium.org/swarming/task/3a010282f9fd8010
>     shows case with 39 .pack files and so bootstrapping + fetch.
>     If you look at prior tasks on the same VM, you'd find this:
>     https://ci.chromium.org/swarming/task/39ffe843d01ed010
>     which spent 8 minutes doing 1 incremental fetch with 39 .pack
>     files.
>
> **Troopers/Sheriffs**: This change is safe to revert.
> However, beware that you should also at the same time revert the recipe
> roll of this CL to the repo, in which the failed builder's recipe is
> located, typically `chromium/tools/build`.
>
> Bug: 749709
> Change-Id: I18e2b63283100d466e9fb981a9094862463f6909
> Reviewed-on: https://chromium-review.googlesource.com/787174
> Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
> Reviewed-by: Takuto Ikuta <tikuta@google.com>

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: 749709
Change-Id: I3052abe4a9b53277a60c0791a85355e7a0bbdf8f
Reviewed-on: https://chromium-review.googlesource.com/823544
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Takuto Ikuta <tikuta@google.com>
8 years ago
Robert Iannucci 41d9d87e96 [recipe_modules/depot_tools] Add on_path helper.
This will be used by various things in build and build_internal which
"need" a depot_tools on PATH (though I suspect most of those should be
refactored to use the depot_tools in chromium/src.git).

R=tandrii@chromium.org

Bug: 789808
Change-Id: I5d0590ccdc5482bd8e3048ad46a39b5aedacbddb
Reviewed-on: https://chromium-review.googlesource.com/828465
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
8 years ago
Robert Iannucci 069dd4fa6b [bot_update] Put depot_tools at the end of $PATH on LUCI.
This should prevent bot_update from overriding vpython by accident.

R=tandrii@chromium.org

Recipe-Nontrivial-Roll: build
Bug: 789808
Change-Id: Ia574204439588862f29f94a80e978b2ff4b57586
Reviewed-on: https://chromium-review.googlesource.com/828069
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
8 years ago
Aaron Gable 7817f02543 Reland "Use core.quotePath=false when git is listing files"
This is a reland of 9219d35688

The original was reverted due to a typo (core,quotePath instead
of core.quotePath). This version is fixed.

Original change's description:
> Use core.quotePath=false when git is listing files
>
> This prevents git from putting quotes around some file names
> (those that have astral-plane characters) and not around others.
>
> R=maruel
>
> Bug: 792302
>
> Change-Id: I3b6a6b36c4720116de811b40177b59aa25c263db
> Reviewed-on: https://chromium-review.googlesource.com/815454
> Commit-Queue: Aaron Gable <agable@chromium.org>
> Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org>

Bug: 792302
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Change-Id: I28d2260948aaf63bd865888c2f60e4cdee9aea48
Reviewed-on: https://chromium-review.googlesource.com/822990
Commit-Queue: Aaron Gable <agable@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
8 years ago
Robbie Iannucci 6aaee3065c Revert "Use core.quotePath=false when git is listing files"
This reverts commit 9219d35688.

Reason for revert: unfortunately this says "core,quotePath" and since it includes recipe changes, we need something that the roller can munch on :(

Original change's description:
> Use core.quotePath=false when git is listing files
> 
> This prevents git from putting quotes around some file names
> (those that have astral-plane characters) and not around others.
> 
> R=​maruel
> 
> Bug: 792302
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Change-Id: I3b6a6b36c4720116de811b40177b59aa25c263db
> Reviewed-on: https://chromium-review.googlesource.com/815454
> Commit-Queue: Aaron Gable <agable@chromium.org>
> Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org>

TBR=maruel@chromium.org,agable@chromium.org

Change-Id: I226388f19024403240a1443eb2b878b9293220e1
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 792302
Reviewed-on: https://chromium-review.googlesource.com/821671
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
8 years ago
Aaron Gable 9219d35688 Use core.quotePath=false when git is listing files
This prevents git from putting quotes around some file names
(those that have astral-plane characters) and not around others.

R=maruel

Bug: 792302
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Change-Id: I3b6a6b36c4720116de811b40177b59aa25c263db
Reviewed-on: https://chromium-review.googlesource.com/815454
Commit-Queue: Aaron Gable <agable@chromium.org>
Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org>
8 years ago
Andrii Shyshkalov 009150b7af bot_update: be explicit when checking out branches.
$ git checkout <name>

may be interpreted by git as checking out a file named <name>
if <name> has slashes. Thus,

$ git checkout infra/config

ends up checking out infra/config directory of the (typically)
master branch that was checked out before. To avoid confusion,
we have to add '--' separator between branch name and file names:

$ git checkout <name> --

This in turns allows to run CQ-based presubmit on infra/config branches.

R=nodir@chromium.org

Bug: 790738
Change-Id: I6ce31a8f0fbd66fd59ac7c2ea9cccd3ff97d1f0e
Reviewed-on: https://chromium-review.googlesource.com/802136
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
8 years ago
Andrii Shyshkalov 1b30125fbc bot_update: don't remove git cache's .lock files.
bot_update and gclient invokes git cache with --ignore-locks.
This should also reduce noisiness of bot_update stdout.

R=iannucci@chromium.org

Bug:
Change-Id: I6baf9e5665d244d08aab58a7d3327e6c3204b2b5
Reviewed-on: https://chromium-review.googlesource.com/791536
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
8 years ago
Andrii Shyshkalov 46a672bafa gclient: don't update cache if needed revision is already there.
Tested end-to-end, for example
  https://ci.chromium.org/swarming/task/3a0147207378b910
which contains:
  src/media/cdm/api (Elapsed: 0:00:01)
  ----------------------------------------
  [0:00:00] Started.
  _____ src\media\cdm\api at ea5df8e78fbd0a4c24cc3a1f3faefefcd1b45237
  [0:00:00] running "git cat-file -e ea5df8e78fbd0a4c24cc3a1f3faefefcd1b45237" in "e:\b\s\w\ir\cache\git\chromium.googlesource.com-chromium-cdm"
  skipping mirror update, it has rev=ea5df8e78fbd0a4c24cc3a1f3faefefcd1b45237 already

thereby saving on needless git fetch (~40s in glcient sync on win trybots),
and reducing the rate of .pack file accumulation inside cache directories.


Risks: silently broken recipes which run gclient sync (or worse, bot_update)
as a means of fetching latest commits in all repos of a solution. I think
the benefit of faster bot_update in chromium CQ is worth the potential risk.

PSA: https://groups.google.com/a/chromium.org/d/msg/infra-dev/UYLdBwAXm1Y/OV9QB6JnBQAJ

Bug: 749709
Change-Id: I7a9e8ab82a5e2b848e450f19a798ac18a0b5e201
Reviewed-on: https://chromium-review.googlesource.com/787331
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
8 years ago
Andrii Shyshkalov 568f473e4b bot_update: add mooooooore chars in logs.
Before, bot_update would print after command:
  Succeeded (Failed) in 0.1 mins
but figuring out which command takes a lot of scrolling/searching
for previous
  Running <cmd with args>
This CL fixes that:
  Succeeded (Failed) in 0.1 mins <cmd with args>



R=machenbach@chromium.org, maruel@chromium.org
TEST=end-to-end with led tool, but only on LUCI stack.

Bug: 749709
Change-Id: Id3031a12b9013ab136ea03cdf6d325930722e50f
Reviewed-on: https://chromium-review.googlesource.com/787175
Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
8 years ago
Andrii Shyshkalov d51ed57edb git_cache: lower max num of .pack files before re-bootstrap on Win.
It used to be 50, I think ~9 gives best results for Chromium on Win:
  on golo VM, it takes <4 minutes to re-boostrap + git fetch small
  delta, assuming zipped git checkout for bootstrap is fresh (~1day).

For other repos, which are significantly smaller, this change should
have minor effect if at all.

Test: I tested this using `led` tool on Win7 machines running LUCI
stack extensively. For example,

  * https://ci.chromium.org/swarming/task/3a0102e8c8657410
    shows case with few .pack files, hence just 1 fetch

  * https://ci.chromium.org/swarming/task/3a010282f9fd8010
    shows case with 39 .pack files and so bootstrapping + fetch.
    If you look at prior tasks on the same VM, you'd find this:
    https://ci.chromium.org/swarming/task/39ffe843d01ed010
    which spent 8 minutes doing 1 incremental fetch with 39 .pack
    files.

**Troopers/Sheriffs**: This change is safe to revert.
However, beware that you should also at the same time revert the recipe
roll of this CL to the repo, in which the failed builder's recipe is
located, typically `chromium/tools/build`.

Bug: 749709
Change-Id: I18e2b63283100d466e9fb981a9094862463f6909
Reviewed-on: https://chromium-review.googlesource.com/787174
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Takuto Ikuta <tikuta@google.com>
8 years ago
Andrii Shyshkalov 0083130c97 Whitespace change inside recipes to kick recipe autoroller.
TBR=iannucci@chromium.org

Bug: 749709
Change-Id: I8aa02f780c5f2a3c2f8c0c408f83b061ba11c672
Reviewed-on: https://chromium-review.googlesource.com/788244
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
8 years ago
Andrii Shyshkalov 25b1da95b3 Revert "Show GIT_TRACE for git command in bot_update.py"
This reverts commit 6e215cefe2.

Reason for revert: seems to have broken bot_update 
https://ci.chromium.org/swarming/task/39ff1814b3bb5b10?server=chromium-swarm.appspot.com

Original change's description:
> Show GIT_TRACE for git command in bot_update.py
> 
> GIT_TRACE=true tells us which internal git command is slow.
> https://git-scm.com/book/en/v2/Git-Internals-Environment-Variables
> 
> I got following results in my local repository.
> Let me confirm the bots show similar behavior.
> 
> C:\src\chromium\src>set GIT_TRACE=true
> 
> C:\src\chromium\src>git rebase-update
> 21:32:36.519698 git.c:576               trace: exec: 'git-rebase-update'
> 21:32:36.519698 run-command.c:626       trace: run_command: 'git-rebase-update'
> 21:32:38.889848 git.c:344               trace: built-in: git 'fetch' '--multiple' 'origin'
> 21:32:38.893846 run-command.c:626       trace: run_command: 'fetch' '--append' 'origin'
> Fetching origin
> 21:32:38.911858 git.c:344               trace: built-in: git 'fetch' '--append' 'origin'
> 21:32:38.983909 run-command.c:626       trace: run_command: 'remote-https' 'origin' 'https://chromium.googlesource.com/chromium/src.git'
> 21:32:39.002923 git.c:576               trace: exec: 'git-remote-https' 'origin' 'https://chromium.googlesource.com/chromium/src.git'
> 21:32:39.003923 run-command.c:626       trace: run_command: 'git-remote-https' 'origin' 'https://chromium.googlesource.com/chromium/src.git'
> 21:32:43.782266 run-command.c:626       trace: run_command: 'rev-list' '--objects' '--stdin' '--not' '--all' '--quiet'
> 21:32:43.827301 run-command.c:626       trace: run_command: 'fetch-pack' '--stateless-rpc' '--stdin' '--lock-pack' '--include-tag' '--thin' 'https://chromium.googlesource.com/chromium/src.git/'
> remote: Counting objects: 74400, done
> remote: Finding sources: 100% (19/19)
> 21:35:49.348951 run-command.c:626       trace: run_command: 'unpack-objects' '--pack_header=2,19'
> 21:35:49.368985 git.c:344               trace: built-in: git 'unpack-objects' '--pack_header=2,19'
> remote: Total 19 (delta 1), reused 18 (delta 1)
> Unpacking objects: 100% (19/19), done.
> 21:35:49.597328 run-command.c:626       trace: run_command: 'rev-list' '--objects' '--stdin' '--not' '--all' '--quiet'
> 21:35:49.612375 git.c:344               trace: built-in: git 'rev-list' '--objects' '--stdin' '--not' '--all' '--quiet'
> From https://chromium.googlesource.com/chromium/src
>    e8994685157b..fd56d3ad0528  master     -> origin/master
> 21:35:50.281845 run-command.c:1452      run_processes_parallel: preparing to run up to 1 tasks
> 21:35:50.287848 run-command.c:1484      run_processes_parallel: done
> 21:35:50.309865 run-command.c:626       trace: run_command: 'gc' '--auto'
> 21:35:50.328880 git.c:344               trace: built-in: git 'gc' '--auto'
> 21:35:50.453967 run-command.c:1452      run_processes_parallel: preparing to run up to 1 tasks
> 21:35:50.460971 run-command.c:1484      run_processes_parallel: done
> 21:35:50.460971 run-command.c:626       trace: run_command: 'gc' '--auto'
> 21:35:50.480985 git.c:344               trace: built-in: git 'gc' '--auto'
> Rebasing: direct_goma
> Rebasing: python_pool
> 
> C:\src\chromium\src>git fetch
> 21:36:23.409703 git.c:344               trace: built-in: git 'fetch'
> 21:36:23.486758 run-command.c:626       trace: run_command: 'remote-https' 'origin' 'https://chromium.googlesource.com/chromium/src.git'
> 21:36:23.506772 git.c:576               trace: exec: 'git-remote-https' 'origin' 'https://chromium.googlesource.com/chromium/src.git'
> 21:36:23.506772 run-command.c:626       trace: run_command: 'git-remote-https' 'origin' 'https://chromium.googlesource.com/chromium/src.git'
> 21:36:28.542596 run-command.c:626       trace: run_command: 'rev-list' '--objects' '--stdin' '--not' '--all' '--quiet'
> 21:36:28.578620 run-command.c:626       trace: run_command: 'fetch-pack' '--stateless-rpc' '--stdin' '--lock-pack' '--include-tag' '--thin' 'https://chromium.googlesource.com/chromium/src.git/'
> 21:36:28.601635 git.c:344               trace: built-in: git 'fetch-pack' '--stateless-rpc' '--stdin' '--lock-pack' '--include-tag' '--thin' 'https://chromium.googlesource.com/chromium/src.git/'
> remote: Counting objects: 67598, done
> remote: Finding sources: 100% (28/28)
> 21:39:29.787441 run-command.c:626       trace: run_command: 'unpack-objects' '--pack_header=2,28'
> 21:39:29.812462 git.c:344               trace: built-in: git 'unpack-objects' '--pack_header=2,28'
> remote: Total 28 (delta 2), reused 20 (delta 2)
> Unpacking objects: 100% (28/28), done.
> 21:39:30.083690 run-command.c:626       trace: run_command: 'rev-list' '--objects' '--stdin' '--not' '--all' '--quiet'
> 21:39:30.108704 git.c:344               trace: built-in: git 'rev-list' '--objects' '--stdin' '--not' '--all' '--quiet'
> From https://chromium.googlesource.com/chromium/src
>    fd56d3ad0528..fbcc09282dc8  master     -> origin/master
> 21:39:30.788186 run-command.c:1452      run_processes_parallel: preparing to run up to 1 tasks
> 21:39:30.795191 run-command.c:1484      run_processes_parallel: done
> 21:39:30.813200 run-command.c:626       trace: run_command: 'gc' '--auto'
> 21:39:30.836212 git.c:344               trace: built-in: git 'gc' '--auto'
> 
> 
> Bug:749709
> Change-Id: I5b94549c42271885365c2354dc3f58399d6aebe0
> Reviewed-on: https://chromium-review.googlesource.com/786691
> Reviewed-by: Ryan Tseng <hinoka@chromium.org>
> Commit-Queue: Takuto Ikuta <tikuta@google.com>

TBR=hinoka@chromium.org,agable@chromium.org,tandrii@chromium.org,tikuta@google.com

Change-Id: I970ddac1fd21e904c69dc18e48169d49293b1800
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 749709
Reviewed-on: https://chromium-review.googlesource.com/786413
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
8 years ago
Takuto Ikuta 6e215cefe2 Show GIT_TRACE for git command in bot_update.py
GIT_TRACE=true tells us which internal git command is slow.
https://git-scm.com/book/en/v2/Git-Internals-Environment-Variables

I got following results in my local repository.
Let me confirm the bots show similar behavior.

C:\src\chromium\src>set GIT_TRACE=true

C:\src\chromium\src>git rebase-update
21:32:36.519698 git.c:576               trace: exec: 'git-rebase-update'
21:32:36.519698 run-command.c:626       trace: run_command: 'git-rebase-update'
21:32:38.889848 git.c:344               trace: built-in: git 'fetch' '--multiple' 'origin'
21:32:38.893846 run-command.c:626       trace: run_command: 'fetch' '--append' 'origin'
Fetching origin
21:32:38.911858 git.c:344               trace: built-in: git 'fetch' '--append' 'origin'
21:32:38.983909 run-command.c:626       trace: run_command: 'remote-https' 'origin' 'https://chromium.googlesource.com/chromium/src.git'
21:32:39.002923 git.c:576               trace: exec: 'git-remote-https' 'origin' 'https://chromium.googlesource.com/chromium/src.git'
21:32:39.003923 run-command.c:626       trace: run_command: 'git-remote-https' 'origin' 'https://chromium.googlesource.com/chromium/src.git'
21:32:43.782266 run-command.c:626       trace: run_command: 'rev-list' '--objects' '--stdin' '--not' '--all' '--quiet'
21:32:43.827301 run-command.c:626       trace: run_command: 'fetch-pack' '--stateless-rpc' '--stdin' '--lock-pack' '--include-tag' '--thin' 'https://chromium.googlesource.com/chromium/src.git/'
remote: Counting objects: 74400, done
remote: Finding sources: 100% (19/19)
21:35:49.348951 run-command.c:626       trace: run_command: 'unpack-objects' '--pack_header=2,19'
21:35:49.368985 git.c:344               trace: built-in: git 'unpack-objects' '--pack_header=2,19'
remote: Total 19 (delta 1), reused 18 (delta 1)
Unpacking objects: 100% (19/19), done.
21:35:49.597328 run-command.c:626       trace: run_command: 'rev-list' '--objects' '--stdin' '--not' '--all' '--quiet'
21:35:49.612375 git.c:344               trace: built-in: git 'rev-list' '--objects' '--stdin' '--not' '--all' '--quiet'
From https://chromium.googlesource.com/chromium/src
   e8994685157b..fd56d3ad0528  master     -> origin/master
21:35:50.281845 run-command.c:1452      run_processes_parallel: preparing to run up to 1 tasks
21:35:50.287848 run-command.c:1484      run_processes_parallel: done
21:35:50.309865 run-command.c:626       trace: run_command: 'gc' '--auto'
21:35:50.328880 git.c:344               trace: built-in: git 'gc' '--auto'
21:35:50.453967 run-command.c:1452      run_processes_parallel: preparing to run up to 1 tasks
21:35:50.460971 run-command.c:1484      run_processes_parallel: done
21:35:50.460971 run-command.c:626       trace: run_command: 'gc' '--auto'
21:35:50.480985 git.c:344               trace: built-in: git 'gc' '--auto'
Rebasing: direct_goma
Rebasing: python_pool

C:\src\chromium\src>git fetch
21:36:23.409703 git.c:344               trace: built-in: git 'fetch'
21:36:23.486758 run-command.c:626       trace: run_command: 'remote-https' 'origin' 'https://chromium.googlesource.com/chromium/src.git'
21:36:23.506772 git.c:576               trace: exec: 'git-remote-https' 'origin' 'https://chromium.googlesource.com/chromium/src.git'
21:36:23.506772 run-command.c:626       trace: run_command: 'git-remote-https' 'origin' 'https://chromium.googlesource.com/chromium/src.git'
21:36:28.542596 run-command.c:626       trace: run_command: 'rev-list' '--objects' '--stdin' '--not' '--all' '--quiet'
21:36:28.578620 run-command.c:626       trace: run_command: 'fetch-pack' '--stateless-rpc' '--stdin' '--lock-pack' '--include-tag' '--thin' 'https://chromium.googlesource.com/chromium/src.git/'
21:36:28.601635 git.c:344               trace: built-in: git 'fetch-pack' '--stateless-rpc' '--stdin' '--lock-pack' '--include-tag' '--thin' 'https://chromium.googlesource.com/chromium/src.git/'
remote: Counting objects: 67598, done
remote: Finding sources: 100% (28/28)
21:39:29.787441 run-command.c:626       trace: run_command: 'unpack-objects' '--pack_header=2,28'
21:39:29.812462 git.c:344               trace: built-in: git 'unpack-objects' '--pack_header=2,28'
remote: Total 28 (delta 2), reused 20 (delta 2)
Unpacking objects: 100% (28/28), done.
21:39:30.083690 run-command.c:626       trace: run_command: 'rev-list' '--objects' '--stdin' '--not' '--all' '--quiet'
21:39:30.108704 git.c:344               trace: built-in: git 'rev-list' '--objects' '--stdin' '--not' '--all' '--quiet'
From https://chromium.googlesource.com/chromium/src
   fd56d3ad0528..fbcc09282dc8  master     -> origin/master
21:39:30.788186 run-command.c:1452      run_processes_parallel: preparing to run up to 1 tasks
21:39:30.795191 run-command.c:1484      run_processes_parallel: done
21:39:30.813200 run-command.c:626       trace: run_command: 'gc' '--auto'
21:39:30.836212 git.c:344               trace: built-in: git 'gc' '--auto'


Bug:749709
Change-Id: I5b94549c42271885365c2354dc3f58399d6aebe0
Reviewed-on: https://chromium-review.googlesource.com/786691
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Commit-Queue: Takuto Ikuta <tikuta@google.com>
8 years ago
Andrii Shyshkalov ddfead3483 bot_update: mooore refactor around _git_checkout.
Bug:
Change-Id: I7141022348cdfd4dafbd2fa00b5981faa0ba79e0
Reviewed-on: https://chromium-review.googlesource.com/771930
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Jao-ke Chin-Lee <jchinlee@chromium.org>
8 years ago
Andrii Shyshkalov 75c0b4b715 bot_update: don't crash badly if there no json outpot.
To illustrate, with this CL it's clear what is broken:
  https://ci.chromium.org/swarming/task/39e5676c13f00710?server=chromium-swarm.appspot.com
while before you get a red herring:
  ncaught Exception: AttributeError("'NoneType' object has no attribute
  'get'",)...
  https://ci.chromium.org/swarming/task/39e55d46c0774610?server=chromium-swarm.appspot.com

R=iannucci@chromium.org

Bug: 786486
Change-Id: I9ed2cdb7261c2e22a5a68116b81f841ece06c5d2
Reviewed-on: https://chromium-review.googlesource.com/777556
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
8 years ago
Andrii Shyshkalov 8762632eb7 Reland "bot_update: make --output_json required in script invocation."
This is a reland of 331a276b54
Original change's description:
> bot_update: make --output_json required in script invocation.
> 
> It's already set by recipe, so this should have 0 difference in prod.
> 
> R=iannucci@chromium.org
> 
> Bug:
> Change-Id: I1e1129b9748d6647e45eb26892c8e892019fe088
> Reviewed-on: https://chromium-review.googlesource.com/769164
> Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>

Change-Id: I165da9650df6b5e8d382aafc30914eff9a524c56
Reviewed-on: https://chromium-review.googlesource.com/777820
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
8 years ago
Robert Flack fe36710a72 Revert "bot_update: make --output_json required in script invocation."
This reverts commit 331a276b54.

Reason for revert: Looks like this is causing bots to fail on the waterfall

Example:
https://logs.chromium.org/v/?s=chromium%2Fbb%2Fchromium%2FWin%2F60913%2F%2B%2Frecipes%2Fsteps%2Fbot_update%2F0%2Fstdout

Original change's description:
> bot_update: make --output_json required in script invocation.
> 
> It's already set by recipe, so this should have 0 difference in prod.
> 
> R=​iannucci@chromium.org
> 
> Bug:
> Change-Id: I1e1129b9748d6647e45eb26892c8e892019fe088
> Reviewed-on: https://chromium-review.googlesource.com/769164
> Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>

TBR=iannucci@chromium.org,tandrii@chromium.org

Change-Id: I65d7665e45dde539f6436ffe8932319e03f96f25
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/777619
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
8 years ago
Andrii Shyshkalov f151da9fac bot_update: refactor _git_checkout moving invariants out of the loop.
Bug:
Change-Id: Ie8781f73236e13c0afedf18dacf770d14563ab9f
Reviewed-on: https://chromium-review.googlesource.com/771857
Reviewed-by: Jao-ke Chin-Lee <jchinlee@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
8 years ago
Andrii Shyshkalov 331a276b54 bot_update: make --output_json required in script invocation.
It's already set by recipe, so this should have 0 difference in prod.

R=iannucci@chromium.org

Bug:
Change-Id: I1e1129b9748d6647e45eb26892c8e892019fe088
Reviewed-on: https://chromium-review.googlesource.com/769164
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
8 years ago
Andrii Shyshkalov f78690b687 bot_update: refactor hiding ugly revision details into lower layers.
Bug:
Change-Id: Ie1caef2d58dce12bafbbec29fa4a5cc65076626c
Reviewed-on: https://chromium-review.googlesource.com/771856
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Jao-ke Chin-Lee <jchinlee@chromium.org>
8 years ago
Andrii Shyshkalov f062cf21e4 bot_update: refactoring improving variable names.
Change-Id: I36c64bb46b967af00adcf69f2e867fd03204d7c2
Reviewed-on: https://chromium-review.googlesource.com/771910
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Jao-ke Chin-Lee <jchinlee@chromium.org>
8 years ago
Andrii Shyshkalov 1f620f0550 bot_update: refactor giant git_checkout into two.
Change-Id: I2c023a538bfe62955238b39034ed7460286789e5
Reviewed-on: https://chromium-review.googlesource.com/771454
Reviewed-by: Jao-ke Chin-Lee <jchinlee@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
8 years ago
Andrii Shyshkalov d7e241a22c bot_update: fix minor pylint issues.
Change-Id: I2f500bb8089039e689d5a989eb77f91374528164
Reviewed-on: https://chromium-review.googlesource.com/771381
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
8 years ago
Andrii Shyshkalov 26e338f40d Revert "Add luci_migration recipe module."
This reverts commit 2a1ca3f68b.

Reason for revert: to be replaced by recipe_engine runtime module
in https://chromium-review.googlesource.com/#/c/infra/luci/recipes-py/+/759677  

Original change's description:
> Add luci_migration recipe module.
> 
> R=​iannucci@chromium.org
> 
> Bug: 782460
> Change-Id: I022deb4d8e3cee8d48e1acccc78f8681490c72b9
> Reviewed-on: https://chromium-review.googlesource.com/757820
> Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
> Reviewed-by: Robbie Iannucci <iannucci@chromium.org>

TBR=iannucci@chromium.org,tandrii@chromium.org

Change-Id: Ieb874ad42b939edb66e35db3cfed0b8b8746e70e
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 782460
Reviewed-on: https://chromium-review.googlesource.com/759357
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
8 years ago
Andrii Shyshkalov 2a1ca3f68b Add luci_migration recipe module.
R=iannucci@chromium.org

Bug: 782460
Change-Id: I022deb4d8e3cee8d48e1acccc78f8681490c72b9
Reviewed-on: https://chromium-review.googlesource.com/757820
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
8 years ago
William Hesse 3b74d3333e Dart: Change url of Dart repository in gclient recipe module Dart config.
Bug:
Change-Id: I031f5787a87a4bddbd3492912814476daf2c9b4f
Reviewed-on: https://chromium-review.googlesource.com/738118
Commit-Queue: William Hesse <whesse@google.com>
Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
8 years ago
Ryan Tseng bb85b3636b bot_update source manifest: fix test_api.py to match the actual output
The test output was incorrect, this matches what it actually should look like.

Bug: 772529
Change-Id: I1ca9ceeb6d6b6f55b2ac7c90667c6a9904d2a43b

Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: skia
Recipe-Nontrivial-Roll: release_scripts
Change-Id: I1ca9ceeb6d6b6f55b2ac7c90667c6a9904d2a43b
Reviewed-on: https://chromium-review.googlesource.com/751694
Commit-Queue: Ryan Tseng <hinoka@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
8 years ago
Ryan Tseng 70dea4270e Reland 2 "bot_update recipe: Upload source manifest"
This relands commit 47b67c426b.
    
Also:
* Fix bot_update.py output of manifest
    
Bug: 772529,776299
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: skiabuildbot
Recipe-Nontrivial-Roll: release_scripts
Recipe-Nontrivial-Roll: skia
Recipe-Nontrivial-Roll: skiabuildbot
Recipe-Nontrivial-Roll: build

Change-Id: I31828ea5bfbc284623be8e553980f83d0889de92
Reviewed-on: https://chromium-review.googlesource.com/748308
Commit-Queue: Ryan Tseng <hinoka@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
8 years ago
Ryan Tseng 5d790d53ea Revert "Reland "bot_update recipe: Upload source manifest""
This reverts commit 47b67c426b.

Reason for revert: Bug in bot_update.py resource, breaks recipe roller.

Original change's description:
> Reland "bot_update recipe: Upload source manifest"
> 
> This reverts commit c3d1208d5c.
> 
> Also:
> * Instead of replacing "manifest", just add a new "source_manifest" 
> to the output JSON.  This allow transition without breakage.
> * Change the test api so test for recipe output changes.
> 
> The plan is to land this first, switch all downstream to "source_manifest",
> and then remove the original "manifest" key.
> 
> Bug: 772529,776299
> Change-Id: Iffb75f18046f8e4c058afe077872d4257b9dd754
> Recipe-Nontrivial-Roll: infra
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Recipe-Nontrivial-Roll: skiabuildbot
> Recipe-Nontrivial-Roll: release_scripts
> Recipe-Nontrivial-Roll: skia
> Recipe-Nontrivial-Roll: skiabuildbot
> Recipe-Nontrivial-Roll: build
> Reviewed-on: https://chromium-review.googlesource.com/731378
> Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> Commit-Queue: Ryan Tseng <hinoka@chromium.org>

TBR=iannucci@chromium.org,hinoka@chromium.org

Change-Id: I7a4ee904075e8b75b8a47f9ef0cd8a633af85a9c
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 772529, 776299
Reviewed-on: https://chromium-review.googlesource.com/748312
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Commit-Queue: Ryan Tseng <hinoka@chromium.org>
8 years ago
Ryan Tseng 47b67c426b Reland "bot_update recipe: Upload source manifest"
This reverts commit c3d1208d5c.

Also:
* Instead of replacing "manifest", just add a new "source_manifest" 
to the output JSON.  This allow transition without breakage.
* Change the test api so test for recipe output changes.

The plan is to land this first, switch all downstream to "source_manifest",
and then remove the original "manifest" key.

Bug: 772529,776299
Change-Id: Iffb75f18046f8e4c058afe077872d4257b9dd754
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: skiabuildbot
Recipe-Nontrivial-Roll: release_scripts
Recipe-Nontrivial-Roll: skia
Recipe-Nontrivial-Roll: skiabuildbot
Recipe-Nontrivial-Roll: build
Reviewed-on: https://chromium-review.googlesource.com/731378
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Ryan Tseng <hinoka@chromium.org>
8 years ago
Jao-ke Chin-Lee 3cf028ea0f [bot_update] Only kick in deadline after the first try.
If the revision has not yet propagated, then we never retry, because the amount of time to fetch HEAD already exceeds the deadline.

Bug: 771007
Change-Id: I91ac30b2b53b4f75729357b3298c52856a4d350e
Reviewed-on: https://chromium-review.googlesource.com/729328
Commit-Queue: Jao-ke Chin-Lee <jchinlee@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
8 years ago
Michael Achenbach c3d1208d5c Revert "bot_update recipe: Upload source manifest"
This reverts commit 037f9b80bb.

Reason for revert: The structural change to manifest breaks the recipe
tester:
https://cs.chromium.org/chromium/infra/recipes/recipes/recipe_roll_tryjob.py?q=recipe_roll_tryjob.py&sq=package:chromium&dr&l=149
https://crbug.com/776299

Original change's description:
> bot_update recipe: Upload source manifest
> 
> Bug: 772529
> Change-Id: I3f833740ddfbc71be42b140c99dcd4bbfeed54fe
> Reviewed-on: https://chromium-review.googlesource.com/706071
> Commit-Queue: Ryan Tseng <hinoka@chromium.org>
> Reviewed-by: Robbie Iannucci <iannucci@chromium.org>

TBR=iannucci@chromium.org,hinoka@chromium.org,agable@chromium.org

Change-Id: Iba9653d73b7f117ca2f94619ae5da900cd0c8c98
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 772529,776299
Reviewed-on: https://chromium-review.googlesource.com/728059
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
8 years ago
Ryan Tseng 037f9b80bb bot_update recipe: Upload source manifest
Bug: 772529
Change-Id: I3f833740ddfbc71be42b140c99dcd4bbfeed54fe
Reviewed-on: https://chromium-review.googlesource.com/706071
Commit-Queue: Ryan Tseng <hinoka@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
8 years ago
Michael Moss 7de54ef0a2 Fix handling of potential empty gitiles result.
This fixes an issue introduced by
crrev.com/b40a45149a73a6b00e92fd1d2df4c0b1543d0005 where a 404 result
might return None contents.

R=agable@google.com, phajdan@google.com

Bug:774355
Change-Id: I993a029adb8d92cb27e043cd3d9b17cc8aee2432
Reviewed-on: https://chromium-review.googlesource.com/717937
Commit-Queue: Michael Moss <mmoss@chromium.org>
Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
8 years ago
Michael Moss b40a45149a Allow specifying alternative "success" codes in gitiles requests.
This is needed in the release recipes because there are times when we
want to check that a file is _not_ there yet (i.e. 404 response).

R=agable@chromium.org, dnj@chromium.org

Change-Id: Iedf646d6c73ebf898e90afd2009840b1e5c5b1ab
Reviewed-on: https://chromium-review.googlesource.com/709916
Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Commit-Queue: Michael Moss <mmoss@chromium.org>
8 years ago
Andrii Shyshkalov 3f1ab05820 Fix gclient config of slave.DEPS and internal.DEPS.
R=smut@chromium.org

Recipe-Nontrivial-Roll: build
Change-Id: If90d6b172c3cd67ca57ad35345a9988e91da2345
Reviewed-on: https://chromium-review.googlesource.com/714776
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: smut <smut@google.com>
8 years ago
Vadim Shtayura 08049e2db2 Revert "git_cache: Remove locks"
This reverts commit c3eb3fa335.

Reason for revert: lots of "runhooks" failure everywhere

Example: https://build.chromium.org/p/chromium.linux/builders/Linux%20Builder/builds/92720

Original change's description:
> git_cache: Remove locks
> 
> These aren't in use, and the original problem they were
> meant to solve has been solved at the gclient.py layer 
> using resource locking:
>   https://codereview.chromium.org/2049583003
> 
> Bug: 773008
> Change-Id: I6609f39d7f15604e0bb3d742a41c4f9fec87a57a
> Reviewed-on: https://chromium-review.googlesource.com/707728
> Reviewed-by: Aaron Gable <agable@chromium.org>
> Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> Commit-Queue: Ryan Tseng <hinoka@chromium.org>

TBR=iannucci@chromium.org,hinoka@chromium.org,agable@chromium.org,phajdan.jr@chromium.org

Change-Id: I31d5fef94f39f3a9f97b9e59121073b1f433d11e
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 773008
Reviewed-on: https://chromium-review.googlesource.com/711054
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
8 years ago
Ryan Tseng c3eb3fa335 git_cache: Remove locks
These aren't in use, and the original problem they were
meant to solve has been solved at the gclient.py layer 
using resource locking:
  https://codereview.chromium.org/2049583003

Bug: 773008
Change-Id: I6609f39d7f15604e0bb3d742a41c4f9fec87a57a
Reviewed-on: https://chromium-review.googlesource.com/707728
Reviewed-by: Aaron Gable <agable@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Ryan Tseng <hinoka@chromium.org>
8 years ago
Michael Moss c634ba5705 Allow gclient solution URLs to be |None|.
This is supported by gclient and has the effect of treating the solution
as a purely local, standalone DEPS (i.e. not as a source repo to
checkout or process in any way). It also prevents the solution from
being added to the output of 'gclient flatten'.

R=agable@chromium.org, phajdan.jr@chromium.org

Change-Id: Ie11246ec7a74097b96ea6a26983a63b3fb3123a5
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build
Reviewed-on: https://chromium-review.googlesource.com/698271
Commit-Queue: Michael Moss <mmoss@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
8 years ago
Michael Moss 210ed0ebfe Expose new gclient.sync() arg in gclient.checkout().
R=phajdan.jr@chromium.org

Change-Id: I7313b8a26a5f2934e26491fc4b9403ba6cea0cee
Reviewed-on: https://chromium-review.googlesource.com/691017
Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Commit-Queue: Michael Moss <mmoss@chromium.org>
8 years ago