Convert monkey patching to class inheritance for OptionParser usage in:
commit_queue.py, gclient.py and trychange.py.
Monkey patching confuses the hell out of pylint.
R=jochen@chromium.org
BUG=
Review URL: https://chromiumcodereview.appspot.com/19552004
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@213423 0039d316-1c4b-4281-b951-d872f2087c98
We can use this to evaluate the usefulness of making hooks run in parallel.
Review URL: https://chromiumcodereview.appspot.com/18851005
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@211446 0039d316-1c4b-4281-b951-d872f2087c98
Sometimes we wish to pull in a complicated dependency but want to
suppress or replace one or more of the hooks rules. Say for example
we want to use a different way of generating the projects, or
have a different set of landmine expectations.
Here we add a custom_hooks section mirroring custom_deps to allow us to
override sections we have identified in the DEPS file. To do so,
we add an optional name to the elements of the hooks list, and overwrite
those whose name matches.
Conventions between included DEPS and the .gclient as to the meanings of the
name are equivalent to the meaning of the customized deps, and so do not
benefit from further structure or definition.
BUG=None
TEST=local unit test
Review URL: https://chromiumcodereview.appspot.com/17742004
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@210573 0039d316-1c4b-4281-b951-d872f2087c98
This enables significant time savings, especially since file_list only exists to
enable file-specific hooks (which, AFAIK, nothing actually uses). On a z620
(linux) using the cached git repos, a first-time `gclient sync --nohooks` takes:
* (with) 131.06s user 14.10s system 117% cpu 2:03.89 total
* (without) 482.13s user 189.35s system 144% cpu 7:45.63 total
This change makes nohooks cause file_list to be None if we don't need to
accumulate it, and updates GitWrapper and SvnWrapper appropriately.
R=szager@chromium.org
BUG=
Review URL: https://chromiumcodereview.appspot.com/18541006
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@210026 0039d316-1c4b-4281-b951-d872f2087c98
Instead of cloning straight into place, clones are made to a global cache dir,
and then local (using --shared) clones are made from the cache to the final
resting place. This means the 'final' clones are full repos with no shenanigans,
meaning that branches, commits, etc. all work, which should allow the rest of
the gclient ecosystem to work without change as well.
The primary benefit is, of course, reduced network IO, and a much lower cost for
'clobber' operations (assuming we don't clobber the cache). It also means that
a given bot can have a greater number of checkouts, since the entire git history
will only be stored once per machine, instead of once per checkout.
R=dpranke@chromium.org, szager@chromium.org
BUG=
Review URL: https://chromiumcodereview.appspot.com/18328003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@210024 0039d316-1c4b-4281-b951-d872f2087c98
This will be used on the bots when calling apply_patch, which commits to git
repos.
R=dpranke@chromium.org,szager@chromium.org
BUG=
Review URL: https://chromiumcodereview.appspot.com/18006002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@209040 0039d316-1c4b-4281-b951-d872f2087c98
This will speed up main waterfall sync. We can
remove this hack tomorrow if svn load is still OK.
TBR=cmp
Review URL: https://chromiumcodereview.appspot.com/17468009
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@207277 0039d316-1c4b-4281-b951-d872f2087c98
SVN traps SIGINT and attempts to clean itself up, but this results in hangs
waiting for TCP. This patch does two things: daemonizes worker threads so they
are culled when the main thread dies (is ctrl-C'd) and keeps track of spawned
subprocesses to kill any remaining ones when the main program is ctrl-C'd.
A user ctrl-C'ing gclient has to manually terminate hung SVN processes, so this
introduces no extra data loss or hazard. stracing a hung SVN process shows that
it is indeed hanging on TCP reads after receiving a SIGINT, implying there is an
underlying but in the SVN binary.
Review URL: https://chromiumcodereview.appspot.com/14759006
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@198205 0039d316-1c4b-4281-b951-d872f2087c98
The 'RemoveDirectory()' function in gclient_utils is deprecated and
rmtree() should be used instead for consistency.
This patch modifies all clients in depot_tools to use rmtree() instead
and removes the RemoveDirectory function.
+ The SVNWrapperTestCase.testRevertNoDotSvn() mocking
expectation has been slightly changed. This was required
because the test invokes code that used to call
gclient_utils.RemoveDirectory() directly, while only
gclient_utils.rmtree() was mocked.
BUG=NONE
R=maruel@chromium.org, ilevy@chromium.org
TEST=manually run gclient_utils_test / gclient_smoketest / scm_unittest / gclient_scm_test
Review URL: https://chromiumcodereview.appspot.com/14134010
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@196133 0039d316-1c4b-4281-b951-d872f2087c98
Here's some background why we need this:
We discovered that google code defines the timestamp of a revision to be the time when a commit was started
rather than when it was finished (apache subversion takes the timestamp when the commit transaction is finished).
This can result in a situation where revision R(i-1) has a higher timestamp than Ri.
See bug: https://code.google.com/p/support/issues/detail?id=30419
When using 'gclient --transitive' we effectively do date-based checkouts.
If a parent has a dependency (without a ...@revision) and that dependency lives in the same repository as the parent does
we'd like to checkout the exact same revision as the parent (if we do a date-based checkout as we do now the google code
bug can result in a situation where we don't get the same revision).
Review URL: https://chromiumcodereview.appspot.com/13814012
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@194852 0039d316-1c4b-4281-b951-d872f2087c98
Some SVN-only trybots get errors when running gclient revert with this
change. I assume it's because the flag is not present for the gclient
revert command, and that the code checking if options.with_branch_heads
is set is common to both revert and sync.
BUG=none
TEST=reverted locally and gclient revert worked again.
Review URL: https://chromiumcodereview.appspot.com/12616006
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@186967 0039d316-1c4b-4281-b951-d872f2087c98
The "branch-heads" are refspecs created from upstream svn branches. This flag
is mostly needed for official builders, and adds about 1/2GB of data to the
checkout, so not currently enabling by default.
Review URL: https://codereview.chromium.org/12546017
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@186598 0039d316-1c4b-4281-b951-d872f2087c98
Support 'git grep <match> <branch>' type searches in gclient grep.
These searches had a slightly different output so needed more
tweeking to the regex. I cleaned it up a bit too.
R=maruel@chromium.org
BUG=
Review URL: https://chromiumcodereview.appspot.com/11316213
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@170039 0039d316-1c4b-4281-b951-d872f2087c98
- This will be used by navabi to have his gclient recurse
command print the current repo.
R=maruel@chromium.org
BUG=
Review URL: https://chromiumcodereview.appspot.com/11308115
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@169812 0039d316-1c4b-4281-b951-d872f2087c98
This allows specificying that a target OS list should override the base
list instead of adding to it. This is useful for iOS builds, where the
desired behavior is to pull only the iOS deps, not the union of the Mac
and iOS deps.
Review URL: https://chromiumcodereview.appspot.com/11363036
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@165671 0039d316-1c4b-4281-b951-d872f2087c98
This will let DEPS files specify a higher local bound for recursion.
Note there is an edge case where a dependency can be initially pulled
in with a lower recursion level and then gets ignored subsequently
when a lower level DEPS overrides it's inclusion level and then
includes the same dependency. I do not solve this case, as
I am intending to use this syntax for top level deps files.
BUG=155780
Review URL: https://chromiumcodereview.appspot.com/11146032
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@162464 0039d316-1c4b-4281-b951-d872f2087c98
- On high core, fast machines, jobs=8 is does not offer good
parallelization. Switch to use number of local cpus.
R=maruel@chromium.org
BUG=
Review URL: https://chromiumcodereview.appspot.com/11140019
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@162072 0039d316-1c4b-4281-b951-d872f2087c98
Let's say there's 2 solutions, solA and solB, both fetching the dependency depA
with different urls, solA has 'depA': 'http://foo/bar' and solB has 'depA':
From('depB', 'bar').
This case used to not work, because LateOverride() was not called in
verify_validity(), so it was not comparing the resolved urls but instead what is
textually specified in both DEPS files.
Also because LateOverride() was not called, verify_validity() was comparing the
original urls even if custom_deps is specified in the solution(s).
Finally, allow one to mismatch the other IFF one is None.
R=szager@chromium.org
BUG=
Review URL: https://chromiumcodereview.appspot.com/11088023
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@161228 0039d316-1c4b-4281-b951-d872f2087c98
This simplifies some of the git-submodule stuff. For example, deps2hooks.bat
can be changed from this:
@echo off
setlocal
set TMP_GCLIENT_FILE=.tmp.gclient
echo solutions=[{"name":".","url":None,"deps_file":".DEPS.git"}] > %TMP_GCLIENT_FILE%
FOR /F "delims=" %%a in ('call gclient hookinfo --gclientfile=%TMP_GCLIENT_FILE%') do set HOOKINFO=%%a
set HOOKINFO="call %HOOKINFO:;= & call%"
cd ..
cmd /c %HOOKINFO%
del /F src\%TMP_GCLIENT_FILE%
... to this:
@echo off
setlocal
call gclient runhooks --spec="solutions=[{'name':'src','url':None,'deps_file':'.DEPS.git'}]"
Review URL: https://chromiumcodereview.appspot.com/10835045
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@149223 0039d316-1c4b-4281-b951-d872f2087c98
Cygwin used to have a bug where the number of lock was very limited, causing it
to crash when --jobs was used with a value different than 1. This bug seems to
have been fixed a few months ago as I can't reproduce the anymore after
extensive usage.
R=cmp@chromium.org
BUG=
TEST=
Review URL: https://chromiumcodereview.appspot.com/10689194
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@146798 0039d316-1c4b-4281-b951-d872f2087c98
into stdout and creating the GitDiffFilterer for git repository
BUG=125894
TEST=
Review URL: http://codereview.chromium.org/10317002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@135324 0039d316-1c4b-4281-b951-d872f2087c98
This ability is useful for versions of Chromium (i.e. Android) which want to
cross-compile to another platform that has a fixed set of custom dependencies.
Review URL: http://codereview.chromium.org/10127004
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@134280 0039d316-1c4b-4281-b951-d872f2087c98
Previous attempts inside the buildbot config missed some places.
This just does it directly in gclient.
Review carefully, this is a depot_tools change.
BUG=None
TEST=None
R=maruel@chromium.org,dschuff@chromium.org
Review URL: http://codereview.chromium.org/10116033
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@132997 0039d316-1c4b-4281-b951-d872f2087c98
This permits running git grep over the dependencies. With --ignore, gclient
recurse continues even if nothing is found in a dependency. Without this flag,
git grep returns 1 when nothing is found, causing gclient to stop processing the
remaining dependencies.
TBR=nsylvain@chromium.org
BUG=
TEST=
Review URL: http://codereview.chromium.org/9600010
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@124914 0039d316-1c4b-4281-b951-d872f2087c98
Support --jobs in 'fetch' and 'recurse' commands.
BUG=115840
TEST=
Review URL: http://codereview.chromium.org/9560010
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@124657 0039d316-1c4b-4281-b951-d872f2087c98
The fetch command is simply an alias for:
gclient recurse -s git git fetch origin
Rework 'recurse' command to use a WorkQueue so that --jobs is supported.
BUG=115840
TEST=
Review URL: http://codereview.chromium.org/9463044
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@124334 0039d316-1c4b-4281-b951-d872f2087c98
An example use case is github over ssh at a pinned revision.
R=petermayo@chromium.org
BUG=115202
TEST=
Review URL: http://codereview.chromium.org/9443025
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@123325 0039d316-1c4b-4281-b951-d872f2087c98
xusydoc@chromium.org
iannucci@chromium.org
maruel@chromium.org
ilevy@chromium.org
petermayo@chromium.org
cmp@chromium.org
szager@chromium.org
digit@chromium.org
kustermann@google.com
jochen@chromium.org
kjellander@chromium.org
mmoss@chromium.org
bcwhite@chromium.org
sivachandra@chromium.org
stuartmorgan@chromium.org
dbeam@chromium.org
haitao.feng@intel.com
peter@chromium.org
bradnelson@google.com
szager@google.com
davidbarr@chromium.org