Commit Graph

154 Commits (8829c52f4cd923f36fde3ec2b4fbf3f87a74ce80)

Author SHA1 Message Date
primiano@chromium.org b9658c343c presubmit: unblacklist third_party/WebKit
The current DEFAULT_BLACKLIST in presubmit_support blacklists by
default third_party/.* in every repo. This requires an adjustement
after the blink merge, as it causing all the blink presubmits to
become noops.

BUG=539768

Review URL: https://codereview.chromium.org/1382993004

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@297030 0039d316-1c4b-4281-b951-d872f2087c98
10 years ago
iannucci@chromium.org d61a4950b2 Changes to improve multiprocessing PRESUBMIT support in Windows
* make RunTest's multiprocessing.Pool in the constructor of InputApi
    to avoid getting tripped up by chdir manipulation.
  * Don't do the split cyclic-import check when the invoker of the
    Pylint presubmit checks explicitly sends cyclic import check
    parameters via extra_args
  * fix pseudobug where ownership of the files variable was unclear,
    and pass all arguments on stdin (instead of mix of CLI + stdin).
  * fix bug in pylint which caused it to manipulate sys.path before
    spawning its subprocesses, which caused multiprocessing to fail
    on windows.
    * Note: This may carry a slight semantic change. Before, pylint would
      add all .py files' directories to sys.path while checking any of
      them. Now in parallel mode, pylint will only add the path of the
      single file to sys.path. This behavior actually mirrors Python's
      own behavior, so the check should be more-correct than before (and
      should cut down on pylint import scanning time with very large
      sys.path's).
    * If someone encounters an issue with this, please note that the
      GetPylint check also includes an extra_paths_list which is
      expressly for this purpose.

R=dpranke@chromium.org, kbr@chromium.org, maruel@chromium.org
BUG=501012

Review URL: https://codereview.chromium.org/1208743002

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@295908 0039d316-1c4b-4281-b951-d872f2087c98
10 years ago
iannucci@chromium.org 0af3bb37eb Parallelize pylint PRESUBMIT checks.
R=maruel@chromium.org
BUG=479837,499650

Review URL: https://codereview.chromium.org/1181103002

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@295664 0039d316-1c4b-4281-b951-d872f2087c98
10 years ago
vadimsh@chromium.org cf6a5d2026 Extract authentication options handling into a separate function.
It is done in preparation for switching to OAuth2 as default (and only)
authentication method. Having all auth options handled by the same code makes it
easier to gradually add OAuth2 support.

As part of this, some options that would no longer work with OAuth2 (and that
are not being used from anywhere now, as far as I can tell) are removed:
  * Passing account password for authentication via command line.
  * Overriding 'Host' header when making requests to Rietveld (won't work with
    SSL anyway).
  * --account_type option (seems to be ClientLogin specific).

R=maruel@chromium.org
BUG=356813

Review URL: https://codereview.chromium.org/1075723002

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@294746 0039d316-1c4b-4281-b951-d872f2087c98
10 years ago
sbc@chromium.org 013731e832 Handle KeyboardInterrupt consistently in python scripts
Handle KeyboardInterrupt gracefully rather the printing a
backtrace. Most users of these tools don't expect a
backtrace when then hit Ctrl-C.

Also, fix a few other inconsistencies found in the python
startup code of these different scripts:
- always call main function 'main' (rather than 'Main')
- always return 0 from main function
- if main takes args never include argv[0]

Review URL: https://codereview.chromium.org/955993006

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@294250 0039d316-1c4b-4281-b951-d872f2087c98
10 years ago
rmistry@google.com 5626a92eec Add ability to specify and run post upload hooks.
Motivation: There are a few things we want to do in Skia infrastructure that is not possible to do without this functionality.
Eg1: If there is a change made to Skia's markdown then automatically include a 'DOCS_PREVIEW_URL=https://skia.org?cl=1234' in the CL's description.
Eg2: Automatically add 'NOTRY=true' for changes impacting things that do not need trybot runs.
Eg3: Include CL specific links to skia's perf (https://perf.skia.org/) and correctness (https://gold.skia.org/) servers.

BUG=chromium:462208

Review URL: https://codereview.chromium.org/949273002

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@294242 0039d316-1c4b-4281-b951-d872f2087c98
10 years ago
thakis@chromium.org c6ef53a877 Revert of Added orphaned compiled Python file checks. (patchset #2 id:20001 of https://codereview.chromium.org/621533002/)
Reason for revert:
Makes `git cl upload` very slow in chrome checkouts.

Original issue's description:
> Added orphaned compiled Python file checks.
> 
> There have been a handful of cases where orphaned compiled Python files (.pyc
> files without an accompanying .py file) have been loaded preferentially by
> the presubmit scripts, causing weird problems. This adds a check and a mode for
> such files, either warning the user or deleting them.
> 
> BUG=None
> TEST=local
>   - Tested against local repository.
> 
> R=iannucci@chromium.org, szager@chromium.org
> 
> Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=292703

TBR=iannucci@chromium.org,szager@chromium.org,dnj@chromium.org
NOTREECHECKS=true
NOTRY=true
BUG=None

Review URL: https://codereview.chromium.org/701683002

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@292818 0039d316-1c4b-4281-b951-d872f2087c98
10 years ago
dnj@chromium.org e917ddc33d Added orphaned compiled Python file checks.
There have been a handful of cases where orphaned compiled Python files (.pyc
files without an accompanying .py file) have been loaded preferentially by
the presubmit scripts, causing weird problems. This adds a check and a mode for
such files, either warning the user or deleting them.

BUG=None
TEST=local
  - Tested against local repository.

R=iannucci@chromium.org, szager@chromium.org

Review URL: https://codereview.chromium.org/621533002

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@292703 0039d316-1c4b-4281-b951-d872f2087c98
10 years ago
pgervais@chromium.org bd0cace8c1 Added os to input_api
BUG=
R=maruel@chromium.org

Review URL: https://codereview.chromium.org/614413003

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@292291 0039d316-1c4b-4281-b951-d872f2087c98
11 years ago
agable@chromium.org 40a3d0b62b Recursively find all tests in a git repo.
This method is necessary to allow good code organization (i.e., unittests live
adjacent to the source files they test) in the new infra.git repository.

R=maruel@chromium.org

Review URL: https://codereview.chromium.org/281013002

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@270564 0039d316-1c4b-4281-b951-d872f2087c98
11 years ago
pgervais@chromium.org e57b09db3a Fixed wrong test
BUG=366554

Review URL: https://codereview.chromium.org/271563002

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@268647 0039d316-1c4b-4281-b951-d872f2087c98
11 years ago
agable@chromium.org ea84ef1828 Get presubmit_support to compute diffs against a given upstream.
R=hinoka@chromium.org
BUG=368720

Review URL: https://codereview.chromium.org/269483002

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@267309 0039d316-1c4b-4281-b951-d872f2087c98
11 years ago
agable@chromium.org 2da1ade7b4 Add --upstream flag to presubmit_support for Git.
R=hinoka@chromium.org
BUG=368720

Review URL: https://codereview.chromium.org/267513002

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@267254 0039d316-1c4b-4281-b951-d872f2087c98
11 years ago
pgervais@chromium.org 2f64f78a07 logged changed paths list
This should help debugging should presubmit failures on a CQ.

BUG=

Review URL: https://codereview.chromium.org/250383006

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@266063 0039d316-1c4b-4281-b951-d872f2087c98
11 years ago
djacques@chromium.org 7b654f5942 Added implicit empty password to 'presubmit_support.py'
BUG=363477
TEST=localtest

Review URL: https://codereview.chromium.org/238603002

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@263781 0039d316-1c4b-4281-b951-d872f2087c98
11 years ago
pgervais@chromium.org 92c300923c Added POST capability to oauth Rietveld
BUG=319446

Review URL: https://codereview.chromium.org/236093002

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@263750 0039d316-1c4b-4281-b951-d872f2087c98
11 years ago
isherman@chromium.org b5cded6951 Infer CL author and reviewer list from local state if the issue has not previously been uploaded.
R=agable@chromium.org
BUG=none

Review URL: https://codereview.chromium.org/195793021

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@259250 0039d316-1c4b-4281-b951-d872f2087c98
11 years ago
mdempsky@chromium.org d59e761ba8 Make missing PRESUBMIT.py warning consistent with others.
Review URL: https://codereview.chromium.org/188013003

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@255125 0039d316-1c4b-4281-b951-d872f2087c98
11 years ago
alokp@chromium.org f634964529 Fix presubmit checks when --trybot-json flag is specified.
Must change current working directory to the presubmit file's directory
to follow the contract of the presubmit scripts.

BUG=348691

Review URL: https://codereview.chromium.org/181063021

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@254618 0039d316-1c4b-4281-b951-d872f2087c98
11 years ago
machenbach@chromium.org 58a69cbe01 Support multiple try masters when sending tries to rietveld.
BUG=

Review URL: https://codereview.chromium.org/178223016

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@254321 0039d316-1c4b-4281-b951-d872f2087c98
11 years ago
maruel@chromium.org 1ce8e665e7 Do not tell people to ask me for help. I won't help them.
Write a small HowTo to help confused people by confusing checks.

R=iannucci@chromium.org
BUG=

Review URL: https://codereview.chromium.org/137653003

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@244717 0039d316-1c4b-4281-b951-d872f2087c98
11 years ago
stip@chromium.org f7d31f5296 Have presubmit.py accept an option to output trybot data as json.
BUG=278558

Review URL: https://codereview.chromium.org/112503005

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@242918 0039d316-1c4b-4281-b951-d872f2087c98
11 years ago
stip@chromium.org 5ca2762dea Allow PRESUBMIT.py files to have old and new-style trybot specifications (as long as each file is consistent).
BUG=278554

Review URL: https://codereview.chromium.org/74163002

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@241590 0039d316-1c4b-4281-b951-d872f2087c98
11 years ago
maruel@chromium.org ffeb2f36d7 Print test duration in verbose mode.
This is useful to diagnose slow tests. Make this multiprocessing aware. Stop
printing when the commands are added but use the proper message system instead.

R=iannucci@chromium.org
BUG=

Review URL: https://codereview.chromium.org/99453003

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@238388 0039d316-1c4b-4281-b951-d872f2087c98
11 years ago
stip@chromium.org 68e0419d2f Rework bot and test parsing to allow receipt of (bot, set(test)) specifications.
This is needed for a further CL to unify TS and CQ using PRESUBMIT.py.

BUG=278554

Review URL: https://codereview.chromium.org/54373011

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@232813 0039d316-1c4b-4281-b951-d872f2087c98
11 years ago
thakis@chromium.org c1c963566a Remove references to sconsbuild.
scons hasn't been used in a long time.

Also remove support for using make on Mac, it's no longer supported. All bots
that were using make on mac are using ninja these days.

BUG=305702

Review URL: https://codereview.chromium.org/26672003

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@227778 0039d316-1c4b-4281-b951-d872f2087c98
12 years ago
danakj@chromium.org 1827852dbe Update hack in presubmit check to reflect new chromium path.
src/base/hash_tables.h has moved to src/base/containers/hash_tables.h in r205241.

R=maruel@chromium.org

Review URL: https://chromiumcodereview.appspot.com/16787005

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@205662 0039d316-1c4b-4281-b951-d872f2087c98
12 years ago
nick@chromium.org ff526198c4 presubmit_support: Call git once per change, to get a diff for all files. Parse this result to generate per-file diffs, which go into a cache that's shared between the AffectedFile instances.
Greatly improves presubmit performance on Blink where there may be rule violations (my test case went from 50s to 5s).

BUG=236206
TEST=new test in presubmit_unittest.py; manual performance test on Mac and Windows after touching hundreds of files in webkit; testing included add/move/edit/delete on both binary and text files.

Review URL: https://chromiumcodereview.appspot.com/15898005

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@205275 0039d316-1c4b-4281-b951-d872f2087c98
12 years ago
ilevy@chromium.org 5678d33ca8 Change parallel test logic to run only with jobs>1
Regular map is preferred when parallization does
not speed presubmit tests.  It makes stacktraces more
understandable and facilitates debugging.  Only use
parallel pool when tests > 1.

R=maruel@chromium.org

Review URL: https://chromiumcodereview.appspot.com/15385002

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@200940 0039d316-1c4b-4281-b951-d872f2087c98
12 years ago
ilevy@chromium.org 9c60ab237a Fix RunTests to not stall on empty test list
Some PRESUBMIT files dynamically generate test list
and so may pass an empty list to RunTests.  There
appears to be a bug in multiprocessing.Pool that
causes a stall in map_async with an empty list.

TBR=maruel

Review URL: https://chromiumcodereview.appspot.com/14962011

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@200613 0039d316-1c4b-4281-b951-d872f2087c98
12 years ago
ilevy@chromium.org e7ceaadef1 Fix uninitialized variable error in presubmit_support
Remove 'out' variable reference.  If subprocess.communicate()
throws an exception, out cannot contain data.

BUG=236073
TBR=cmp

Review URL: https://chromiumcodereview.appspot.com/14386022

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@196904 0039d316-1c4b-4281-b951-d872f2087c98
12 years ago
iannucci@chromium.org 720fd7ae8d Allow presubmit_support to fetch issue data from Rietveld.
R=ilevy@chromium.org, maruel@chromium.org
BUG=

Review URL: https://chromiumcodereview.appspot.com/13966016

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@196023 0039d316-1c4b-4281-b951-d872f2087c98
12 years ago
ilevy@chromium.org bc11731a9e Add support for parallel presubmit unit testing.
Enable parallel tests on depot_tools.
On Z620 presubmit times: 3m -> 35s.

Review URL: https://codereview.chromium.org/14247012

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@195377 0039d316-1c4b-4281-b951-d872f2087c98
12 years ago
maruel@chromium.org c6f60e8906 Fix R= line handling when there is no value and improve presubmit TAG line regex
Make it more consistent across the tool. Using \s also includes \n, which
confuses the tool.

Add more tests.

R=iannucci@chromium.org
BUG=

Review URL: https://chromiumcodereview.appspot.com/14265007

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@195214 0039d316-1c4b-4281-b951-d872f2087c98
12 years ago
enne@chromium.org e72c5f519a Add cpplint to presubmit input_api
This will let clients that don't use the heavily filtered canned cpplint
check and want to behave more strictly (aka cc/) reuse any common
settings applied to the cpplint module.

R=maruel@chromium.org
BUG=none

Review URL: https://codereview.chromium.org/13866044

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@194264 0039d316-1c4b-4281-b951-d872f2087c98
12 years ago
wez@chromium.org a6d011e8dc Add PresubmitPromptOrNotify helper.
PresubmitPromptOrNotify prints the message and prompts the user only
if the presubmit is on upload, not commit.

This is useful to provide upload-time sanity checks without blocking
the CL from being landed via the commit-queue.


Review URL: https://chromiumcodereview.appspot.com/12676031

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@190711 0039d316-1c4b-4281-b951-d872f2087c98
12 years ago
iannucci@chromium.org 8a4a2bc611 Add skip_canned option to presubmit_support.
This will replace the hack in commit-queue/verification/presubmit_shim, and will
be used on the presubmit trybot.

R=maruel@chromium.org
BUG=

Review URL: https://codereview.chromium.org/12481002

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@186922 0039d316-1c4b-4281-b951-d872f2087c98
12 years ago
maruel@chromium.org 4bac4b5517 Create CachingRietveld to automatically cache results for presubmit checks.
Multiple presubmit checks may call the same function multiple times, so it's
worth caching the results to speed up the presubmit check run.

Convert presubmit_support, git-cl and gcl to use it.

R=dpranke@chromium.org
BUG=


Review URL: https://chromiumcodereview.appspot.com/11280143

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@169726 0039d316-1c4b-4281-b951-d872f2087c98
12 years ago
dpranke@chromium.org 17cc244cdb implement per-file OWNERS support
R=maruel@chromium.org
BUG=119394


Review URL: https://chromiumcodereview.appspot.com/11114005

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@162529 0039d316-1c4b-4281-b951-d872f2087c98
13 years ago
gavinp@chromium.org 656326d43d Permit google app engine to be installed in testing_support.
I recently needed to run presubmit tests in depot_tools, and I was helpfully told:

$ gcl presubmit mychange
... eliding stuff ...
Install google_appengine sdk in /scratch/scratch/chrome/depot_tools_git/testing_support or higher up

This was a lie. If you put google_appengine there, the same presubmit
tools that told you to install it there will fail you for numerous
style violations contained within appengine itself. This change makes
us ignore those by blacklisting google_appengine inside of
testing_support.

Review URL: https://chromiumcodereview.appspot.com/10829293

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@151210 0039d316-1c4b-4281-b951-d872f2087c98
13 years ago
maruel@chromium.org 4f6852cca4 Remove python 2.5 compatibility code.
R=cmp@chromium.org
BUG=
TEST=


Review URL: http://codereview.chromium.org/10165007

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@133265 0039d316-1c4b-4281-b951-d872f2087c98
13 years ago
maruel@chromium.org 3ecc8ea477 Adds saner behavior for git try -t foo to not fail in a inexplicable way.
Enforces that each builder and builder:test association must be in its own list item in PRESUBMIT.py/GetPreferredTrySlaves().

If a user run git try -t foo, trychange.py will now take the list of all the
slaves, will skip any :compile builder, and apply the test filter to them.

Currently, git try -t foo on a chromium checkout throws an exception because of
PRESUBMIT.py files containing test specification.

R=petermayo@chromium.org
BUG=
TEST=


Review URL: http://codereview.chromium.org/9664015

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@125975 0039d316-1c4b-4281-b951-d872f2087c98
13 years ago
estade@chromium.org ae7af92aba Add CSS as a source file type.
this forces license headers, no line-end whitespace, etc.

Also update the default license header regex to allow concluding */ on the final line.

Review URL: http://codereview.chromium.org/9109001

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@119448 0039d316-1c4b-4281-b951-d872f2087c98
13 years ago
maruel@chromium.org 9b31f1678b Fix running presubmit_support.py directly
R=cmp@chromium.org
BUG=
TEST=


Review URL: http://codereview.chromium.org/9234038

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@119264 0039d316-1c4b-4281-b951-d872f2087c98
13 years ago
maruel@chromium.org 3bbf29439c Cache diff to speed up when the diff is processed multiple times
R=dpranke@chromium.org
BUG=
TEST=


Review URL: http://codereview.chromium.org/9127026

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@117040 0039d316-1c4b-4281-b951-d872f2087c98
13 years ago
maruel@chromium.org d579fcf951 Enforces using cwd in all svn calls.
That helps weed out some issues faces with svn plus helped me figure out some
misuses.

Most of the commands have been implicitly depending on os.getcwd(). This change
makes it always consistent and clear when dependence on the current directory is
needed.

Remove default arguments to scm.SVN.GenerateDiff and a few other calls to be
sure the refactoring was done right.

R=dpranke@chromium.org
BUG=
TEST=make sure most commands aren't broke


Review URL: http://codereview.chromium.org/8771042

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@114262 0039d316-1c4b-4281-b951-d872f2087c98
13 years ago
maruel@chromium.org 428342a0a7 Standardize the sys.path fix up and fix a few pylint warnings.
Disable temporarily W0403, will be reenabled on the next CL

R=dpranke@chromium.org
BUG=
TEST=

Review URL: http://codereview.chromium.org/8508017

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@109435 0039d316-1c4b-4281-b951-d872f2087c98
14 years ago
maruel@chromium.org 7ccb4bb83e Blacklist .diff and .patch files
Otherwise, silly checks like EOL whitespace triggers constantly.

R=dpranke@chromium.org
BUG=
TEST=


Review URL: http://codereview.chromium.org/8494013

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@108899 0039d316-1c4b-4281-b951-d872f2087c98
14 years ago
maruel@chromium.org e085d81a36 Add assert to make sure the files argument type is valid.
This would have caught a bug in the CQ instead of silently failing.

R=dpranke@chromium.org
BUG=
TEST=


Review URL: http://codereview.chromium.org/8203002

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@104767 0039d316-1c4b-4281-b951-d872f2087c98
14 years ago
asvitkine@chromium.org 1516995e79 Support for |change| argument to |GetPreferredTrySlaves()|, try 2.
Needed to make it so changes containing only *.mm are only sent to Mac trybots by default.

BUG=none
TEST=none

Review URL: http://codereview.chromium.org/8059009

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@102930 0039d316-1c4b-4281-b951-d872f2087c98
14 years ago