Commit Graph

422 Commits (main)

Author SHA1 Message Date
Gavin Mak 9f715f8017 Revert "Add AffectedFile.Extension() in presubmit_support"
This reverts commit d447f41294.

Reason for revert: "AttributeError: 'str' object has no attribute 'Extension'"

Original change's description:
> Add AffectedFile.Extension() in presubmit_support
>
> Add convenience method AffectedFile.Extension() for extracting file
> extension in a standard way. Add a test for this new functionality.
> Add a similar test for AffectedFile.UnixLocalPath().
>
> Bug: None
> Change-Id: If591e751fb2e4fb5355ad0b6f93f310667849d68
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6273341
> Commit-Queue: Anton Bershanskyi <bershanskyi@gmail.com>
> Reviewed-by: Dirk Pranke <dpranke@google.com>
> Reviewed-by: Gavin Mak <gavinmak@google.com>

Bug: None
Change-Id: Ic51b0942e3a3267194fb73af5e92c902b2098c4c
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6279837
Auto-Submit: Gavin Mak <gavinmak@google.com>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
1 week ago
Anton Bershanskyi d447f41294 Add AffectedFile.Extension() in presubmit_support
Add convenience method AffectedFile.Extension() for extracting file
extension in a standard way. Add a test for this new functionality.
Add a similar test for AffectedFile.UnixLocalPath().

Bug: None
Change-Id: If591e751fb2e4fb5355ad0b6f93f310667849d68
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6273341
Commit-Queue: Anton Bershanskyi <bershanskyi@gmail.com>
Reviewed-by: Dirk Pranke <dpranke@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
1 week ago
Dirk Pranke 7cffe56af4 Add UnixLocalPath*() in presubmit_support.
This adds AffectedFile.UnixLocalPath(), Change.UnixLocalPaths(), and
input_api.UnixLocalPaths() to the presubmit support APIs, so that we can
check paths in a platform-independent manner (i.e., not have to worry
about whether we're running on Windows and might need to handle
backslashes).

Change-Id: I1a424825384cb950ef2d4d3008c9c6e60028cf07
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6236404
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Dirk Pranke <dpranke@google.com>
3 weeks ago
Gavin Mak 77a800d729 Print branch name when running presubmits
This implements the proposed solution B in https://crbug.com/380877175#comment5
and will help distinguish presubmit results when working with stacked changes.

Bug: 380877175
Change-Id: I7f0807f7ce03109288976069c57c28488d6cadfd
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6196130
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
1 month ago
Gavin Mak a6ce588715 Use ProvidedDiffChange if diff is empty string
A ProvidedDiffChange should be able to use an diff, even if it is empty.

Bug: 372936485
Change-Id: If6bea0063e92c8942aed080e4c26b8d032b90faa
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5926311
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Scott Lee <ddoman@chromium.org>
4 months ago
Bruce Dawson 9157dd872e Remove extraneous Python version from presubmit warnings
When a presubmit warning is triggered this message is printed:

    There were Python 3 presubmit warnings.

The Python version information was useful when we were transitioning
our presubmits from Python 2 to Python 3, but that transition is long
finished, and now that information is just noise.

This change deletes the excess information.

Change-Id: I4e43e1c9daf36daaaa9430d76ad557c800cf962b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5907924
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Auto-Submit: Bruce Dawson <brucedawson@chromium.org>
5 months ago
Joanna Wang d43a3eb748 Add end_commit for calculating GitChange diffs.
Bug: b/365162784

Change-Id: I97b69b1ae5183bbdaa7d4de0b1f23e62f4e1d8d0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5853908
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
6 months ago
Gavin Mak 37a737c1bc Update presubmit feedback message for Cog
We shouldn't be suggesting people run unsupported commands on cog.

Change-Id: I0a15b825b32a3f9bc40c65e0b51be33f06de54ad
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5782965
Auto-Submit: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
7 months ago
Gavin Mak 7e1a7e0fda Show message every 30s for long running presubmits
Slow presubmits like CheckUnwantedDependencies can often take over 30s
to run. Print a "still running" message so users don't think something
hung.

Bug: b/358457858
Change-Id: I47f5589af43d8fb0f384e6f7c633e9d0addd568a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5782588
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Scott Lee <ddoman@chromium.org>
7 months ago
Gavin Mak 495e14b7b6 Add flush_cache flag to input_api.change.NewContents
NewContents always reads from cache, but it is sometimes useful flush
cache and use NewContents as a way to read the updatedcontents of a
file.

For example, some presubmit checks run formatters and check if any files
are modified. Many of these use `git diff` but that isn't available in
a non-git workspace, so checks will have to rely on reading contents
directly.

Bug: b/333744051
Change-Id: I3b3f4c88cc130607020ad599bbc15616d559725c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5740609
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
7 months ago
Gavin Mak 8ac3425344 Create scm.DIFF.GetAllFiles
For a ProvidedDiffChange, the AllFiles method naively returns all files
with rglob("*"). The returned list includes files in nested submodules.
This does not match the behavior of GitChange's AllFiles which uses
git ls-files to find all files.

Implement a new SCM that stops iterating recursively when it
sees a submodule from .gitmodules in the repo root.

Bug: b/323243527
Change-Id: I170d0f1bc4a838acea04779dee3df7fca0bce359
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5648616
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
8 months ago
Gavin Mak 47a841a204 Handle empty diff in _diffs_to_change_files
An valid empty diff can be passed into _diffs_to_change_files if the
modified file's contents is same as upstream. In this case,
_diffs_to_change_files gets an IndexError.

Bug: b/336555565
Change-Id: I848e6016a1e8089473ff8a72d2e0142fbabfda9f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5635166
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
8 months ago
Gavin Mak 1a61eb625d Provide useful submodule info for all Change classes
GitChange uses 'scm.GIT.ListSubmodules' to filter out files under repo
root that belong to a submodule. ProvidedDiffChange does not use this
so doesn't filter out submoduled files.

'scm.GIT.ListSubmodules' just reads '.gitmodules' at repo root and
can be run outside a git workspace. Use this to provide useful
submodule info for Change/ProvidedDiffChange.

Bug: b/333744051
Change-Id: Idaead11c69681e86276a29b0dc58090e7c4ec2c6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5466527
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
11 months ago
Gavin Mak 715d94600f Add --generate_diff flag to presubmit_support.py
This flag allows a diff to be generated based on an upstream commit
for provided files and for checks to be run with on that diff.

Bug: b/323243527
Change-Id: Ia3f4ec50b15eabb90d391b9bc795fc4b35a35a08
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5376402
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
12 months ago
Brian Ryner 1a92126938 Delete vpython, leaving only vpython3.
Reland of https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5187892

Bug: 1376538
Change-Id: I6947cf2a54ec2b45b2540ad0784fafe73f44c441
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5343135
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Eli Ribble <eliribble@google.com>
Commit-Queue: Brian Ryner <bryner@google.com>
12 months ago
Gavin Mak cac08dbf68 Implement UpstreamBranch in Change class
This method was moved to GitChange in https://crrev.com/c/5286636 but
should be implemented for all Change classes for CheckPatchFormatted.

Bug: 323243527
Change-Id: Iafdf8be2f402a93cfe7285cb5e7d0d1e01cca049
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5332873
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
1 year ago
Gavin Mak 00915b6874 Implement _ProvidedDiffCache.GetOldContents
Get old contents for a file using the provided diff.

Bug: b/323243527
Change-Id: Ib0e42cb1efc218002fae0b07c248b2806816b7c3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5304396
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
1 year ago
Gavin Mak 62dc9f462e Implement support for change info thru diff file
This will be used in cases where git workspace is unavailable.

Bug: b/323243527
Change-Id: I51056cfbac50f76a0de1b98cfec488d8f133e353
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5263021
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
1 year ago
Gavin Mak 4c331a1ee2 Reland "Reland "Refactor git functionality out of Change and _DiffCache""
This is a reland of commit 4d2864f3a1

The previous change initiated "self._diff_cache()" per file.

Original change's description:
> Reland "Refactor git functionality out of Change and _DiffCache"
>
> This is a reland of commit dd1a596c3e
>
> It is not a pure reland and adds support for caching submodules.
>
> Original change's description:
> > Refactor git functionality out of Change and _DiffCache
> >
> > A followup change will add support for change diff provided as user
> > input through stdin/file.
> >
> > Bug: b/323243527
> > Change-Id: I8d3420370e134859c61e35e23d76803227e4a506
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5254364
> > Reviewed-by: Joanna Wang <jojwang@chromium.org>
> > Commit-Queue: Gavin Mak <gavinmak@google.com>
> > Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
>
> Bug: b/323243527
> Change-Id: I74bbb179d4cbda7431101a2d707131fab2093029
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5280620
> Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
> Commit-Queue: Gavin Mak <gavinmak@google.com>

Bug: b/323243527
Change-Id: I4733b6870935d4200cb32255214c57cdb55e84f0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5286636
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
1 year ago
Arthur Sonzogni aa475e6a12 Revert "Reland "Refactor git functionality out of Change and _DiffCache""
This reverts commit 4d2864f3a1.

Reason for revert: This still doesn't scale well with large commit:
https://logs.chromium.org/logs/chromium/buildbucket/cr-buildbucket/8756578662173359185/+/u/presubmit/stdout?format=raw

This is now a different instruction called repeatedly needlessly:
```
git -c core.quotePath=false diff -p --no-color --no-prefix --no-ext-diff f0b151b0e519ceb3efae2ad8951d8a9ce86e4fbd...HEAD --no-renames;  cwd=D:\b\s\w\ir\cache\builder\win_presubmit\src
```
called for each affected files.

We probably want to cache the output for this too?

Original change's description:
> Reland "Refactor git functionality out of Change and _DiffCache"
>
> This is a reland of commit dd1a596c3e
>
> It is not a pure reland and adds support for caching submodules.
>
> Original change's description:
> > Refactor git functionality out of Change and _DiffCache
> >
> > A followup change will add support for change diff provided as user
> > input through stdin/file.
> >
> > Bug: b/323243527
> > Change-Id: I8d3420370e134859c61e35e23d76803227e4a506
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5254364
> > Reviewed-by: Joanna Wang <jojwang@chromium.org>
> > Commit-Queue: Gavin Mak <gavinmak@google.com>
> > Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
>
> Bug: b/323243527
> Change-Id: I74bbb179d4cbda7431101a2d707131fab2093029
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5280620
> Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
> Commit-Queue: Gavin Mak <gavinmak@google.com>

Bug: b/323243527, b/324293047
Change-Id: Iffde46facefa9d2e3a0d3a9ab5f8e753c53ea6d4
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5272317
Auto-Submit: Arthur Sonzogni <arthursonzogni@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
1 year ago
Gavin Mak 4d2864f3a1 Reland "Refactor git functionality out of Change and _DiffCache"
This is a reland of commit dd1a596c3e

It is not a pure reland and adds support for caching submodules.

Original change's description:
> Refactor git functionality out of Change and _DiffCache
>
> A followup change will add support for change diff provided as user
> input through stdin/file.
>
> Bug: b/323243527
> Change-Id: I8d3420370e134859c61e35e23d76803227e4a506
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5254364
> Reviewed-by: Joanna Wang <jojwang@chromium.org>
> Commit-Queue: Gavin Mak <gavinmak@google.com>
> Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>

Bug: b/323243527
Change-Id: I74bbb179d4cbda7431101a2d707131fab2093029
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5280620
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
1 year ago
Arthur Sonzogni b1d51e6113 Revert "Refactor git functionality out of Change and _DiffCache"
This reverts commit dd1a596c3e.

Reason for revert: This doesn't scale well with large commit:
https://issues.chromium.org/issues/324293047

Original change's description:
> Refactor git functionality out of Change and _DiffCache
>
> A followup change will add support for change diff provided as user
> input through stdin/file.
>
> Bug: b/323243527
> Change-Id: I8d3420370e134859c61e35e23d76803227e4a506
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5254364
> Reviewed-by: Joanna Wang <jojwang@chromium.org>
> Commit-Queue: Gavin Mak <gavinmak@google.com>
> Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>

Bug: b/323243527
Change-Id: Ic5179de6744a2ead629f512a5098f6e8c63d01c8
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5277255
Commit-Queue: Gavin Mak <gavinmak@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Auto-Submit: Arthur Sonzogni <arthursonzogni@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
1 year ago
Gavin Mak dd1a596c3e Refactor git functionality out of Change and _DiffCache
A followup change will add support for change diff provided as user
input through stdin/file.

Bug: b/323243527
Change-Id: I8d3420370e134859c61e35e23d76803227e4a506
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5254364
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
1 year ago
Aravind Vasudevan 1f441eb375 [fix_encoding] Remove fix_encoding
The fix_encoding module within depot_tools was included back in the python2[1] days to as a be all encoding fix boilerplate that is called across depot_tools scripts.

However, now that depot_tools officially deprecated support for py2 and support >= 3.8[2], the boilerplate is not needed anymore.

* `fix_win_codec()`[3] The 'cp65001' codec issue this fixes is fixed in python 3.3[4].
* `fix_default_encoding()`[5] python3 defaults to utf8.
* `fix_win_sys_argv()`[6] sys.srgv unicode issue is fixed in python3[7].
* `fix_win_console()`[8] Fixed[9].

[1] https://codereview.chromium.org/6721029
[2] https://crrev.com/371aa997c04791d21e222ed43a1a0d55b450dd53/README.md
[3] https://source.chromium.org/chromium/chromium/tools/depot_tools/+/main:fix_encoding.py;l=123-132;drc=cfa826c9845122d445dce4f51f556381865dbed3
[4] https://github.com/python/cpython/issues/57425#issuecomment-1093559969
[5] https://source.chromium.org/chromium/chromium/tools/depot_tools/+/main:fix_encoding.py;l=29-66;drc=cfa826c9845122d445dce4f51f556381865dbed3
[6] https://crsrc.org/d/fix_encoding.py;l=73-120;drc=cfa826c9845122d445dce4f51f556381865dbed3
[7] https://github.com/python/cpython/issues/46381#issuecomment-1093409968
[8] https://source.chromium.org/chromium/chromium/tools/depot_tools/+/main:fix_encoding.py;l=315-344;drc=cfa826c9845122d445dce4f51f556381865dbed3
[9] https://github.com/python/cpython/issues/45943#issuecomment-1093402603

Bug: 1501984
Change-Id: I1d512a4b1bfe14e680ac0aa08027849b999cc638
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5263016
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
1 year ago
Gavin Mak 120efcb475 Remove _MergeMasters
This was added in https://codereview.chromium.org/178223016 and used
for rietveld which is no longer used.

Change-Id: Idbfdb50a61dba884f5212c1de904f83324a449f7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5254600
Auto-Submit: Gavin Mak <gavinmak@google.com>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
1 year ago
Rupert Wiser af6eabff53 Revert "Delete vpython, leaving only vpython3."
This reverts commit db0dcbbe7c.

Reason for revert: `gclient sync` no longer works on my local checkout with this change due to the error `No such file or directory: 'vpython'.`.

Original change's description:
> Delete vpython, leaving only vpython3.
>
> Bug: 1376538
> Change-Id: Ia7795f75fd05db63f155c5feffcc43822fb45bb1
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5187892
> Reviewed-by: Mike Frysinger <vapier@chromium.org>
> Commit-Queue: Brian Ryner <bryner@google.com>
> Reviewed-by: George Engelbrecht <engeg@google.com>
> Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
> Reviewed-by: Chenlin Fan <fancl@chromium.org>

Bug: 1376538
Change-Id: I7c32eea1f8ae07644ab81a35213de5c68c64dfb6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5260042
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Andy Perelson <ajp@chromium.org>
Commit-Queue: Brian Ryner <bryner@google.com>
Reviewed-by: Brian Ryner <bryner@google.com>
1 year ago
Brian Ryner db0dcbbe7c Delete vpython, leaving only vpython3.
Bug: 1376538
Change-Id: Ia7795f75fd05db63f155c5feffcc43822fb45bb1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5187892
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Queue: Brian Ryner <bryner@google.com>
Reviewed-by: George Engelbrecht <engeg@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Chenlin Fan <fancl@chromium.org>
1 year ago
Joanna Wang c341d58921 Include submodules when looking for PRESUBMIT files.
Bug: 1475770
Change-Id: I3a4b10a8dc68db9f0507bb3cefa62d233df53b68
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5219691
Reviewed-by: Scott Lee <ddoman@chromium.org>
Auto-Submit: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Scott Lee <ddoman@chromium.org>
1 year ago
Joanna Wang 46cb7d0aca Add AffectedSubmodules() to presubmit change.
+ Change scm GetAllFiles back to returning all files
  + should be safe to do since it's only used by presubmit code
+  add ignore_submodules option to CaptureStatus
  + It's used by other parts of depot_tools code
+ Add AffectedSubmodules to return submodules only
+ AffectedFiles still only returns non-submodule files.

Bug: 1475770
Change-Id: I457ad96099fb20eff4cc1a4fc84f59e7ae707abe
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5216005
Auto-Submit: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
1 year ago
Joanna Wang c3daf77a40 Move ListSubmodules into InputApi
Change-Id: Ia37291aca59b429b59bdc6424a2f36394d558e3f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5214265
Auto-Submit: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
1 year ago
Joanna Wang 5400d9ef5a Add ListSubmodules helper for presubmits.
We need this for this presubmit otherwise we cannot remove git deps from DEPS: http://shortn/_P7Xkt33SRx
We will also be able to reuse this and replace: http://shortn/_WYBIUC9pa4

Bug: 1475770
Change-Id: I4cea689d130df77c344d82d4d8f9db02d42cd847
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5209098
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
1 year ago
Josip Sokcevic 31a99001bd Remove dead code
R=gavinmak@google.com

Change-Id: If27b290ebb575db5ec93b4abcb7f432bedd6f18a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5153638
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
1 year ago
Gavin Mak edba22d4eb Fix multiline comment formatting
Many incorrectly formatted comments exist from the switch to
4 space indent: https://crrev.com/c/4836379

Bug: 1514505
Change-Id: I6366f9da812919bd35b999f18fa8a49b7a66c09b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5153633
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
1 year ago
Josip Sokcevic 1bdc5c8db4 [presubmit] Don't replace os.environ
Assigning os.environ the result of os.environ.copy() replaces it
with a standard dict. This breaks reflection into the actual
environment, as well as features such as case-insensitivity on
Windows, leading to undefined standard library behavior.

R=bryner, gavinmak

Bug: 1511316
Change-Id: I63d026387104ae92669256567854f8bf9d2397a2
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5133887
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Brian Ryner <bryner@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
1 year ago
Liviu Rau 7f22330c69 Make rdb status calculation reusable
Bug:none
Change-Id: Ie7058e40d6e0174af382ef8f4ecfb62403683890
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5017798
Reviewed-by: Scott Lee <ddoman@chromium.org>
Commit-Queue: Liviu Rau <liviurau@google.com>
1 year ago
Gavin Mak f936d540e1 Remove __future__ imports
All __future__ imports (unicode_literals, print_function) are already
mandatory in py3. Also remove an outdated py2 comment in
presubmit_canned_checks.py

Bug: 1475402
Change-Id: I27cf6a8268f6dd1081f22af782c4c29a975376ae
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4867135
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
1 year ago
Mike Frysinger 124bb8e53c switch to 4 space indent
Leave the recipes/ code at 2 space to match the rest of the recipes
project in other repos.

Reformatted using:
files=( $(
	git ls-tree -r --name-only HEAD | \
		grep -Ev -e '^(third_party|recipes)/' | \
		grep '\.py$';
	git grep -l '#!/usr/bin/env.*python' | grep -v '\.py$'
) )
parallel ./yapf -i -- "${files[@]}"
~/chromiumos/chromite/contrib/reflow_overlong_comments "${files[@]}"

The files that still had strings that were too long were manually
reformatted because they were easy and only a few issues.
autoninja.py
clang_format.py
download_from_google_storage.py
fix_encoding.py
gclient_utils.py
git_cache.py
git_common.py
git_map_branches.py
git_reparent_branch.py
gn.py
my_activity.py
owners_finder.py
presubmit_canned_checks.py
reclient_helper.py
reclientreport.py
roll_dep.py
rustfmt.py
siso.py
split_cl.py
subcommand.py
subprocess2.py
swift_format.py
upload_to_google_storage.py

These files still had lines (strings) that were too long, so the pylint
warnings were suppressed with a TODO.
auth.py
gclient.py
gclient_eval.py
gclient_paths.py
gclient_scm.py
gerrit_util.py
git_cl.py
presubmit_canned_checks.py
presubmit_support.py
scm.py

Change-Id: Ia6535c4f2c48d46b589ec1e791dde6c6b2ea858f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4836379
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Auto-Submit: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Gavin Mak a94d8feecb Drop py2 support in presubmit files
python3 is the only supported version of python in depot_tools.

Bug: 1475402
Change-Id: I62ef9979a57bedc3be6886239068de5cd73f0504
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4814439
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2 years ago
Yiwei Zhang 422f4d59a4 config: use lucicfg validate to validate the config files
The new flow is that if any affected file is in the config directory
of a config set, the script will call `lucicfg validate` to validate
the whole config directory and then filter out the validation result
that does not belong to affected file(s). This will introduce a bit
more latency in valdiation because before, only affected files are
sent to LUCI Config for validation. However, this is not latency
senstively operation and always validating the full snapshot will
faciliate us breaking down the config as it's possible in order to
validate the affected file, LUCI Config will need an unaffected file.

R=iannucci, yuanjunh

Bug: 1449933
Change-Id: Id330973f59b882569a8ece36edb4b6a8ff0ed2d0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4591233
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Yuanjun Huang <yuanjunh@google.com>
2 years ago
Bruce Dawson 05ebc9e3e0 Allow calling tag functions from PostUploadHook
Calling CheckChangeHasNoUnwantedTags and
CheckChangeHasBugFieldFromChange from a PostUploadHook is useful because
it ensures that users will see a message if they add a bad tag when
creating the commit message during upload. Allowing this requires
creating entry points for these functions that take a change parameter
instead of an input_api parameter. It also requires fixing our use of
the deprecated inspect.getargspec function.

In order to not tell users twice (once before and once after upload)
that they haven't used a Bug: tag (it is optional, after all) a flag
to make these suggestions optional is also added.

This change is required in support of crrev.com/c/4490087.

Bug: 1434702
Change-Id: I3fd909a71c6c7d7671d6f986ad60d5375143a2f1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4519532
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
2 years ago
Bruce Dawson b6cb9e0b9a Remove Python 2 support for presubmit Commands
PRESUBMIT.py files can invoke other scripts, and those scripts may be
run under Python 2 or Python 3. There are multiple mechanisms that
govern which version will be used, sometimes requiring several flags
to be passed.

This change removes support for Python 2 from this system, thus making
it simpler to invoke Python 3 scripts.

The function parameters that are used to select Python versions are
passed in from multiple places so they still need to be supported, but
they are now ignored. The parameters are deleted to prevent accidental
use.

This change was tested by running this command in a Chromium repo:

  git cl presubmit --force --all

Bug: 1207012
Change-Id: I4adc7164417e155ff80d3a039cf36ed030756456
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4328470
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
2 years ago
Josip Sokcevic a4b871ebd9 Disable showing callstacks when running async jobs
When using async jobs, information about presubmit is not present in
callstack and it's just noise.

Change-Id: Ib95af92f9e83f1fb8b8d68c7aece6bbb4b03e61c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4505697
Auto-Submit: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2 years ago
Daniel Cheng 541638fd23 Fix the duplicate CC issue, hopefully once and for all.
Change-Id: Idd08d28ddeab8b66df1ae9a51d581ceaf62ec984
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4522489
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Daniel Cheng <dcheng@chromium.org>
2 years ago
Robert Iannucci 3d6d2d2500 [git-cl][presubmit] Remove references to LUCI_OMIT_PYTHON2.
This changes all references to the LUCI_OMIT_PYTHON2 environment
variable to instead assume that it is `'true'`.

R=sokcevic, tikuta

Bug: 1441784
Recipe-Nontrivial-Roll: build
Change-Id: Iea161a372adb68cdcb330d131df2c19ca2fe7b37
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4522480
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Josip Sokcevic 967cf672eb Revert "Reland "Remove Python 2 support for PRESUBMIT.py""
This reverts commit 119cff3d2f.

Reason for revert: some builders still call py2 presubmit
http://go/bbid/8781466817788761185

We need to clean up recipe first before relanding this

Original change's description:
> Reland "Remove Python 2 support for PRESUBMIT.py"
>
> This reverts commit d4c6cbeb61.
>
> Reason for revert: Python 2 invocations of the presubmit system
> should be gone now.
>
> Original change's description:
> > Revert "Remove Python 2 support for PRESUBMIT.py"
> >
> > This reverts commit 8454fc2458.
> >
> > Reason for revert: post submit hooks failing
> > https://crbug.com/1422416
> >
> > Original change's description:
> > > Remove Python 2 support for PRESUBMIT.py
> > >
> > > The presubmit system still supports invoking PRESUBMIT.py files using
> > > Python 2. This has recently been turned off on the bots so this change
> > > removes support more completely.
> > >
> > > There are still some python3 parameters being passed around - it seemed
> > > better to do the simplest possible removal now, with a follow-up change
> > > to remove more support code after this has sat for a while.
> > >
> > > Tests run from PRESUBMIT.py files could still be run using Python 2, but
> > > those should also have been addressed already. Removing support for that
> > > will be done in a subsequent change.
> > >
> > > Bug: 1207012
> > > Change-Id: Id244d547a04438f83734dba269c3cc180c148b37
> > > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4315183
> > > Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
> > > Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
> >
> > Bug: 1207012
> > Bug: 1422416
> > Change-Id: Iaf3102e63ec3c698d0258fac5746dbd92c30edbb
> > No-Presubmit: true
> > No-Tree-Checks: true
> > No-Try: true
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4317176
> > Auto-Submit: Josip Sokcevic <sokcevic@chromium.org>
> > Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
> > Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
>
> Bug: 1207012
> Bug: 1422416
> Change-Id: I9521095989c708188fca0251fa13881e086b1395
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4516015
> Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
> Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
> Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>

Bug: 1207012
Bug: 1422416
Change-Id: Ifa775ca19f8c7b3c166eb89a255c25c20aaf4a98
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4521578
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
2 years ago
Bruce Dawson 119cff3d2f Reland "Remove Python 2 support for PRESUBMIT.py"
This reverts commit d4c6cbeb61.

Reason for revert: Python 2 invocations of the presubmit system
should be gone now.

Original change's description:
> Revert "Remove Python 2 support for PRESUBMIT.py"
>
> This reverts commit 8454fc2458.
>
> Reason for revert: post submit hooks failing
> https://crbug.com/1422416
>
> Original change's description:
> > Remove Python 2 support for PRESUBMIT.py
> >
> > The presubmit system still supports invoking PRESUBMIT.py files using
> > Python 2. This has recently been turned off on the bots so this change
> > removes support more completely.
> >
> > There are still some python3 parameters being passed around - it seemed
> > better to do the simplest possible removal now, with a follow-up change
> > to remove more support code after this has sat for a while.
> >
> > Tests run from PRESUBMIT.py files could still be run using Python 2, but
> > those should also have been addressed already. Removing support for that
> > will be done in a subsequent change.
> >
> > Bug: 1207012
> > Change-Id: Id244d547a04438f83734dba269c3cc180c148b37
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4315183
> > Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
> > Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
>
> Bug: 1207012
> Bug: 1422416
> Change-Id: Iaf3102e63ec3c698d0258fac5746dbd92c30edbb
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4317176
> Auto-Submit: Josip Sokcevic <sokcevic@chromium.org>
> Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
> Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>

Bug: 1207012
Bug: 1422416
Change-Id: I9521095989c708188fca0251fa13881e086b1395
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4516015
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Josip Sokcevic a9a7eecf37 Revert "Extract searching relevat files in utils function"
This reverts commit 512fd3bd85.

Reason for revert: breaks some presubmits
https://crbug.com/1421441

Original change's description:
> Extract searching relevat files in utils function
>
> This change introduces lib/ directory which will be used for classes
> that are considered libraries and used by multiple executables.
>
> Change-Id: I3da126778bf5ffdb28d7a3c2b966c85de08ba717
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4289679
> Reviewed-by: Joanna Wang <jojwang@chromium.org>
> Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>

Bug: 1421441
Change-Id: I20e40f452a544a1188a258131a620e2afcd64f13
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4328468
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Josip Sokcevic 48d8e90bd4 Revert "Reland "Remove Python 2 support for PRESUBMIT.py""
This reverts commit e9ece0f581.

Reason for revert: broke infra presubmits:
https://ci.chromium.org/p/infra/builders/try/infra-try-presubmit

Original change's description:
> Reland "Remove Python 2 support for PRESUBMIT.py"
>
> This is a reland of commit 8454fc2458
>
> Original change's description:
> > Remove Python 2 support for PRESUBMIT.py
> >
> > The presubmit system still supports invoking PRESUBMIT.py files using
> > Python 2. This has recently been turned off on the bots so this change
> > removes support more completely.
> >
> > There are still some python3 parameters being passed around - it seemed
> > better to do the simplest possible removal now, with a follow-up change
> > to remove more support code after this has sat for a while.
> >
> > Tests run from PRESUBMIT.py files could still be run using Python 2, but
> > those should also have been addressed already. Removing support for that
> > will be done in a subsequent change.
> >
> > Bug: 1207012
> > Change-Id: Id244d547a04438f83734dba269c3cc180c148b37
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4315183
> > Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
> > Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
>
> Bug: 1207012
> Change-Id: If542cac21d8ec8704b28d03fd8407c5c2899ca2c
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4317177
> Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
> Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
> Auto-Submit: Josip Sokcevic <sokcevic@chromium.org>

Bug: 1207012
Change-Id: Iebf76d9e2580761fc773791bac07439160503c98
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4323198
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Auto-Submit: Josip Sokcevic <sokcevic@chromium.org>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
2 years ago
Josip Sokcevic e9ece0f581 Reland "Remove Python 2 support for PRESUBMIT.py"
This is a reland of commit 8454fc2458

Original change's description:
> Remove Python 2 support for PRESUBMIT.py
>
> The presubmit system still supports invoking PRESUBMIT.py files using
> Python 2. This has recently been turned off on the bots so this change
> removes support more completely.
>
> There are still some python3 parameters being passed around - it seemed
> better to do the simplest possible removal now, with a follow-up change
> to remove more support code after this has sat for a while.
>
> Tests run from PRESUBMIT.py files could still be run using Python 2, but
> those should also have been addressed already. Removing support for that
> will be done in a subsequent change.
>
> Bug: 1207012
> Change-Id: Id244d547a04438f83734dba269c3cc180c148b37
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4315183
> Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
> Commit-Queue: Bruce Dawson <brucedawson@chromium.org>

Bug: 1207012
Change-Id: If542cac21d8ec8704b28d03fd8407c5c2899ca2c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4317177
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Auto-Submit: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Josip Sokcevic d4c6cbeb61 Revert "Remove Python 2 support for PRESUBMIT.py"
This reverts commit 8454fc2458.

Reason for revert: post submit hooks failing
https://crbug.com/1422416

Original change's description:
> Remove Python 2 support for PRESUBMIT.py
>
> The presubmit system still supports invoking PRESUBMIT.py files using
> Python 2. This has recently been turned off on the bots so this change
> removes support more completely.
>
> There are still some python3 parameters being passed around - it seemed
> better to do the simplest possible removal now, with a follow-up change
> to remove more support code after this has sat for a while.
>
> Tests run from PRESUBMIT.py files could still be run using Python 2, but
> those should also have been addressed already. Removing support for that
> will be done in a subsequent change.
>
> Bug: 1207012
> Change-Id: Id244d547a04438f83734dba269c3cc180c148b37
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4315183
> Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
> Commit-Queue: Bruce Dawson <brucedawson@chromium.org>

Bug: 1207012
Bug: 1422416
Change-Id: Iaf3102e63ec3c698d0258fac5746dbd92c30edbb
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4317176
Auto-Submit: Josip Sokcevic <sokcevic@chromium.org>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
2 years ago