Commit Graph

1841 Commits (7592f9326fc96770ea5615f16854c23087ac6a49)

Author SHA1 Message Date
Ben Pastene d410c664e6 gclient: Preserve ANSI color codes when calling hooks.
If a hook prints error/warning output that's color-coded, gclient
will cause the coloring to be disabled since those hooks aren't
called directly from a terminal.

By emulating a terminal when launching subprocs from gclient, we can
convince them to keep the color escape codes.

LED builds with both //third_party/depot_tools rolled to this CL, as
well as depot_tools in the recipe bundle rolled to this CL:
linux: https://ci.chromium.org/swarming/task/4e40237985888310
mac: https://ci.chromium.org/swarming/task/4e4023ea0c829710
win: https://ci.chromium.org/swarming/task/4e4024612e03dc10

Bug: 1034063
Change-Id: I4150f66ef215ece06f4c32482dcd4ded14eb1bfe
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2368435
Reviewed-by: Dirk Pranke <dpranke@google.com>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
5 years ago
Saagar Sanghavi 531d992f00 Changed naming convention
presubmit:gerrit_host/folder/to/repo:path/to/file/:CheckName
Example
presubmit:chromium-review.googlesource.com/chromium/src/:services/viz/

Bug: 1112667
Change-Id: I095a2c04e73c64d67fa1bb3dbf7e6dfd1d93fe26
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2335873
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Saagar Sanghavi <saagarsanghavi@google.com>
5 years ago
Saagar Sanghavi 03b15131d3 [Backward Compatible] Timing and RDB for Lower-level checks
Allows presubmit_support to perform Timing and RDB reporting for each of
the lower-level "CheckXYZ" functions, provided that
ENABLE_NEW_PRESUBMIT_CHECKS is called in the PRESUBMIT.py
file that we are running. I am using the variable to avoid the breakage
caused by the similar previous CL (2332321) which was not backwards
compatible.
In addition, asks for "-realm" with ResultDB so it can maintain the
security boundary.

Bug: 1106943
Change-Id: Ia49e8884e2653bee7de7ed7d254452d55dd5c617
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2341828
Commit-Queue: Saagar Sanghavi <saagarsanghavi@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Ben Pastene 8351dc1fb7 Revert "[Fixed from older branch] Timing/RDB for lower-level checks"
This reverts commit dba1d65a19.

Reason for revert: "Check* takes exactly 3 arguments (2 given)" presubmit errors
eg: https://ci.chromium.org/p/chromium/builders/try/chromium_presubmit/933062?
https://crbug.com/1113506

Original change's description:
> [Fixed from older branch] Timing/RDB for lower-level checks
> 
> presubmit: Report timing to rdb for individual checks
> Rather than using CheckChangeOnCommit() or CheckChangeOnUpload() as the
> sole entry points for PRESUBMIT.py files, this cl enables presubmit_support.py
> to use any CheckXYZ[OnCommit/Upload] function to serve as an entry point.
> 
> This way, we can perform timing and RDB reporting for each of the
> presubmit check functions and have check-specific data.
> 
> Bug: 1106943
> Change-Id: Ifdabd68c0904a6d70a828f12de157369c6c1571d
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2332321
> Commit-Queue: Saagar Sanghavi <saagarsanghavi@google.com>
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Dirk Pranke <dpranke@google.com>

TBR=dpranke@google.com,estaab@chromium.org,ehmaldonado@chromium.org,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,saagarsanghavi@google.com

Change-Id: Ibd80f70661d2196f903f0c6b79eb74eda6c94d44
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 1106943, 1113506
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2340692
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
5 years ago
Saagar Sanghavi dba1d65a19 [Fixed from older branch] Timing/RDB for lower-level checks
presubmit: Report timing to rdb for individual checks
Rather than using CheckChangeOnCommit() or CheckChangeOnUpload() as the
sole entry points for PRESUBMIT.py files, this cl enables presubmit_support.py
to use any CheckXYZ[OnCommit/Upload] function to serve as an entry point.

This way, we can perform timing and RDB reporting for each of the
presubmit check functions and have check-specific data.

Bug: 1106943
Change-Id: Ifdabd68c0904a6d70a828f12de157369c6c1571d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2332321
Commit-Queue: Saagar Sanghavi <saagarsanghavi@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@google.com>
5 years ago
Lei Zhang 8a0efc1862 Handle BUG_PREFIX correctly when populating Bug: lines.
Right now git_cl.py inserts an extra colon if the bug prefix ends with
a colon. Fix this by detecting whether the bug prefix ends with a colon
or not, and only inserting the colon when necessary.

Bug: 1111955
Change-Id: I3db199435087abb17b9600fc208b11d2d8e6a615
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2333426
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Lei Zhang <thestig@chromium.org>
5 years ago
Saagar Sanghavi 98b332f2db Fixed relative path to work when presubmit run from any directory
Bug: 1111829
Change-Id: I75c616f4fa9c16c19e732e43ba740bb67f3eb827
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2327839
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Saagar Sanghavi <saagarsanghavi@google.com>
5 years ago
Edward Lesmes a58287b8ee presubmit: Also validate metadata in OWNERS files.
Bug: 1102997
Change-Id: Id847129d320004fcd121941f2e14cf9ad56ab4a7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2318335
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Saagar Sanghavi 9949ab7a4b Preliminary changes to git cl and presubmit_support
Added an rdb wrapper and use it to time CheckChangeOn{Commit, Upload} from every PRESUBMIT.py file, then reports the results to ResultSink.

Bug: http://crbug.com/1107588

Change-Id: I9bc8a18ba615a76f3e0611eae5b2c4adc5924276
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2242097
Commit-Queue: Saagar Sanghavi <saagarsanghavi@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Garrett Beaty <gbeaty@chromium.org>
5 years ago
Corentin Wallez 801c20219b gclient: deprecated use_relative_hooks
And merge it in use_relative_paths. In all Chromium repos,
use_relative_hooks is True iff:

 - use_relative_paths is True, and
 - there is at least one hook

It makes sense that you'd want to use relative hooks if you use relative
paths so this CL merges both flags into use_relative_paths.

Bug: chromium:1107325
Change-Id: I0fe40eba1e1c61be26c812c7ca4329efb72c7f90
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2306795
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Auto-Submit: Corentin Wallez <cwallez@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lesmes 6d938a0d85 presubmit_canned_checks: Add DIR_METADATA presubmit checks.
- Add a check to validate DIR_METADATA files.
- Add a check to enforce that OWNERS files contain no metadata if a
  DIR_METADATA file is present in the same directory.

Bug: 1102997
Change-Id: Iedb09941001577dc81b1dda2241481ff667b2568
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2298330
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
5 years ago
Corentin Wallez 271a78ac35 gclient: Make gclient_gn_args_file honor use_relative_paths
Without this repos that use the use_relative_paths still need to use
their absolute path for gclient_gn_args_file.

Bug: chromium:1102833
Change-Id: I470096625061a28abf495f4d1035121edbcd581f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2279874
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Corentin Wallez <cwallez@chromium.org>
5 years ago
Ayu Ishii 0be4724e2f CodeInclusion: Rename whitelist/blacklist -> allowlist/blocklist
Bug: 1097674
Change-Id: Ib1a4a3fe3eca37a1a3a90d62c19eeda5fe4bc499
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2293134
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Ayu Ishii <ayui@chromium.org>
5 years ago
local_bot 6402141d1f Rename blocklist/allowlist to skip_list/check_list
This change is backwards compatible.

R=dpranke@google.com

Bug: 1098560
Change-Id: Ic9447041a68b8c379f65900af11c89b006e3f806
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2285156
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Dirk Pranke <dpranke@google.com>
5 years ago
Dirk Pranke fdd2cd6e5f Reland "Add a Str() function to gclient for use in DEPS files."
This relands c7eed83 with a fix to the way variables are
propagated from parent dependencies into child dependencies.

The original CL description from c7eed83 was:
> gclient's existing functionality for handling variables is
> ambiguous: the value of a variable can either be a string literal
> or an expression fragment. The implementation is required to
> parse a value as an expression, and, if it is legal, treat it
> as an expression instead of a literal. This means that
>
>   gclient_gn_args_file = 'src/build/args.gni'
>   gclient_gn_args = ['xcode_version']
>   vars = {
>     'xcode_version': 'xcode-12'
>   }
>
> would cause a problem because gclient would try to parse the
> variable as an expression, and 'xcode' would not be defined.
>
> This patch adds a workaround for this, where you can instead
> use the Str() function to explicitly tell gclient to treat the
> value as a string and not a potential expression.
>
> The above example would be changed to:
>
>   gclient_gn_args_file = 'src/build/args.gni'
>   gclient_gn_args = ['xcode_version']
>   vars = {
>     'xcode_version': Str('xcode-12')
>   }
>
> The variable may still be used in every context where it was legal
> to be used before.
>
This reverts commit 84431987dd384c79c84515004d19db67345a1c00.

Bug: 1099242
TBR=ehmaldonado@chromium.org

Change-Id: I047b871df47c367c1f34a3985e5813504e3c5c6f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2274152
Commit-Queue: Dirk Pranke <dpranke@google.com>
Reviewed-by: Ben Pastene <bpastene@chromium.org>
5 years ago
Dirk Pranke ac93e6d567 Revert "Add a Str() function to gclient for use in DEPS files."
This reverts commit c7eed83f96.

Reason for revert: I'm getting reports of internal iOS checkouts being broken. Reverting while I reproduce / debug it.

Original change's description:
> Add a Str() function to gclient for use in DEPS files.
> 
> gclient's existing functionality for handling variables is
> ambiguous: the value of a variable can either be a string literal
> or an expression fragment. The implementation is required to
> parse a value as an expression, and, if it is legal, treat it
> as an expression instead of a literal. This means that
> 
>   gclient_gn_args_file = 'src/build/args.gni'
>   gclient_gn_args = ['xcode_version']
>   vars = {
>     'xcode_version': 'xcode-12'
>   }
> 
> would cause a problem because gclient would try to parse the
> variable as an expression, and 'xcode' would not be defined.
> 
> This patch adds a workaround for this, where you can instead
> use the Str() function to explicitly tell gclient to treat the
> value as a string and not a potential expression.
> 
> The above example would be changed to:
> 
>   gclient_gn_args_file = 'src/build/args.gni'
>   gclient_gn_args = ['xcode_version']
>   vars = {
>     'xcode_version': Str('xcode-12')
>   }
> 
> The variable may still be used in every context where it was legal
> to be used before.
> 
> Bug: 1099242
> 
> Change-Id: Ic2a17eea5f7098113bdba0557fe29e1a931a74b8
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2268406
> Reviewed-by: Ben Pastene <bpastene@chromium.org>
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
> Commit-Queue: Dirk Pranke <dpranke@google.com>

TBR=thakis@chromium.org,dpranke@google.com,ehmaldonado@chromium.org,bpastene@chromium.org,apolito@google.com,infra-scoped@luci-project-accounts.iam.gserviceaccount.com

Change-Id: Iac2b003f32acdbca15a19f821b61423e34b3466c
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 1099242
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2273978
Reviewed-by: Dirk Pranke <dpranke@google.com>
Commit-Queue: Dirk Pranke <dpranke@google.com>
5 years ago
Dirk Pranke c7eed83f96 Add a Str() function to gclient for use in DEPS files.
gclient's existing functionality for handling variables is
ambiguous: the value of a variable can either be a string literal
or an expression fragment. The implementation is required to
parse a value as an expression, and, if it is legal, treat it
as an expression instead of a literal. This means that

  gclient_gn_args_file = 'src/build/args.gni'
  gclient_gn_args = ['xcode_version']
  vars = {
    'xcode_version': 'xcode-12'
  }

would cause a problem because gclient would try to parse the
variable as an expression, and 'xcode' would not be defined.

This patch adds a workaround for this, where you can instead
use the Str() function to explicitly tell gclient to treat the
value as a string and not a potential expression.

The above example would be changed to:

  gclient_gn_args_file = 'src/build/args.gni'
  gclient_gn_args = ['xcode_version']
  vars = {
    'xcode_version': Str('xcode-12')
  }

The variable may still be used in every context where it was legal
to be used before.

Bug: 1099242

Change-Id: Ic2a17eea5f7098113bdba0557fe29e1a931a74b8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2268406
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Dirk Pranke <dpranke@google.com>
5 years ago
local_bot 37ce201d92 Add setters to legacy input_api.DEFAULT_*
Some PRESUBMIT.py scripts set DEFAULT_WHITE_LIST and DEFAULT_BLACK_LIST.
This patch adds support to override those defaults during transition
period.

R=apolito@google.com, ehmaldonado@chromium.org

Bug: 1098562
Change-Id: I5dfff253e61485d354a83f3f509d8a6d4d87dec6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2270717
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
5 years ago
local_bot 30f774ef87 Rename whitelist/blacklist to allowlist/blocklist
R=apolito@google.com, ehmaldonado@chromium.org

Bug: 1098562
Bug: 1098560
Change-Id: I45dc739f9e6d1a2e9bfcd9fa0587f4bb4a115c5f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2265255
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
5 years ago
Leandro Lovisolo 44de5e3f65 presubmit_canned_checks.py: Ignore long TypeScript imports.
Change-Id: If9e133002e2cdcff42917a487a677f3ed73d4d3a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2241184
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Leandro Lovisolo <lovisolo@google.com>
5 years ago
Josip Sokcevic 14a83aec56 Reland "Use OS level locking in git_cache.py"
This is a reland of d3affaa624

Original change's description:
> Use OS level locking in git_cache.py
> 
> Without OS level locking it's possible to leave "lock" files on disk
> which will prevent next run to acquire those locks. This can easily
> happen if SIGKIL is issued.
> 
> R=apolito@google.com, ehmaldonado@chromium.org
> 
> Bug: 1049610
> Change-Id: Id87aa1376b9ea5ff0c2d14f3603636493ed1dd5b
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2189333
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> Commit-Queue: Josip Sokcevic <sokcevic@google.com>

Bug: 1049610
Change-Id: I58e65a10f7c779e0de1121ba7167c694996e390c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2211189
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
5 years ago
Josip Sokcevic 8f6bfe301f Revert "Use OS level locking in git_cache.py"
This reverts commit d3affaa624.

Reason for revert: no attribute ignore_lock

Original change's description:
> Use OS level locking in git_cache.py
> 
> Without OS level locking it's possible to leave "lock" files on disk
> which will prevent next run to acquire those locks. This can easily
> happen if SIGKIL is issued.
> 
> R=​apolito@google.com, ehmaldonado@chromium.org
> 
> Bug: 1049610
> Change-Id: Id87aa1376b9ea5ff0c2d14f3603636493ed1dd5b
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2189333
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> Commit-Queue: Josip Sokcevic <sokcevic@google.com>

TBR=iannucci@chromium.org,ehmaldonado@chromium.org,apolito@google.com,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com

Change-Id: Iecc963e0a99d7f59f3f8801e529839346f9fbaf3
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 1049610
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2211186
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
5 years ago
Josip Sokcevic d3affaa624 Use OS level locking in git_cache.py
Without OS level locking it's possible to leave "lock" files on disk
which will prevent next run to acquire those locks. This can easily
happen if SIGKIL is issued.

R=apolito@google.com, ehmaldonado@chromium.org

Bug: 1049610
Change-Id: Id87aa1376b9ea5ff0c2d14f3603636493ed1dd5b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2189333
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
5 years ago
Yuly Novikov 64dab2dbab Print the SHA of files downloaded from GS
Bug: angleproject:4621
Change-Id: I85aefeeffa05712a63a62e9acef8df27b1933e34
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2190862
Commit-Queue: Yuly Novikov <ynovikov@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Auto-Submit: Yuly Novikov <ynovikov@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Josip Sokcevic 6afaa6ca97 Prune branches that no longer exist on remote
If a local branch tracks removed remote tracking branch (e.g. foo) and
remote has a branch that starts with such name (e.g. foo/bar), git fetch
will fail. --prune flag removes any remote-tracking branches that no
longer exist.

R=apolito@google.com, ehmaldonado@chromium.org

Bug: 1079483
Change-Id: I9bc31bf961d52a86b6fa2342249971b99a003666
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2190341
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lesmes 4de54135e8 Revert "git-cl: turn GERRIT_SQUASH_UPLOADS into a warning"
This reverts commit 31a538a3a9.

Reason for revert:
Breaks ANGLE's workflow.

Original change's description:
> git-cl: turn GERRIT_SQUASH_UPLOADS into a warning
> 
> Now that repos have dropped this from their codereview.settings file,
> change this logic to issue a warning if it's ever seen again.
> 
> Drop checking local gerrit.override-squash-uploads config too since
> it's no longer relevant.
> 
> Bug: 993518
> Change-Id: Id91bbc94b0890ca21c51a274f6acc41f2ae19b78
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1756159
> Commit-Queue: Mike Frysinger <vapier@chromium.org>
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>

TBR=vapier@chromium.org,ehmaldonado@chromium.org,infra-scoped@luci-project-accounts.iam.gserviceaccount.com

Change-Id: Iaa8b3341b189f356082ae25a1557898e25820566
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 993518
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2182019
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Mike Frysinger 31a538a3a9 git-cl: turn GERRIT_SQUASH_UPLOADS into a warning
Now that repos have dropped this from their codereview.settings file,
change this logic to issue a warning if it's ever seen again.

Drop checking local gerrit.override-squash-uploads config too since
it's no longer relevant.

Bug: 993518
Change-Id: Id91bbc94b0890ca21c51a274f6acc41f2ae19b78
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1756159
Commit-Queue: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Josip Sokcevic 5e18b605d3 Fix git cl upload on Windows
Use ISO8601 time format standard to avoid localized result and need to
encode.

R=ehmaldonado@chromium.org

Bug: 1074002
Change-Id: I95c2ddd55a9661351f2b45601f7bba24c2951c31
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2163401
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
5 years ago
Bruce Dawson 7a0b07a8a5 Fix slash direction sensitivity in git cl owners
Slashes and backslashes are interchangeable on Windows in many contexts.
But not all. In particular, if you add a path to a dictionary and then
try looking it up with an equivalent path that has different slashes
then - not surprisingly - the lookup will fail. Therefore it is
important to always use slashes when populating _paths_to_owners. This
is now done, and documented.

Additionally, while fnmatch.fnmatch handles comparing slash separated
paths to backslash separated paths, fnmatch.translate does not. This is
arguably a bug in fnmatch.translate, or at least something that should
be documented. This bug is worked around by having _fnmatch sanitize
file names to always use slashes, and asserting that the patterns do
not contain backslashes.

With these changes this command:
  git cl owners --ignore-current --show-all chrome/browser/BUILD.gn
now correctly gives the same results as this command on Windows:
  git cl owners --ignore-current --show-all chrome\browser\BUILD.gn

The modified test fails without the other changes, passes with them.

Bug: 1009104
Change-Id: I416c7131281f00e352c1d2b85c30fcc463417fa5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1915410
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Robert Iannucci bffd63b243 [cipd] Take cipd.ps1 out of %PATH%.
Having it in %PATH% means that powershell users will accidentally
run 'cipd.ps1' when they should be running 'cipd.bat'; This would
cause them to get requests for input parameters (like the backend URL)
which they have no idea how to supply.

R=agable@chromium.org, nodir@chromium.org, vadimsh@google.com

Bug: 1073529
Change-Id: I62a3bd9e00229a2047293dba1ba480d28f77bf63
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2161645
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
5 years ago
George Engelbrecht 888c0fe768 ReadHttpResponse: Handle 429s with retries and adjust retry count
We've had reliability issues with gerrit, primarily related to
429 status codes but also DDoS bans. The light DDoS bans can be
very short lived and we extend retries to handle this. Longer
term bans can take up to an hour to lift.

BUG=chromium:1071590
TEST=./gerrit_client_test.py

Change-Id: Iaf68c0d9cc7375aa58367ec0447d54a99f8ebf39
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2153089
Commit-Queue: George Engelbrecht <engeg@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
5 years ago
Andrii Shyshkalov 8aebb60a8d git cl patch: support short CL URLs.
Change-Id: I71ae8e94ec0cc5d880c002c7187351ac6ee2c2b3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2152218
Auto-Submit: Andrii Shyshkalov <tandrii@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lesmes e1a9c8db7e Reland "metrics: Use vpython3 instead of sys.executable"
This is a reland of 310e1e814d

Use subprocess2, which will set shell=True when running on Windows.

Original change's description:
> metrics: Use vpython3 instead of sys.executable
>
> Apparently, uploading metrics might happen after sys.executable
> has been deleted.
>
> Bug: 1068263
> Change-Id: Id4ab465bb97d146b688f42fb2002e29372e65555
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2148879
> Commit-Queue: Anthony Polito <apolito@google.com>
> Reviewed-by: Anthony Polito <apolito@google.com>

Bug: 1068263
Change-Id: I19c915f3e8862ae3248b01e0eccdbdf0bfda7369
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2150170
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lesmes 8073a50204 gclient: Set with_branch_heads if asked to sync to refs/branch-heads/*
The bug is caused because we already have refs/branch-heads/XXXX locally,
so we don't attempt to fetch it again, even if it has been updated.

This solution is a bit of a hack to make sure that we fetch
refs/branch-heads/*, which is the most common case for this to fail.

Bug: 1070545
Change-Id: Iec7cfff18219794230188e2b9e901c587acd274b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2149849
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
5 years ago
Edward Lesmes 3820520acb Revert "metrics: Use vpython3 instead of sys.executable"
This reverts commit 310e1e814d.

Reason for revert:
"vpython3" doesn't work on Windows without shell

Original change's description:
> metrics: Use vpython3 instead of sys.executable
> 
> Apparently, uploading metrics might happen after sys.executable
> has been deleted.
> 
> Bug: 1068263
> Change-Id: Id4ab465bb97d146b688f42fb2002e29372e65555
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2148879
> Commit-Queue: Anthony Polito <apolito@google.com>
> Reviewed-by: Anthony Polito <apolito@google.com>

TBR=ehmaldonado@chromium.org,apolito@google.com,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com

Change-Id: I6b9998de0d3afeda142971f889361fb1c8a37a0d
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 1068263
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2149762
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lesmes 310e1e814d metrics: Use vpython3 instead of sys.executable
Apparently, uploading metrics might happen after sys.executable
has been deleted.

Bug: 1068263
Change-Id: Id4ab465bb97d146b688f42fb2002e29372e65555
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2148879
Commit-Queue: Anthony Polito <apolito@google.com>
Reviewed-by: Anthony Polito <apolito@google.com>
5 years ago
Johnny(Jianning) Ding 08f4d59e0b Decode the returned encoded bytes of subprocess.check_all/check_output.
Bug: 1068779
Change-Id: Ie399e134577732b441c37ca2083d72a66bb631c3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2142745
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Johnny (Jianning) Ding <jnd@chromium.org>
5 years ago
Quinten Yearsley 925cedba7b Run spellchecker on depot_tools
This was made by running `codespell` and `scspell`
and then checking the results.

Change-Id: I169fd5b40294f83015075b4a899fbca263821f25
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2144602
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
Auto-Submit: Quinten Yearsley <qyearsley@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lesmes 5a5537dc63 git-cl: Fix retrieving gitcookies path.
Decode the result of reading http.cookiefile from git config

Bug: 1066992
Change-Id: I1d4f2c8e54f717ef61ab4e630f3f66206c56047e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2132774
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lesmes 994bed5496 Fix Python3 integer division issue in roll-dep.
Change-Id: I43d1b9cecee3066c206b2cfdb2fa9d37bc5001e5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2132232
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Auto-Submit: Corentin Wallez <cwallez@chromium.org>
5 years ago
Edward Lesmes 50da7706ce Reland "git-cl: Remove unused and duplicate functions."
This is a reland of e3a49aa405

GetRelativeRoot was fixed to use '.' instead of None

Original change's description:
> git-cl: Remove unused and duplicate functions.
>
> Remove unused functions, and use scm.GIT where possible to reduce
> duplication.
>
> Change-Id: I24f05e7b3ae04743e97b6665ee668f44d6acc294
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2116938
> Reviewed-by: Anthony Polito <apolito@google.com>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

Change-Id: I54bd1d34d3501a38bf8af2f9e70059d58fa538a5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2125293
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Josip Sokcevic 6a7e234b58 Fix Windows unit tests for roll-dep and use py3
R=ehmaldonado@chromium.org

Change-Id: I5bf1aa21a5ae9b90dbab4cc0439ca19e73de337f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2125221
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
5 years ago
Josip Sokcevic b3e593c1ba Escape AST string before writing to tokens
This patch enables more unit tests on Windows.

R=ehmaldonado@chromium.org

Change-Id: I8667777fc6eef11568aa2aead9850f7e37757e58
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2123808
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
5 years ago
Edward Lesmes 0dd5482c52 depot_tools: Restore appending title before change description.
On an initial upload, git-cl upload -t <title> should append <title>
before the change description.

This broke because options.squash is set to a default value very late
into code execution, so its value was not available when computing the
initial description.

options.squash is now set when parsing the options, early into code
execution.

Bug: 1064905
Change-Id: I04a18c572dd5b04a3013d39f71bc43d026ea85d3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2122324
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lesmes 17aaea21d4 Reland "depot_tools: Run using Python 3 by default."
This reverts commit e2ac022f0b.

git-cache issues were solved by https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2120966

Original change's description:
> Reland "depot_tools: Run using Python 3 by default."
>
> This is a reland of 01ed358917
>
> * python_runner.sh, gclient, roll-dep and fetch will call vpython
>   (instead of vpython3) when running on Windows under git-bash.
> * vpython3 now detects when running on Windows under git-bash and
>   executes vpython3.bat instead.
> * vpython3.bat calls python3.exe directly instead of calling python3.bat.
>
> Original change's description:
> > depot_tools: Run using Python 3 by default.
> >
> > Run gclient, roll-dep, fetch and custom git commands (i.e. git-cl, git-rebase-update, git-new-branch, etc.)
> > using vpython3 by default.
> >
> > Change-Id: I4eecddafa6ca4c5f82ec097615c79d2a741613e7
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2113550
> > Reviewed-by: Anthony Polito <apolito@google.com>
> > Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
>
> Change-Id: I9829141d7ea26a67e655264430151f493e73a930
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2118418
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Anthony Polito <apolito@google.com>

Change-Id: Iff02533f0a11062f335a010c074043c720d5b2ba
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2121304
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lesmes 34f71abca1 git-cache: Also bootstrap in the case that there are 0 pack files
This can happen if the cache repo was init'd, but has no pack files;
previously it would try to fetch from scratch.

Change-Id: I71689e30bdede392588c69e118e9297d86a134a3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2120281
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lesmes 7d1655b880 depot_tools: Split gclient_smoketests.
gclient_smoketest takes the longest to run among all tests.
Split it so that presubmit takes less time overall, and is
less likely to exceed the time limit.

Change-Id: I5a4f714b98f59a2e46bae944a043ad41b6786c25
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2121154
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lesmes 4c3eb70358 git-cache: Decode subprocess.check_output
Always decode subprocess.check_output.

Bug: 1064547
Change-Id: I459a236235a2355f9dd9813dafc12dff44866529
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2120966
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Josip Sokcevic e2ac022f0b Revert "Reland "depot_tools: Run using Python 3 by default.""
This reverts commit e9013eeb3d.

Reason for revert: https://crbug.com/1064547

Original change's description:
> Reland "depot_tools: Run using Python 3 by default."
> 
> This is a reland of 01ed358917
> 
> * python_runner.sh, gclient, roll-dep and fetch will call vpython
>   (instead of vpython3) when running on Windows under git-bash.
> * vpython3 now detects when running on Windows under git-bash and
>   executes vpython3.bat instead.
> * vpython3.bat calls python3.exe directly instead of calling python3.bat.
> 
> Original change's description:
> > depot_tools: Run using Python 3 by default.
> >
> > Run gclient, roll-dep, fetch and custom git commands (i.e. git-cl, git-rebase-update, git-new-branch, etc.)
> > using vpython3 by default.
> >
> > Change-Id: I4eecddafa6ca4c5f82ec097615c79d2a741613e7
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2113550
> > Reviewed-by: Anthony Polito <apolito@google.com>
> > Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> 
> Change-Id: I9829141d7ea26a67e655264430151f493e73a930
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2118418
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Anthony Polito <apolito@google.com>

TBR=ehmaldonado@chromium.org,ajp@chromium.org,apolito@google.com,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com

Change-Id: Ia382d6e4e330f5a7a7a62e4d369c3fa16fc17333
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2120956
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
5 years ago
Tibor Goldschwendt 7c5efb2a7e Add format string option to git cl archive
This option lets users specify a format of the archive tags. E.g.
  git cl archive -p 'archived/{issue}-{branch}'
makes tags in the format
  archived/1234-foo-feature

Change-Id: Icb74cc68781cda21a70c802bd640543e92ae97a9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2116723
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Tibor Goldschwendt <tiborg@chromium.org>
5 years ago