Commit Graph

205 Commits (9f668bbff99167ba797888c6d322dd309f338aee)

Author SHA1 Message Date
Gavin Mak edba22d4eb Fix multiline comment formatting
Many incorrectly formatted comments exist from the switch to
4 space indent: https://crrev.com/c/4836379

Bug: 1514505
Change-Id: I6366f9da812919bd35b999f18fa8a49b7a66c09b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5153633
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
1 year ago
Josip Sokcevic 31a590e0cd [scm] Cache ResolveCommit call
The cache won't be used if ResolveCommit throws an exception or when
non-sha1 is used.

This optimization saves about 7% of overall gclient sync on
chromium/src.

R=gavinmak@google.com

Change-Id: I9c36d937dc7b8553818888e2e9c4eecd029a978c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5147496
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
1 year ago
Josip Sokcevic 4cf6518959 scm: Remove usage of distutil
The package is deprecated and will be removed in py 3.12.

R=ddoman@google.com

Bug: 1500415
Change-Id: Ia130c556ad3f2135fac4a9b1d9bb4ecc088cfe9d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5030818
Reviewed-by: Scott Lee <ddoman@chromium.org>
Reviewed-by: Jonathan Nieder <jrn@google.com>
Auto-Submit: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Scott Lee <ddoman@chromium.org>
Commit-Queue: Jonathan Nieder <jrn@google.com>
1 year ago
Joanna Wang 60adf7b6d9 sync: Ensure child dependencies are always removed first.
Fixed: chromium:1454643, chromium:1486677
Change-Id: I3f8f5f9a2e2419830e94fa1419292717820830c8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4914238
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
1 year ago
Mike Frysinger 124bb8e53c switch to 4 space indent
Leave the recipes/ code at 2 space to match the rest of the recipes
project in other repos.

Reformatted using:
files=( $(
	git ls-tree -r --name-only HEAD | \
		grep -Ev -e '^(third_party|recipes)/' | \
		grep '\.py$';
	git grep -l '#!/usr/bin/env.*python' | grep -v '\.py$'
) )
parallel ./yapf -i -- "${files[@]}"
~/chromiumos/chromite/contrib/reflow_overlong_comments "${files[@]}"

The files that still had strings that were too long were manually
reformatted because they were easy and only a few issues.
autoninja.py
clang_format.py
download_from_google_storage.py
fix_encoding.py
gclient_utils.py
git_cache.py
git_common.py
git_map_branches.py
git_reparent_branch.py
gn.py
my_activity.py
owners_finder.py
presubmit_canned_checks.py
reclient_helper.py
reclientreport.py
roll_dep.py
rustfmt.py
siso.py
split_cl.py
subcommand.py
subprocess2.py
swift_format.py
upload_to_google_storage.py

These files still had lines (strings) that were too long, so the pylint
warnings were suppressed with a TODO.
auth.py
gclient.py
gclient_eval.py
gclient_paths.py
gclient_scm.py
gerrit_util.py
git_cl.py
presubmit_canned_checks.py
presubmit_support.py
scm.py

Change-Id: Ia6535c4f2c48d46b589ec1e791dde6c6b2ea858f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4836379
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Auto-Submit: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Joanna Wang e36c6bba14 Make gclient getdep work with submodules.
Bug: 1475770
Change-Id: I341910e75195d87b91defd98f2c6ba25262995b1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4828423
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
2 years ago
Josip Sokcevic 2d5c673fdb [scm] Fix list all files when filename has space
Off by one - we should have 4 parts, so we need maxsplit=3.

R=aravindvasudev@google.com, jojwang

Fixed: 1474283
Change-Id: I13b568feb8ee2aa307a2267c43765c1749b77c90
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4799773
Auto-Submit: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
2 years ago
Josip Sokcevic 49b24e41da [scm] Remove gitlinks from list of affected files
Presubmit checks expect files to be actual files. Gitlink entries are
directories (if initialized), and that breaks various existing tests.

Since we don't run presubmits from git dependencies, it's safe to ignore
any updates to gitlinks and omit them from All Files.

R=jojwang

Change-Id: I670e5caef4067a0203578e9be50c4fabf23b3153
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4795116
Auto-Submit: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
2 years ago
Josip Sokcevic 7958e30579 Revert "Move scm.py to lib/"
This reverts commit 688adfe276.

Reason for revert: breaks try jobs

Original change's description:
> Move scm.py to lib/
>
> It belongs there.
>
> Change-Id: I286b50c43c587c39557fc508175efbd4b0f51d51
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4289680
> Reviewed-by: Joanna Wang <jojwang@chromium.org>
> Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>

Change-Id: I662907b8bb4b2e7db82409bd34c3905db7961a12
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4300261
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
2 years ago
Josip Sokcevic 688adfe276 Move scm.py to lib/
It belongs there.

Change-Id: I286b50c43c587c39557fc508175efbd4b0f51d51
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4289680
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Joanna Wang b46232e729 [stacked_changes] Add Changelist._PrepareChange() common func called before creating commits.
Bug: b/265929888
Change-Id: I18a0c5ba6757aef91e750b9703079c96b090dc1e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4178920
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Joanna Wang ab9c6ba27f Make cwd optional for scm.IsAncestor()
Bug:b/265929888
Change-Id: I9f25cd26eaf2d708ce3ac7d86503bb2c295c8406
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4178351
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2 years ago
Aravind Vasudevan c5f0cbb865 Use pylint 2.7 for depot_tools
This includes a few fixes for specific errors, and disables several new
warnings introduced in this version, in order to allow for an incremental migration.

Bug:1262286
Change-Id: I4b8f8fc521386419a3121bbb07edc8ac83170a94
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3413679
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
3 years ago
Josip Sokcevic 42c5bbbc96 Revert "Use pylint 2.7 for depot_tools"
This reverts commit 22bf605bb6.

Reason for revert: breaks gclient sync

Original change's description:
> Use pylint 2.7 for depot_tools
>
> This includes a few fixes for specific errors, and disables several new
> warnings introduced in this version, in order to allow for an incremental migration.
>
> Bug:1262286
> Change-Id: Ie97d686748c9c952e87718a65f401c5f6f80a5c9
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3400616
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>

Bug: 1262286
Change-Id: Ieb946073c7886c7bf056ce843a5a48e82becf7a5
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3413672
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
3 years ago
Aravind Vasudevan 22bf605bb6 Use pylint 2.7 for depot_tools
This includes a few fixes for specific errors, and disables several new
warnings introduced in this version, in order to allow for an incremental migration.

Bug:1262286
Change-Id: Ie97d686748c9c952e87718a65f401c5f6f80a5c9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3400616
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
3 years ago
Josip Sokcevic e1482c5548 Refresh remote HEAD if matches legacy default
If remote HEAD returns legacy default branch, refresh HEAD by running
set-head to check if there are any changes.

R=gavinmak@google.com

Change-Id: I5891a899b512e81fccfb086a056d497c906df4b2
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3152593
Auto-Submit: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
4 years ago
Josip Sokcevic 7e133ff177 Use main branch as the default branch
R=apolito@google.com

Change-Id: I6a2ed557b9e8d45194b2fe4ea5327ce62020733c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3000134
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
4 years ago
Josip Sokcevic 091f5ac0a6 Use real default branch in gclient
Currently, gclient sync assumes the default branch is master, and
it doesn't work at all if such branch doesn't exist. This change queries
local git copy to get remote HEAD. If local git version is not
available, it queries remote git server using ls-remote.

This change requires git version 2.28 (depot_tools comes with 2.29).

R=ehmaldonado@chromium.org

Bug: 1156318
Change-Id: Id348e0f1004093f395139e8f4d62adb66b94ca9c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2628359
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
4 years ago
Edward Lesmes cf06cad1be [subprocess2] Replace VOID with DEVNULL
subprocess.DEVNULL was introduced in Python3 to serve same purpose
as subprocess2.VOID, so rename VOID to DEVNULL in subprocess2.

Change-Id: I6dade3306ffc3bc2441ac6083f362b099c2427e9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2587758
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
4 years ago
Josip Sokcevic 9c0dc3026e Replace non-inclusive terms
Bug: 1118410, 1118413, 1118422, 1118424, 1118426
Bug: 1118428, 1118430, 1118434, 1118438, 1118442
Bug: 1118453, 1118454
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: build
Change-Id: I43e6305f8c3136774f01b57f12bb442bcb863371
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2551388
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
4 years ago
Josip Sokcevic 5bdfcd8f33 Add `main` when guessing origin remote
This fixes presubmit checks that rely on FetchUpstreamTuple which is
used in patching.

R=ehmaldonado@chromium.org

Bug: 1143724
Change-Id: I163fa0ea6f1b356280d497196869bb206463ad4f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2515268
Auto-Submit: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
4 years ago
Quinten Yearsley 925cedba7b Run spellchecker on depot_tools
This was made by running `codespell` and `scspell`
and then checking the results.

Change-Id: I169fd5b40294f83015075b4a899fbca263821f25
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2144602
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
Auto-Submit: Quinten Yearsley <qyearsley@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lesmes 56dbf9a4b5 depot_tools: Don't use carets when checking if a revision exists.
On Windows, git can be either an executable or batch script, each of
which requires carets (^) to be escaped in different ways.

This makes it hard to use `git rev-parse --verify REV^{commit}`, so we
truncate full hash revisions to achieve the same effect instead.

Change-Id: I01578620706a1bab75a2decc55d9b4f1ac3afe28
Bug: 1065307
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2127968
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lesmes 50da7706ce Reland "git-cl: Remove unused and duplicate functions."
This is a reland of e3a49aa405

GetRelativeRoot was fixed to use '.' instead of None

Original change's description:
> git-cl: Remove unused and duplicate functions.
>
> Remove unused functions, and use scm.GIT where possible to reduce
> duplication.
>
> Change-Id: I24f05e7b3ae04743e97b6665ee668f44d6acc294
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2116938
> Reviewed-by: Anthony Polito <apolito@google.com>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

Change-Id: I54bd1d34d3501a38bf8af2f9e70059d58fa538a5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2125293
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lesmes cba460828f Revert "git-cl: Remove unused and duplicate functions."
This reverts commit e3a49aa405.

Reason for revert:

TypeError: expected str, bytes or os.PathLike object, not NoneType

Original change's description:
> git-cl: Remove unused and duplicate functions.
> 
> Remove unused functions, and use scm.GIT where possible to reduce
> duplication.
> 
> Change-Id: I24f05e7b3ae04743e97b6665ee668f44d6acc294
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2116938
> Reviewed-by: Anthony Polito <apolito@google.com>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

TBR=ehmaldonado@chromium.org,apolito@google.com,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com

Change-Id: I334a6289eb2c1f3e20feddd428307542d2aa38a9
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2119411
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lesmes e3a49aa405 git-cl: Remove unused and duplicate functions.
Remove unused functions, and use scm.GIT where possible to reduce
duplication.

Change-Id: I24f05e7b3ae04743e97b6665ee668f44d6acc294
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2116938
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lemur d52eddaad2 scm: Fix bug when checking for valid revision and add tests.
When running `git rev-parse REV^{commit}` in Windows, `^` must be
escaped.
However, it was escaped more times than necessary.

Split IsValidRevision to call ResolveCommit, add tests to ResolveCommit,
and run scm.py tests on Windows.

Change-Id: I761a820394c8b5410d68b6ccd6c352c41c30c88c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2092904
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lemur 851532894f git-cl: Use scm.GIT.{Get,Set}BranchConfig and {Get,Set}BranchRef instead of git calls.
scm.GIT functions are tested, and using them makes it easier to mock
git calls on git-cl tests.

Bug: 1051631
Change-Id: If067aa3f71b7478cafd7985d3020dacc0c6a41c1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2055464
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lemur 15a9b8cc5d git-cl: Use scm.GIT.FetchUpstreamTuple.
scm:
- Add methods to deal with git configs and tests for them.
  Will make it easier to mock git config calls as opposed
  to all git calls.
- Add tests to FetchUpstreamTuple

git-cl:
- Mock out settings.GetChangeRoot()
- Use scm.GIT.FetchUpstreamTuple() to reduce code duplication,
  and mock it out on tests.

Bug: 1051631
Change-Id: I1a3220b4c0f3e6221b3c00550259a433c2775798
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2052552
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Anthony Polito <apolito@google.com>
5 years ago
Edward Lemur 7f6dec0169 git-cl: Simplify Change object construction.
The `author` parameter is never set.
The `local_description` parameter was only set with the
purpose of getting the local description, so separate it into its
own method and call that instead.
Use the absolute root when getting the list of modified files instead.

Bug: 1042324
Change-Id: I8505172edaacee0656dda98ebbd5f8ebfacb3cef
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2038050
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lemur 98cfac11bd scm: Add GetAllFiles method.
Will be used by presubmit_support to run checks over all files on a repo.

Bug: 1042324
Change-Id: I872b4eb7f287f3a4b14d753cad73d0c5d7beb00c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2002961
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lemur c9144527d9 depot_tools: Don't use basestring in scm.CaptureStatus.
basestring doesn't exist in Python 3. scm.Capture status doesn't need to support strings either.

Bug: 1002209
Change-Id: Ie9edfdfd09b789bfaace038a6ad8f3a7da344bc4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1890925
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Quinten Yearsley 3841eaa164 Remove unused imports in depot_tools/*.py
I know that sometimes imports can have side-effects,
so unused imports shouldn't always be removed, but these
ones look like they could be.

Change-Id: Iea9f82afa99b0ea35f29a28f20ce0493b579cfee
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1819860
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
6 years ago
Raul Tambre 6a9b00e433 scm: Replace invalid UTF-8 in Git output
This may occur when git diff is ran with binary changes in the diff.
Example: https://chromium-review.googlesource.com/c/chromium/src/+/1606595

Bug: 942522
Change-Id: I44caef559de44076ef394caa5abc3967db2bb0b9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1607840
Auto-Submit: Raul Tambre <raul@tambre.ee>
Commit-Queue: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
6 years ago
Raul Tambre cd862e37ea gclient_scm: Decode Git output centrally for better Python 3 support
This avoids the need to do the decoding everywhere separately.
Also a small fix for filter() no longer returning a list on Py3.

The scripts still work with Python 2.
There are no intended behaviour changes.

Bug: 942522
Change-Id: Id0f5153011b2ef1b64394359087864cd9434e45e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1595685
Commit-Queue: Dirk Pranke <dpranke@chromium.org>
Auto-Submit: Raul Tambre <raul@tambre.ee>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
6 years ago
Edward Lemur 8c66565649 gclient: Fix applying patches to branch heads.
Bug: 956807
Change-Id: I2eed6db1f338806812fca778986d51b0a007eddd
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1592577
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
6 years ago
Bruce Dawson 9c06201209 Fix multiple confusingly escaped regex strings
Python (prior to 3.8) treats meaningless string escape sequences as if
they were a slash followed by the character. That is, '\w' == '\\w'.
Python 3.8 rejects this, and it's confusing. This change fixes seven of these
regex strings found in depot_tools (through a regex search, natch). Most of
the fixes don't actually change the value of the strings, and this was
manually verified:

>>> '(/c(/.*/\+)?)?/(\d+)(/(\d+)?/?)?$' == r'(/c(/.*/\+)?)?/(\d+)(/(\d+)?/?)?$'
True
>>> '#\s*OWNERS_STATUS\s+=\s+(.+)$' == r'#\s*OWNERS_STATUS\s+=\s+(.+)$'
True
>>> 'COM\d' == r'COM\d'
True
>>> '^\s+Change-Id:\s*(\S+)$' == r'^\s+Change-Id:\s*(\S+)$'
True
>>> 'ETag:\s+([a-z0-9]{32})' == r'ETag:\s+([a-z0-9]{32})'
True

Two exceptions were the regex expressions in filter_demo_output.py and scm.py.
These were turned into raw strings despite this changing the value of the
string passed to re. This works because re supports the \x, \d, \w, \t, and
other escape sequences needed to make this work.

TL;DR - use raw strings for regex to avoid melting your brain. If bulk changing
regex strings to raw watch out for double-slashes.

Bug: 958138
Change-Id: Ic45264cfc63e8bae9cfcffe2cd88a57c2d3dcdae
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1590534
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
6 years ago
Edward Lemur 9a5e3bd817 gclient_scm: Simplify fetching refs.
Bug: 874501, 942229
Change-Id: Ie9896e8a289e32a1b468ed5fa51c95a81970bdf8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1544802
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
6 years ago
Raul Tambre b946b2389c Improve gclient Python 3 compatibility
This enables gclient sync and gclient runhooks to run, barring hook script failures.
git cl upload also now works.

The scripts still work with Python 2.
There are no intended behaviour changes.

Bug: 942522
Change-Id: I2ac587b5f803ba7f5bb5e412337ce049f4b1a741
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1524583
Commit-Queue: Raul Tambre <raul@tambre.ee>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
6 years ago
Michael Spang 2f3c820ae3 Simplify GIT.IsValidRevision
We should pass --verify to tell rev-parse that we're just testing,
and use the ^{commit} dereference operator to avoid the hack of
removing a character from the commit hash to see if it's really
in the object database.

Bug: 938627

Change-Id: Ic6ea898b0a5a6a1a5d706c7586c7208ec8ca2ce2
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1504104
Commit-Queue: Michael Spang <spang@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
6 years ago
Andrii Shyshkalov 2f727917ac git cl: add reviewers and ccs to git push command if possible.
Also fix a typo in ValidateEmail function which didn't support
email addresses with '-' in them, e.g., infra-dev@chromium.org.

R=ehmaldonado

Bug: 875089
Change-Id: I2d73c1473527c9bf62e25e9f88250196b783fcb0
Reviewed-on: https://chromium-review.googlesource.com/c/1242849
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
6 years ago
Edward Lemur ca7d881540 Reland "Reland "gclient_scm: Use cherry-picking instead of rebasing.""
Abort any cherry-picks before applying the patch, so that if the bots are in a
bad state, we don't fail.

Original change's description:
> Reland "gclient_scm: Use cherry-picking instead of rebasing."
>
> The failures were caused by:
>  1 - When one change (call it #2) has been uploaded on top of another (#1),
>      and (#1) has already landed, git cherry-pick complains that the range
>      '<merge-base>..<change #2>' contains empty commits, since the contents
>      of (#1) are already present in the tree.
>  2 - We did not abort the cherry-picking when 'git cherry-pick' failed,
>      so a failure made all further CLs in that bot fail.
>
> This CL fixes it and prevents further regressions.
>
> Original change's description:
> > gclient_scm: Use cherry-picking instead of rebasing.
> >
> > Currently gclient might include extra commits when applying patches.
> > For example, in this case we checkout |patch| and rebase it on top of |base|,
> > thus including an |extra commit| that we shouldn't.
> >
> > o master
> > |
> > . o patch
> > |/
> > o extra commit
> > |
> > o base (what gclient synced src at)
> >
> > This change uses the merge-base between |patch| and |master| to cherry-pick only
> > the changes belonging to the patch.
> >
> > Bug: 850812
> > Change-Id: I138192f96bc62b1bb19b0e1ad952c8f8c67631c4
> > Reviewed-on: https://chromium-review.googlesource.com/1137052
> > Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> > Reviewed-by: Aaron Gable <agable@chromium.org>
>
> Bug: 850812
> Change-Id: I83f38d0a258df3f5cd89e277f0d648badff29a22
> Reviewed-on: https://chromium-review.googlesource.com/1139554
> Reviewed-by: Aaron Gable <agable@chromium.org>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

Bug: 850812
Change-Id: Ic65bda67c792bd7af5ec013a62d9615d1498eb3a
Reviewed-on: https://chromium-review.googlesource.com/1142805
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
John Budorick 2c984a03ca Revert "Reland "gclient_scm: Use cherry-picking instead of rebasing.""
This reverts commit f9afc77f28.

Reason for revert: https://bugs.chromium.org/p/chromium/issues/detail?id=864301#c10

Original change's description:
> Reland "gclient_scm: Use cherry-picking instead of rebasing."
> 
> The failures were caused by:
>  1 - When one change (call it #2) has been uploaded on top of another (#1),
>      and (#1) has already landed, git cherry-pick complains that the range
>      '<merge-base>..<change #2>' contains empty commits, since the contents
>      of (#1) are already present in the tree.
>  2 - We did not abort the cherry-picking when 'git cherry-pick' failed,
>      so a failure made all further CLs in that bot fail.
> 
> This CL fixes it and prevents further regressions.
> 
> Original change's description:
> > gclient_scm: Use cherry-picking instead of rebasing.
> >
> > Currently gclient might include extra commits when applying patches.
> > For example, in this case we checkout |patch| and rebase it on top of |base|,
> > thus including an |extra commit| that we shouldn't.
> >
> > o master
> > |
> > . o patch
> > |/
> > o extra commit
> > |
> > o base (what gclient synced src at)
> >
> > This change uses the merge-base between |patch| and |master| to cherry-pick only
> > the changes belonging to the patch.
> >
> > Bug: 850812
> > Change-Id: I138192f96bc62b1bb19b0e1ad952c8f8c67631c4
> > Reviewed-on: https://chromium-review.googlesource.com/1137052
> > Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> > Reviewed-by: Aaron Gable <agable@chromium.org>
> 
> Bug: 850812
> Change-Id: I83f38d0a258df3f5cd89e277f0d648badff29a22
> Reviewed-on: https://chromium-review.googlesource.com/1139554
> Reviewed-by: Aaron Gable <agable@chromium.org>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

TBR=agable@chromium.org,ehmaldonado@chromium.org,jbudorick@chromium.org

Change-Id: I2138757899abd6eb9bece2dac3a25676effafdf3
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 850812
Reviewed-on: https://chromium-review.googlesource.com/1142624
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: John Budorick <jbudorick@chromium.org>
7 years ago
Edward Lemur f9afc77f28 Reland "gclient_scm: Use cherry-picking instead of rebasing."
The failures were caused by:
 1 - When one change (call it #2) has been uploaded on top of another (#1),
     and (#1) has already landed, git cherry-pick complains that the range
     '<merge-base>..<change #2>' contains empty commits, since the contents
     of (#1) are already present in the tree.
 2 - We did not abort the cherry-picking when 'git cherry-pick' failed,
     so a failure made all further CLs in that bot fail.

This CL fixes it and prevents further regressions.

Original change's description:
> gclient_scm: Use cherry-picking instead of rebasing.
>
> Currently gclient might include extra commits when applying patches.
> For example, in this case we checkout |patch| and rebase it on top of |base|,
> thus including an |extra commit| that we shouldn't.
>
> o master
> |
> . o patch
> |/
> o extra commit
> |
> o base (what gclient synced src at)
>
> This change uses the merge-base between |patch| and |master| to cherry-pick only
> the changes belonging to the patch.
>
> Bug: 850812
> Change-Id: I138192f96bc62b1bb19b0e1ad952c8f8c67631c4
> Reviewed-on: https://chromium-review.googlesource.com/1137052
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Aaron Gable <agable@chromium.org>

Bug: 850812
Change-Id: I83f38d0a258df3f5cd89e277f0d648badff29a22
Reviewed-on: https://chromium-review.googlesource.com/1139554
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
John Budorick 40bacee96a Revert "gclient_scm: Use cherry-picking instead of rebasing."
This reverts commit fb78b368de.

Reason for revert: https://bugs.chromium.org/p/chromium/issues/detail?id=864301

Original change's description:
> gclient_scm: Use cherry-picking instead of rebasing.
> 
> Currently gclient might include extra commits when applying patches.
> For example, in this case we checkout |patch| and rebase it on top of |base|,
> thus including an |extra commit| that we shouldn't.
> 
> o master
> |
> . o patch
> |/
> o extra commit
> |
> o base (what gclient synced src at)
> 
> This change uses the merge-base between |patch| and |master| to cherry-pick only
> the changes belonging to the patch.
> 
> Bug: 850812
> Change-Id: I138192f96bc62b1bb19b0e1ad952c8f8c67631c4
> Reviewed-on: https://chromium-review.googlesource.com/1137052
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Aaron Gable <agable@chromium.org>

TBR=agable@chromium.org,ehmaldonado@chromium.org

Change-Id: I57299e60e58eac5656dc88937c622d0d14c4ba37
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 850812
Reviewed-on: https://chromium-review.googlesource.com/1139553
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: John Budorick <jbudorick@chromium.org>
7 years ago
Edward Lemur fb78b368de gclient_scm: Use cherry-picking instead of rebasing.
Currently gclient might include extra commits when applying patches.
For example, in this case we checkout |patch| and rebase it on top of |base|,
thus including an |extra commit| that we shouldn't.

o master
|
. o patch
|/
o extra commit
|
o base (what gclient synced src at)

This change uses the merge-base between |patch| and |master| to cherry-pick only
the changes belonging to the patch.

Bug: 850812
Change-Id: I138192f96bc62b1bb19b0e1ad952c8f8c67631c4
Reviewed-on: https://chromium-review.googlesource.com/1137052
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
7 years ago
Aaron Gable f4068aa3ea Use core.quotePath=false for more git diffs
This includes a minor refactor so that some gclient_scm methods
can all share the same core.quotePath specifier.

R=iannucci

Bug: 792302
Change-Id: Iaadf190f5c0666787cf7c2ccda88d6dba9aace9b
Reviewed-on: https://chromium-review.googlesource.com/823131
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Aaron Gable <agable@chromium.org>
7 years ago
Aaron Gable 7817f02543 Reland "Use core.quotePath=false when git is listing files"
This is a reland of 9219d35688

The original was reverted due to a typo (core,quotePath instead
of core.quotePath). This version is fixed.

Original change's description:
> Use core.quotePath=false when git is listing files
>
> This prevents git from putting quotes around some file names
> (those that have astral-plane characters) and not around others.
>
> R=maruel
>
> Bug: 792302
>
> Change-Id: I3b6a6b36c4720116de811b40177b59aa25c263db
> Reviewed-on: https://chromium-review.googlesource.com/815454
> Commit-Queue: Aaron Gable <agable@chromium.org>
> Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org>

Bug: 792302
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Change-Id: I28d2260948aaf63bd865888c2f60e4cdee9aea48
Reviewed-on: https://chromium-review.googlesource.com/822990
Commit-Queue: Aaron Gable <agable@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
7 years ago
Robbie Iannucci 6aaee3065c Revert "Use core.quotePath=false when git is listing files"
This reverts commit 9219d35688.

Reason for revert: unfortunately this says "core,quotePath" and since it includes recipe changes, we need something that the roller can munch on :(

Original change's description:
> Use core.quotePath=false when git is listing files
> 
> This prevents git from putting quotes around some file names
> (those that have astral-plane characters) and not around others.
> 
> R=​maruel
> 
> Bug: 792302
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Change-Id: I3b6a6b36c4720116de811b40177b59aa25c263db
> Reviewed-on: https://chromium-review.googlesource.com/815454
> Commit-Queue: Aaron Gable <agable@chromium.org>
> Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org>

TBR=maruel@chromium.org,agable@chromium.org

Change-Id: I226388f19024403240a1443eb2b878b9293220e1
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 792302
Reviewed-on: https://chromium-review.googlesource.com/821671
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
7 years ago
Aaron Gable 9219d35688 Use core.quotePath=false when git is listing files
This prevents git from putting quotes around some file names
(those that have astral-plane characters) and not around others.

R=maruel

Bug: 792302
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Change-Id: I3b6a6b36c4720116de811b40177b59aa25c263db
Reviewed-on: https://chromium-review.googlesource.com/815454
Commit-Queue: Aaron Gable <agable@chromium.org>
Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org>
7 years ago