This fixes an issue on Windows where running `gclient setdep` would convert the DEPS file from LF to CRLF line endings.
From the python 2.7 docs:
" On Windows, 'b' appended to the mode opens the file in binary mode, so there are also modes like 'rb', 'wb', and 'r+b'. Python on Windows makes a distinction between text and binary files; the end-of-line characters in text files are automatically altered slightly when data is read or written. "
- https://docs.python.org/2.7/tutorial/inputoutput.html#reading-and-writing-files
Change-Id: I94183918789a8cdd4aa655db4b3dadfaae23d13a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2067477
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: John Emau <John.Emau@microsoft.com>
Using git daemon for smoke tests times out on Windows.
This CL adds support for local directory URLs to gclient so we can tests
using local directories as remotes instead.
Bug: 1024683
Change-Id: I6dcefa6eb0a93713cf46ea1e4c6b29311ad37565
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1915416
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Also add a better warning if 'gclient sync -D' fails due to a modified file.
I've tested this code via deleting a DEP and running gclient sync -D.
Bug: 981149
Change-Id: I97035ac238d163ccb1684c3ee423c223ed0f6299
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1891830
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Anthony Polito <apolito@google.com>
Ran:
vi $(git grep --name-only iteritems | grep -v third_party)
vi $(git grep --name-only itervalues | grep -v third_party)
vi $(git grep --name-only 'print ' | grep -v third_party)
and edited the files quickly with adhoc macros. Then ran in recipes/:
./recipes.py test train
There was only a small subset of files that had been updated to use
six.iteritems() and six.itervalues(). Since the dataset size that is
being used in gclient is small (pretty much always below 200 items),
it's better to just switch to .items() right away and take the temporary
performance hit, so that we don't need to come back to rewrite the code.
Recipe-Nontrivial-Roll: build
Bug: 984182
Change-Id: I5faf11486b66b0d73c9098ab0f2ce1b15a45c53e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1854900
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Auto-Submit: Marc-Antoine Ruel <maruel@chromium.org>
An increasing number of Windows developers are "accidentally" ending up
with Python 3 installed on their machine through the Windows Store. If
this version is first in the path then gclient may fail in non-obvious
ways. This adds a python version check to give a clear warning message.
R=vapier@chromium.org
Change-Id: I7a292fba2ab7d059a6149644d0b1fd92d2371dce
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1825945
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
- Merge CheckCallAndFilter[AndHeader]
- print_stdout will cause command output to be redirected to sys.stdout.
- Made compatible with Python 3.
Bug: 984182
Change-Id: Ida30e295b872c8c1a1474a376a90aecea924f307
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1727404
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Ran "2to3 -w -n -f print ./" and manually added imports.
Ran "^\s*print " and "\s+print " to find batch/shell scripts, comments and the like with embedded code, and updated them manually.
Also manually added imports to files, which used print as a function, but were missing the import.
The scripts still work with Python 2.
There are no intended behaviour changes.
Bug: 942522
Change-Id: Id777e4d4df4adcdfdab1b18bde89f235ef491b9f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1595684
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Dirk Pranke <dpranke@chromium.org>
Auto-Submit: Raul Tambre <raul@tambre.ee>
Previously, gclient will exit if it found a tilde in any element from
PATH. This CL changes that to first check if the path is the actual
depot_tools directory.
Also check for '~', instead of '~/', in case the path looks like
'~user/path/to/depot_tools/.
Bug: 952865
Change-Id: Ied07444e44edb655b5c8837b5e51eeb0dcebba6d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1581419
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Henrique Ferreiro <hferreiro@igalia.com>
Bash and its descendants allow having $PATH entries containing literal tildes,
like:
...:~/depot_tools:...
If the user has their shell configured this way, their shell will be able to
invoke `fetch` and other depot_tools commands, but depot_tools will not in turn
be able to invoke subcommands via popen because popen does not have this
bash-specific behavior when not in shell mode.
This change has gclient detect this configuration problem and error out with a
descriptive error message. Not doing so leads to a puzzling condition where
gclient complains about being unable to find a program that (from the user's
perspective) is in their $PATH and invokable interactively.
It also has 'fetch' suppress its own failure stack trace if it runs gclient and
gclient fails; the 'fetch' stack trace pushes the messages from gclient (which
are actually diagnostic) off the screen.
Bug: chromium:952865
Change-Id: Ibba4d2fccee405aa68392ce141493f1de21ec018
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1573204
Commit-Queue: Elly Fong-Jones <ellyjones@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
A previous CL replaced "basestring" with "str", because basestring
does not exist in Python 3. However, this broke Python 2's ability
to interoperate with unicode strings. This CL introduces a workaround
(defining basestring to be equivalent to string, if it doesn't exist
already), and restores the references to basestring. This workaround
can be fixed when we're 100% on Python 3.
It also undoes some unnecessary and harder-to-read formatting changes.
Bug: 942522
Change-Id: I4a31ee46dc048134c2e4832b6c44ea00ce341899
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1572441
Commit-Queue: Aaron Gable <agable@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Restore previous behavior and keep emitting a warning when an existing
gclient entry is not longer part of the client. Additionally, suggest
the usage of `gclient sync -D` to automatically remove them.
Change-Id: I62d4662ae4d0886d340230019419b68debffc5ba
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1483031
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Michael Moss <mmoss@chromium.org>
Commit-Queue: Henrique Ferreiro <hferreiro@igalia.com>
This is no longer supported to make the move to Python 3 easier.
Only some very old/weird user configurations may have this, so this provides a clear actionable error message.
Bug: 942522
Change-Id: I88106089c53d84b19c4f77b7e60fe251082122d7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1539957
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Raul Tambre <raul@tambre.ee>
This enables gclient sync and gclient runhooks to run, barring hook script failures.
git cl upload also now works.
The scripts still work with Python 2.
There are no intended behaviour changes.
Bug: 942522
Change-Id: I2ac587b5f803ba7f5bb5e412337ce049f4b1a741
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1524583
Commit-Queue: Raul Tambre <raul@tambre.ee>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
This reverts commit 2d29261e94.
Reason for revert: Both jbudorick and I asked you not to land this today :(.
Original change's description:
> Remove exception for relative CIPD
>
> This seems to no longer be necessary, as relative paths 'just work' for
> CIPD deps now.
>
> Change-Id: I3843548dc74d29af90e0c402f0459dbe60934423
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1529132
> Reviewed-by: Dirk Pranke <dpranke@chromium.org>
> Reviewed-by: John Budorick <jbudorick@chromium.org>
> Commit-Queue: Brandon Tolsch <btolsch@chromium.org>
TBR=dpranke@chromium.org,jbudorick@chromium.org,btolsch@chromium.org
Change-Id: I1d9a771ef34dae0dec15499d7cf28e4ba5daf2ff
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1531193
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Dirk Pranke <dpranke@chromium.org>
This seems to no longer be necessary, as relative paths 'just work' for
CIPD deps now.
Change-Id: I3843548dc74d29af90e0c402f0459dbe60934423
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1529132
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Brandon Tolsch <btolsch@chromium.org>
gclient_utils.py is a kitchen sink and is for that reason expensive
to import. Move the comparatively cheap and simple path routines
to a new gclient_paths module and use that in gn.py, clang_format.py,
dart_format.py.
(To be able to move FindGclientRoot() to gclient_paths.py,
make it use io.open() instead of FileRead(). FileRead() tries
to paper over invalid utf-8, but that was added for presubmits,
not for .gclient files, so this is hopefully fine.)
Cuts gn.py overhead in half (on my Windows laptop from 0.6s to 0.25s,
still high; on my Mac laptop from 0.1s to 0.05s), and probably helps
the other two too.
Completely remove PathDifference() since it's unused.
Bug: 939959
Change-Id: I6a70f6e4c16062b622fb2df8778e8a598d4cc956
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1512058
Commit-Queue: Nico Weber <thakis@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
This CL shows bit more detail of value collision.
Change-Id: I612f1a80cf160e7eebb2b48f0ddd1b8f9aaa81b1
Reviewed-on: https://chromium-review.googlesource.com/c/1475265
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
This makes 'checkout_chromeos' conditions visible to '--deps=all'
processing.
R=ehmaldonado@google.com
Bug: 928829
Change-Id: I1fe3d73117203d549c7039f7621eecadd3ac344e
Reviewed-on: https://chromium-review.googlesource.com/c/1454849
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Bug: 906114
Change-Id: Ib8614086f7c456c04d27f33c73809671ed073bf4
Reviewed-on: https://chromium-review.googlesource.com/c/1341862
Reviewed-by: Eric Boren <borenet@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
This is a reland of 5705acabe0
If a gclient config is not found, then built-in variables wont
be supported.
Original change's description:
> [gclient] Make getdep and setdep to provide builtin vars
>
> Bug: 906114
> Change-Id: I069cc21343911f7fdb3c91ecbd8fcba53fc8099f
> Reviewed-on: https://chromium-review.googlesource.com/c/1340461
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
> Commit-Queue: Eric Boren <borenet@chromium.org>
Bug: 906114
Change-Id: I72f30d10b5f0180fd5c616a42393f5b12055ce8e
Reviewed-on: https://chromium-review.googlesource.com/c/1341039
Reviewed-by: Eric Boren <borenet@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Bug: 906114
Change-Id: I069cc21343911f7fdb3c91ecbd8fcba53fc8099f
Reviewed-on: https://chromium-review.googlesource.com/c/1340461
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Eric Boren <borenet@chromium.org>
Make it possible to refer to built-in variables without having to declare
then in DEPS files.
Bug: None
Change-Id: I5403963052463befc074f29750de56cce13927ce
Reviewed-on: https://chromium-review.googlesource.com/c/1312234
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
This is a reland of 67ef3f67e8
We no longer call cipd unconditionally.
Original change's description:
> gclient: delete unversioned directory before adding cipd dep for the same path
>
> Bug: 882611
> Change-Id: I46e41cc9693b90874b5d6569a12ec638eaac1050
> Reviewed-on: https://chromium-review.googlesource.com/1228655
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Bug: 882611
Change-Id: I683bfc62bd1eebfec0853583f96f3981c2c6bdf2
Reviewed-on: https://chromium-review.googlesource.com/1232891
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
When a recursive dependency has use_relative_paths it also makes sense
to have the hooks working directory by the dependency's directory.
Otherwise if a hook uses one of the relative dependencies it is impossible
to know which path prefix to use.
However we cannot change the behavior of hooks with use_relative_paths
because it would break existing projects that use_relative_paths but
hardcoded the prefix for hooks. Instead we add a second boolean,
use_relative_hooks that triggers the behavior.
Adds tests for the new behavior and a test for existing interactio
between hooks and recursedeps.
BUG=chromium:875245
Change-Id: Ie4c526baa425ff887b3be54e0feca7c597ededec
Reviewed-on: https://chromium-review.googlesource.com/1213327
Commit-Queue: Corentin Wallez <cwallez@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Also add shortcuts, so we don't have to look through all the branches
if the patch is based on a common branch (i.e. master, infra/config, lkgr)
Bug: 870279
Change-Id: I625a8481dccac9a475b096b926e6fab7efe676b0
Reviewed-on: https://chromium-review.googlesource.com/1161094
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Edward Lemur
John Emau
Edward Lemur
Anthony Polito
Marc-Antoine Ruel
Bruce Dawson
James Darpinian
Raul Tambre
Henrique Ferreiro
Elly Fong-Jones
Aaron Gable
btolsch
Dirk Pranke
Nico Weber
Takuto Ikuta
Michael Moss
Sergiy Byelozyorov
Eric Boren
Andrii Shyshkalov
Corentin Wallez