Commit Graph

20 Commits (main)

Author SHA1 Message Date
Lei Zhang 664154a905 ninja: Improve depot_tools directory detection
Tweak the code in ninja.py so it would only skip directories in $PATH
that are named "depot_tools". The existing detection logic skips
"foo_depot_tools". Also rename functions in the file to follow Python
Style and auto-format the file.

Change-Id: I631958768168f0d673b824c8e14783d2c6f65563
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6259139
Reviewed-by: Junji Watanabe <jwata@google.com>
Commit-Queue: Lei Zhang <thestig@chromium.org>
2 weeks ago
Fumitoshi Ukai 356ef0324e ninja: error if trying to build for use_remoteexec=true
Change-Id: Ia32dd3cba1d1874401c6614f792f212b2cfa60a1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5660200
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Junji Watanabe <jwata@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
8 months ago
Andrew Grieve e1385296c4 Add "ninja -t targets" to the command allowlist
It's used here: https://source.chromium.org/chromium/chromium/src/+/main:build/android/list_java_targets.py;drc=911128ac37a470797e6e24251598e4d0f3d3c21c;l=82

Change-Id: I31c687d36393a36b3b7d75b4668720214011aa1f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5527347
Auto-Submit: Andrew Grieve <agrieve@chromium.org>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
10 months ago
Fumitoshi Ukai df437783c2 Reland "ninja.py check siso_marker"
This reverts ninja.py in commit 6ed0d7b86c

Allow `-t list`, `-t commands` and `-t inputs` even in siso out dir.

Bug: b/338414465, b/339320220
Change-Id: Ia8cdaf53680207625b485e6e326fbe4db8297b96
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5524253
Reviewed-by: Junji Watanabe <jwata@google.com>
Reviewed-by: Andrew Grieve <agrieve@chromium.org>
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
10 months ago
Andrew Grieve 6ed0d7b86c Revert "Reland "siso.py: check ninja marker""
This reverts commit c9fb389f4b.

Reason for revert: Breaks ability to do "ninja -t commands"

Original change's description:
> Reland "siso.py: check ninja marker"
>
> This reverts commit b1c6497e19.
>
> Reason for revert: Fix var name
>
> ```
> ukai@ukai ...src/chromium/src % ~/src/depot_tools/depot_tools/siso ninja -C out/ninja
> depot_tools/siso.py: Using Siso binary from SISO_PATH: /usr/local/google/home/ukai/go/bin/siso.
> depot_tools/siso.py: out/ninja contains Ninja state file.
> Use `autoninja` to use reclient,
> or run `gn clean out/ninja` to switch from ninja to siso
>
> ukai@ukai ...src/chromium/src % ~/src/depot_tools/depot_tools/siso ninja -C out/siso base
> depot_tools/siso.py: Using Siso binary from SISO_PATH: /usr/local/google/home/ukai/go/bin/siso.
>  4.63s init credentials
> reapi instance: projects/rbe-chrome-untrusted/instances/default_instance
>  1.63s loading fs state
>  3.42s load build.ninja
> 15.32s Regenerating ninja files
>  3.64s reloading
> build finished
> local:3 remote:0 cache:0 fallback:0 skip:3026
> reapi: ops: 17(err:1) / r:0(err:0) 0B / w:0(err:0) 0B
> fs: ops: 12092(err:192) / r:350(err:0) 189.04MiB / w:1(err:0) 13.96KiB
> 27.29s Build Succeeded: 3 steps - 0.11/s
>
> ```
>
> Original change's description:
> > Revert "siso.py: check ninja marker"
> >
> > This reverts commit 41bbfb8b90.
> >
> > Reason for revert:
> > ukai@ukai ...src/chromium/src % siso ninja -C out/siso
> > depot_tools/siso.py: Using Siso binary from SISO_PATH: /usr/local/google/home/ukai/go/bin/siso.
> > Traceback (most recent call last):
> >   File "/usr/local/google/home/ukai/depot_tools/siso.py", line 115, in <module>
> >     sys.exit(main(sys.argv))
> >              ^^^^^^^^^^^^^^
> >   File "/usr/local/google/home/ukai/depot_tools/siso.py", line 104, in main
> >     checkOutdir(args[1:])
> >   File "/usr/local/google/home/ukai/depot_tools/siso.py", line 21, in checkOutdir
> >     for i, arg in enumerate(ninja_args):
> >                             ^^^^^^^^^^
> > NameError: name 'ninja_args' is not defined. Did you mean: 'ninja_marker'?
> >
> >
> > Original change's description:
> > > siso.py: check ninja marker
> > >
> > > If out dir was built by Ninja, don't build by Siso.
> > >
> > > (still allow `siso query` etc)
> > >
> > > Bug: b/338414465
> > > Change-Id: I48f8fbb5e83714fd3d27ff3a2bfffe929e4ef121
> > > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5518396
> > > Reviewed-by: Junji Watanabe <jwata@google.com>
> > > Commit-Queue: Junji Watanabe <jwata@google.com>
> > > Auto-Submit: Fumitoshi Ukai <ukai@google.com>
> > > Commit-Queue: Fumitoshi Ukai <ukai@google.com>
> >
> > Bug: b/338414465
> > Change-Id: Ie69ca1b6c1d3dc47475d9356beab29a917356479
> > No-Presubmit: true
> > No-Tree-Checks: true
> > No-Try: true
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5514902
> > Reviewed-by: Philipp Wollermann <philwo@chromium.org>
> > Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
> > Commit-Queue: Fumitoshi Ukai <ukai@google.com>
>
> Bug: b/338414465
> Change-Id: Idb18ad9191a062f83b538c1e2a76ee77200c4df0
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5518397
> Reviewed-by: Philipp Wollermann <philwo@chromium.org>
> Auto-Submit: Fumitoshi Ukai <ukai@google.com>
> Commit-Queue: Fumitoshi Ukai <ukai@google.com>

Bug: b/338414465
Change-Id: I0a24db3675762b92e96ab37b629fa12b4af89240
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5522633
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Auto-Submit: Andrew Grieve <agrieve@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
10 months ago
Fumitoshi Ukai 6247b7d939 ninja.py: check siso_marker
If out dir was built by siso, don't build by ninja

Bug: b/338414465
Change-Id: I121beaa6c17409cc7a4b4535e158ed89c47cac89
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5514899
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Junji Watanabe <jwata@google.com>
Reviewed-by: Junji Watanabe <jwata@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
10 months ago
Takuto Ikuta df3e577855 apply black and `git cl format` for ninja related python files
This is made by
$ black --line-length 79 *ninja*.py
$ git cl format

Change-Id: Ic446898a5461ae536542f6312cae2ce126dfe82a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5035265
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Junji Watanabe <jwata@google.com>
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Philipp Wollermann <philwo@chromium.org>
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
1 year ago
Junji Watanabe c5505889cf [gn] Fallback GN in PATH
This allows us to call GN outside of gclient checkout.
e.g PATH=$PATH:~/chromium/src/buildtools/linux64 gn

Change-Id: Ib4b6d09fbd4d3a12ea1407fcea0eb5494c808fb5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4886535
Auto-Submit: Junji Watanabe <jwata@google.com>
Commit-Queue: Junji Watanabe <jwata@google.com>
Reviewed-by: Daniel Cheng <dcheng@chromium.org>
1 year ago
Mike Frysinger 124bb8e53c switch to 4 space indent
Leave the recipes/ code at 2 space to match the rest of the recipes
project in other repos.

Reformatted using:
files=( $(
	git ls-tree -r --name-only HEAD | \
		grep -Ev -e '^(third_party|recipes)/' | \
		grep '\.py$';
	git grep -l '#!/usr/bin/env.*python' | grep -v '\.py$'
) )
parallel ./yapf -i -- "${files[@]}"
~/chromiumos/chromite/contrib/reflow_overlong_comments "${files[@]}"

The files that still had strings that were too long were manually
reformatted because they were easy and only a few issues.
autoninja.py
clang_format.py
download_from_google_storage.py
fix_encoding.py
gclient_utils.py
git_cache.py
git_common.py
git_map_branches.py
git_reparent_branch.py
gn.py
my_activity.py
owners_finder.py
presubmit_canned_checks.py
reclient_helper.py
reclientreport.py
roll_dep.py
rustfmt.py
siso.py
split_cl.py
subcommand.py
subprocess2.py
swift_format.py
upload_to_google_storage.py

These files still had lines (strings) that were too long, so the pylint
warnings were suppressed with a TODO.
auth.py
gclient.py
gclient_eval.py
gclient_paths.py
gclient_scm.py
gerrit_util.py
git_cl.py
presubmit_canned_checks.py
presubmit_support.py
scm.py

Change-Id: Ia6535c4f2c48d46b589ec1e791dde6c6b2ea858f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4836379
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Auto-Submit: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Wan-Teh Chang ef6c306221 Fix a spelling error in a comment
The comment was originally added in
https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4149962

Based on the commit message, I guess "those", not "these", was the
intended word.

Bug: 1403485
Change-Id: If3a72486b4d15a099ce1c2c4835d23a364c1e768
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4392153
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Nico Weber <thakis@chromium.org>
Reviewed-by: Junji Watanabe <jwata@google.com>
Commit-Queue: Wan-Teh Chang <wtc@google.com>
2 years ago
Wan-Teh Chang badcbc5268 Minor edits of error message in fallback()
The error message was originally added in
https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4194897

Bug: 1340825
Change-Id: I6415fc81113f0d876067bd3e3e28f8c35cd164d9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4392152
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Commit-Queue: Wan-Teh Chang <wtc@google.com>
Reviewed-by: Junji Watanabe <jwata@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2 years ago
Edman Anjos 543fdf32d1 Fix typo in ninja.py: oterweise -> otherwise
Bug: N/A
Change-Id: I04e9a60ff2047ae8d77e2504505aef3cf22e6050
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4295213
Auto-Submit: Edman Anjos <edman@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Loic Sharma e3fe027009 [Windows] Ignore extra spaces in ninja args
The `ninja.py` script fails for this invocation:

```
ninja -C C:\Code\f\engine\src\out\host_debug_unopt\
 flutter_windows_unittests.exe
```

However, this works as expected:

```
ninja.exe -C C:\Code\f\engine\src\out\host_debug_unopt\
 flutter_windows_unittests.exe
```

Notice how there are extra spaces between the DIR and target. This bug breaks the Flutter engine's build integration with Visual Studio after `ninja.exe` was removed from depot_tools.

Here is an example MSBuild target that runs into this issue:

```xml
  <Target Name="Build">
    <Exec Command="call ninja -C $(OutDir)  flutter_windows_unittests.exe" />
  </Target>
```

Change-Id: Ib0022bc48151de438ac209fd07e0183872538ac6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4215121
Reviewed-by: Junji Watanabe <jwata@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Loïc Sharma <loicsharma@google.com>
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
2 years ago
Shelley Vohr 106754b269 Account for managed solution in src when finding ninja
At present when looking for ninja, main only searches in two locations, gclient root and src. The Electron project lists Chromium’s src as a dependency in its DEPS file, so Ninja won’t be in the primary solution’s path (as it’s a transitive dependency, not a direct dependency of Chromium). This change adapts existing logic to account for that case.

Change-Id: I35aea29135d5968722a308bfb6dbb8a727550b42
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4203207
Commit-Queue: Junji Watanabe <jwata@google.com>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
2 years ago
Junji Watanabe 8f41177abf Remove legacy ninja binaries
PSA has already been sent to internal/external devs.
Internal: http://g/chromium-dev-internal/n2IL0Mbh0E8
External: https://groups.google.com/a/chromium.org/g/chromium-dev/c/027jM6DLkIk

Bug: 1340825
Change-Id: I9f3b4b2cb20147fd48ba379b257c7280ba9c53f8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4194897
Commit-Queue: Junji Watanabe <jwata@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2 years ago
Junji Watanabe 3e7206ff0a Remove CPATH,LIBRARY_PATH,SDKROOT env vars set by macOS python3
macOS python3 sets those env vars implicitly. https://openradar.appspot.com/radar?id=5608755232243712

Removing them to avoid changing clang's behaviors.

Fixed: 1403485
Change-Id: I2ecf0cad8a1f69f5d7d180d2a2a003ce97365f3f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4149962
Reviewed-by: Nico Weber <thakis@chromium.org>
Commit-Queue: Nico Weber <thakis@chromium.org>
Auto-Submit: Junji Watanabe <jwata@google.com>
2 years ago
Junji Watanabe 2b1aa8dcab Add single file compile support for Windows in ninja wrappers
ninja.bat can't pass '^' for single file compile.
This change fixes ninja.bat to apply double quotas to args to preserve "^".
Note that ninja.bat still needs "^^" since single "^" is already removed  as an escape character.
This is the same implementation with autoninja.{bat, py}.

Test: http://go/paste/6182226814304256

Bug: 1340825
Change-Id: I183c7890634a77471a617c4cc2b9c8c33bc8487d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4060603
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
Auto-Submit: Junji Watanabe <jwata@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Junji Watanabe <jwata@google.com>
2 years ago
Junji Watanabe 22bbba13ce Do not show python stack trace in ninja.py when KeyboardInterrupt happens
Before
```
❯ ninja -C out/Default
ninja: Entering directory `out/Default'
Traceback (most recent call last):
  File "/Users/jwata/depot_tools/ninja.py", line 85, in <module>
    sys.exit(main(sys.argv))
  File "/Users/jwata/depot_tools/ninja.py", line 79, in main
    return subprocess.call([ninja_path] + args[1:])
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/subprocess.py", line 351, in call
    return p.wait(timeout=timeout)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/subprocess.py", line 1189, in wait
    return self._wait(timeout=timeout)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/subprocess.py", line 1917, in _wait
    (pid, sts) = self._try_wait(0)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/subprocess.py", line 1875, in _try_wait
    (pid, sts) = os.waitpid(self.pid, wait_flags)
KeyboardInterrupt
```

After
```
❯ ninja -C out/Default
ninja: Entering directory `out/Default'
interrupted
```

Fixed: 1398687
Change-Id: I7cbc80f2214d46b492c6e9ade7b0d78766a8a6f7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4085366
Reviewed-by: Nico Weber <thakis@chromium.org>
Auto-Submit: Junji Watanabe <jwata@google.com>
Commit-Queue: Junji Watanabe <jwata@google.com>
2 years ago
Brian Osman a0cf432171 Tweak ninja fallback logic
In Skia (for example), GetPrimarySolutionPath works, but FindGclientRoot
does not (users tend not to use gclient to sync DEPS, etc).

Without this change, we'd get to the loop. If a user had pulled ninja to
third_party/ninja/ninja, then it would work just fine. If they hadn't,
python would throw an exception on the os.path.join line, because we'd
be passing a NoneType as the first argument.

Bug: chromium:1340825
Change-Id: Iab86450c5b6f00bc9c2716b9504debc9a56d9bb1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4034942
Commit-Queue: Brian Osman <brianosman@google.com>
Reviewed-by: Nico Weber <thakis@chromium.org>
2 years ago
Junji Watanabe 1f67d5573f Add ninja wrappers to trigger DEPS ninja
This is the first step to deprecate ninja binaries in depot_tools.

depot_tools/{ninja, ninja.bat} will call ninja.py, which finds CIPD ninja under src/third_party in the current gclient source tree. https://crrev.com/c/3869740

If it fails to find, it does fallback to an existing ninja binary in depot_tools, until removing the references to the legacy ninja binaries.

# For those who see the deprecation message,
You need to install ninja using DEPS at your project. (https://crrev.com/c/3869740 is an example.)

If your project doesn't have DEPS, you need to install ninja and include it in PATH manually. e.g. sudo apt install ninja-build

Test:
Linux builder: https://ci.chromium.org/swarming/task/5e9cfc917aa3f110?server=chromium-swarm.appspot.com
Windows builder: https://ci.chromium.org/swarming/task/5e9d48885c460b10?server=chromium-swarm.appspot.com

Bug: 1340825
Change-Id: I70d1863d72ddfa65b87a62c9bde8ff63f7641d13
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3925341
Reviewed-by: Nico Weber <thakis@chromium.org>
Commit-Queue: Junji Watanabe <jwata@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
2 years ago