Commit Graph

17 Commits (main)

Author SHA1 Message Date
Mohamed Heikal 9b4d1e485d Integrate autoninja.py with fast_local_dev_server.py
- Starts the build server when the build starts.
- Writes tty filename to env variable.
- Tells the build server about the current build so it does not exit
  until autoninja does, even if idle.
- Cancels pending tasks on Ctrl+c.

Change-Id: I86bb9852bd0975f381b049b9ff21c38eef7cef9d
Bug: 370589852
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5917985
Reviewed-by: Junji Watanabe <jwata@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
Auto-Submit: Mohamed Heikal <mheikal@chromium.org>
Commit-Queue: Mohamed Heikal <mheikal@chromium.org>
3 months ago
Fumitoshi Ukai f5e1092339 siso: print error if siso binary doesn't exist in third_party/siso
If build/config/siso/.sisoenv exists, but third_party/siso/cipd/siso
doesn't exist, it prints error

depot_tools/siso.py: Could not find .sisoenv under build/config/siso of the current project. Did you run gclient sync?

But it should print error as siso binary not in third_party/siso
instead.

Change-Id: I6a6a920b0e6d877551f0fa7c650ab7c8165180bf
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5802659
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
6 months ago
Fumitoshi Ukai 06ab4cbb26 siso: call checkOutdir to prevent from running siso in ninja's out
checkOutdir should be called before run `siso ninja`
to not run `siso ninja` where `ninja` is used.

Change-Id: I55453b21d35396b59924be5398b47f67b592eca1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5802658
Reviewed-by: Philipp Wollermann <philwo@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
6 months ago
Fumitoshi Ukai 4eb5cc887d siso: allow run $SISO_PATH even if .sisoenv not found
When $SISO_PATH is set, allow to run it even if .sisoenv not found.
Useful for `siso fetch`, `siso recall` etc, which would not be
needed to run in chromium checkout dir.

Change-Id: If3bfcf879cc67dc41bedf17cd771c94de836c949
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5804427
Commit-Queue: Richard Wang <richardwa@google.com>
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
Reviewed-by: Philipp Wollermann <philwo@google.com>
Reviewed-by: Richard Wang <richardwa@google.com>
6 months ago
Josip Sokcevic 43c6415bce Add a new possible siso path
We are relocating siso binary from third_party/siso/siso to
third_party/siso/cipd/siso to make it compatible with Cog, which
requires cipd packages to be placed in directories with no Git content.

Bug: 346837573
Change-Id: If40e6033e341dcedfee5670f07d68c14bd923f7b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5625091
Reviewed-by: Philipp Wollermann <philwo@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
9 months ago
Fumitoshi Ukai 951578030e siso: set PYTHONDONTWRITEBYTECODE=1 if not set
during build, lots of python scripts run and generates
lots of pycache *.pyc in workspace.
To keep workspace clean, set PYTHONDONTWRITEBYTECODE=1
in siso.py

Change-Id: I725c6745c44410eef4a780b8ffc30fdafd899334
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5606824
Reviewed-by: Philipp Wollermann <philwo@google.com>
Reviewed-by: Erik Staab <estaab@chromium.org>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
9 months ago
Fumitoshi Ukai 08d5eb6d68 Reland "siso.py check ninja marker"
This reverts siso.py in commit 6ed0d7b86c
siso.py doesn't affect `ninja -t commands`

If out dir was built by Ninja, don't build by Siso.
(still allow `siso query` etc)

Bug: b/338414465
Change-Id: I3b317ca40153cee3987390d8b62ac070b6f85243
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5524213
Commit-Queue: Andrew Grieve <agrieve@chromium.org>
Reviewed-by: Andrew Grieve <agrieve@chromium.org>
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
10 months ago
Andrew Grieve 6ed0d7b86c Revert "Reland "siso.py: check ninja marker""
This reverts commit c9fb389f4b.

Reason for revert: Breaks ability to do "ninja -t commands"

Original change's description:
> Reland "siso.py: check ninja marker"
>
> This reverts commit b1c6497e19.
>
> Reason for revert: Fix var name
>
> ```
> ukai@ukai ...src/chromium/src % ~/src/depot_tools/depot_tools/siso ninja -C out/ninja
> depot_tools/siso.py: Using Siso binary from SISO_PATH: /usr/local/google/home/ukai/go/bin/siso.
> depot_tools/siso.py: out/ninja contains Ninja state file.
> Use `autoninja` to use reclient,
> or run `gn clean out/ninja` to switch from ninja to siso
>
> ukai@ukai ...src/chromium/src % ~/src/depot_tools/depot_tools/siso ninja -C out/siso base
> depot_tools/siso.py: Using Siso binary from SISO_PATH: /usr/local/google/home/ukai/go/bin/siso.
>  4.63s init credentials
> reapi instance: projects/rbe-chrome-untrusted/instances/default_instance
>  1.63s loading fs state
>  3.42s load build.ninja
> 15.32s Regenerating ninja files
>  3.64s reloading
> build finished
> local:3 remote:0 cache:0 fallback:0 skip:3026
> reapi: ops: 17(err:1) / r:0(err:0) 0B / w:0(err:0) 0B
> fs: ops: 12092(err:192) / r:350(err:0) 189.04MiB / w:1(err:0) 13.96KiB
> 27.29s Build Succeeded: 3 steps - 0.11/s
>
> ```
>
> Original change's description:
> > Revert "siso.py: check ninja marker"
> >
> > This reverts commit 41bbfb8b90.
> >
> > Reason for revert:
> > ukai@ukai ...src/chromium/src % siso ninja -C out/siso
> > depot_tools/siso.py: Using Siso binary from SISO_PATH: /usr/local/google/home/ukai/go/bin/siso.
> > Traceback (most recent call last):
> >   File "/usr/local/google/home/ukai/depot_tools/siso.py", line 115, in <module>
> >     sys.exit(main(sys.argv))
> >              ^^^^^^^^^^^^^^
> >   File "/usr/local/google/home/ukai/depot_tools/siso.py", line 104, in main
> >     checkOutdir(args[1:])
> >   File "/usr/local/google/home/ukai/depot_tools/siso.py", line 21, in checkOutdir
> >     for i, arg in enumerate(ninja_args):
> >                             ^^^^^^^^^^
> > NameError: name 'ninja_args' is not defined. Did you mean: 'ninja_marker'?
> >
> >
> > Original change's description:
> > > siso.py: check ninja marker
> > >
> > > If out dir was built by Ninja, don't build by Siso.
> > >
> > > (still allow `siso query` etc)
> > >
> > > Bug: b/338414465
> > > Change-Id: I48f8fbb5e83714fd3d27ff3a2bfffe929e4ef121
> > > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5518396
> > > Reviewed-by: Junji Watanabe <jwata@google.com>
> > > Commit-Queue: Junji Watanabe <jwata@google.com>
> > > Auto-Submit: Fumitoshi Ukai <ukai@google.com>
> > > Commit-Queue: Fumitoshi Ukai <ukai@google.com>
> >
> > Bug: b/338414465
> > Change-Id: Ie69ca1b6c1d3dc47475d9356beab29a917356479
> > No-Presubmit: true
> > No-Tree-Checks: true
> > No-Try: true
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5514902
> > Reviewed-by: Philipp Wollermann <philwo@chromium.org>
> > Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
> > Commit-Queue: Fumitoshi Ukai <ukai@google.com>
>
> Bug: b/338414465
> Change-Id: Idb18ad9191a062f83b538c1e2a76ee77200c4df0
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5518397
> Reviewed-by: Philipp Wollermann <philwo@chromium.org>
> Auto-Submit: Fumitoshi Ukai <ukai@google.com>
> Commit-Queue: Fumitoshi Ukai <ukai@google.com>

Bug: b/338414465
Change-Id: I0a24db3675762b92e96ab37b629fa12b4af89240
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5522633
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Auto-Submit: Andrew Grieve <agrieve@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
10 months ago
Fumitoshi Ukai c9fb389f4b Reland "siso.py: check ninja marker"
This reverts commit b1c6497e19.

Reason for revert: Fix var name

```
ukai@ukai ...src/chromium/src % ~/src/depot_tools/depot_tools/siso ninja -C out/ninja
depot_tools/siso.py: Using Siso binary from SISO_PATH: /usr/local/google/home/ukai/go/bin/siso.
depot_tools/siso.py: out/ninja contains Ninja state file.
Use `autoninja` to use reclient,
or run `gn clean out/ninja` to switch from ninja to siso

ukai@ukai ...src/chromium/src % ~/src/depot_tools/depot_tools/siso ninja -C out/siso base
depot_tools/siso.py: Using Siso binary from SISO_PATH: /usr/local/google/home/ukai/go/bin/siso.
 4.63s init credentials
reapi instance: projects/rbe-chrome-untrusted/instances/default_instance
 1.63s loading fs state
 3.42s load build.ninja
15.32s Regenerating ninja files
 3.64s reloading
build finished
local:3 remote:0 cache:0 fallback:0 skip:3026
reapi: ops: 17(err:1) / r:0(err:0) 0B / w:0(err:0) 0B
fs: ops: 12092(err:192) / r:350(err:0) 189.04MiB / w:1(err:0) 13.96KiB
27.29s Build Succeeded: 3 steps - 0.11/s

```

Original change's description:
> Revert "siso.py: check ninja marker"
>
> This reverts commit 41bbfb8b90.
>
> Reason for revert:
> ukai@ukai ...src/chromium/src % siso ninja -C out/siso
> depot_tools/siso.py: Using Siso binary from SISO_PATH: /usr/local/google/home/ukai/go/bin/siso.
> Traceback (most recent call last):
>   File "/usr/local/google/home/ukai/depot_tools/siso.py", line 115, in <module>
>     sys.exit(main(sys.argv))
>              ^^^^^^^^^^^^^^
>   File "/usr/local/google/home/ukai/depot_tools/siso.py", line 104, in main
>     checkOutdir(args[1:])
>   File "/usr/local/google/home/ukai/depot_tools/siso.py", line 21, in checkOutdir
>     for i, arg in enumerate(ninja_args):
>                             ^^^^^^^^^^
> NameError: name 'ninja_args' is not defined. Did you mean: 'ninja_marker'?
>
>
> Original change's description:
> > siso.py: check ninja marker
> >
> > If out dir was built by Ninja, don't build by Siso.
> >
> > (still allow `siso query` etc)
> >
> > Bug: b/338414465
> > Change-Id: I48f8fbb5e83714fd3d27ff3a2bfffe929e4ef121
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5518396
> > Reviewed-by: Junji Watanabe <jwata@google.com>
> > Commit-Queue: Junji Watanabe <jwata@google.com>
> > Auto-Submit: Fumitoshi Ukai <ukai@google.com>
> > Commit-Queue: Fumitoshi Ukai <ukai@google.com>
>
> Bug: b/338414465
> Change-Id: Ie69ca1b6c1d3dc47475d9356beab29a917356479
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5514902
> Reviewed-by: Philipp Wollermann <philwo@chromium.org>
> Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
> Commit-Queue: Fumitoshi Ukai <ukai@google.com>

Bug: b/338414465
Change-Id: Idb18ad9191a062f83b538c1e2a76ee77200c4df0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5518397
Reviewed-by: Philipp Wollermann <philwo@chromium.org>
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
10 months ago
Fumitoshi Ukai b1c6497e19 Revert "siso.py: check ninja marker"
This reverts commit 41bbfb8b90.

Reason for revert: 
ukai@ukai ...src/chromium/src % siso ninja -C out/siso
depot_tools/siso.py: Using Siso binary from SISO_PATH: /usr/local/google/home/ukai/go/bin/siso.
Traceback (most recent call last):
  File "/usr/local/google/home/ukai/depot_tools/siso.py", line 115, in <module>
    sys.exit(main(sys.argv))
             ^^^^^^^^^^^^^^
  File "/usr/local/google/home/ukai/depot_tools/siso.py", line 104, in main
    checkOutdir(args[1:])
  File "/usr/local/google/home/ukai/depot_tools/siso.py", line 21, in checkOutdir   
    for i, arg in enumerate(ninja_args):
                            ^^^^^^^^^^
NameError: name 'ninja_args' is not defined. Did you mean: 'ninja_marker'?


Original change's description:
> siso.py: check ninja marker
>
> If out dir was built by Ninja, don't build by Siso.
>
> (still allow `siso query` etc)
>
> Bug: b/338414465
> Change-Id: I48f8fbb5e83714fd3d27ff3a2bfffe929e4ef121
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5518396
> Reviewed-by: Junji Watanabe <jwata@google.com>
> Commit-Queue: Junji Watanabe <jwata@google.com>
> Auto-Submit: Fumitoshi Ukai <ukai@google.com>
> Commit-Queue: Fumitoshi Ukai <ukai@google.com>

Bug: b/338414465
Change-Id: Ie69ca1b6c1d3dc47475d9356beab29a917356479
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5514902
Reviewed-by: Philipp Wollermann <philwo@chromium.org>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
10 months ago
Fumitoshi Ukai 41bbfb8b90 siso.py: check ninja marker
If out dir was built by Ninja, don't build by Siso.

(still allow `siso query` etc)

Bug: b/338414465
Change-Id: I48f8fbb5e83714fd3d27ff3a2bfffe929e4ef121
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5518396
Reviewed-by: Junji Watanabe <jwata@google.com>
Commit-Queue: Junji Watanabe <jwata@google.com>
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
10 months ago
Fumitoshi Ukai 9f668bbff9 siso: use stderr to print message from siso.py
siso binary will print json etc (e.g. `siso fs export`)
and usually used to pipe to jq or so.

but siso.py print message when $SISO_PATH is set
and corrupt json message.

use stderr for such message.

Change-Id: Ic35ecabc159fb2053a8a29816c276d914f8d2468
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5197566
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
1 year ago
Junji Watanabe 9ed72f91a9 [siso] Propagte SIGINT to siso process
siso's signal handler doesn't run as expected with siso/autosiso
wrappers.
This CL sets signal handlers for the Python wrappers to propagate
signals to siso.

Note that Siso immediately exists at the 2nd Ctrl-C.
So developers can stop Siso forcibly when they want.

Bug: b/308734805
Change-Id: Idd8cc1d99931d609cfc3bedfb4c0bb4319cd20b3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4996768
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Junji Watanabe <jwata@google.com>
1 year ago
Mike Frysinger 124bb8e53c switch to 4 space indent
Leave the recipes/ code at 2 space to match the rest of the recipes
project in other repos.

Reformatted using:
files=( $(
	git ls-tree -r --name-only HEAD | \
		grep -Ev -e '^(third_party|recipes)/' | \
		grep '\.py$';
	git grep -l '#!/usr/bin/env.*python' | grep -v '\.py$'
) )
parallel ./yapf -i -- "${files[@]}"
~/chromiumos/chromite/contrib/reflow_overlong_comments "${files[@]}"

The files that still had strings that were too long were manually
reformatted because they were easy and only a few issues.
autoninja.py
clang_format.py
download_from_google_storage.py
fix_encoding.py
gclient_utils.py
git_cache.py
git_common.py
git_map_branches.py
git_reparent_branch.py
gn.py
my_activity.py
owners_finder.py
presubmit_canned_checks.py
reclient_helper.py
reclientreport.py
roll_dep.py
rustfmt.py
siso.py
split_cl.py
subcommand.py
subprocess2.py
swift_format.py
upload_to_google_storage.py

These files still had lines (strings) that were too long, so the pylint
warnings were suppressed with a TODO.
auth.py
gclient.py
gclient_eval.py
gclient_paths.py
gclient_scm.py
gerrit_util.py
git_cl.py
presubmit_canned_checks.py
presubmit_support.py
scm.py

Change-Id: Ia6535c4f2c48d46b589ec1e791dde6c6b2ea858f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4836379
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Auto-Submit: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Junji Watanabe 4480f863c5 [siso] check presence of .sisoenv file
`set([primary_solution_path, gclient_root_path, gclient_src_root_path])` doesn't guarantee order.
This is problematic when `SISO_PATH` is used because it doesn't check the presence of `third_party/siso/siso` binary.

This CL changes siso.py to check the presence of `.sisoenv` to detect the correct `base_dir`.

Change-Id: Ieb1acc44269037d43c793eb28cd2f12ad0179b17
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4625315
Auto-Submit: Junji Watanabe <jwata@google.com>
Commit-Queue: Junji Watanabe <jwata@google.com>
Reviewed-by: Philipp Wollermann <philwo@google.com>
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
2 years ago
Richard Wang 4992184cd1 [autosiso] add SISO_PATH env var
Bug: b/287895730
Change-Id: Ibbf658f802d46a14fc82a622f7f853dcadc25e5f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4624217
Reviewed-by: Philipp Wollermann <philwo@google.com>
Commit-Queue: Richard Wang <richardwa@google.com>
2 years ago
Fumitoshi Ukai 3ca8d0d0ca add siso wrapper
Bug: b/277855544, b/277859568
Change-Id: I520834647881745db7a42fd357bc829a7c6779d8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4413517
Reviewed-by: Philipp Wollermann <philwo@google.com>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Junji Watanabe <jwata@google.com>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Junji Watanabe <jwata@google.com>
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
2 years ago