Commit Graph

24 Commits (00790d319fb17066fdd85238e93d42ba4c2f18f2)

Author SHA1 Message Date
Peter Wen 4ee32ce4cb [ninjalog uploader] Collect more android GN args
Chrome on Android is going to use android_static_analysis to control
how static analysis runs (values can be: on, off, or build_server). This
arg matters a great deal for build performance.

Bug: 1202732
Change-Id: I0f7437452605f59c9bd01ff75aba56274f115a5b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3427859
Auto-Submit: Peter Wen <wnwen@chromium.org>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
3 years ago
Takuto Ikuta 98cf932814 ninjalog_uploader: do not redirect stderr
Fixed: 1292542
Change-Id: Ia312eb7e4e3fd8d4c2b0fdaa10b7cc0290d38b93
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3428659
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
3 years ago
Takuto Ikuta 18a2844b53 ninjalog_uploader: do not use capture_output and text
capture_output and text args are not available in Python3.6.

Bug: 1290210
Change-Id: Id3c70ad456c5ca56df9935d64229f599d7aae4b5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3406721
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: David Munro <davidmunro@google.com>
Commit-Queue: David Munro <davidmunro@google.com>
3 years ago
Takuto Ikuta 3dc0172a6b fix ninjalog_uploader.py
This is to fix https://crbug.com/1288639#c5

Bug: 1288639
Change-Id: Ife224fb5d5fe4faa4a78785828d4fa488fda3c6e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3403475
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
3 years ago
Takuto Ikuta a657331e90 ninjalog_uploader: use goma_auth to detect googler
Bug: 1288639
Change-Id: I447e2f66603ffb8d68599dcf22023fd7857dc4fd
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3400398
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
3 years ago
Victor Hugo Vianna Silva 787f2f0eb5 [ninjalog uploader] Collect more performance-relevant GN args
Collect the Android args from the "Faster Edit/Deploy" section of the
build instructions [1], as well as blink_symbol_level. A TODO about
collecing enable_js_type_check is deleted, since that argument will
disappear in the near future as Chrome migrates to typescript.

[1] https://source.chromium.org/chromium/chromium/src/+/main:docs/android_build_instructions.md;l=364;drc=bbc1a1ee3eab2d5e4a5bf7c8b3b7e8d64f83e45d

R=tikuta@chromium.org

Bug: 1263441
Change-Id: I55dbc37d8543443d8767127523dfa8cb281fce6c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3245375
Auto-Submit: Victor Vianna <victorvianna@google.com>
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
3 years ago
Takuto Ikuta 4bb3a7d00d ninjalog_uploader: ignore http.client.RemoteDisconnected
Bug: b/190562569
Change-Id: Iaa8d119ad643e64bdf231fe5ae43ea4daa69996f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2947272
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
4 years ago
Takuto Ikuta 84e43fa2c4 ninjalog uploader: migrate to python3
This removes dependency to httplib2 by using urllib.
https://docs.python.org/3/library/http.client.html#httpresponse-objects

Use six for both py2/py3 unittest.

Bug: b/177465438
Change-Id: I48383559842615f97abea45b55ca0acf4d7c8bd3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2629087
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Yoshisato Yanagisawa <yyanagisawa@google.com>
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
4 years ago
Takuto Ikuta c603339365 autoninja: use pure python for ninjalog_uploader_wrapper.py
I'll update to python3 separately.

Bug: b/177465438
Change-Id: Id204789f2d0854a796e5408597700d90dc8cd191
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2626863
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
Reviewed-by: Yoshisato Yanagisawa <yyanagisawa@google.com>
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
4 years ago
Gavin Mak e6a623392d [depot_tools] Use logging.warning
logging.warn is deprecated and should be replaced with logging.warning.

Bug:1097402
Change-Id: I55ce007bec47608eeafcf3cd9ac2b90f60073115
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2575621
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
4 years ago
Bruce Dawson 9e633036da Fix reference to enable_js_type_check
A TODO in ninjalog_uploader.py referred to the closure_compile gn arg
which was recently renamed (as of crrev.com/c/2248564) to
enable_js_type_check. This updates it (and adds a missing space and a
bug number). This is purely a comment fix.

Change-Id: I117fa7910663491fd7c6e017fe98e2a977a75f13
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2336176
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Auto-Submit: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
5 years ago
Takuto Ikuta 6c7b829e54 remove reference to jumbo build
Bug: 994387
Change-Id: I0c675bb6aca04d64593de420c7936cb88ff6a051
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2334503
Commit-Queue: Dirk Pranke <dpranke@google.com>
Reviewed-by: Dirk Pranke <dpranke@google.com>
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
5 years ago
Takuto Ikuta a2e91dbb61 apply yapf for ninjalog*.py
This is preparation to fix the issue.

Bug: 992282
Change-Id: I64b237670bf460ff765df3eaf069b0e0bdcfb31d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2237268
Commit-Queue: Ye Kuang <yekuang@google.com>
Reviewed-by: Ye Kuang <yekuang@google.com>
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
5 years ago
Edward Lemur 59a3b2fd5d Run autoninja scripts using vpython.
Bug: 1002153
Change-Id: I769a5370d726f4ce478035f59bfd1df50fef452e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1992315
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lesmes 1961043388 Revert "Move httplib2 import above third_party."
This reverts commit cd1331ea69.

Reason for revert:
Original CL was reverted.

Original change's description:
> Move httplib2 import above third_party.
> 
> 7fd6347be3 removed the third_party/httplib2 files, but left the pyc files. This is breaking CQ for nest.
> 
> move httplib2 import above adding third_party to the path in ninjalog_uploader_wrapper to fix.
> 
> Change-Id: Id28f68ce7e3eb94783bdd5ee7ce15ae5930fde7e
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1988690
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
> Commit-Queue: Mark Orchard <morchard@google.com>

TBR=ehmaldonado@webrtc.org,ehmaldonado@chromium.org,morchard@google.com,apolito@google.com

Change-Id: Ic7775cdb4abae1855358711335349fc39eb37dfb
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1988739
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Mark Orchard cd1331ea69 Move httplib2 import above third_party.
7fd6347be3 removed the third_party/httplib2 files, but left the pyc files. This is breaking CQ for nest.

move httplib2 import above adding third_party to the path in ninjalog_uploader_wrapper to fix.

Change-Id: Id28f68ce7e3eb94783bdd5ee7ce15ae5930fde7e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1988690
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Mark Orchard <morchard@google.com>
5 years ago
Quinten Yearsley 3841eaa164 Remove unused imports in depot_tools/*.py
I know that sometimes imports can have side-effects,
so unused imports shouldn't always be removed, but these
ones look like they could be.

Change-Id: Iea9f82afa99b0ea35f29a28f20ce0493b579cfee
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1819860
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
6 years ago
Takuto Ikuta 5c00a22f22 [ninjalog_uploader] add path for httplib2
This is to let httplib2 import its sock.py in __init__.py correctly.
This is pointed out in
https://groups.google.com/a/chromium.org/d/msg/chromium-dev/8BF_cywS9JY/8g-HHmuqEQAJ

Also removed an unnecessary httplib2 import.

Change-Id: Iad0377ad282acd9e1e222890f522e4e62086dcaf
Reviewed-on: https://chromium-review.googlesource.com/c/1475255
Reviewed-by: Yoshisato Yanagisawa <yyanagisawa@google.com>
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
6 years ago
Takuto Ikuta 36248fcd63 [ninjalog] don't upload ninja log of no-op build
Check timestamp of .ninja_log before upload and don't upload if mtime is very old.

Bug: 900161
Change-Id: I0cf00ae29e23a9f609564e4c12b4b518320ca23a
Reviewed-on: https://chromium-review.googlesource.com/c/1404611
Commit-Queue: Shinya Kawanaka <shinyak@chromium.org>
Reviewed-by: Shinya Kawanaka <shinyak@chromium.org>
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
6 years ago
Takuto Ikuta c8069af594 [ninjalog] fully anonymize build stats before sending
We cannot send PII data.

This CL tried to remove following data,
* build config that can contain build directory path
* build directory path
* hostname
* cwd

I use per build uuid instead of per user uuid.


Bug: 900161
Change-Id: Id533762749806ad8616b7dc07f21b27dfe632c9a
Reviewed-on: https://chromium-review.googlesource.com/c/1369473
Reviewed-by: Shinya Kawanaka <shinyak@chromium.org>
Reviewed-by: Fumitoshi Ukai <ukai@chromium.org>
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
6 years ago
Takuto Ikuta cf56a4bfb0 [ninjalog_uploader] Add BuildTargetFromCommandLine function
This function extracts build target from command line.

Bug: 900161
Change-Id: I38d37689a42437de76933ea3c9afb9def5a1253c
Reviewed-on: https://chromium-review.googlesource.com/c/1381911
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Shinya Kawanaka <shinyak@chromium.org>
6 years ago
Takuto Ikuta 61cb9d6ba7 [ninjalog_uploader] add tests
Use unittest module instead of doctest.

Bug: 900161
Change-Id: I2e8265afb66345814c4c4b8e94618171b7aaaaf3
Reviewed-on: https://chromium-review.googlesource.com/c/1379441
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Shinya Kawanaka <shinyak@chromium.org>
6 years ago
Takuto Ikuta 96fdf7cbe6 [ninja] collect build configs
This is to know what build configs are popular among chromium developer.

Bug: 900161
Change-Id: Id07722e70fc407b3f8668080d49033da6e1a7f34
Reviewed-on: https://chromium-review.googlesource.com/c/1358034
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Shinya Kawanaka <shinyak@chromium.org>
Reviewed-by: Fumitoshi Ukai <ukai@chromium.org>
6 years ago
Takuto Ikuta 9af233a508 introduce ninjalog_uploader to autoninja
This CL introduces ninjalog_uploader to autoninja for posix environment.
This is to collect build statistics from googler to find user side build performance bottleneck.

ninjalog_uploader_wrapper is used to notify user and manage small config of upload script.

Bug: 900161
Change-Id: I48ac8cd1d52f64e8fdafaec43636d2d79ef9040d
Reviewed-on: https://chromium-review.googlesource.com/c/1345255
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Shinya Kawanaka <shinyak@chromium.org>
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
6 years ago