Commit Graph

36 Commits (main)

Author SHA1 Message Date
Allen Li 04a2e71a92 [metrics] Add new auth metric
Bug: b/358442026
Change-Id: If91494b52fe27177f68e42ec40977fd8ca1fd4cc
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5786166
Commit-Queue: Allen Li <ayatane@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
6 months ago
Richard Wang 6f180c0a23 Use $HOME/.config/depot_tools on linux for .cfg files
Or $XDG_CONFIG_HOME/.config/depot_tools if set

A followup CL can add support for win/macos.

Bug: b/345092320
Change-Id: I877baa4d7fd912b42cfcd88ad0aa347b700a89f2
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5604606
Commit-Queue: Richard Wang <richardwa@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
9 months ago
Peter Boström 74a6ca92bb Clean up stacked-changes dogfood
This removes logspam from `git cl` when a user (like me) hasn't opted
either in or out. DOGFOOD_STACKED_CHANGES=1 is the codified behavior.
This removes the DOGFOOD_STACKED_CHANGES=0 opt-out.

Bug: b/265929888
Change-Id: Ia159f898512d815e3a4b76cc8859c599016a7bec
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5274266
Auto-Submit: Peter Boström <pbos@chromium.org>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
1 year ago
Gavin Mak edba22d4eb Fix multiline comment formatting
Many incorrectly formatted comments exist from the switch to
4 space indent: https://crrev.com/c/4836379

Bug: 1514505
Change-Id: I6366f9da812919bd35b999f18fa8a49b7a66c09b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5153633
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
1 year ago
Gavin Mak f936d540e1 Remove __future__ imports
All __future__ imports (unicode_literals, print_function) are already
mandatory in py3. Also remove an outdated py2 comment in
presubmit_canned_checks.py

Bug: 1475402
Change-Id: I27cf6a8268f6dd1081f22af782c4c29a975376ae
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4867135
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
1 year ago
Mike Frysinger 124bb8e53c switch to 4 space indent
Leave the recipes/ code at 2 space to match the rest of the recipes
project in other repos.

Reformatted using:
files=( $(
	git ls-tree -r --name-only HEAD | \
		grep -Ev -e '^(third_party|recipes)/' | \
		grep '\.py$';
	git grep -l '#!/usr/bin/env.*python' | grep -v '\.py$'
) )
parallel ./yapf -i -- "${files[@]}"
~/chromiumos/chromite/contrib/reflow_overlong_comments "${files[@]}"

The files that still had strings that were too long were manually
reformatted because they were easy and only a few issues.
autoninja.py
clang_format.py
download_from_google_storage.py
fix_encoding.py
gclient_utils.py
git_cache.py
git_common.py
git_map_branches.py
git_reparent_branch.py
gn.py
my_activity.py
owners_finder.py
presubmit_canned_checks.py
reclient_helper.py
reclientreport.py
roll_dep.py
rustfmt.py
siso.py
split_cl.py
subcommand.py
subprocess2.py
swift_format.py
upload_to_google_storage.py

These files still had lines (strings) that were too long, so the pylint
warnings were suppressed with a TODO.
auth.py
gclient.py
gclient_eval.py
gclient_paths.py
gclient_scm.py
gerrit_util.py
git_cl.py
presubmit_canned_checks.py
presubmit_support.py
scm.py

Change-Id: Ia6535c4f2c48d46b589ec1e791dde6c6b2ea858f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4836379
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Auto-Submit: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Gavin Mak 5f955dfff1 Drop py2 support from metrics files
python3 is the only supported version of python in depot_tools.

Bug: 1475402
Change-Id: Ic925af3a79311b11a953f1d0d4602754488a9de3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4824480
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Aravind Vasudevan ed935cf29b Remove deadcode from depot_tools scripts
Removing some deadcode from depot_tools scripts. More follow-up CLs will
be made after this to remove more dead code.

R=sokcevic,gavinmak

Change-Id: Iad20e9e1737e49c1370248d32c6317cf2be63de4
Bug: 1475776
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4811216
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2 years ago
Joanna Wang 79499cf439 Update env_vars metrics name.
Bug: b/265929888
Change-Id: Iaf2fd1d509f4390d26c8f4f74f863bbe3551600c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4219773
Reviewed-by: Gavin Mak <gavinmak@google.com>
Auto-Submit: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2 years ago
Joanna Wang 5ed21de943 Change env_variables to repeated metric.
Bug: b/265929888
Change-Id: I7787e053ca0785801817e4916d9b25fa2ff0e8d7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4219772
Auto-Submit: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2 years ago
Joanna Wang 92d7df88aa Add way to track env variables.
Bug:b/265929888
Change-Id: Ibf24e9aae7bff1d9af36c15fc7f9bdad9dbba526
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4209320
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Jonas Termansen bf7eb5292c [portability] Support unknown operating systems in gclient.
Detect the name of unknown operating systems using uname, if available,
since it doesn't append the operating system version.

Change-Id: Idab7bd0db65a8d424ec2fd48f06247405b6649e4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4169240
Auto-Submit: Jonas Termansen <sortie@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Josip Sokcevic 4de5deacd4 Explicitly run everything with python3
R=aravindvasudev@google.com, gavinmak@google.com

Change-Id: Iaa5e960c6226dea3a0814c7cb778d952eafb1502
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3525960
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
3 years ago
Edward Lesmes 73faeeae3f metrics: Wait until upload is complete on bots.
Otherwise, when the step completes, it won't be able to close
files used for stdout/stderr, since the metrics process is still
running.

Bug: 1206429
Change-Id: Id88578cbd9dab070e07f92e545ec137245566d16
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2877608
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Jamie Madill <jmadill@chromium.org>
4 years ago
Edward Lesmes 9c34906c59 metrics: Don't print notice if reporting build metrics.
Change-Id: Ib7e23cd1b57c6d0497f0e0ff1fe31b38f11bfc00
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2876082
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
4 years ago
Edward Lesmes 1e59a247ab metrics: Report bot metrics if DEPOT_TOOLS_REPORT_BUILD is set
If the DEPOT_TOOLS_REPORT_BUILD envvar is set, Depot Tools will
report information about the builder running the command
(e.g. buildbucket project, bucket, builder and build id).

It will also authenticate to the metrics server, and ignore any
requests not made by ChOps service accounts.

Change-Id: I078a4c2170b4226086c42f289fa449bdebc87179
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2861213
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
4 years ago
Milad Fa 52fdd1ffce Adding AIX
Change-Id: Ibf023a1f6b635872c849bda885b47ea1dbf7dda3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2410733
Reviewed-by: Dirk Pranke <dpranke@google.com>
Commit-Queue: Dirk Pranke <dpranke@google.com>
5 years ago
Edward Lesmes e1a9c8db7e Reland "metrics: Use vpython3 instead of sys.executable"
This is a reland of 310e1e814d

Use subprocess2, which will set shell=True when running on Windows.

Original change's description:
> metrics: Use vpython3 instead of sys.executable
>
> Apparently, uploading metrics might happen after sys.executable
> has been deleted.
>
> Bug: 1068263
> Change-Id: Id4ab465bb97d146b688f42fb2002e29372e65555
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2148879
> Commit-Queue: Anthony Polito <apolito@google.com>
> Reviewed-by: Anthony Polito <apolito@google.com>

Bug: 1068263
Change-Id: I19c915f3e8862ae3248b01e0eccdbdf0bfda7369
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2150170
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lesmes 3820520acb Revert "metrics: Use vpython3 instead of sys.executable"
This reverts commit 310e1e814d.

Reason for revert:
"vpython3" doesn't work on Windows without shell

Original change's description:
> metrics: Use vpython3 instead of sys.executable
> 
> Apparently, uploading metrics might happen after sys.executable
> has been deleted.
> 
> Bug: 1068263
> Change-Id: Id4ab465bb97d146b688f42fb2002e29372e65555
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2148879
> Commit-Queue: Anthony Polito <apolito@google.com>
> Reviewed-by: Anthony Polito <apolito@google.com>

TBR=ehmaldonado@chromium.org,apolito@google.com,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com

Change-Id: I6b9998de0d3afeda142971f889361fb1c8a37a0d
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 1068263
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2149762
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lesmes 310e1e814d metrics: Use vpython3 instead of sys.executable
Apparently, uploading metrics might happen after sys.executable
has been deleted.

Bug: 1068263
Change-Id: Id4ab465bb97d146b688f42fb2002e29372e65555
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2148879
Commit-Queue: Anthony Polito <apolito@google.com>
Reviewed-by: Anthony Polito <apolito@google.com>
5 years ago
Quinten Yearsley 925cedba7b Run spellchecker on depot_tools
This was made by running `codespell` and `scspell`
and then checking the results.

Change-Id: I169fd5b40294f83015075b4a899fbca263821f25
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2144602
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
Auto-Submit: Quinten Yearsley <qyearsley@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lemur 73065b2067 depot_tools: Make some changes to make metrics collection compatible with Python 3.
Bug: 984182
Change-Id: I55e9e83d01d5a86464cc234c083e4212f0ba4a1e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1713217
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
6 years ago
Edward Lemur 9b51f3e394 metrics: Convert timestamp and depot_tools_age to integers.
TBR=ajp@chromium.org

Change-Id: I839997c132ad7b9642e795b9562bf4fea18211f4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1585271
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
6 years ago
Edward Lemur 18df41e0cf metrics: Collect full timestamps.
Update metrics.README.md and add a metrics_version field.

Bug: None
Change-Id: I17d9032bd787af2a6fc49f80d59e05152d5223b7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1406256
Reviewed-by: Andy Perelson <ajp@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
6 years ago
Raul Tambre b946b2389c Improve gclient Python 3 compatibility
This enables gclient sync and gclient runhooks to run, barring hook script failures.
git cl upload also now works.

The scripts still work with Python 2.
There are no intended behaviour changes.

Bug: 942522
Change-Id: I2ac587b5f803ba7f5bb5e412337ce049f4b1a741
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1524583
Commit-Queue: Raul Tambre <raul@tambre.ee>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
6 years ago
Edward Lemur 861640f6c8 metrics: Collect git version.
Bug: None
Change-Id: I73545ad59134c6e5dbeb47fb2e8168a5afc0e497
Reviewed-on: https://chromium-review.googlesource.com/c/1296861
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
6 years ago
Edward Lemur 149834e67d metrics: Add function to collect repeated metrics.
Bug: 897394
Change-Id: Iece4507c7bf92d7a8aad045d31b4f06e54b3af15
Reviewed-on: https://chromium-review.googlesource.com/c/1292243
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
6 years ago
Edward Lemur 488362624b metrics: Add a mechanism to notify users when we want to collect additional metrics.
When we change the version number in metrics_utils:

If the user is not a Googler, or has opted out explicitly, nothing happens
and we still don't collect metrics.

If we're collecting metrics from the user, we stop collecting metrics
and display a notice telling them what has changed.
That notice will be displayed ten times, after which we will
resume collecting metrics. A notice telling them we're collecting metrics
will still be displayed.

Bug: None
Change-Id: If1cc12b2fc06f0d6237714c4f182367b1afdf9fb
Reviewed-on: https://chromium-review.googlesource.com/c/1285395
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
6 years ago
Tomasz Śniatowski bc56d8cfab Fix typo in a metrics.py message
Change-Id: Ifb598565e0a048cc0acbb8afa506762ab05ce42c
Reviewed-on: https://chromium-review.googlesource.com/1192443
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
7 years ago
Edward Lemur dd5051fa52 metrics: Deal with permission denied when writing metrics.cfg.
If we don't have permission to create the metrics.cfg file, print a
notice and disable metrics collection.

Bug: 870231
Change-Id: I784e988ed021daef0fb07c08f1da44718581b1b9
Reviewed-on: https://chromium-review.googlesource.com/1166322
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Edward Lemur 6f812e132d metrics: Separate metrics collection and print notice logic.
As it is right now, when a function is collecting metrics it prints
the notice and calls sys.exit() to ensure the notice is the last thing
that is printed.

git-cl split has to call git-cl upload multiple times, but once it has
been called once, it exits, so only the first cl is uploaded.

This separates metrics collection from notice printing, so that the
function that is collecting metrics behaves like a function that isn't.
It also makes sure we don't collect metrics for multiple functions at
the same time.

Bug: 868280
Change-Id: Ic58ebe7d19e09ed85fa8b0af76dcbf608ee4c9bc
Reviewed-on: https://chromium-review.googlesource.com/1153503
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Edward Lemur ad463c9517 git-cl: Report known project urls.
Bug: 832386
Change-Id: I80bffe3df1b3097ca2d3939d101eb617f3077901
Reviewed-on: https://chromium-review.googlesource.com/1150442
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Edward Lemur 5ba1e9caee git cl: Start reporting metrics.
Bug: 832386
Change-Id: Iccb71c020aea31db33fc16050cb1100ccf5a7fbb
Reviewed-on: https://chromium-review.googlesource.com/1145902
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Edward Lemur 7fa0f19f73 metrics: Fix countdown to metrics collection.
We were printing the notice when the decorator was called, so that if
several functions are decorated, the countdown will increase more than once
during each execution.

This change fixes that.

Bug: 832386
Change-Id: I5d6dd2d793137e1e2c83b6cc765b6245080b9d91
Reviewed-on: https://chromium-review.googlesource.com/1141073
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Edward Lemur 3298e7b592 gclient: Start reporting metrics.
Bug: 832386
Change-Id: I6d1167802f077bcd67bf004ccc417661d8fff3c7
Reviewed-on: https://chromium-review.googlesource.com/1135903
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Edward Lemur 32e3d1e37c Add a library for monitoring.
Change-Id: I64c3d143186be938042c12e2455bdb034d3bcba8
Reviewed-on: https://chromium-review.googlesource.com/1079639
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
7 years ago