post_build_ninja_summary.py normally sorts its output by weighted time,
which is elapsed time divided by how many steps are running. Sometimes
it is more appropriate to sort by elapsed (wall clock) time instead.
This adds a -e/--elapsed_time_sorting option to do this.
Change-Id: I13a1b094f8fd7f6795a871bee91932ce0ddab79d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4285697
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Makes it easier to run on linux/mac.
Change-Id: I0ebf512cd576a5c8aafbddc7c6b6f8e2bd6919d9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3971922
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Auto-Submit: Andrew Grieve <agrieve@chromium.org>
In Python 3 dict.values() is not a sortable type so it must be converted
to a list prior to returning targets data from the ReadTargets function.
In Python 3 you cannot compare arbitrary objects. This could happen when
sorting task_start_stop_times if the time and types were identical. This
is fixed by using the first two entries in the tuple as the sorting key.
When a .ninja_log file is corrupt it triggers a diagnostic message which
has only caused confusion. This changes that message to suggest the most
likely root cause (corruption) since that is the only cause I have seen.
If a build step generates multiple targets with long names then wrapping
of lines may occur. To avoid that the target description size is capped.
Bug: 941669
Change-Id: I2a808d2629a639e231ce2dbf2dab41251110b156
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2432409
Auto-Submit: Bruce Dawson <brucedawson@chromium.org>
Commit-Queue: Dirk Pranke <dpranke@google.com>
Reviewed-by: Dirk Pranke <dpranke@google.com>
Increase the number of long times by extension to report from 5 to 10
since java build steps produce a number of different extension types. It
is difficult to distinguish between android vs non-android builds, so
increase this default for all platforms.
Prioritize .jar outputs for java targets and allow up to two levels of
extensions to be recognized.
e.g.
file.interface.jar would have .interface.jar as its full extension.
file.javac.jar would have .javac.jar as its full extension.
Some of these build steps require .jar outputs, which is why these
targets use a secondary extension to differentiate.
Example output: https://crrev.com/c/2142395
Bug: chromium:1067273
Change-Id: Id5780980f60841c3384d91bf96121c6dec3e8bed
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2142163
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Commit-Queue: Peter Wen <wnwen@chromium.org>
Auto-Submit: Peter Wen <wnwen@chromium.org>
The post_build_ninja_summary.py script, which autoninja automatically
runs if NINJA_SUMMARIZE_BUILD=1 is set, summarizes build time by file
extension. This is a good default summary but it can be handy to create
additional output categories. This change allows using fn_match style
wildcards to create additional categories which will then be summarized.
For instance:
python post_build_ninja_summary.py -C out/Def -s prec*.obj;blink*.obj
In addition the chromium_step_types environment variable can be set so
that when autoninja runs this script there will be additional summary
categories.
This has been used most recently when measuring the cost of creating and
using blink precompiled header files.
Bug: 1061326
Change-Id: I080b4ddd06d045a2351220e202cd9eec111bf00e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2093145
Auto-Submit: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
The build summary script was overly aggressive about summarizing .mojom
files under the mojo category. This was intended to measure the cost of
generating source from mojo files but it accidentally pulled in the
.mojom object files.
Categorizing different types of object files might be worthwhile but
that should be done intentionally and consistently.
Change-Id: Iab6b7e94797ce7f1ed46805034b4f274c88617e0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2067647
Commit-Queue: Aaron Gable <agable@chromium.org>
Auto-Submit: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
The "CPU time" values printed by post_build_ninja_summary.py are not
actually measure of CPU time, they are measures of wall-clock time or
elapsed time. This just changes the labeling of the numbers - the actual
numbers printed are unchanged.
The "weighted time" number continues to be the more important number
since it is an approximation of how much the build step(s) are affecting
build time. See the comments in the script for details.
Change-Id: Ibdb4efdd327ece34492ab10337c234a826514197
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1900019
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Ran "2to3 -w -n -f print ./" and manually added imports.
Ran "^\s*print " and "\s+print " to find batch/shell scripts, comments and the like with embedded code, and updated them manually.
Also manually added imports to files, which used print as a function, but were missing the import.
The scripts still work with Python 2.
There are no intended behaviour changes.
Bug: 942522
Change-Id: Id777e4d4df4adcdfdab1b18bde89f235ef491b9f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1595684
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Dirk Pranke <dpranke@chromium.org>
Auto-Submit: Raul Tambre <raul@tambre.ee>
post_build_ninja_summary.py detects a new incremental build by looking
for the end time stamps to go backwards. This means that repeated builds
that have a single long step will not be reliably detected and will, in
some cases, be missed entirely. This hits me sometimes when doing link
testing - delete an output DLL, rebuild, and the old results may be
displayed again.
This change updates the script to check for a different duration for an
existing record as well as an earlier end time should handle almost all
cases.
This also renames the targets local variable to avoid confusion due
to the targets member variable of the Targets class.
Bug: chromium:787983
Change-Id: I60a371df75d6cb7a55bd46b38156cb109feb8f15
Reviewed-on: https://chromium-review.googlesource.com/1061413
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Sometimes the log files are not named .ninja_log and then it's
good if you can give the script the actual name and path.
Change-Id: I59e9812c81a83ffac3ecdc8b29887e99bf71b60f
Reviewed-on: https://chromium-review.googlesource.com/1012112
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
Commit-Queue: Daniel Bratell <bratell@opera.com>
post_build_ninja_summary.py gives a summary of a ninja build. It can be
run standalone or it can be run automatically by autoninja. This CL
updates the Python script and the autoninja bash script to make this
work on Linux. This includes removing a zero-value assert, and ensuring
that .so files get categorized as such.
Change-Id: I2d59ab129f5ce70117beeb119719f8432bfbab7c
Reviewed-on: https://chromium-review.googlesource.com/915053
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
post_build_ninja_summary.py analyzes the .ninja_log file in an output
directory to summarize build performance. It lists the most expensive
build steps, and build-step types, printing both their elapsed time and,
more importantly, their "weighted" time which is a measure of their
contribution to the total build time.
For each time segment within the build the running build steps
accumulate weighted time that is proportional to the elapsed time
divided by the number of tasks running. If a thousand compilation steps
are running in parallel then they will each be "charged" for 1/1,000th
of the elapsed time. If a single link step is running alone then it is
charged with all of the elapsed time.
Bug: chromium:787983
Change-Id: Id5aea715f798a16415dd0365a27f0051202668e5
Reviewed-on: https://chromium-review.googlesource.com/871988
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Erik Staab <estaab@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Bruce Dawson
Andrew Grieve
Takuto Ikuta
Peter Wen
Edward Lemur
Raul Tambre
Daniel Bratell