You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
 
Go to file
Mark Mentovai 4005bb7410 mac-arm64: Provide an arm64-native ninja-mac for builds in ¼ of the time
ninja-mac is expanded to become a universal (“fat”) executable. The
existing x86_64 slice was taken from the x86_64 ninja-mac as it
previously existed in this location, and a new arm64 slice was added.

This is an aid to native development on arm64. In macOS, subprocesses
inherit their parent’s “binpref,” meaning that children of a translated
process will, by default, also be translated, if there’s a choice
between native and translated. Where ninja-mac is x86_64-only and runs
translated on arm64, all tools it invokes will also run translated, even
if native versions are available. This has an impact on any part of the
toolchain for which universal versions are available: in Chrome’s case,
tools such as ld64 are affected. In Crashpad’s build, which uses the
compiler provided by Xcode, it even affects the compiler, leading to
significant differences in build time on arm64.

mark@arm-and-hammer zsh% file $(which ninja-mac)
…/ninja-mac: Mach-O 64-bit executable x86_64
mark@arm-and-hammer zsh% time ninja -C out/release
ninja: Entering directory `out/release'
[1/1] Regenerating ninja files
[511/511] LINK ./crashpad_minidump_test
ninja -C out/release  1384.03s user 111.05s system 393% cpu 6:19.53 total

mark@arm-and-hammer zsh% file $(which ninja-mac)
…/ninja-mac: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64:Mach-O 64-bit executable arm64]
…/ninja-mac (for architecture x86_64):	Mach-O 64-bit executable x86_64
…/ninja-mac (for architecture arm64):	Mach-O 64-bit executable arm64
mark@arm-and-hammer zsh% time ninja -C out/release
ninja: Entering directory `out/release'
[1/1] Regenerating ninja files
[511/511] LINK ./crashpad_minidump_test
ninja -C out/release  606.80s user 34.71s system 701% cpu 1:31.41 total

This differs from the previous attempt, e64cd3aeff
(https://chromium-review.googlesource.com/c/2288417) in that the arm64
version was built with Xcode 12.2b1 (12B5018i). The linker in this
version of Xcode produces linker-signed output for arm64 by default. As
of macOS 11.0db6 (20A5364e), all arm64 code must be signed. More
information at
https://developer.apple.com/documentation/macos-release-notes/macos-big-sur-11-universal-apps-beta-release-notes#Code-Signing.
In the previous attempt, an older Xcode was used and the output was not
signed at all. Because it did not run on systems upgraded to 11.0db6,
this version of ninja-mac was backed out in a070dd18d7
(https://chromium-review.googlesource.com/c/2393775).

To create this ninja-mac:

% git clone https://github.com/ninja-build/ninja.git
% cd ninja
% git checkout v1.8.2
% mkdir ../build
% cd ../build
% LDFLAGS=-Wl,-s ../ninja/configure.py --bootstrap
% lipo -create -output ninja-mac $(which ninja-mac) ninja

An explanation of the LDFLAGS: e64cd3aeff used “strip” on the output,
matching the stripped x86_64 slice, but “strip” is not currently
compatible with linker-signed executables. This is documented at
https://developer.apple.com/documentation/xcode-release-notes/xcode-12_2-beta-release-notes#Apple-Clang-Compiler.
I was unable to use the recommended workaround in this case. Rather than
leave the output unstripped, I asked ld64 to strip symbols using its -s
option, which it believes is obsolete, but in fact provides the intended
behavior.

The x86_64 slice is unsigned, as it was previously. The new arm64 slice
is linker-signed with an ad-hoc signature. The x86_64 slice was built
with the macOS 10.12 SDK and runs on Mac OS X 10.6 or later. The arm64
slice was built with and has a minimum runtime requirement of macOS
11.0. In practice, the difference doesn’t matter, as the 11.0 SDK is
required to build for arm64, and no macOS version earlier than 11.0 will
run on arm64.

Bug: chromium:1103315
Change-Id: I9f201d0d6f476e187acc0008681c3fd2c1a72aa0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2427051
Reviewed-by: Nico Weber <thakis@chromium.org>
Commit-Queue: Mark Mentovai <mark@chromium.org>
4 years ago
bootstrap Roll git to 2.28.0 on Windows 5 years ago
fetch_configs
git-templates
infra Roll recipe dependencies (trivial). 5 years ago
man Modify git map-branches to show the number of commits on a branch 5 years ago
python-bin
recipes Add xcode version for Mac OS 10.15.4 in osx_sdk 5 years ago
testing_support gclient: deprecated use_relative_hooks 5 years ago
tests Use rstrip when getting footer tags 5 years ago
third_party Revert "Improve git cl split" 5 years ago
win_toolchain Print packaging file size/count stats always 5 years ago
zsh-goodies
.cipd_impl.ps1 [cipd] Take cipd.ps1 out of %PATH%. 5 years ago
.gitattributes
.gitignore
.style.yapf Update "git cl format --python" to default to PEP-8 5 years ago
.vpython Preliminary changes to git cl and presubmit_support 5 years ago
.vpython3 Preliminary changes to git cl and presubmit_support 5 years ago
CROS_OWNERS CROS_OWNERS: add Michael 5 years ago
GOMA_OWNERS Add tikuta, <user>@chromium.org to GOMA_OWNERS 5 years ago
LICENSE
LUCI_OWNERS tandrii: switch OWNERS to @google.com account. 5 years ago
OWNERS [resultdb] add ddoman@ to depot_tools/LUCI_OWNERS 5 years ago
PRESUBMIT.py Warn only if non-inclusive term is used 5 years ago
README.gclient.md
README.git-cl.md
README.md add clarification for Python 3 5 years ago
WATCHLISTS WATCHLISTS: CC chops-source-team@ on all depot_tools changes. 5 years ago
auth.py
autoninja Fix a python invocation that wasn't replaced with vpython. 5 years ago
autoninja.bat Run autoninja scripts using vpython. 5 years ago
autoninja.py Pass large -j value to ninja when use_rbe = true 5 years ago
bb
bb.bat
bootstrap_python3 Add quotes around paths in bash scripts. 5 years ago
breakpad.py
cbuildbot
chrome_set_ver
cipd [cipd] Always bootstrap mac-amd64, even on arm64. 5 years ago
cipd.bat [cipd] Take cipd.ps1 out of %PATH%. 5 years ago
cipd_bin_setup.bat
cipd_bin_setup.sh
cipd_client_version [cipd] Rollback latest CIPD deployment. 5 years ago
cipd_client_version.digests [cipd] Rollback latest CIPD deployment. 5 years ago
cipd_manifest.txt Roll lucicfg tool to @ 8803da5d 5 years ago
cipd_manifest.versions Roll lucicfg tool to @ 8803da5d 5 years ago
cit
cit.bat
cit.py
clang-format
clang-format.bat
clang_format.py
clang_format_merge_driver
clang_format_merge_driver.bat
clang_format_merge_driver.py
codereview.settings
compile_single_file
compile_single_file.bat
compile_single_file.py remove reference to jumbo build 5 years ago
cpplint.bat
cpplint.py CodeInclusion: Rename whitelist/blacklist -> allowlist/blocklist 5 years ago
cpplint_chromium.py
cros cros: raise python version requirement to 3.6 5 years ago
cros_sdk
crosjobs
detect_host_arch.py
dirmd [dirmd] Add dirmd executables 5 years ago
dirmd.bat [dirmd] Add dirmd executables 5 years ago
download_from_google_storage
download_from_google_storage.bat
download_from_google_storage.py Improve Python-3 compatibility of download_from_google_storage.py 5 years ago
ensure_bootstrap
fetch Make Python 3 default on Windows 5 years ago
fetch.bat Abort fetch, gclient, gsutil.py when update_depot_tools fails on Windows 5 years ago
fetch.py Make fetch compatible with py 3.5 5 years ago
fix_encoding.py Run spellchecker on depot_tools 5 years ago
gclient Make Python 3 default on Windows 5 years ago
gclient-new-workdir.py
gclient.bat Abort fetch, gclient, gsutil.py when update_depot_tools fails on Windows 5 years ago
gclient.py Revert "Remove use_relative_hooks now that all deprecated uses are removed." 5 years ago
gclient_completion.sh
gclient_eval.py Revert "Remove use_relative_hooks now that all deprecated uses are removed." 5 years ago
gclient_paths.py depot_tools: Add tests for gclient_paths 5 years ago
gclient_scm.py Xcode 12b3 (and later) emits arm64e binaries; try that architecture. 5 years ago
gclient_utils.py gclient: Don't try opening pseudo-terminal pipes on AIX 5 years ago
gerrit_client.py Add gerrit_client unit tests 5 years ago
gerrit_util.py ReadHttpResponse: Handle 429s with retries and adjust retry count 5 years ago
git-cache Add quotes around paths in bash scripts 5 years ago
git-cl Add quotes around paths in bash scripts. 5 years ago
git-crrev-parse
git-drover Add quotes around paths in bash scripts 5 years ago
git-find-releases Add quotes around paths in bash scripts 5 years ago
git-footers Add quotes around paths in bash scripts 5 years ago
git-freeze Add quotes around paths in bash scripts 5 years ago
git-gs
git-hyper-blame Add quotes around paths in bash scripts 5 years ago
git-map Add quotes around paths in bash scripts 5 years ago
git-map-branches Add quotes around paths in bash scripts 5 years ago
git-mark-merge-base Add quotes around paths in bash scripts 5 years ago
git-nav-downstream Add quotes around paths in bash scripts 5 years ago
git-nav-upstream
git-new-branch Add quotes around paths in bash scripts 5 years ago
git-number Add quotes around paths in bash scripts 5 years ago
git-rebase-update Add quotes around paths in bash scripts 5 years ago
git-rename-branch Add quotes around paths in bash scripts 5 years ago
git-reparent-branch Add quotes around paths in bash scripts 5 years ago
git-retry Add quotes around paths in bash scripts 5 years ago
git-runhooks
git-squash-branch Add quotes around paths in bash scripts 5 years ago
git-thaw Add quotes around paths in bash scripts 5 years ago
git-upstream-diff Add quotes around paths in bash scripts 5 years ago
git_cache.py Try to be more efficient with git-cache gargage collecting. 5 years ago
git_cl.py Allow 'main' as a local main branch name. 5 years ago
git_cl_completion.sh Add git completion helper to cl upload 5 years ago
git_common.py Modify git map-branches to show the number of commits on a branch 5 years ago
git_dates.py
git_drover.py git-drover: Don't attempt to decode output of check_call. 5 years ago
git_find_releases.py Add unit tests for git_find_releases 5 years ago
git_footers.py Use rstrip when getting footer tags 5 years ago
git_freezer.py
git_hyper_blame.py Add support for git_hyper_blame for Windows 5 years ago
git_map.py git-map: Refactor and add simple tests 5 years ago
git_map_branches.py Modify git map-branches to show the number of commits on a branch 5 years ago
git_mark_merge_base.py
git_nav_downstream.py depot_tools: Use gclient_utils.AskForData instead of raw_input. 5 years ago
git_new_branch.py git-cl: Fix invocation of git new-branch 5 years ago
git_number.py Run spellchecker on depot_tools 5 years ago
git_rebase_update.py Added option to keep empty branches in rebase-update. 5 years ago
git_rename_branch.py
git_reparent_branch.py
git_retry.py
git_squash_branch.py
git_upstream_diff.py
gn
gn.bat
gn.py
goma_auth
goma_auth.bat Fix goma_auth.bat to use nul instead of null. 5 years ago
goma_ctl
goma_ctl.bat Fixed goma_ctl.bat to pipe to nul instead of creating a file called null. 5 years ago
gsutil.py
gsutil.py.bat gsutil.py.bat: Skip updating depot_tools if DEPOT_TOOLS_UPDATE=0. 5 years ago
gsutil.vpython gsutil.vpython: Set vpython_version: 2.7 5 years ago
led
led.bat
lockfile.py Reland "Use OS level locking in git_cache.py" 5 years ago
luci-auth
luci-auth.bat
lucicfg
lucicfg.bat
mac_toolchain
metrics.README.md metrics: Update documentation 5 years ago
metrics.py Adding AIX 5 years ago
metrics_utils.py
my_activity.py my_activity.py: add coreboot gerrit 5 years ago
ninja
ninja-linux32
ninja-linux64
ninja-mac mac-arm64: Provide an arm64-native ninja-mac for builds in ¼ of the time 4 years ago
ninja.exe
ninjalog.README.md
ninjalog_uploader.py Fix reference to enable_js_type_check 5 years ago
ninjalog_uploader_wrapper.py use CREATE_NEW_PROCESS_GROUP to invoke ninjalog_uploader 5 years ago
owners.py don't let 'anyone' be a reviewer 5 years ago
owners_finder.py Print clarifying message when no new owners needed 5 years ago
post_build_ninja_summary.py More detailed build-step type summary for android 5 years ago
presubmit_canned_checks.py CheckDirMetadataFormat: LocalPath() -> AbsoluteLocalPath() 5 years ago
presubmit_support.py Output a newline when there are warnings and no prompt. 5 years ago
profile.xml
prpc
prpc.bat
pylint
pylint-1.5
pylint-1.6
pylint-1.7
pylint-1.8
pylint-1.9
pylint.bat
pylint_main.py
pylintrc CodeInclusion: Rename whitelist/blacklist -> allowlist/blocklist 5 years ago
python_runner.sh Make Python 3 default on Windows 5 years ago
rdb [resultdb] Add rdb and rdb.bat 5 years ago
rdb.bat [resultdb] Add rdb and rdb.bat 5 years ago
rdb_wrapper.py Changed naming convention 5 years ago
repo repo: update to 2.8 launcher 5 years ago
roll-dep Make Python 3 default on Windows 5 years ago
roll-dep.bat Make Python 3 default on Windows 5 years ago
roll_dep.py Get roll-dep to handle \ separators 5 years ago
scm.py Run spellchecker on depot_tools 5 years ago
setup_color.py
split_cl.py don't let 'anyone' be a reviewer 5 years ago
subcommand.py Run spellchecker on depot_tools 5 years ago
subprocess2.py Remove rev-parse --verify FETCH_HEAD 5 years ago
update_depot_tools Add quotes around paths in bash scripts 5 years ago
update_depot_tools.bat Fix command prompt title after update_depot_tools 5 years ago
update_depot_tools_toggle.py
upload_metrics.py
upload_to_google_storage.py
vpython
vpython.bat Reland "vpython: Specify interpreter to run." 5 years ago
vpython3 depot_tools: Simplify MinGW check. 5 years ago
vpython3.bat Fix git-* commands when using python3 on Win 5 years ago
watchlists.py
weekly
win32imports.py Reland "Use OS level locking in git_cache.py" 5 years ago
wtf
yapf update yapf to 0.27.0 5 years ago
yapf.bat

README.md

depot_tools

Tools for working with Chromium development. It requires python 2.7 or 3.8 for python 3 support.

Tools

The most important tools are:

  • fetch: A gclient wrapper to checkout a project. Use fetch --help for more details.
  • gclient: A meta-checkout tool. Think repo or git submodules, except that it support OS-specific rules, e.g. do not checkout Windows only dependencies when checking out for Android. Use gclient help for more details and README.gclient.md.
  • git cl: A code review tool to interact with Rietveld or Gerrit. Use git cl help for more details and README.git-cl.md.
  • roll-dep: A gclient dependency management tool to submit a dep roll, updating a dependency to a newer revision.

There are a lot of git utilities included.

Updating

depot_tools updates itself automatically when running gclient tool. To disable auto update, set the environment variable DEPOT_TOOLS_UPDATE=0.

To update package manually, run update_depot_tools.bat on Windows, or ./update_depot_tools on Linux or Mac.

On Windows only, running gclient will install git and python.

Contributing

To contribute change for review:

git new-branch <somename>
# Hack
git add .
git commit -a -m "Fixes goat teleporting"
# find reviewers
git cl owners
git log -- <yourfiles>

# Request a review.
git cl upload -r reviewer1@chromium.org,reviewer2@chromium.org --send-mail

# Edit change description if needed.
git cl desc

# If change is approved, flag it to be committed.
git cl set-commit

# If change needs more work.
git rebase-update
...
git cl upload -t "Fixes goat teleporter destination to be Australia"

See also open bugs, open reviews, forum or report problems.

cpplint.py

Until 2018, our cpplint.py was a copy of the upstream version at https://github.com/google/styleguide/tree/gh-pages/cpplint. Unfortunately, that repository is not maintained any more. If you want to update cpplint.py in depot_tools, just upload a patch to do so. We will figure out a long-term strategy via issue https://crbug.com/916550.

Note that the cpplint.py here is also used by the Tricium analyzer, so if the cpplint.py here changes, we should also update the copy used there.