ninja-mac is expanded to become a universal (“fat”) executable. The
existing x86_64 slice was taken from the x86_64 ninja-mac as it
previously existed in this location, and a new arm64 slice was added.
This is an aid to native development on arm64. In macOS, subprocesses
inherit their parent’s “binpref,” meaning that children of a translated
process will, by default, also be translated, if there’s a choice
between native and translated. Where ninja-mac is x86_64-only and runs
translated on arm64, all tools it invokes will also run translated, even
if native versions are available. This has an impact on any part of the
toolchain for which universal versions are available: in Chrome’s case,
tools such as ld64 are affected. In Crashpad’s build, which uses the
compiler provided by Xcode, it even affects the compiler, leading to
significant differences in build time on arm64.
mark@arm-and-hammer zsh% file $(which ninja-mac)
…/ninja-mac: Mach-O 64-bit executable x86_64
mark@arm-and-hammer zsh% time ninja -C out/release
ninja: Entering directory `out/release'
[1/1] Regenerating ninja files
[511/511] LINK ./crashpad_minidump_test
ninja -C out/release 1384.03s user 111.05s system 393% cpu 6:19.53 total
mark@arm-and-hammer zsh% file $(which ninja-mac)
…/ninja-mac: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64:Mach-O 64-bit executable arm64]
…/ninja-mac (for architecture x86_64): Mach-O 64-bit executable x86_64
…/ninja-mac (for architecture arm64): Mach-O 64-bit executable arm64
mark@arm-and-hammer zsh% time ninja -C out/release
ninja: Entering directory `out/release'
[1/1] Regenerating ninja files
[511/511] LINK ./crashpad_minidump_test
ninja -C out/release 606.80s user 34.71s system 701% cpu 1:31.41 total
This differs from the previous attempt, e64cd3aeff
(https://chromium-review.googlesource.com/c/2288417) in that the arm64
version was built with Xcode 12.2b1 (12B5018i). The linker in this
version of Xcode produces linker-signed output for arm64 by default. As
of macOS 11.0db6 (20A5364e), all arm64 code must be signed. More
information at
https://developer.apple.com/documentation/macos-release-notes/macos-big-sur-11-universal-apps-beta-release-notes#Code-Signing.
In the previous attempt, an older Xcode was used and the output was not
signed at all. Because it did not run on systems upgraded to 11.0db6,
this version of ninja-mac was backed out in a070dd18d7
(https://chromium-review.googlesource.com/c/2393775).
To create this ninja-mac:
% git clone https://github.com/ninja-build/ninja.git
% cd ninja
% git checkout v1.8.2
% mkdir ../build
% cd ../build
% LDFLAGS=-Wl,-s ../ninja/configure.py --bootstrap
% lipo -create -output ninja-mac $(which ninja-mac) ninja
An explanation of the LDFLAGS: e64cd3aeff used “strip” on the output,
matching the stripped x86_64 slice, but “strip” is not currently
compatible with linker-signed executables. This is documented at
https://developer.apple.com/documentation/xcode-release-notes/xcode-12_2-beta-release-notes#Apple-Clang-Compiler.
I was unable to use the recommended workaround in this case. Rather than
leave the output unstripped, I asked ld64 to strip symbols using its -s
option, which it believes is obsolete, but in fact provides the intended
behavior.
The x86_64 slice is unsigned, as it was previously. The new arm64 slice
is linker-signed with an ad-hoc signature. The x86_64 slice was built
with the macOS 10.12 SDK and runs on Mac OS X 10.6 or later. The arm64
slice was built with and has a minimum runtime requirement of macOS
11.0. In practice, the difference doesn’t matter, as the 11.0 SDK is
required to build for arm64, and no macOS version earlier than 11.0 will
run on arm64.
Bug: chromium:1103315
Change-Id: I9f201d0d6f476e187acc0008681c3fd2c1a72aa0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2427051
Reviewed-by: Nico Weber <thakis@chromium.org>
Commit-Queue: Mark Mentovai <mark@chromium.org>
This reverts commit e64cd3aeff.
Reason for revert: A never macOS 11 beta made it so that
native arm binaries need to be signed, and the Xcode beta used
to build this arm64 slice here didn't do ad-hoc signing by default
yet, see https://crbug.com/1125021
Original change's description:
> mac: Add an arm64 slice to ninja-mac.
>
> To build the arm ninja binary, I downloaded a zip with the ninja 1.8.2
> code, and ran `./configure --bootstrap` on an arm mac, with Xcode 12 beta
> installed, followed by `strip ninja`. That gave me a working ninja 1.8.2
> arm binary. I then copied it to my intel mac and ran
>
> lipo -create -output ninja-mac ninja-mac ~/Downloads/ninja
>
> to make a fat binary.
>
> No intended behavior change on non-arm macs.
>
> (This means the x86_64 slice, which is the previously existing binary,
> has a LC_VERSION_MIN_MACOSX version 10.6, while the arm64 slice has
> LC_BUILD_VERSION minos 11.0, according to `otool -arch ax86_64 -l ninja-mac`
> and `otool -arch arm64 -l ninja-mac`.)
>
> Bug: 1103315,762354
> Change-Id: I2c4bea53a01d7a6135f9cd5554e3123dba960664
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2288417
> Auto-Submit: Nico Weber <thakis@chromium.org>
> Reviewed-by: Mark Mentovai <mark@chromium.org>
> Commit-Queue: Mark Mentovai <mark@chromium.org>
> Commit-Queue: Nico Weber <thakis@chromium.org>
TBR=thakis@chromium.org,dpranke@google.com,mark@chromium.org,infra-scoped@luci-project-accounts.iam.gserviceaccount.com
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: 1103315
Bug: 762354
Change-Id: Icd968a2fecb9509722c43d9cb03db92e6e3829e4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2393775
Reviewed-by: Nico Weber <thakis@chromium.org>
Commit-Queue: Nico Weber <thakis@chromium.org>
To build the arm ninja binary, I downloaded a zip with the ninja 1.8.2
code, and ran `./configure --bootstrap` on an arm mac, with Xcode 12 beta
installed, followed by `strip ninja`. That gave me a working ninja 1.8.2
arm binary. I then copied it to my intel mac and ran
lipo -create -output ninja-mac ninja-mac ~/Downloads/ninja
to make a fat binary.
No intended behavior change on non-arm macs.
(This means the x86_64 slice, which is the previously existing binary,
has a LC_VERSION_MIN_MACOSX version 10.6, while the arm64 slice has
LC_BUILD_VERSION minos 11.0, according to `otool -arch ax86_64 -l ninja-mac`
and `otool -arch arm64 -l ninja-mac`.)
Bug: 1103315,762354
Change-Id: I2c4bea53a01d7a6135f9cd5554e3123dba960664
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2288417
Auto-Submit: Nico Weber <thakis@chromium.org>
Reviewed-by: Mark Mentovai <mark@chromium.org>
Commit-Queue: Mark Mentovai <mark@chromium.org>
Commit-Queue: Nico Weber <thakis@chromium.org>
New binaries for Mac, Linux32, Linux64, Windows. Also update shell script.
See the bug for how these were built.
The main "new" thing is fix for old cmake.
Bug: 762354, 762407
Change-Id: I4f825e357796e1cb52d2efa041bde46d68cbe152
Reviewed-on: https://chromium-review.googlesource.com/659477
Reviewed-by: Nico Weber <thakis@chromium.org>
Commit-Queue: Takuto Ikuta <tikuta@google.com>
New binaries for Mac, Linux32, Linux64, Windows. Also update shell script.
See the bug for how these were built.
The main "new" thing is performance improvement when building chrome with goma.
After confirming the the behavior of ninja 1.8.0 on buildbot, I will roll ninja 1.8.1 includes fix for non-deterministic test.
TBR=dpranke@chromium.org
Bug: 761691, 738186
Change-Id: I108996dccfdf5d95d8815d6cece5eba46f3e8178
Reviewed-on: https://chromium-review.googlesource.com/648372
Commit-Queue: Takuto Ikuta <tikuta@google.com>
Reviewed-by: Nico Weber <thakis@chromium.org>
New binaries for Mac, Linux32, Linux64, Windows. Also update shell script.
See the bug for how these were built.
The main "new" thing is a regression fix where 1.7.1 broke %e %r in
NINJA_STATUS.
BUG=663749
Review-Url: https://codereview.chromium.org/2489023002
See the bug for how these were built (comment 16 onwards).
See the bug comment 0 also for what's new in this version.
All binaries built by thakis@chromium.org.
The Windows binary is now (as of 1.7.x) 64-bit instead of 32-bit.
BUG=607268
TBR=scottmg@chromium.org
Review URL: https://codereview.chromium.org/1925133002 .
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@300281 0039d316-1c4b-4281-b951-d872f2087c98
See the bug for how these were built. See the bug comment 0 also
for what's new in this version.
The linux binaries were built by thestig@chromium.org.
The windows and mac binaries were built by thakis@chromium.org.
The Windows binary is now 64-bit instead of 32-bit.
BUG=607268
R=scottmg@chromium.org
Review URL: https://codereview.chromium.org/1919773006 .
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@300251 0039d316-1c4b-4281-b951-d872f2087c98
Might have caused
ninja: error: '..\..\third_party\qunit\src', needed by 'remoting\unittests\qunit\src', missing and no known rule to make it
on the windows bots.
BUG=389683
Review URL: https://codereview.chromium.org/341253004
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@280434 0039d316-1c4b-4281-b951-d872f2087c98
On mac, I ran:
git checkout v1.3.3
./bootstrap.py
CXX=path/to/chromiums/clang++ ./configure --with-gtest=path/to/gtest
./ninja ninja_test
./ninja_test # all passed
strip ninja
Empty build perf is ~7% slower (https://github.com/martine/ninja/issues/544),
but this allows us to enable deps mode by default which will make empty builds
twice as fast.
Mac binary size grew by ~16kB.
The linux binaries were built by thestig@chromium.org.
The windows binary was built by scottmg@chromium.org.
New stuff since 1.1:
- NINJA_STATUS now allows %e, for elapsed time.
- "pool" feature
- "compdb" tool for writing clang compilation databases
- faster deps mode (needs to be explicitly enabled in gyp, not done yet)
BUG=241628
Review URL: https://codereview.chromium.org/15490003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@201143 0039d316-1c4b-4281-b951-d872f2087c98
On mac, I ran:
git checkout v1.1.0
ninja
./ninja ninja_test
./ninja_test # all passed
strip ninja
I verified that the empty build speed doesn't change.
Mac binary size grew by ~8kB.
The linux binaries were built by tony@chromium.org.
The windows binary was built by scottmg@chromium.org.
BUG=167996
Review URL: https://codereview.chromium.org/11734004
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@174818 0039d316-1c4b-4281-b951-d872f2087c98
On both mac and linux, I ran:
git checkout v1.0.0
ninja
./ninja ninja_test
./ninja_test # all passed
strip ninja
Changes since the last push:
* NINJA_STATUS can now print rates via the o and c format chars
* The version switch is now --version, -V no longer works
* -t list now works without a build.ninja
* Old .ninja_log files now trigger a rebuild instead of an error
* Missing .d files make the corresponding .o file dirty
* Allow @ and = in paths in depfiles
* Ninja's output is now lower-case ("error:" instead of "ERROR:" etc)
* Lots of Windows features (which doesn't affect Mac and Linux)
On Linux, I switched from gcc4.4.3 to clang r163126. The clang-built
binary is 125856 bytes (compared to 129944 bytes previously), and it does
an empty 'chrome' build about 70ms faster (0.94s instead of 1.01s).
The speed difference is almost completely in faster depfile parsing:
clang-built (0.943s):
$ time ../../ninja/ninja -C out/Release chrome -d stats
ninja: Entering directory `out/Release'
ninja: no work to do.
metric count avg (us) total (ms)
.ninja parse 716 507.8 363.6
canonicalize str 113486 0.2 27.7
canonicalize path 1305641 0.1 184.1
lookup node 1305641 0.2 200.9
.ninja_log load 1 11144.0 11.1
node stat 37828 2.6 98.0
depfile load 9266 80.9 749.3
path->node hash load 0.98 (48241 entries / 49157 buckets)
real 0m1.162s
user 0m0.980s
sys 0m0.170s
gcc-built (1.013s):
$ time ninja -C out/Release chrome -d stats
ninja: Entering directory `out/Release'
ninja: no work to do.
metric count avg (us) total (ms)
.ninja parse 716 516.4 369.7
canonicalize str 113486 0.2 27.6
canonicalize path 1305641 0.1 181.0
lookup node 1305641 0.2 260.0
.ninja_log load 1 11012.0 11.0
node stat 37828 2.6 98.2
depfile load 9266 86.3 799.3
path->node hash load 0.98 (48241 entries / 49157 buckets)
real 0m1.220s
user 0m0.970s
sys 0m0.240s
On Mac, I alsow used clang r163126. The binary size stayed at 125104 bytes,
performance didn't change. An empty build of chrome takes 1.26s. While
node stat is a bit slower, and there are more defiles to parse compared
to linux, most of the time difference is in slower .ninja parsing:
mac (1.263s):
$ time ../../ninja/ninja -C out/Release/ chrome -d stats
ninja: Entering directory `out/Release/'
ninja: no work to do.
metric count avg (us) total (ms)
.ninja parse 687 823.4 565.7
canonicalize str 115002 0.3 31.1
canonicalize path 1120853 0.2 175.6
lookup node 1120853 0.2 202.9
.ninja_log load 1 54126.0 54.1
node stat 38860 4.1 160.4
depfile load 9649 86.1 830.8
path->node hash load 0.99 (48522 entries / 49157 buckets)
real 0m1.478s
user 0m1.121s
sys 0m0.351s
TBR=scottmg
Review URL: https://codereview.chromium.org/10918242
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@156751 0039d316-1c4b-4281-b951-d872f2087c98
On both mac and linux, I ran:
git checkout release-120715
ninja
./ninja ninja_test
./ninja_test # all passed
strip ninja
Changes since the last push:
* ninja now has a functional -V flag that prints version information
* ninja is a lot faster. On my MBP, empty builds go from 1.8s to 1.1s,
on my linux box from 1.4s to 0.9s
* ninja now has a -l N flag to do parallelsim based on load average
On linux, the binary grew 24 bytes, from 129920 to 129944. I used
gcc 4.4.3 to build ninja.
On mac, it grew from 4200 byte from 120904 to 125104 (likely caused by me
using a newer clang). I used clang r159409 to build ninja.
TBR=maruel@chromium.org
Review URL: https://chromiumcodereview.appspot.com/10810025
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@147557 0039d316-1c4b-4281-b951-d872f2087c98
On Linux, I did a full ninja checkout and then ran:
python bootstrap.py
./configure.py --with-gtest=../chrome/src/testing/gtest
./ninja ninja_test
./ninja_test # (all tests passed)
strip ninja
I timed empty builds of 'chrome' with the old and the new version,
both took 1.1s for that. Binary size comparison:
121632 bytes old binary
129920 bytes new binary
I used gcc 4.4.3 to build ninja.
On Mac, I updated an existing checkout to the latest version and then ran:
CXX=../chrome-git/src/third_party/llvm-build/Release+Asserts/bin/clang++ \
./configure.py --with-gtest=../chrome-git/src/testing/gtest
./ninja
./ninja ninja_test # (all tests passed)
strip ninja
I timed empty builds of 'chrome' with the old and the new version,
both took 1.1s for that. Binary size comparison:
128976 bytes old binary
120904 bytes new binary
I used clang r155352 to build ninja. Last time I used gcc 4.2, which
explains the size drop.
The major new features in this update are:
* ninja cleans up files when interrupted mid-build
* NINJA_STATUS environment variable to tweak the status line
(see ninja manual)
Also set svn:mimetype to application/octet-stream on both files.
Review URL: https://chromiumcodereview.appspot.com/10221020/
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@134307 0039d316-1c4b-4281-b951-d872f2087c98
Built by running `./ninja && strip ninja` on
a (64bit) 10.6 Mac and on a 64bit linux box.
TBR=evan
Review URL: https://chromiumcodereview.appspot.com/9262002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@118205 0039d316-1c4b-4281-b951-d872f2087c98
The shell script currently just assumes Linux, but can be
fixed to run the Mac binary when appropriate.
Review URL: http://codereview.chromium.org/9212017
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@117723 0039d316-1c4b-4281-b951-d872f2087c98
Mark Mentovai
Nico Weber
Takuto Ikuta
thakis@chromium.org
evan@chromium.org