You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
depot_tools/recipes/recipe_modules/gsutil/properties.proto

13 lines
396 B
Protocol Buffer

Add a method to gsutil recipe_module to disable multiprocessing on mac In https://crbug.com/1327371, we started encountering hanging gsutil calls. These were triggered under very specific circumstances, but were essentially due to a multi-processing bug in python on MacOS: https://bugs.python.org/issue33725 When running gsutil on mac, it explicitly suggests disabling multi-processing for this reason: https://source.chromium.org/chromium/chromium/src/+/main:third_party/catapult/third_party/gsutil/gslib/command.py;drc=3a12d6ccdec28da8bda09d9ff826aae1f9504e59;l=1331 So this CL simply puts that suggestion into practice. There are two options for doing so: adding a line to an active Boto file, or adding a "-o" arg to every gsutil invocation. Since chromium has multiple copies of gsutil and multiple different locations where `gsutil` is invoked, the latter option is considered intractable. Instead, we choose the Boto file option by adding a context manager to the gsutil recipe module that will insert a custom Boto file into the environment that only disables multi-threading. Due to the fact that the BOTO_PATH env var takes multiple paths, we can safely incorporate both our custom Boto along with LUCI's per-build Boto file in the same gsutil invocation. And when the context manager exits, the environment should revert back to its original state. Chromium incorporates this method in crrev.com/c/3662023. See it take effect during the "gclient runhooks" step in this led build: https://ci.chromium.org/swarming/task/5b0c5174e70cb010?server=chromium-swarm.appspot.com Bug: 1327371 Change-Id: I75cfdf16d0ec023bf81ea57991dde996694a46c6 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3661949 Reviewed-by: Josip Sokcevic <sokcevic@google.com> Commit-Queue: Ben Pastene <bpastene@chromium.org>
3 years ago
// Copyright 2022 The LUCI Authors. All rights reserved.
// Use of this source code is governed under the Apache License, Version 2.0
// that can be found in the LICENSE file.
syntax = "proto3";
package recipe_modules.depot_tools.gsutil;
message EnvProperties {
// Env vars that gsutil uses for both authentication and configuration.
string BOTO_CONFIG = 1;
string BOTO_PATH = 2;
}