Commit Graph

38 Commits (816c2b3548fbb7c48a876484faac3e5f39c9416d)

Author SHA1 Message Date
Raul Tambre e9730d75a0 subprocess2: Ensure environment keys and values are strings on Python 3
Example error during "git cl upload":
Traceback (most recent call last):
  File "C:\Google\depot_tools\presubmit_support.py", line 220, in CallCommand
    returncode, stdout = self._RunWithTimeout(cmd, test.stdin, test.kwargs)
  File "C:\Google\depot_tools\presubmit_support.py", line 204, in _RunWithTimeout
    p = subprocess.Popen(cmd, **kwargs)
  File "C:\Google\depot_tools\subprocess2.py", line 143, in __init__
    super(Popen, self).__init__(args, **kwargs)
  File "C:\Google\depot_tools\bootstrap-3_8_0b1_chromium_1_bin\python\bin\Lib\subprocess.py", line 390, in __init__
    errread, errwrite)
  File "C:\Google\depot_tools\bootstrap-3_8_0b1_chromium_1_bin\python\bin\Lib\subprocess.py", line 640, in _execute_child
    startupinfo)
TypeError: environment can only contain strings

Bug: 984182
Change-Id: Id996d73a80845aaeaa061107ed123627091cb600
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1815872
Commit-Queue: Raul Tambre <raul@tambre.ee>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Auto-Submit: Raul Tambre <raul@tambre.ee>
5 years ago
Edward Lemur a81450253f depot_tools: Use mock from vpython (or unittest.mock) instead of third_party/mock
Change-Id: I3a188b34ae5f62649108afe08fe0e389a408c2ac
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1947933
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Anthony Polito <apolito@google.com>
5 years ago
Edward Lemur a877ee62b1 depot_tools: Run some tests on Python 3 on the bots.
Bug: 984182
Change-Id: I0b7f1e3e056548a27a416c9b4078e54c9e5d60d0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1769400
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@google.com>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
6 years ago
Edward Lemur 192e8a67cf subprocess2: Rewrite tests to be Python 3 compatible.
Also replace mox with mock and move the test script to a different file.

Bug: 984182
Change-Id: I9005a523c2abd82c38a7c61501c7cbfd4201a5b7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1745412
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
6 years ago
Edward Lemur 1556fbc353 depot_tools: Simplify subprocess2.
Remove support for unused features:
- timeout
- nag_timer and nag_max
- Popen start property
- callback functions for stdout and stderr

Bug: 984182
Change-Id: Ib2327119508a89d1e60f4a42c64b78d050f48092
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1745406
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
6 years ago
Raul Tambre 7c93846385 Convert except statements to be Python 3 compatible
Ran "2to3 -w -n -f except ./".

The scripts still work with Python 2.
There are no intended behaviour changes.

Bug: 942522
Change-Id: Ifa274cb83f74cfa8ce092fffbb88f3ab5309e72c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1607841
Commit-Queue: Raul Tambre <raul@tambre.ee>
Auto-Submit: Raul Tambre <raul@tambre.ee>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
6 years ago
Quinten Yearsley b2cc4a94da depot_tools: Replace pylint error numbers with symbolic names.
This affects a bunch of files, but only changes comments,
and shouldn't make any difference to behavior.

The purpose is to slightly improve readability of pylint
disable comments.

Change-Id: Ic6cd0f8de792b31d91c6125f6da2616450b30f11
Reviewed-on: https://chromium-review.googlesource.com/420412
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
8 years ago
Quinten Yearsley 442fb64cab depot_tools: Run automatic spell-checker.
Specifically, this CL was made by running codespell
(https://github.com/lucasdemarchi/codespell), manually filtering
for changes in non-third-party files that appear correct.

Change-Id: Ia16c1b29483d777744450d7bea45a178cf877a25
Reviewed-on: https://chromium-review.googlesource.com/420871
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
8 years ago
maruel@chromium.org 9717013017 Fix many (but not all[1]) depot_tools tests on Windows.
[1] *scm_test.py are deeply broken, svnadmin is missing for svn_bin\.
And the git test are failing at shutdown.

Also Fix SvnCheckout.testMove flakiness.

TBR=ilevy@chromium.org,szager@chromium.org
BUG=

Review URL: https://chromiumcodereview.appspot.com/14729012

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@198900 0039d316-1c4b-4281-b951-d872f2087c98
12 years ago
szager@chromium.org 12b07e7e55 Refactor nag functionality in to NagTimer class.
Add default 30 second nag timer to gclient subprocesses.

BUG=227537

Review URL: https://chromiumcodereview.appspot.com/14826003

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@198207 0039d316-1c4b-4281-b951-d872f2087c98
12 years ago
szager@chromium.org e0558e6502 Add nag_timer.
BUG=227537
R=maruel@chromium.org,cmp@chromium.org
NOTRY=true

Review URL: https://chromiumcodereview.appspot.com/14798003

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@197790 0039d316-1c4b-4281-b951-d872f2087c98
12 years ago
szager@chromium.org b054ebc8f3 Don't discard stderr.
R=maruel@chromium.org
NOTRY=true

Review URL: https://codereview.chromium.org/14643009

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@197419 0039d316-1c4b-4281-b951-d872f2087c98
12 years ago
maruel@chromium.org 0e023174cf Fix variable aliasing. Fixes pylint W0623.
I'm not sure why it hasn't triggered before.

TBR=dpranke@chromium.org
BUG=
TEST=


Review URL: http://codereview.chromium.org/9172002

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@117054 0039d316-1c4b-4281-b951-d872f2087c98
13 years ago
maruel@chromium.org 740a6c0bd5 Using lists is faster than cStringIO.
Clean up stdin management.
Remove stale comments.

R=dpranke@chromium.org
BUG=
TEST=


Review URL: http://codereview.chromium.org/8808002

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@113065 0039d316-1c4b-4281-b951-d872f2087c98
13 years ago
maruel@chromium.org bc3a53cb7b Improve subprocess2 stdin tests.
No significant code change.

R=dpranke@chromium.org
TEST=
BUG=


Review URL: http://codereview.chromium.org/8801019

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@113062 0039d316-1c4b-4281-b951-d872f2087c98
13 years ago
maruel@chromium.org 94c712fa1b Reimplement r109239 but using Popen.communicate() instead.
Enables threaded callback handler for subprocess.communicate().

R=dpranke@chromium.org
BUG=
TEST=


Review URL: http://codereview.chromium.org/8749015

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@112465 0039d316-1c4b-4281-b951-d872f2087c98
13 years ago
maruel@chromium.org a8e8163a06 Add Popen.shell, add more subprocess2 tests and make it more compact.
R=dpranke@chromium.org
BUG=
TEST=


Review URL: http://codereview.chromium.org/8750003

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@112351 0039d316-1c4b-4281-b951-d872f2087c98
13 years ago
maruel@chromium.org 14e37ad7ad Silence pylint on Mac OSX 10.6
Also, increase the wait for socket binding to 0.2 seconds because OSX10.6 is
broken _and_ slow.

R=dpranke@chromium.org
BUG=
TEST=


Review URL: http://codereview.chromium.org/8749005

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@112277 0039d316-1c4b-4281-b951-d872f2087c98
13 years ago
maruel@chromium.org db59bfc139 Improve testing by comparing behavior of subprocess to subprocess2
This makes sure no regression relative to subprocess is introduced in
subprocess2.

R=dpranke@chromium.org
BUG=
TEST=


Review URL: http://codereview.chromium.org/8722001

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@112217 0039d316-1c4b-4281-b951-d872f2087c98
13 years ago
maruel@chromium.org dd9837f9d9 Add Popen.start property.
R=dpranke@chromium.org
BUG=
TEST=


Review URL: http://codereview.chromium.org/8694006

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@112102 0039d316-1c4b-4281-b951-d872f2087c98
13 years ago
maruel@chromium.org 7bb06bbaa8 Move auto_stub.py from commit-queue
Use it in subprocess2 and trial_dir to remove manually bookeeping.

R=dpranke@chromium.org
BUG=
TEST=


Review URL: http://codereview.chromium.org/8688011

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@111941 0039d316-1c4b-4281-b951-d872f2087c98
13 years ago
maruel@chromium.org 93e2137df5 Add more subprocess2 tests with stderr=subproces2.STDOUT.
No code change beside tests. This is mainly a regression test to make sure I
don't regress that code path.

TBR=dpranke@chromium.org
BUG=
TEST=


Review URL: http://codereview.chromium.org/8695003

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@111535 0039d316-1c4b-4281-b951-d872f2087c98
13 years ago
maruel@google.com ef77f9e63a Make subprocess2.Popen a class instead of a function.
This will be necessary to override member functions eventually. It also better
replicates what subprocess.Popen is.

R=dpranke@chromium.org
BUG=
TEST=


Review URL: http://codereview.chromium.org/8570005

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@111530 0039d316-1c4b-4281-b951-d872f2087c98
13 years ago
maruel@chromium.org 4942e4a587 Add most of testing improvements without the tee-specific tests
Change return value from list to tuple for communicate(timeout != None).

R=dpranke@chromium.org
BUG=
TEST=


Review URL: http://codereview.chromium.org/8539015

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@110094 0039d316-1c4b-4281-b951-d872f2087c98
13 years ago
maruel@chromium.org 428342a0a7 Standardize the sys.path fix up and fix a few pylint warnings.
Disable temporarily W0403, will be reenabled on the next CL

R=dpranke@chromium.org
BUG=
TEST=

Review URL: http://codereview.chromium.org/8508017

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@109435 0039d316-1c4b-4281-b951-d872f2087c98
14 years ago
maruel@chromium.org 7eda862b1f Revert r109283, r109282 and r109239.
"Add callback support for stdout and stderr."
"Add tests for both universal_newlines=True and False."
"Implement accelerated tee support for POSIX."

The reason is that the semantic of
subprocess2.check_call(cmd, timeout=60, stderr=subprocess2.STDOUT) changed,
stderr wasn't redirected to VOID anymore and for an unknown reason, it was
hanging until the timeout happened. Will reapply r109239 with a new regression
test to detect that bug.

TBR=dpranke@chromium.org
BUG=
TEST=


Review URL: http://codereview.chromium.org/8505046

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@109369 0039d316-1c4b-4281-b951-d872f2087c98
14 years ago
maruel@chromium.org fefff18b55 Implement accelerated tee support for POSIX.
This removes all the need of threading, which removes the contention on the GIL
lock.

Taking S2Test.test_check_output_tee_large as baseline, numbers are
real/user/sys in seconds:
Ubuntu workstation: ~25x  2.4/1.9/1.5 -> 0.10/0.70/0.02
OSX 10.6 laptop:    ~40x  6.4/5.3/3.9 -> 0.15/0.80/0.07
Cygwin on win7:      ~4x  2.8/2.2/1.3 -> 0.60/0.16/0.30

R=dpranke@chromium.org
BUG=
TEST=


Review URL: http://codereview.chromium.org/8462008

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@109283 0039d316-1c4b-4281-b951-d872f2087c98
14 years ago
maruel@chromium.org f2dca4e174 Add tests for both universal_newlines=True and False.
Fix stdout+stderr on Windows to binary mode in the child test process to make
sure testing is done correctly.

R=dpranke@chromium.org
BUG=
TEST=manually tested on OSX, Windows, cygwin and ubuntu


Review URL: http://codereview.chromium.org/8472002

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@109282 0039d316-1c4b-4281-b951-d872f2087c98
14 years ago
maruel@chromium.org 65be6f6bcf Add callback support for stdout and stderr.
It's currently an inefficient thread implementation. Interestingly
enough, callback support is significantly faster on cygwin than on
native python.

Writing an efficient implementation is punted for a later change,
one per implementation.

Stops using a temporary file since it's not necessary anymore.

The goal is to reduce the number of places where a similar paradigm
is used by having a canonical generic implementation.

R=dpranke@chromium.org
BUG=
TEST=Tested manually on Windows, cygwin, linux, OSX 10.6


Review URL: http://codereview.chromium.org/8374026

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@109239 0039d316-1c4b-4281-b951-d872f2087c98
14 years ago
maruel@chromium.org 37ca0b1cdd Fix tests on Windows
R=dpranke@chromium.org
BUG=
TEST=


Review URL: http://codereview.chromium.org/8366042

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@106866 0039d316-1c4b-4281-b951-d872f2087c98
14 years ago
maruel@chromium.org 87e6d33103 Update subprocess2.check_output() to behave like subprocess.check_output().
stderr is not redirected by default. stdout is not allowed.
Both were oversight.
Do not override stdin=None in case the user would response to stderr output for
example.

Increase test coverage.

R=dpranke@chromium.org
BUG=
TEST=


Review URL: http://codereview.chromium.org/7860041

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@100456 0039d316-1c4b-4281-b951-d872f2087c98
14 years ago
maruel@chromium.org 0bcd1d3430 Make subprocess2.check_call() compliant with subprocess.check_call().
Rename check_call to check_call_out. It's a quite specific need when stderr is
needed or when the user doesn't want the default arguments of check_output.

R=dpranke@chromium.org
BUG=
TEST=

Review URL: http://codereview.chromium.org/6882127

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@83023 0039d316-1c4b-4281-b951-d872f2087c98
14 years ago
maruel@chromium.org 1f063db16a Make subprocess2.call() returned values to be the same as subprocess.call().
Renamed the previous subprocess2.call() to subprocess2.communicate().

R=dpranke@chromium.org
BUG=
TEST=

Review URL: http://codereview.chromium.org/6877005

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@81976 0039d316-1c4b-4281-b951-d872f2087c98
14 years ago
maruel@chromium.org 4a982271a7 Defaults stdin to VOID for capture and check_output()
Since no output is user visible anyway, causing a hang with no clue about what
to type in.

R=dpranke@chromium.org
BUG=
TEST=

Review URL: http://codereview.chromium.org/6823091

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@81302 0039d316-1c4b-4281-b951-d872f2087c98
14 years ago
maruel@chromium.org 1d9f62946b Set returncode to subprocess2.TIMED_OUT instead of -9 when a process times out.
R=dpranke@chromium.org
BUG=
TEST=

Review URL: http://codereview.chromium.org/6806009

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@80775 0039d316-1c4b-4281-b951-d872f2087c98
14 years ago
maruel@chromium.org c98c0c5249 Make more tests pass on Windows.
Also fix a few issues found along the way.

Tests had regressed a lot. Add a lot of tweaks to make most test pass.

R=dpranke@chromium.org
BUG=
TEST=

Review URL: http://codereview.chromium.org/6792060

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@80618 0039d316-1c4b-4281-b951-d872f2087c98
14 years ago
maruel@chromium.org f08b09c2af Fix automatic shell detection, the check was reversed
Improved tests and added regression test.

R=dpranke@chromium.org
BUG=
TEST=

Review URL: http://codereview.chromium.org/6793044

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@80614 0039d316-1c4b-4281-b951-d872f2087c98
14 years ago
maruel@chromium.org eba4022de8 Add subprocess2 unit tests
R=dpranke@chromium.org
BUG=
TEST=

Review URL: http://codereview.chromium.org/6693078

git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@80454 0039d316-1c4b-4281-b951-d872f2087c98
14 years ago