Analyze DEPS autorolls

This new logic tries to figure out which files really changed in a DEPS roll by checking out the old DEPS file and recursively running git diff on all of the repos listed in DEPS. When successful, affected_files turns from just ['DEPS'] into an actual list of affected files. When not successful, it falls back to the original logic.

When DEPS is autorolled, typically the change is just an updated git revision. We don't apply this logic to manual DEPS rolls, as they can contain changes to other variables, whose effects are not easily discernible.

MUST be applied in tandem with https://crrev.com/c/2250928

Bug: 923016
Change-Id: I9e33461ea4ae81dc056d229f8604943745e30389
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2283855
Commit-Queue: Gregory Guterman <guterman@google.com>
Reviewed-by: Garrett Beaty <gbeaty@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>

recipes/README.recipes.md

@@ -26,6 +26,7 @@
   * [depot_tools:examples/full](#recipes-depot_tools_examples_full)
   * [fetch_end_to_end_test](#recipes-fetch_end_to_end_test)
   * [gclient:examples/full](#recipes-gclient_examples_full)
+  * [gclient:tests/diff_deps](#recipes-gclient_tests_diff_deps)
   * [gclient:tests/patch_project](#recipes-gclient_tests_patch_project)
   * [gclient:tests/sync_failure](#recipes-gclient_tests_sync_failure)
   * [gerrit:examples/full](#recipes-gerrit_examples_full)
@@ -259,28 +260,32 @@ Returns (Path): The "depot_tools" root directory.
 &emsp; **@property**<br>&mdash; **def [upload\_to\_google\_storage\_path](/recipes/recipe_modules/depot_tools/api.py#17)(self):**
 ### *recipe_modules* / [gclient](/recipes/recipe_modules/gclient)
 
-[DEPS](/recipes/recipe_modules/gclient/__init__.py#1): [gitiles](#recipe_modules-gitiles), [tryserver](#recipe_modules-tryserver), [recipe\_engine/context][recipe_engine/recipe_modules/context], [recipe\_engine/json][recipe_engine/recipe_modules/json], [recipe\_engine/path][recipe_engine/recipe_modules/path], [recipe\_engine/platform][recipe_engine/recipe_modules/platform], [recipe\_engine/properties][recipe_engine/recipe_modules/properties], [recipe\_engine/python][recipe_engine/recipe_modules/python], [recipe\_engine/step][recipe_engine/recipe_modules/step]
+[DEPS](/recipes/recipe_modules/gclient/__init__.py#1): [git](#recipe_modules-git), [gitiles](#recipe_modules-gitiles), [tryserver](#recipe_modules-tryserver), [recipe\_engine/context][recipe_engine/recipe_modules/context], [recipe\_engine/json][recipe_engine/recipe_modules/json], [recipe\_engine/path][recipe_engine/recipe_modules/path], [recipe\_engine/platform][recipe_engine/recipe_modules/platform], [recipe\_engine/properties][recipe_engine/recipe_modules/properties], [recipe\_engine/python][recipe_engine/recipe_modules/python], [recipe\_engine/raw\_io][recipe_engine/recipe_modules/raw_io], [recipe\_engine/step][recipe_engine/recipe_modules/step]
 
-#### **class [GclientApi](/recipes/recipe_modules/gclient/api.py#65)([RecipeApi][recipe_engine/wkt/RecipeApi]):**
+#### **class [GclientApi](/recipes/recipe_modules/gclient/api.py#68)([RecipeApi][recipe_engine/wkt/RecipeApi]):**
 
-&mdash; **def [\_\_call\_\_](/recipes/recipe_modules/gclient/api.py#75)(self, name, cmd, infra_step=True, \*\*kwargs):**
+&emsp; **@property**<br>&mdash; **def [DepsDiffException](/recipes/recipe_modules/gclient/api.py#426)(self):**
+
+&mdash; **def [\_\_call\_\_](/recipes/recipe_modules/gclient/api.py#78)(self, name, cmd, infra_step=True, \*\*kwargs):**
 
 Wrapper for easy calling of gclient steps.
 
-&mdash; **def [break\_locks](/recipes/recipe_modules/gclient/api.py#271)(self):**
+&mdash; **def [break\_locks](/recipes/recipe_modules/gclient/api.py#274)(self):**
 
 Remove all index.lock files. If a previous run of git crashed, bot was
 reset, etc... we might end up with leftover index.lock files.
 
-&mdash; **def [checkout](/recipes/recipe_modules/gclient/api.py#231)(self, gclient_config=None, revert=RevertOnTryserver, inject_parent_got_revision=True, extra_sync_flags=None, \*\*kwargs):**
+&mdash; **def [checkout](/recipes/recipe_modules/gclient/api.py#234)(self, gclient_config=None, revert=RevertOnTryserver, inject_parent_got_revision=True, extra_sync_flags=None, \*\*kwargs):**
 
 Return a step generator function for gclient checkouts.
 
-&emsp; **@staticmethod**<br>&mdash; **def [config\_to\_pythonish](/recipes/recipe_modules/gclient/api.py#120)(cfg):**
+&emsp; **@staticmethod**<br>&mdash; **def [config\_to\_pythonish](/recipes/recipe_modules/gclient/api.py#123)(cfg):**
+
+&mdash; **def [diff\_deps](/recipes/recipe_modules/gclient/api.py#365)(self, cwd):**
 
-&mdash; **def [get\_config\_defaults](/recipes/recipe_modules/gclient/api.py#114)(self):**
+&mdash; **def [get\_config\_defaults](/recipes/recipe_modules/gclient/api.py#117)(self):**
 
-&mdash; **def [get\_gerrit\_patch\_root](/recipes/recipe_modules/gclient/api.py#293)(self, gclient_config=None):**
+&mdash; **def [get\_gerrit\_patch\_root](/recipes/recipe_modules/gclient/api.py#296)(self, gclient_config=None):**
 
 Returns local path to the repo where gerrit patch will be applied.
 
@@ -293,7 +298,7 @@ Instead, properly map a repository to a local path using repo_path_map.
 TODO(nodir): remove this. Update all recipe tests to specify a git_repo
 matching the recipe.
 
-&mdash; **def [get\_repo\_path](/recipes/recipe_modules/gclient/api.py#320)(self, repo_url, gclient_config=None):**
+&mdash; **def [get\_repo\_path](/recipes/recipe_modules/gclient/api.py#323)(self, repo_url, gclient_config=None):**
 
 Returns local path to the repo checkout given its url.
 
@@ -301,7 +306,7 @@ Consults cfg.repo_path_map and fallbacks to urls in configured solutions.
 
 Returns None if not found.
 
-&emsp; **@staticmethod**<br>&mdash; **def [got\_revision\_reverse\_mapping](/recipes/recipe_modules/gclient/api.py#125)(cfg):**
+&emsp; **@staticmethod**<br>&mdash; **def [got\_revision\_reverse\_mapping](/recipes/recipe_modules/gclient/api.py#128)(cfg):**
 
 Returns the merged got_revision_reverse_mapping.
 
@@ -309,7 +314,7 @@ Returns (dict): A mapping from property name -> project name. It merges the
     values of the deprecated got_revision_mapping and the new
     got_revision_reverse_mapping.
 
-&mdash; **def [inject\_parent\_got\_revision](/recipes/recipe_modules/gclient/api.py#206)(self, gclient_config=None, override=False):**
+&mdash; **def [inject\_parent\_got\_revision](/recipes/recipe_modules/gclient/api.py#209)(self, gclient_config=None, override=False):**
 
 Match gclient config to build revisions obtained from build_properties.
 
@@ -319,22 +324,22 @@ Args:
   override (bool) - If True, will forcibly set revision and custom_vars
     even if the config already contains values for them.
 
-&mdash; **def [resolve\_revision](/recipes/recipe_modules/gclient/api.py#142)(self, revision):**
+&mdash; **def [resolve\_revision](/recipes/recipe_modules/gclient/api.py#145)(self, revision):**
 
-&mdash; **def [runhooks](/recipes/recipe_modules/gclient/api.py#265)(self, args=None, name='runhooks', \*\*kwargs):**
+&mdash; **def [runhooks](/recipes/recipe_modules/gclient/api.py#268)(self, args=None, name='runhooks', \*\*kwargs):**
 
-&mdash; **def [set\_patch\_repo\_revision](/recipes/recipe_modules/gclient/api.py#350)(self, gclient_config=None):**
+&mdash; **def [set\_patch\_repo\_revision](/recipes/recipe_modules/gclient/api.py#353)(self, gclient_config=None):**
 
 Updates config revision corresponding to patched project.
 
 Useful for bot_update only, as this is the only consumer of gclient's config
 revision map. This doesn't overwrite the revision if it was already set.
 
-&emsp; **@spec_alias.deleter**<br>&mdash; **def [spec\_alias](/recipes/recipe_modules/gclient/api.py#110)(self):**
+&emsp; **@spec_alias.deleter**<br>&mdash; **def [spec\_alias](/recipes/recipe_modules/gclient/api.py#113)(self):**
 
-&mdash; **def [sync](/recipes/recipe_modules/gclient/api.py#147)(self, cfg, extra_sync_flags=None, \*\*kwargs):**
+&mdash; **def [sync](/recipes/recipe_modules/gclient/api.py#150)(self, cfg, extra_sync_flags=None, \*\*kwargs):**
 
-&emsp; **@use_mirror.setter**<br>&mdash; **def [use\_mirror](/recipes/recipe_modules/gclient/api.py#97)(self, val):**
+&emsp; **@use_mirror.setter**<br>&mdash; **def [use\_mirror](/recipes/recipe_modules/gclient/api.py#100)(self, val):**
 ### *recipe_modules* / [gerrit](/recipes/recipe_modules/gerrit)
 
 [DEPS](/recipes/recipe_modules/gerrit/__init__.py#1): [recipe\_engine/context][recipe_engine/recipe_modules/context], [recipe\_engine/json][recipe_engine/recipe_modules/json], [recipe\_engine/path][recipe_engine/recipe_modules/path], [recipe\_engine/python][recipe_engine/recipe_modules/python], [recipe\_engine/raw\_io][recipe_engine/recipe_modules/raw_io], [recipe\_engine/step][recipe_engine/recipe_modules/step]
@@ -953,6 +958,11 @@ Raises:
 [DEPS](/recipes/recipe_modules/gclient/examples/full.py#5): [gclient](#recipe_modules-gclient), [recipe\_engine/buildbucket][recipe_engine/recipe_modules/buildbucket], [recipe\_engine/context][recipe_engine/recipe_modules/context], [recipe\_engine/path][recipe_engine/recipe_modules/path], [recipe\_engine/properties][recipe_engine/recipe_modules/properties], [recipe\_engine/step][recipe_engine/recipe_modules/step]
 
 &mdash; **def [RunSteps](/recipes/recipe_modules/gclient/examples/full.py#55)(api):**
+### *recipes* / [gclient:tests/diff\_deps](/recipes/recipe_modules/gclient/tests/diff_deps.py)
+
+[DEPS](/recipes/recipe_modules/gclient/tests/diff_deps.py#7): [gclient](#recipe_modules-gclient), [recipe\_engine/assertions][recipe_engine/recipe_modules/assertions], [recipe\_engine/buildbucket][recipe_engine/recipe_modules/buildbucket], [recipe\_engine/path][recipe_engine/recipe_modules/path], [recipe\_engine/platform][recipe_engine/recipe_modules/platform], [recipe\_engine/properties][recipe_engine/recipe_modules/properties], [recipe\_engine/raw\_io][recipe_engine/recipe_modules/raw_io]
+
+&mdash; **def [RunSteps](/recipes/recipe_modules/gclient/tests/diff_deps.py#18)(api):**
 ### *recipes* / [gclient:tests/patch\_project](/recipes/recipe_modules/gclient/tests/patch_project.py)
 
 [DEPS](/recipes/recipe_modules/gclient/tests/patch_project.py#9): [gclient](#recipe_modules-gclient), [recipe\_engine/buildbucket][recipe_engine/recipe_modules/buildbucket], [recipe\_engine/path][recipe_engine/recipe_modules/path], [recipe\_engine/properties][recipe_engine/recipe_modules/properties]
@@ -1036,6 +1046,7 @@ Move things around in a loop!
 
 &mdash; **def [RunSteps](/recipes/recipe_modules/windows_sdk/examples/full.py#13)(api):**
 
+[recipe_engine/recipe_modules/assertions]: https://chromium.googlesource.com/infra/luci/recipes-py.git/+/59e7d3f289e8f160db10bbcd311b299805080c4b/README.recipes.md#recipe_modules-assertions
 [recipe_engine/recipe_modules/buildbucket]: https://chromium.googlesource.com/infra/luci/recipes-py.git/+/59e7d3f289e8f160db10bbcd311b299805080c4b/README.recipes.md#recipe_modules-buildbucket
 [recipe_engine/recipe_modules/cipd]: https://chromium.googlesource.com/infra/luci/recipes-py.git/+/59e7d3f289e8f160db10bbcd311b299805080c4b/README.recipes.md#recipe_modules-cipd
 [recipe_engine/recipe_modules/commit_position]: https://chromium.googlesource.com/infra/luci/recipes-py.git/+/59e7d3f289e8f160db10bbcd311b299805080c4b/README.recipes.md#recipe_modules-commit_position

recipes/recipe_modules/gclient/__init__.py

@@ -1,4 +1,5 @@
 DEPS = [
+  'git',
   'gitiles',
   'recipe_engine/context',
   'recipe_engine/json',
@@ -6,6 +7,7 @@ DEPS = [
   'recipe_engine/platform',
   'recipe_engine/properties',
   'recipe_engine/python',
+  'recipe_engine/raw_io',
   'recipe_engine/step',
   'tryserver',
 ]

recipes/recipe_modules/gclient/api.py

@@ -2,8 +2,11 @@
 # Use of this source code is governed by a BSD-style license that can be
 # found in the LICENSE file.
 
+import re
 from recipe_engine import recipe_api
 
+class DepsDiffException(Exception):
+  pass
 
 class RevisionResolver(object):
   """Resolves the revision based on build properties."""
@@ -358,3 +361,68 @@ class GclientApi(recipe_api.RecipeApi):
     path, revision = cfg.repo_path_map.get(repo_url, (None, None))
     if path and revision and path not in cfg.revisions:
       cfg.revisions[path] = revision
+
+  def diff_deps(self, cwd):
+    cwd = cwd.join(self.get_gerrit_patch_root())
+    with self.m.context(cwd=cwd):
+      step_result = self.m.git(
+          '-c',
+          'core.quotePath=false',
+          'checkout',
+          'HEAD~',
+          '--',
+          'DEPS',
+          name='checkout the previous DEPS',
+          stdout=self.m.raw_io.output()
+      )
+
+      try:
+        cfg = self.c
+
+        step_result = self(
+            'recursively git diff all DEPS',
+            [
+                'recurse',
+                self.resource('diff_deps.py'),
+            ],
+            stdout=self.m.raw_io.output_text(add_output_log=True),
+        )
+
+        paths = []
+        # gclient recurse prepends a number and a > to each line
+        # Let's take that out
+        for line in step_result.stdout.strip().splitlines():
+          if 'fatal: bad object' in line:
+            msg = "Couldn't checkout previous ref: %s" % line
+            step_result.presentation.logs['DepsDiffException'] = msg
+            raise self.DepsDiffException(msg)
+          elif re.match('\d+>', line):
+            paths.append(line[line.index('>') + 1:])
+
+
+        # Normalize paths
+        if self.m.platform.is_win:
+          # Looks like "analyze" wants POSIX slashes even on Windows (since git
+          # uses that format even on Windows).
+          paths = [path.replace('\\', '/') for path in paths]
+
+        if len(paths) > 0:
+          return paths
+        else:
+          msg = 'Unexpected result: autoroll diff found 0 files changed'
+          step_result.presentation.logs['DepsDiffException'] = msg
+          raise self.DepsDiffException(msg)
+
+      finally:
+        self.m.git(
+            '-c',
+            'core.quotePath=false',
+            'checkout',
+            'HEAD',
+            '--',
+            'DEPS',
+            name="checkout the original DEPS")
+
+  @property
+  def DepsDiffException(self):
+    return DepsDiffException

recipes/recipe_modules/gclient/resources/diff_deps.py

@@ -0,0 +1,16 @@
+#!/usr/bin/env python
+import os
+import subprocess
+import sys
+
+if '@' in os.environ.get('GCLIENT_URL'):
+  ref = os.environ.get('GCLIENT_URL').split('@')[-1]
+else:
+  sys.exit(0)
+
+diff = subprocess.check_output("git diff --cached --name-only %s" % ref, shell=True)
+
+dep_path = os.environ.get('GCLIENT_DEP_PATH', '')
+for line in diff.splitlines():
+  if line:
+    print(os.path.join(dep_path,  line))

recipes/recipe_modules/gclient/test_api.py

@@ -7,6 +7,10 @@ import hashlib
 from recipe_engine import recipe_test_api
 
 class GclientTestApi(recipe_test_api.RecipeTestApi):
+  def diff_deps_test_data(self, files):
+    return self.m.raw_io.stream_output(
+        '\n'.join(['10>%s' % fname for fname in files]))
+
   def output_json(self, projects):
     """Deterministically synthesize json.output test data for gclient's
     --output-json option.

recipes/recipe_modules/gclient/tests/diff_deps.expected/basic.json

@@ -0,0 +1,55 @@
+[
+  {
+    "cmd": [
+      "git",
+      "-c",
+      "core.quotePath=false",
+      "checkout",
+      "HEAD~",
+      "--",
+      "DEPS"
+    ],
+    "cwd": "[CACHE]/src",
+    "infra_step": true,
+    "name": "checkout the previous DEPS"
+  },
+  {
+    "cmd": [
+      "python",
+      "-u",
+      "RECIPE_REPO[depot_tools]/gclient.py",
+      "recurse",
+      "RECIPE_MODULE[depot_tools::gclient]/resources/diff_deps.py"
+    ],
+    "cwd": "[CACHE]/src",
+    "env_suffixes": {
+      "PATH": [
+        "RECIPE_REPO[depot_tools]"
+      ]
+    },
+    "infra_step": true,
+    "name": "gclient recursively git diff all DEPS",
+    "~followup_annotations": [
+      "@@@STEP_LOG_LINE@raw_io.output_text@10>third_party/mockfile1@@@",
+      "@@@STEP_LOG_LINE@raw_io.output_text@10>third_party/mockfile2@@@",
+      "@@@STEP_LOG_END@raw_io.output_text@@@"
+    ]
+  },
+  {
+    "cmd": [
+      "git",
+      "-c",
+      "core.quotePath=false",
+      "checkout",
+      "HEAD",
+      "--",
+      "DEPS"
+    ],
+    "cwd": "[CACHE]/src",
+    "infra_step": true,
+    "name": "checkout the original DEPS"
+  },
+  {
+    "name": "$result"
+  }
+]

recipes/recipe_modules/gclient/tests/diff_deps.expected/dont have revision yet.json

@@ -0,0 +1,84 @@
+[
+  {
+    "cmd": [
+      "git",
+      "-c",
+      "core.quotePath=false",
+      "checkout",
+      "HEAD~",
+      "--",
+      "DEPS"
+    ],
+    "cwd": "[CACHE]/src",
+    "infra_step": true,
+    "name": "checkout the previous DEPS"
+  },
+  {
+    "cmd": [
+      "python",
+      "-u",
+      "RECIPE_REPO[depot_tools]/gclient.py",
+      "recurse",
+      "RECIPE_MODULE[depot_tools::gclient]/resources/diff_deps.py"
+    ],
+    "cwd": "[CACHE]/src",
+    "env_suffixes": {
+      "PATH": [
+        "RECIPE_REPO[depot_tools]"
+      ]
+    },
+    "infra_step": true,
+    "name": "gclient recursively git diff all DEPS",
+    "~followup_annotations": [
+      "@@@STEP_LOG_LINE@raw_io.output_text@fatal: bad object abcdef1234567890@@@",
+      "@@@STEP_LOG_END@raw_io.output_text@@@",
+      "@@@STEP_LOG_LINE@DepsDiffException@Couldn't checkout previous ref: fatal: bad object abcdef1234567890@@@",
+      "@@@STEP_LOG_END@DepsDiffException@@@"
+    ]
+  },
+  {
+    "cmd": [
+      "git",
+      "-c",
+      "core.quotePath=false",
+      "checkout",
+      "HEAD",
+      "--",
+      "DEPS"
+    ],
+    "cwd": "[CACHE]/src",
+    "infra_step": true,
+    "name": "checkout the original DEPS"
+  },
+  {
+    "cmd": [],
+    "name": "RECIPE CRASH (Uncaught exception)",
+    "~followup_annotations": [
+      "@@@STEP_EXCEPTION@@@",
+      "The recipe has crashed at point 'Uncaught exception'!",
+      "",
+      "Traceback (most recent call last):",
+      "  File \"RECIPE_REPO[recipe_engine]/recipe_engine/internal/engine.py\", in run_steps",
+      "    raw_result = recipe_obj.run_steps(api, engine)",
+      "  File \"RECIPE_REPO[recipe_engine]/recipe_engine/internal/recipe_deps.py\", in run_steps",
+      "    properties_def, api=api)",
+      "  File \"RECIPE_REPO[recipe_engine]/recipe_engine/internal/property_invoker.py\", in invoke_with_properties",
+      "    arg_names, **additional_args)",
+      "  File \"RECIPE_REPO[recipe_engine]/recipe_engine/internal/property_invoker.py\", in _invoke_with_properties",
+      "    return callable_obj(*props, **additional_args)",
+      "  File \"RECIPE_REPO[depot_tools]/recipes/recipe_modules/gclient/tests/diff_deps.py\", line 33, in RunSteps",
+      "    affected_files = api.gclient.diff_deps(api.path['cache'])",
+      "  File \"RECIPE_REPO[recipe_engine]/recipe_engine/recipe_api.py\", in _inner",
+      "    return func(*a, **kw)",
+      "  File \"RECIPE_REPO[depot_tools]/recipes/recipe_modules/gclient/api.py\", line 398, in diff_deps",
+      "    raise self.DepsDiffException(msg)",
+      "DepsDiffException: Couldn't checkout previous ref: fatal: bad object abcdef1234567890"
+    ]
+  },
+  {
+    "failure": {
+      "humanReason": "Uncaught Exception: DepsDiffException(\"Couldn't checkout previous ref: fatal: bad object abcdef1234567890\",)"
+    },
+    "name": "$result"
+  }
+]

recipes/recipe_modules/gclient/tests/diff_deps.expected/no change, exception.json

@@ -0,0 +1,83 @@
+[
+  {
+    "cmd": [
+      "git",
+      "-c",
+      "core.quotePath=false",
+      "checkout",
+      "HEAD~",
+      "--",
+      "DEPS"
+    ],
+    "cwd": "[CACHE]/src",
+    "infra_step": true,
+    "name": "checkout the previous DEPS"
+  },
+  {
+    "cmd": [
+      "python",
+      "-u",
+      "RECIPE_REPO[depot_tools]/gclient.py",
+      "recurse",
+      "RECIPE_MODULE[depot_tools::gclient]/resources/diff_deps.py"
+    ],
+    "cwd": "[CACHE]/src",
+    "env_suffixes": {
+      "PATH": [
+        "RECIPE_REPO[depot_tools]"
+      ]
+    },
+    "infra_step": true,
+    "name": "gclient recursively git diff all DEPS",
+    "~followup_annotations": [
+      "@@@STEP_LOG_END@raw_io.output_text@@@",
+      "@@@STEP_LOG_LINE@DepsDiffException@Unexpected result: autoroll diff found 0 files changed@@@",
+      "@@@STEP_LOG_END@DepsDiffException@@@"
+    ]
+  },
+  {
+    "cmd": [
+      "git",
+      "-c",
+      "core.quotePath=false",
+      "checkout",
+      "HEAD",
+      "--",
+      "DEPS"
+    ],
+    "cwd": "[CACHE]/src",
+    "infra_step": true,
+    "name": "checkout the original DEPS"
+  },
+  {
+    "cmd": [],
+    "name": "RECIPE CRASH (Uncaught exception)",
+    "~followup_annotations": [
+      "@@@STEP_EXCEPTION@@@",
+      "The recipe has crashed at point 'Uncaught exception'!",
+      "",
+      "Traceback (most recent call last):",
+      "  File \"RECIPE_REPO[recipe_engine]/recipe_engine/internal/engine.py\", in run_steps",
+      "    raw_result = recipe_obj.run_steps(api, engine)",
+      "  File \"RECIPE_REPO[recipe_engine]/recipe_engine/internal/recipe_deps.py\", in run_steps",
+      "    properties_def, api=api)",
+      "  File \"RECIPE_REPO[recipe_engine]/recipe_engine/internal/property_invoker.py\", in invoke_with_properties",
+      "    arg_names, **additional_args)",
+      "  File \"RECIPE_REPO[recipe_engine]/recipe_engine/internal/property_invoker.py\", in _invoke_with_properties",
+      "    return callable_obj(*props, **additional_args)",
+      "  File \"RECIPE_REPO[depot_tools]/recipes/recipe_modules/gclient/tests/diff_deps.py\", line 33, in RunSteps",
+      "    affected_files = api.gclient.diff_deps(api.path['cache'])",
+      "  File \"RECIPE_REPO[recipe_engine]/recipe_engine/recipe_api.py\", in _inner",
+      "    return func(*a, **kw)",
+      "  File \"RECIPE_REPO[depot_tools]/recipes/recipe_modules/gclient/api.py\", line 414, in diff_deps",
+      "    raise self.DepsDiffException(msg)",
+      "DepsDiffException: Unexpected result: autoroll diff found 0 files changed"
+    ]
+  },
+  {
+    "failure": {
+      "humanReason": "Uncaught Exception: DepsDiffException('Unexpected result: autoroll diff found 0 files changed',)"
+    },
+    "name": "$result"
+  }
+]

recipes/recipe_modules/gclient/tests/diff_deps.expected/windows.json

@@ -0,0 +1,55 @@
+[
+  {
+    "cmd": [
+      "git",
+      "-c",
+      "core.quotePath=false",
+      "checkout",
+      "HEAD~",
+      "--",
+      "DEPS"
+    ],
+    "cwd": "[CACHE]\\src",
+    "infra_step": true,
+    "name": "checkout the previous DEPS"
+  },
+  {
+    "cmd": [
+      "python",
+      "-u",
+      "RECIPE_REPO[depot_tools]\\gclient.py",
+      "recurse",
+      "RECIPE_MODULE[depot_tools::gclient]\\resources\\diff_deps.py"
+    ],
+    "cwd": "[CACHE]\\src",
+    "env_suffixes": {
+      "PATH": [
+        "RECIPE_REPO[depot_tools]"
+      ]
+    },
+    "infra_step": true,
+    "name": "gclient recursively git diff all DEPS",
+    "~followup_annotations": [
+      "@@@STEP_LOG_LINE@raw_io.output_text@10>third_party/mockfile1@@@",
+      "@@@STEP_LOG_LINE@raw_io.output_text@10>third_party/mockfile2@@@",
+      "@@@STEP_LOG_END@raw_io.output_text@@@"
+    ]
+  },
+  {
+    "cmd": [
+      "git",
+      "-c",
+      "core.quotePath=false",
+      "checkout",
+      "HEAD",
+      "--",
+      "DEPS"
+    ],
+    "cwd": "[CACHE]\\src",
+    "infra_step": true,
+    "name": "checkout the original DEPS"
+  },
+  {
+    "name": "$result"
+  }
+]

recipes/recipe_modules/gclient/tests/diff_deps.py

@@ -0,0 +1,88 @@
+# Copyright 2020 The Chromium Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+from recipe_engine import post_process
+
+DEPS = [
+  'recipe_engine/assertions',
+  'recipe_engine/buildbucket',
+  'recipe_engine/path',
+  'recipe_engine/platform',
+  'recipe_engine/properties',
+  'recipe_engine/raw_io',
+
+  'gclient',
+]
+
+def RunSteps(api):
+  src_cfg = api.gclient.make_config(CACHE_DIR=api.path['cache'].join('git'))
+
+  soln = src_cfg.solutions.add()
+  soln.name = 'src'
+  soln.url = 'https://chromium.googlesource.com/chromium/src.git'
+  src_cfg.repo_path_map.update({
+      'https://chromium.googlesource.com/src': ('src', 'HEAD'),
+      'https://chromium.googlesource.com/v8/v8': ('src/v8', 'HEAD'),
+      # non-canonical URL
+      'https://webrtc.googlesource.com/src.git': (
+          'src/third_party/webrtc', 'HEAD'),
+  })
+
+  api.gclient.c = src_cfg
+  affected_files = api.gclient.diff_deps(api.path['cache'])
+
+  api.assertions.assertEqual(
+      affected_files,
+      list(api.properties.get('diff_deps_files')),
+  )
+
+def GenTests(api):
+  test_files = (
+      'third_party/mockfile1',
+      'third_party/mockfile2'
+  )
+  no_test_files = []
+
+  yield api.test(
+      'basic',
+      api.buildbucket.try_build(),
+      api.properties(diff_deps_files=test_files),
+      api.override_step_data(
+          'gclient recursively git diff all DEPS',
+          api.gclient.diff_deps_test_data(test_files),
+      ),
+      api.post_process(post_process.StatusSuccess),
+  )
+
+  yield api.test(
+      'no change, exception',
+      api.buildbucket.try_build(),
+      api.properties(diff_deps_files=no_test_files),
+      api.override_step_data(
+          'gclient recursively git diff all DEPS',
+          api.gclient.diff_deps_test_data(no_test_files),
+      ),
+      api.expect_exception('DepsDiffException')
+  )
+  yield api.test(
+      'dont have revision yet',
+      api.buildbucket.try_build(),
+      api.properties(diff_deps_files=test_files),
+      api.override_step_data(
+          'gclient recursively git diff all DEPS',
+          api.raw_io.stream_output('fatal: bad object abcdef1234567890'),
+      ),
+      api.expect_exception('DepsDiffException')
+  )
+  yield api.test(
+      'windows',
+      api.buildbucket.try_build(),
+      api.properties(diff_deps_files=test_files),
+      api.platform.name('win'),
+      api.override_step_data(
+          'gclient recursively git diff all DEPS',
+          api.gclient.diff_deps_test_data(test_files),
+      ),
+      api.post_process(post_process.StatusSuccess),
+  )