This is because the linux-presubmit ci job does not run with all
dependencies checked out. See bug for details.
Bug: 398970704
Change-Id: Ia562cc6de7e586f947ccc9d351e9fc5feafa9f22
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6300962
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Jordan Brown <rop@google.com>
Ideally we would track these separately but practically they fall into
the same category as the restricted license. It is on the OWNERS and the
reviewers to make sure that they comply with the terms of a Patent file
or any license. Currently we are just generating presubmit warnings that
are being ignored.
This will also enable future tooling that utilises this list for
auditing restricted licenses to surface Patent files.
Bug: 381146326
Change-Id: I0f091bef9649d3a9f7b03940c8634e56bee9541f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6290872
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Jordan Brown <rop@google.com>
Add the following items to the allowlsted licenses based on the
requirement of crrev.com/c/6239492 :
- PngSuite
- Spencer-86
- unicode_org
Bug: 378472917
Change-Id: I6dfa79d612131eae154e5030ec22b787ebc57060
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6243133
Commit-Queue: Jordan Brown <rop@google.com>
Auto-Submit: Ramin Halavati <rhalavati@chromium.org>
Reviewed-by: Jordan Brown <rop@google.com>
This adds a new way to report CVEs that includes an accompanying
description. It also adds a new validation check that ensures that the
CVE description is present for every entry listed in the 'Mitigated:'
field.
Bug: b/392026683
Change-Id: Ie55595970b49d705ac532f1f8c41ff47d959f56c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6211644
Auto-Submit: Jordan Brown <rop@google.com>
Reviewed-by: Jiewei Qian <qjw@chromium.org>
Commit-Queue: Jiewei Qian <qjw@chromium.org>
This field stores a list of comma-separated CVE IDs that the dependency mitigates.
The field is validated to contain only valid CVE IDs.
Bug: b/392026683
Change-Id: I9578fc709086131695cfa7eee51e717c24440853
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6197756
Reviewed-by: Jiewei Qian <qjw@chromium.org>
Commit-Queue: Jordan Brown <rop@google.com>
Currently using a license in the WITH_PERMISSION_ONLY list will create a
warning. By making an ALL_LICENSE list including this list and also
allowing it when checking for open source compatible licenses, it will
no longer create warnings.
This will enable us to change the current warnings into errors.
Bug: b/388620886
Change-Id: I883a3d3c825f0f1903b62d0b93810218b1f42bb9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6188501
Commit-Queue: Jordan Brown <rop@google.com>
Reviewed-by: Rachael Newitt <renewitt@google.com>
Adding a special entry for dependencies using this license id.
The Android Software Development Kit License is a special case.
It can introduce licensing complexities due to the potentially extensive
transitive dependency chain. Developers should carefully review the
licenses of all dependencies.
Change-Id: I8626391ce04f921a9efa519a5305afce62a5f1c2
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6174215
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Jordan Brown <rop@google.com>
The documentation aims to cover how to make, and review changes to this file.
Change-Id: I887ce938df71119f5d04f7cec30350beddcdef47
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6168292
Commit-Queue: Jordan Brown <rop@google.com>
Reviewed-by: Nico Weber <thakis@google.com>
Reviewed-by: Rick Byers <rbyers@chromium.org>
This change introduces a new error for license fields that use any of
the following `["/", ";", " and ", " or "]`.
I chose to include the offending character/s in the error message
because I find it easier to parse error messages that tell me exactly
which character is the bad one. Similarly I've included conditions in
the reason to handle the plural cases correctly, generating either:
`License contains a bad delimiter character ...`, or
`License contains bad delimiter characters ...`
I realise this means that any downstream rules looking to detect this
error will need to check for a common subset, e.g 'bad delimiter
character', however I think it's worth it for the improved user
experience of receiving the error.
I've also anticipated that most of these errors will be due to
situations where multiple licenses are offered, and included additional
text explaining that only the most permissive of the choices should be
included.
This will affect 9 dependencies and they need to choose between multiple licenses anyway so it's okay to generate an error and have partybug file bugs.
Bug: http://b/374850412
Change-Id: I6eb53a8a3bd541a1801dff133884b719dcdfe04d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6181848
Reviewed-by: Jiewei Qian <qjw@chromium.org>
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Jordan Brown <rop@google.com>
Adds "BLAS", "LicenseRef-base64-cpp", and "SolarDesigner", to the allow
listed licenses.
Bug: 365320654
Change-Id: Id156b2cc881167e4dac88a9e671b21e5de8263bb
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6172448
Reviewed-by: Jordan Brown <rop@google.com>
Commit-Queue: Jordan Brown <rop@google.com>
This makes the allowlist files easier to read. It also prevents accidental changes when adding new entries.
Note that this CL does not modify the actual entries. A duplicate of ISC was removed.
Change-Id: Id64e64dbc934af1b412fce0e0c7c34a8d7b4c5f8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6168290
Commit-Queue: Jordan Brown <rop@google.com>
Reviewed-by: Rachael Newitt <renewitt@google.com>
Auto-Submit: Jordan Brown <rop@google.com>
Reciprocal licenses can only be used in open source projects.
This change updates the presubmit validation checks to accept an
optional flag `allow_reciprocal_licenses`. When True, the allowlist is
extended to include reciprocal licenses.
Bug: 385020146
Change-Id: I0374658207bc87ffd74e033762ee4973c6e83b3b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6107863
Reviewed-by: Jordan Brown <rop@google.com>
Auto-Submit: Jordan Brown <rop@google.com>
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Rachael Newitt <renewitt@google.com>
This is a list of licenses classified as 'reciprocal'. Due to the requirements of licenses of this type we can only allow their use in open source projects. This change introduces the variable 'OPEN_SOURCE_SPDX_LICENSES' with an initial set of reciprocal licenses currently used in chromium.
Change-Id: I376a7623e3685d67edd63ceb3088ca68c9d2fb7e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6107860
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Jordan Brown <rop@google.com>
Add support for validating licenses that aren't in the official SPDX license list but are commonly used in our codebase. These are the values output by the license classifier. This introduces a new EXTENDED_LICENSE_CLASSIFIERS set that contains these custom license identifiers, organized by restrictiveness level similar to ALLOWED_SPDX_LICENSES.
Bug: 379977497
Change-Id: I18e0b38572ee4df783573ea338b55ac237d8134d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6107859
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Jordan Brown <rop@google.com>
These are the licenses that are currently in chromium and are 'unencumbered', 'permissive', or 'notice' licenses
Change-Id: I40281cdb2e8b7e1af7a06e0ff234e2005903d8c1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6055002
Reviewed-by: Jordan Brown <rop@google.com>
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Jordan Brown <rop@google.com>
.. to show license classifications via comments
Change-Id: Id18ee6a597457ecaaafbd24373d78aca09ba7058
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6055001
Commit-Queue: Jordan Brown <rop@google.com>
Reviewed-by: Jordan Brown <rop@google.com>
Reviewed-by: Rachael Newitt <renewitt@google.com>
Change the delimiter for license field from allowing complex cases using "and", "or", and "/" to only allowing a single comma separated list of licenses that are in use.
When given a choice of licenses OWNERS should choose the most appropriate and list this one. In nearly all cases this should be 'whichever is the least restrictive'.
Corresponding change in documentation: https://crrev.com/c/6068628
Change-Id: Ic30dfacb9ba586137b9493cec878b636107a55f4
Bug: 311097536
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6055313
Reviewed-by: Jordan Brown <rop@google.com>
Commit-Queue: Rachael Newitt <renewitt@google.com>
Auto-Submit: Jordan Brown <rop@google.com>
Reviewed-by: Rachael Newitt <renewitt@google.com>
Use the "official" XML schema validation pattern from published CPE
schema.
Add a error message to tell owners that they need to provide at least
one component (other than part) in CPE URN format.
Bug: 378273455
Change-Id: I5ac957f02a0f899d069161cdce54fff499fb35f3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6073136
Commit-Queue: Jiewei Qian <qjw@chromium.org>
Reviewed-by: Anne Redulla <aredulla@google.com>
When a license validation warning is generated, link to the allowlist file for easy fixing.
Bug: 382745675
Change-Id: I4899401a687ce06e74cd18a95e1a28a2684a8027
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6076211
Reviewed-by: Jordan Brown <rop@google.com>
Commit-Queue: Jordan Brown <rop@google.com>
Reviewed-by: Rachael Newitt <renewitt@google.com>
Auto-Submit: Jordan Brown <rop@google.com>
This reverts commit 9dd0755b69.
Reason for revert: 'Supplement:Apache-2.0' is considered to be allowed, but is not technically a valid SPDX identifier.
Original change's description:
> Adding Supplement:Apache-2.0 to chromium license allowlist.
>
> The 'Supplement:Apache-2.0' contains the full license text and is the preferred version over just 'Apache-2.0'.
>
> Change-Id: Ie4566ffdcd20771a0170190270d8d45e1b77980d
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6040532
> Reviewed-by: Jordan Brown <rop@google.com>
> Reviewed-by: Rachael Newitt <renewitt@google.com>
> Commit-Queue: Rachael Newitt <renewitt@google.com>
> Auto-Submit: Jordan Brown <rop@google.com>
Change-Id: Ic6529514304b826dbc8840a468bab47321c30471
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6040754
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Auto-Submit: Jordan Brown <rop@google.com>
The 'Supplement:Apache-2.0' contains the full license text and is the preferred version over just 'Apache-2.0'.
Change-Id: Ie4566ffdcd20771a0170190270d8d45e1b77980d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6040532
Reviewed-by: Jordan Brown <rop@google.com>
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Rachael Newitt <renewitt@google.com>
Auto-Submit: Jordan Brown <rop@google.com>
This CL introduces a validation rule for "Revision: DEPS" syntax
so dependencies managed by DEPS and autorolled can use it to declare
their versioning metadata.
Bug: b/335761679
Change-Id: I0b4f99d281543f9295b122ac71036b06205a6168
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5904321
Commit-Queue: Jiewei Qian <qjw@chromium.org>
Reviewed-by: Rachael Newitt <renewitt@google.com>
If Chromium is the canonical repository, the version of the dependency
is essentially Chromium.
Change-Id: Ifcb80dbeee0d36bf71234f8f48423e8f7aa9dcf0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5772151
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Jiewei Qian <qjw@chromium.org>
Adds support to report line numbers when validation fails.
Change-Id: Iba94c5b3582d7e51f15d266d188909d3a82b75cb
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5740963
Reviewed-by: Jordan Brown <rop@google.com>
Commit-Queue: Jiewei Qian <qjw@chromium.org>
Reviewed-by: Anne Redulla <aredulla@google.com>
This permit downstream clients to retrieve the "source" text and do
their own formatting instead of relying on the format coded in
get_message().
Change-Id: Ia36cbd064ed0781bda76b09b064b97f6dc5e899e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5686730
Commit-Queue: Jiewei Qian <qjw@chromium.org>
Reviewed-by: Anne Redulla <aredulla@google.com>
This CL changes CPEPrefix field to return None for the validate field
accessor.
There's little reason to return a special "unknown" string in property
accessor (which is used for tooling automation).
We still allow specifying "unknown" in that field.
Bug: b/321154076
Change-Id: Ib4cbc017d6b6df179ccfb008bd5ec9477913764b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5465016
Reviewed-by: Anne Redulla <aredulla@google.com>
Commit-Queue: Jiewei Qian <qjw@chromium.org>
This CL adds a typed interface that exposes parsed metadata for
downstream consumption.
Conventionally:
- A validated field should be retrieved by the property of the same name
- A validated field returns "None" if said field is not provided, or is
clearly invalid (e.g. "Unknown" values)
- Raw values can still be retrieved with get_entries()
When using the properties accessor, fields are normalized and/or coerced to a suitable type (e.g. list of str, str of a particular format).
Bug: b/321154076
Change-Id: Ia56969a838e682a7b7eb1dc0781d48e1e38a2ff0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5446637
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Jiewei Qian <qjw@chromium.org>
This CL adds a "early terminate the field based on field value" parser
mechanism to end the field as soon as the field value provides an
unambiguous answer to the question we care about.
This is to prevent over-extraction over certain fields (specifically,
local modifications) which can either be a definitive answer (e.g. No
modification) or multi-line free-form texts (which may contain unknown
fields that we don't care about at this stage).
This mitigates over extraction of README.chromium files like:
```
Local Modifications:
None
How to Uprev:
Steps...
```
Where the old parser would extract "None\n\nHow to Uprev:\nSteps..."
This CL also refactors single line fields to use the same early
termination mechanism since single line field simply ends as soon as
the line is parsed.
Union[Something, None] is changed to Optional[Something] based on
styleguide.
Bug: b/324149233
Change-Id: I3fca80eaceb071263f8ae8730afda230fff0bbb0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5394917
Reviewed-by: Anne Redulla <aredulla@google.com>
Commit-Queue: Jiewei Qian <qjw@chromium.org>
This CL makes scan.py to sort result summary (keyed on the type of
validation results), so the output summary will have a predictable
order for easier diffing.
Change-Id: I0eda8a439e159eae667729512424b813eb0f5e11
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5390925
Auto-Submit: Jiewei Qian <qjw@chromium.org>
Commit-Queue: Jiewei Qian <qjw@chromium.org>
Reviewed-by: Anne Redulla <aredulla@google.com>
This CL adds a "structured" concept to the parser. In a structured
field, the parser will proactively look for field-like patterns to
start a new field (even if they aren't known fields).
This mitigates the issue when an unknown field immediately
follows a multi-line text field, such as:
URL: https://example.com
UnknownField: abc
And URL field value parses to
"https://example.com<newline>UnknownField:abc".
Bug: b/324149233
Change-Id: I54807bd7b242fc14c679483453ade83f8fd20225
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5379679
Reviewed-by: Anne Redulla <aredulla@google.com>
Commit-Queue: Jiewei Qian <qjw@chromium.org>
Explicitly sorts the discovered metadata file, and defines
ordering of validation messages, so the tools will output
will be self-consistent.
Change-Id: I9b263a16b151c014e5950638f066376469c701df
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5379678
Commit-Queue: Jiewei Qian <qjw@chromium.org>
Reviewed-by: Anne Redulla <aredulla@google.com>
os.walk is more efficient than the current handwritten traversal.
Measured the time to scan reduced from 30s+ to 8s on p920 on
chromium/src.
`followlinks=True` is set to preserve behavior that os.path.isdir
returns True for symlink to directories, and the current traversal code
will descend into those.
Change-Id: I941eec9105a46d6538ca484fbb5249a75888e38a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5379945
Reviewed-by: Anne Redulla <aredulla@google.com>
Commit-Queue: Jiewei Qian <qjw@chromium.org>
This CL updates the License field validation so that the warning to use
the standard comma separator is only returned if processing the license
value resulted in multiple license types.
Bug: b:309712938
Change-Id: Ic9189b8dd76e60bc3d546dea41fdb36faae8dbb4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5003558
Auto-Submit: Anne Redulla <aredulla@google.com>
Commit-Queue: Anne Redulla <aredulla@google.com>
Reviewed-by: Dan Le Febvre <dlf@google.com>
Commit-Queue: Dan Le Febvre <dlf@google.com>
This CL expands on the date format validation for third party
metadata. Now, values that are recognized to be using a different format
from the preferred format of YYYY-MM-DD will only return a warning,
instead of an error.
Bug: b:285453019
Change-Id: I344dc863601b4e03e801cdfb3cc5912cfe13b762
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4961973
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Anne Redulla <aredulla@google.com>
All __future__ imports (unicode_literals, print_function) are already
mandatory in py3. Also remove an outdated py2 comment in
presubmit_canned_checks.py
Bug: 1475402
Change-Id: I27cf6a8268f6dd1081f22af782c4c29a975376ae
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4867135
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
This CL adds a message prefix to metadata validation issues that will
be presubmit errors, but are currently returned as warnings while
metadata quality is still in the process of being uplifted.
Bug: b:285453019
Change-Id: I2d93b7af6724f945bed3be8f1acb86fc0fddad92
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4876061
Commit-Queue: Anne Redulla <aredulla@google.com>
Reviewed-by: Rachael Newitt <renewitt@google.com>
This CL updates the third party metadata field License File, so that
missing license filepaths are reported as relative to the repo root,
instead of the absolute filepath.
Bug: b:277147404
Change-Id: I16ac23d86b0ef12ab6020d8e81525965489888ac
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4876394
Auto-Submit: Anne Redulla <aredulla@google.com>
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Anne Redulla <aredulla@google.com>
This CL changes what is considered valid versioning info. Instead of
both Date and Revision being required if Version was unknown, now only
one of Date or Revision has to be specified.
Bug: b:277147404
Change-Id: Iedb06e2d55f0cd0ef0a2931013a2a52b15befd75
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4852699
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Anne Redulla <aredulla@google.com>
Reviewed-by: Rachael Newitt <renewitt@google.com>
Leave the recipes/ code at 2 space to match the rest of the recipes
project in other repos.
Reformatted using:
files=( $(
git ls-tree -r --name-only HEAD | \
grep -Ev -e '^(third_party|recipes)/' | \
grep '\.py$';
git grep -l '#!/usr/bin/env.*python' | grep -v '\.py$'
) )
parallel ./yapf -i -- "${files[@]}"
~/chromiumos/chromite/contrib/reflow_overlong_comments "${files[@]}"
The files that still had strings that were too long were manually
reformatted because they were easy and only a few issues.
autoninja.py
clang_format.py
download_from_google_storage.py
fix_encoding.py
gclient_utils.py
git_cache.py
git_common.py
git_map_branches.py
git_reparent_branch.py
gn.py
my_activity.py
owners_finder.py
presubmit_canned_checks.py
reclient_helper.py
reclientreport.py
roll_dep.py
rustfmt.py
siso.py
split_cl.py
subcommand.py
subprocess2.py
swift_format.py
upload_to_google_storage.py
These files still had lines (strings) that were too long, so the pylint
warnings were suppressed with a TODO.
auth.py
gclient.py
gclient_eval.py
gclient_paths.py
gclient_scm.py
gerrit_util.py
git_cl.py
presubmit_canned_checks.py
presubmit_support.py
scm.py
Change-Id: Ia6535c4f2c48d46b589ec1e791dde6c6b2ea858f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4836379
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Auto-Submit: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
All files in metadata/ are new, so they should follow the PEP-8 style.
Change-Id: I5d8424536c3d7b703e6b8087e0e2d70c06a1549c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4834909
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Rachael Newitt <renewitt@google.com>
Jordan Brown
Ramin Halavati
Joshua Pawlicki
Jiewei Qian
Andrew Grieve
Anne Redulla
Gavin Mak
Mike Frysinger