Commit Graph

28 Commits (fccf516910abbe98dbb967606cb73406f156567e)

Author SHA1 Message Date
Jordan Brown 833f488142 Revert "Adding Supplement:Apache-2.0 to chromium license allowlist."
This reverts commit 9dd0755b69.

Reason for revert: 'Supplement:Apache-2.0' is considered to be allowed, but is not technically a valid SPDX identifier.

Original change's description:
> Adding Supplement:Apache-2.0 to chromium license allowlist.
>
> The 'Supplement:Apache-2.0' contains the full license text and is the preferred version over just 'Apache-2.0'.
>
> Change-Id: Ie4566ffdcd20771a0170190270d8d45e1b77980d
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6040532
> Reviewed-by: Jordan Brown <rop@google.com>
> Reviewed-by: Rachael Newitt <renewitt@google.com>
> Commit-Queue: Rachael Newitt <renewitt@google.com>
> Auto-Submit: Jordan Brown <rop@google.com>

Change-Id: Ic6529514304b826dbc8840a468bab47321c30471
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6040754
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Auto-Submit: Jordan Brown <rop@google.com>
12 months ago
Jordan 9dd0755b69 Adding Supplement:Apache-2.0 to chromium license allowlist.
The 'Supplement:Apache-2.0' contains the full license text and is the preferred version over just 'Apache-2.0'.

Change-Id: Ie4566ffdcd20771a0170190270d8d45e1b77980d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6040532
Reviewed-by: Jordan Brown <rop@google.com>
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Rachael Newitt <renewitt@google.com>
Auto-Submit: Jordan Brown <rop@google.com>
12 months ago
Jordan b5eb54d5c1 Updating presubmit to check license is a valid spdx identifier.
Further details are available in https://docs.google.com/document/d/1x4GLly7KJ2xmlJRvsswJlmVSLLjBjk5NOxWZ-Dryg_A

Bug: 358504615
Change-Id: I76997454140ee63aea6b6f492669800efce271be
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6021876
Reviewed-by: Joey Scarr <jsca@google.com>
Commit-Queue: Jordan Brown <rop@google.com>
Reviewed-by: Andrew Grieve <agrieve@chromium.org>
Reviewed-by: Rick Byers <rbyers@chromium.org>
Reviewed-by: Rachael Newitt <renewitt@google.com>
1 year ago
Jiewei Qian 6af18461ad metadata: allow a special "Revision: DEPS" syntax
This CL introduces a validation rule for "Revision: DEPS" syntax
so dependencies managed by DEPS and autorolled can use it to declare
their versioning metadata.

Bug: b/335761679
Change-Id: I0b4f99d281543f9295b122ac71036b06205a6168
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5904321
Commit-Queue: Jiewei Qian <qjw@chromium.org>
Reviewed-by: Rachael Newitt <renewitt@google.com>
1 year ago
Andrew Grieve caa77da956 Do not warn about using "and" / "or" in License: fields
"and" and "or" are semantically different, so we should not discourage
their use.

See: https://crbug.com/355672411#comment11

Bug: 355672411
Change-Id: I95aa204da5550747ec67ccb6e42b87e6ac042b0b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5783166
Auto-Submit: Andrew Grieve <agrieve@chromium.org>
Reviewed-by: Anne Redulla <aredulla@google.com>
Commit-Queue: Anne Redulla <aredulla@google.com>
1 year ago
Jordan 7e547050e8 Updating Revision presubmit to check for valid commit hash
Bug:b/349274008
Change-Id: I6ce8f1993bcf78514111d709b69b109e481d18b7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5652700
Reviewed-by: Jiewei Qian <qjw@chromium.org>
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Jordan Brown <rop@google.com>
1 year ago
Jiewei Qian bba165a25c metadata: treats CPEPrefix "unknown" as None
This CL changes CPEPrefix field to return None for the validate field
accessor.

There's little reason to return a special "unknown" string in property
accessor (which is used for tooling automation).

We still allow specifying "unknown" in that field.

Bug: b/321154076
Change-Id: Ib4cbc017d6b6df179ccfb008bd5ec9477913764b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5465016
Reviewed-by: Anne Redulla <aredulla@google.com>
Commit-Queue: Jiewei Qian <qjw@chromium.org>
2 years ago
Jiewei Qian b7ed76a09d metadata: define a clear DependencyMetadata interface
This CL adds a typed interface that exposes parsed metadata for
downstream consumption.

Conventionally:
- A validated field should be retrieved by the property of the same name
- A validated field returns "None" if said field is not provided, or is
  clearly invalid (e.g. "Unknown" values)
- Raw values can still be retrieved with get_entries()

When using the properties accessor, fields are normalized and/or coerced to a suitable type (e.g. list of str, str of a particular format).

Bug: b/321154076
Change-Id: Ia56969a838e682a7b7eb1dc0781d48e1e38a2ff0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5446637
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Jiewei Qian <qjw@chromium.org>
2 years ago
Jiewei Qian 79cfa048c0 metadata: early terminate certain fields to avoid over extraction
This CL adds a "early terminate the field based on field value" parser
mechanism to end the field as soon as the field value provides an
unambiguous answer to the question we care about.

This is to prevent over-extraction over certain fields (specifically,
local modifications) which can either be a definitive answer (e.g. No
modification) or multi-line free-form texts (which may contain unknown
fields that we don't care about at this stage).

This mitigates over extraction of README.chromium files like:

```
Local Modifications:
None

How to Uprev:
Steps...
```

Where the old parser would extract "None\n\nHow to Uprev:\nSteps..."

This CL also refactors single line fields to use the same early
termination mechanism since single line field simply ends as soon as
the line is parsed.

Union[Something, None] is changed to Optional[Something] based on
styleguide.

Bug: b/324149233
Change-Id: I3fca80eaceb071263f8ae8730afda230fff0bbb0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5394917
Reviewed-by: Anne Redulla <aredulla@google.com>
Commit-Queue: Jiewei Qian <qjw@chromium.org>
2 years ago
Jiewei Qian d76c4d6045 metadata: add "structured" field parsing
This CL adds a "structured" concept to the parser. In a structured
field, the parser will proactively look for field-like patterns to
start a new field (even if they aren't known fields).

This mitigates the issue when an unknown field immediately
follows a multi-line text field, such as:

URL: https://example.com
UnknownField: abc

And URL field value parses to
"https://example.com<newline>UnknownField:abc".

Bug: b/324149233
Change-Id: I54807bd7b242fc14c679483453ade83f8fd20225
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5379679
Reviewed-by: Anne Redulla <aredulla@google.com>
Commit-Queue: Jiewei Qian <qjw@chromium.org>
2 years ago
Anne Redulla a83e690662 [ssci] Add condition on license field separator warning
This CL updates the License field validation so that the warning to use
the standard comma separator is only returned if processing the license
value resulted in multiple license types.

Bug: b:309712938
Change-Id: Ic9189b8dd76e60bc3d546dea41fdb36faae8dbb4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5003558
Auto-Submit: Anne Redulla <aredulla@google.com>
Commit-Queue: Anne Redulla <aredulla@google.com>
Reviewed-by: Dan Le Febvre <dlf@google.com>
Commit-Queue: Dan Le Febvre <dlf@google.com>
2 years ago
Anne Redulla 10cd8e406d [ssci] Recognize other date formats for third party metadata
This CL expands on the date format validation for third party
metadata. Now, values that are recognized to be using a different format
from the preferred format of YYYY-MM-DD will only return a warning,
instead of an error.

Bug: b:285453019
Change-Id: I344dc863601b4e03e801cdfb3cc5912cfe13b762
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4961973
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Anne Redulla <aredulla@google.com>
2 years ago
Anne Redulla 4dac5d6b4b [ssci] Return relative license filepath instead of absolute
This CL updates the third party metadata field License File, so that
missing license filepaths are reported as relative to the repo root,
instead of the absolute filepath.


Bug: b:277147404
Change-Id: I16ac23d86b0ef12ab6020d8e81525965489888ac
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4876394
Auto-Submit: Anne Redulla <aredulla@google.com>
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Anne Redulla <aredulla@google.com>
2 years ago
Anne Redulla 36bd52621f [ssci] Support alias for Shipped field
Bug: b:297823626
Change-Id: Ib4be88567040d147f6cdba4f6c7d2b37a0f3898b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4871939
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Anne Redulla <aredulla@google.com>
2 years ago
Mike Frysinger 124bb8e53c switch to 4 space indent
Leave the recipes/ code at 2 space to match the rest of the recipes
project in other repos.

Reformatted using:
files=( $(
	git ls-tree -r --name-only HEAD | \
		grep -Ev -e '^(third_party|recipes)/' | \
		grep '\.py$';
	git grep -l '#!/usr/bin/env.*python' | grep -v '\.py$'
) )
parallel ./yapf -i -- "${files[@]}"
~/chromiumos/chromite/contrib/reflow_overlong_comments "${files[@]}"

The files that still had strings that were too long were manually
reformatted because they were easy and only a few issues.
autoninja.py
clang_format.py
download_from_google_storage.py
fix_encoding.py
gclient_utils.py
git_cache.py
git_common.py
git_map_branches.py
git_reparent_branch.py
gn.py
my_activity.py
owners_finder.py
presubmit_canned_checks.py
reclient_helper.py
reclientreport.py
roll_dep.py
rustfmt.py
siso.py
split_cl.py
subcommand.py
subprocess2.py
swift_format.py
upload_to_google_storage.py

These files still had lines (strings) that were too long, so the pylint
warnings were suppressed with a TODO.
auth.py
gclient.py
gclient_eval.py
gclient_paths.py
gclient_scm.py
gerrit_util.py
git_cl.py
presubmit_canned_checks.py
presubmit_support.py
scm.py

Change-Id: Ia6535c4f2c48d46b589ec1e791dde6c6b2ea858f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4836379
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Auto-Submit: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Anne Redulla 6715758ed9 [ssci] PEP8 formatting for metadata directory
All files in metadata/ are new, so they should follow the PEP-8 style.

Change-Id: I5d8424536c3d7b703e6b8087e0e2d70c06a1549c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4834909
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Rachael Newitt <renewitt@google.com>
2 years ago
Anne Redulla 693e0b3121 [ssci] Check CPE metadata field for both 2.3 and 2.2 formats
Bug: b:277147404
Change-Id: Ib817f374b8a33ab7463d93ccef7ad852b16dd9d1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4834898
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Anne Redulla <aredulla@google.com>
2 years ago
Anne Redulla 427f0f43ad [ssci] Script to run validation on all metadata files
Adds script metadata/scan.py which can be used to search for and
validate Chromium dependency metadata files, given a repository
root directory.

Bug: b:277147404
Change-Id: Ibde0eeb7babe0b1e3f9c7f887bece629d390974a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4823596
Commit-Queue: Anne Redulla <aredulla@google.com>
Reviewed-by: Rachael Newitt <renewitt@google.com>
2 years ago
Anne Redulla c7aca34c8e Define main metadata validation functions
This is a reland of commit a1cfc693af

The original commit was reverted do to `ModuleNotFoundError`s. I believe this was due to not specifying `metadata` to be part of the `depot_tools` recipe bundle. I have updated `.gitattributes` for this, and also added `__init__.py` files.

I will put the changes to `presubmit_canned_checks.py` in a later CL, once I can confirm `metadata` is being bundled.

Original change's description:
> [ssci] Added CheckChromiumMetadataFiles in presubmit_canned_checks
>
> Bug: b:277147404
> Change-Id: I14a2f11b256bc85fdfe225443ef533c38463ca3e
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4796694
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Reviewed-by: Rachael Newitt <renewitt@google.com>
> Commit-Queue: Anne Redulla <aredulla@google.com>

Bug: b:277147404
Change-Id: Ibd9efd5970a5393c157ca8763f97064d7c167803
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4803385
Reviewed-by: Rachael Newitt <renewitt@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Anne Redulla <aredulla@google.com>
2 years ago
Anne Redulla 3aeb682373 [ssci] Added validate method for single dependencies
Bug: b:277147404
Change-Id: I54c9c82d093cb11813e1c224da125b8d555f1b29
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4797050
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Anne Redulla <aredulla@google.com>
2 years ago
Anne Redulla 2b583af7e1 [ssci] Added parser for README validator
Bug: b:277147404
Change-Id: I7ee0fe35e1017eb477255f12045d00e855f7dfb4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4787830
Reviewed-by: Rachael Newitt <renewitt@google.com>
Auto-Submit: Anne Redulla <aredulla@google.com>
Commit-Queue: Rachael Newitt <renewitt@google.com>
2 years ago
Anne Redulla c7350c496a [ssci] Defined License metadata field
Bug: b:277147404
Change-Id: I498537328c0908417955eda34ad48ca46f4275aa
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4776734
Reviewed-by: Rachael Newitt <renewitt@google.com>
Auto-Submit: Anne Redulla <aredulla@google.com>
Commit-Queue: Rachael Newitt <renewitt@google.com>
2 years ago
Anne Redulla 7d26320c4d [ssci] Defined License File metadata field
Bug: b:277147404
Change-Id: If0a9394a3cac3aa6819ed1ac2ad875564648a8f5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4776733
Commit-Queue: Anne Redulla <aredulla@google.com>
Reviewed-by: Rachael Newitt <renewitt@google.com>
2 years ago
Anne Redulla e95696ff53 [ssci] Defined URL metadata field
Bug: b:277147404
Change-Id: I4e1fb86afb991fd3b6bf59a2a96d620fc7eea469
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4776732
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Anne Redulla <aredulla@google.com>
2 years ago
Anne Redulla 0f405ea312 [ssci] Defined Version metadata field
Bug: b:277147404
Change-Id: Idcdff48e5ef2052057fa7fdff7204b50a1c3c4d9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4776731
Commit-Queue: Anne Redulla <aredulla@google.com>
Reviewed-by: Rachael Newitt <renewitt@google.com>
2 years ago
Anne Redulla a29922791a [ssci] Defined CPEPrefix metadata field
Bug: b:277147404
Change-Id: I24a0324f707b13d89beaad0c65c06457375a251f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4776730
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Anne Redulla <aredulla@google.com>
2 years ago
Anne Redulla 378fe5fb91 [ssci] Defined Date metadata field
Bug: b:277147404
Change-Id: I26fbc670ff402c2ba26ba62b344fc3f3ccd61d24
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4776729
Reviewed-by: Rachael Newitt <renewitt@google.com>
Auto-Submit: Anne Redulla <aredulla@google.com>
Commit-Queue: Anne Redulla <aredulla@google.com>
2 years ago
Anne Redulla 51690612da [ssci] Defined basic metadata fields
Bug: b:277147404
Change-Id: Iabba43add61ce5090be50ac8f0245f624028c44b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4776728
Reviewed-by: Rachael Newitt <renewitt@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Anne Redulla <aredulla@google.com>
2 years ago