Skip to content

Add support for npm package-lock.json v3#49

Merged
dvail merged 2 commits intomasterfrom
dv/add-support-for-npm-package-lock
Jul 26, 2024
Merged

Add support for npm package-lock.json v3#49
dvail merged 2 commits intomasterfrom
dv/add-support-for-npm-package-lock

Conversation

@dvail
Copy link
Contributor

@dvail dvail commented Jul 19, 2024

Adds support for npm's package-lock.json (v3) as an alternative to yarn's custom yarn.lock file. This would allow the stackrox UI to drop yarn in favor of npm in order to enable hermetic builds in the konflux pipelines.

This update includes dev dependencies as well as nested transitive dependencies in the output. Essentially if a module is resolved somewhere in the application's node_modules folder, it will be included in the ossls notice output with a LICENSE file and license.json metadata. It would not be difficult to drop either of these dependency categories if needed, but I didn't see the harm in including them.

(As an aside, this means the dependencies included by the current yarn version could change depending on how the tree is resolved. e.g. If we depend on yaml1.7.2 at the top level and yaml2.2.1 as a nested dependency, the latter would not be included in the license output. In theory a dependency change could invert the two, in which case the former would not be included in the output.)

How it works

The npm package-lock.json v3 contains a packages key with a value object where:

  1. The first object key is "", which contains metadata about the root project. This is ignored.
  2. All other keys are in the form of "node_modules/<package-name>" or "node_modules/<package-name>/node_modules/<nested-package-name>". The values to these keys contain metadata about the package, the only data which we care about being the version field.

This code update reads all of the keys from 2 above and treats:

  1. The full key, containing all node_modules segments as the expected path of the module on disk
  2. The substring of the key removing all prefix up to and including the last "node_modules" segment as the package name
  3. The version field of the object as the package version

Once the paths, names, and versions for all modules are obtained, control is returned back to the existing ossls code which then reads metadata and license information for all of these packages and outputs the results on disk.

Testing

Manual testing against the full package-lock.json in stackrox/stackrox#11967 and compare to similar output against the yarn equivalent in stackrox@master.

# With the package-lock.json and updated .ossls.yml config in #11967

go build && mv ossls ~/bin
cd ~/projects/stackrox
rm -rf image/rhel/THIRD_PARTY_NOTICES
make ossls-notice
ls image/rhel/THIRD_PARTY_NOTICES
...
workbox-strategies6.5.4
workbox-streams6.5.4
workbox-sw6.5.4
workbox-webpack-plugin6.5.4
workbox-window6.5.4
workerpool6.5.1
wrap-ansi6.2.0
wrap-ansi7.0.0
wrappy1.0.2
write-file-atomic4.0.2
ws8.17.1
xml-name-validator4.0.0
xml1.0.1
xmlchars2.2.0
xtend4.0.2
y18n5.0.5
yallist3.1.1
yallist4.0.0
yaml-ast-parser0.0.43
yaml1.10.2
yaml1.7.2
yaml2.2.1
yargs-parser20.2.9
yargs-parser21.1.1
yargs-unparser2.0.0
yargs16.2.0
yargs17.7.2
yauzl2.10.0
yocto-queue0.1.0
yup1.4.0
zen-observable-ts1.2.5
zen-observable0.8.15
$ cat yup1.4.0/*
# LICENSE

The MIT License (MIT)

Copyright (c) 2014 Jason Quense

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

# license-info.json

{
  "license": "MIT",
  "metadata": {
    "author": {
      "name": "@monasticpanic Jason Quense"
    },
    "name": "yup",
    "repository": {
      "type": "git",
      "url": "git+https://github.com/jquense/yup.git"
    }
  }
}

Automated unit test included with this PR.

@dvail dvail force-pushed the dv/add-support-for-npm-package-lock branch from 76ee567 to 658b244 Compare July 19, 2024 16:27
@dvail dvail marked this pull request as ready for review July 19, 2024 19:27
@dvail dvail requested a review from pedrottimark July 22, 2024 18:07
Copy link

@davdhacs davdhacs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1
I ran the ossls built from this against stackrox master and stackrox/stackrox#11967 and compared the THIRD_PARTY_NOTICES.

$ go build && mv ossls ~/.local/bin
$ cd ../stackrox
$ make ossls-notice
$ ls -1 image/rhel/THIRD_PARTY_NOTICES | tee yarn.3rdparty.list
$ git switch dv/ROX-25287-replace-yarn-w-npm-for-konflux
$ rm -rf image/rhel/THIRD_PARTY_NOTICES
# un-comment npm lines in .ossls.yml
$ make -C ui clean deps
$ make ossls-notice
$ ls -1 image/rhel/THIRD_PARTY_NOTICES | tee npm.3rdparty.list

Copy link

@RTann RTann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this isn't a main part of the product, I won't keep going with nitpicks. If it works, it works :)

@dvail dvail merged commit 2f29d92 into master Jul 26, 2024
@dvail dvail deleted the dv/add-support-for-npm-package-lock branch July 26, 2024 17:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants