FedoraReview

#475 Replace licensecheck with askalono
Closed: not-a-bug by msuchy. Opened by msuchy.

Closed: not-a-bug
Reopen Issue

The licensecheck output seems to be less and less useful. Askalono (from askalono-cli package) seems to provide a better result.

The content of tarball can be scanned as

askalono crawl DIR

Or we can run both tool. Or configure which one to use.

What makes askalono any better?

licensecheck match based on the regexp. See /usr/share/perl5/vendor_perl/Regexp/Pattern/License.pm

askalono (and other modern tools) base the decision on the editing distance of the whole text. And provides you with a confidence level.

@msuchy
I was looking at Askalono and licensecheck and both are problematic in many cases.
When I run askalono to examples in the licensecheck, it failed many times.
Seems that the licensecheck has better results for now.

Some example:
We have Perl module Crypt::Argon2 (https://metacpan.org/pod/Crypt::Argon2)
askalono crawl Crypt-Argon2-0.019

./LICENSE
License: Apache-2.0 (original text)
Score: 0.996

But there are files like https://metacpan.org/release/LEONT/Crypt-Argon2-0.019/source/src/argon2.c
where is:

/*
 * Argon2 reference source code package - reference C implementations
 *
 * Copyright 2015
 * Daniel Dinu, Dmitry Khovratovich, Jean-Philippe Aumasson, and Samuel Neves
 *
 * You may use this work under the terms of a Creative Commons CC0 1.0
 * License/Waiver or the Apache Public License 2.0, at your option. The terms of
 * these licenses can be found at:
 *
 * - CC0 1.0 Universal : https://creativecommons.org/publicdomain/zero/1.0
 * - Apache 2.0        : https://www.apache.org/licenses/LICENSE-2.0
 *
 * You should have received a copy of both of these licenses along with this
 * software. If not, they may be obtained at the above URLs.
 */

BTW: licensecheck not detect this license too.
licensecheck -r Crypt-Argon2-0.019

./Build.PL: *No copyright* UNKNOWN
./Changes: *No copyright* UNKNOWN
./INSTALL: *No copyright* UNKNOWN
./LICENSE: Apache License 2.0
./MANIFEST: *No copyright* UNKNOWN [generated file]
./META.json: *No copyright* UNKNOWN
./META.yml: *No copyright* UNKNOWN
./README: Apache License 2.0
./dist.ini: *No copyright* UNKNOWN
./typemap: *No copyright* UNKNOWN
./include/argon2.h: UNKNOWN
./opt/opt.c: UNKNOWN
./ref/ref.c: UNKNOWN
./script/argon2-calibrate: Apache License 2.0
./src/argon2.c: UNKNOWN
./src/core.c: UNKNOWN
./src/core.h: UNKNOWN
./src/encoding.c: Creative Commons CC0 1.0
./src/encoding.h: UNKNOWN
./src/thread.c: UNKNOWN
./src/thread.h: UNKNOWN
./t/test.t: *No copyright* UNKNOWN
./lib/Crypt/Argon2.pm: Apache License 2.0
./lib/Crypt/Argon2.xs: *No copyright* UNKNOWN
./src/blake2/blake2-impl.h: UNKNOWN
./src/blake2/blake2.h: UNKNOWN
./src/blake2/blake2b.c: UNKNOWN
./src/blake2/blamka-round-opt.h: UNKNOWN
./src/blake2/blamka-round-ref.h: UNKNOWN
./xt/author/pod-coverage.t: *No copyright* UNKNOWN [generated file]
./xt/author/pod-syntax.t: *No copyright* UNKNOWN [generated file]

I imported askalano for Fedora to use with go2rpm in Golang packages. It is definitely imperfect. It work decently for normal licence files but will fail on license files with multiple licenses included. And it's hit or miss with headers in files.

I have more trust in licensecheck than askolono because Debian develops and uses licensecheck for their package license audits. It was built for distro license evaluation, and askolono does not seem to be good enough compared to it.

Also askalono hasn't been touch since 2002. And there are glaring omission, like https://github.com/jpeddicord/askalono/issues/91 that I wasn't aware until this morning until you sent me a PR @msuchy regarding an exception.

OK, then close this one.

Metadata Update from @msuchy:
- Issue close_status updated to: not-a-bug
- Issue status updated to: Closed (was: Open)

@msuchy I've got great results with https://github.com/nexB/scancode-toolkit

I'll see if I can package it.

 "packages": [
    {
      "type": "golang",
      "namespace": "gopkg.in",
      "name": "retry.v1",
      "version": null,
      "qualifiers": {},
      "subpath": null,
      "primary_language": "Go",
      "description": null,
      "release_date": null,
      "parties": [],
      "keywords": [],
      "homepage_url": "https://pkg.go.dev/gopkg.in/retry.v1",
      "download_url": null,
      "size": null,
      "sha1": null,
      "md5": null,
      "sha256": null,
      "sha512": null,
      "bug_tracking_url": null,
      "code_view_url": null,
      "vcs_url": "https://gopkg.in/retry.v1.git",
      "copyright": null,
      "holder": null,
      "declared_license_expression": "lgpl-3.0-plus WITH lgpl-3.0-linking-exception",
      "declared_license_expression_spdx": "LGPL-3.0-or-later WITH LGPL-3.0-linking-exception",
      "license_detections": [
        {
          "license_expression": "lgpl-3.0-plus WITH lgpl-3.0-linking-exception",
          "matches": [
            {
              "score": 100.0,
              "start_line": 2,
              "end_line": 186,
              "matched_length": 1376,
              "match_coverage": 100.0,
              "matcher": "2-aho",
              "license_expression": "lgpl-3.0-plus WITH lgpl-3.0-linking-exception",
              "rule_identifier": "lgpl-3-plus-linking2.RULE",
              "rule_relevance": 100,
              "rule_url": "https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/rules/lgpl-3-plus-linking2.RULE"
            }
          ],
          "identifier": "lgpl_3_0_plus_with_lgpl_3_0_linking_exception-aa096e42-664a-55f1-9554-e8b85268f339"
        }
      ],
      "other_license_expression": null,
      "other_license_expression_spdx": null,
      "other_license_detections": [],
      "extracted_license_statement": null,
      "notice_text": null,
      "source_packages": [],
      "extra_data": {},
      "repository_homepage_url": "https://pkg.go.dev/gopkg.in/retry.v1",
      "repository_download_url": null,
      "api_data_url": null,
      "package_uid": "pkg:golang/gopkg.in/retry.v1?uuid=56c22e4c-0307-4e3b-9485-dfb197a24851",
      "datafile_paths": [
        "golang-gopkg-retry-1/retry-1.0.3.tar.gz-extract/retry-1.0.3/go.mod"
      ],
      "datasource_ids": [
        "go_mod"
      ],
      "purl": "pkg:golang/gopkg.in/retry.v1"
    }

The result on argon2.c:

    {
      "path": "Crypt-Argon2-0.019/src/argon2.c",
      "type": "file",
      "name": "argon2.c",
      "base_name": "argon2",
      "extension": ".c",
      "size": 14473,
      "date": "2023-05-24",
      "sha1": "2c4d397d0eb11a083c0885955832f0bf235b02f1",
      "md5": "dff5a3ac05342d2cd355ab44126d2b94",
      "sha256": "b1289ec7134e8502e9113396fdac89402bf2575ee1b35e33fb7410f2fb63bb6d",
      "mime_type": "text/x-c",
      "file_type": "C source, ASCII text",
      "programming_language": "C",
      "is_binary": false,
      "is_text": true,
      "is_archive": false,
      "is_media": false,
      "is_source": true,
      "is_script": false,
      "package_data": [],
      "for_packages": [],
      "detected_license_expression": "cc0-1.0 OR apache-2.0",
      "detected_license_expression_spdx": "CC0-1.0 OR Apache-2.0",
      "license_detections": [
        {
          "license_expression": "cc0-1.0 OR apache-2.0",
          "matches": [
            {
              "score": 100.0,
              "start_line": 7,
              "end_line": 15,
              "matched_length": 81,
              "match_coverage": 100.0,
              "matcher": "2-aho",
              "license_expression": "cc0-1.0 OR apache-2.0",
              "rule_identifier": "cc0-1.0_or_apache-2.0_5.RULE",
              "rule_relevance": 100,
              "rule_url": "https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/rules/cc0-1.0_or_apache-2.0_5.RULE"
            }
          ],
          "identifier": "cc0_1_0_or_apache_2_0-39dacec7-e5b6-589b-09f2-109cf5beebde"
        }
      ],
      "license_clues": [],
      "percentage_of_license_text": 4.34,
      "copyrights": [
        {
          "copyright": "Copyright 2015 Daniel Dinu, Dmitry Khovratovich, Jean-Philippe Aumasson, and Samuel Neves",
          "start_line": 4,
          "end_line": 5
        }
      ],
      "holders": [
        {
          "holder": "Daniel Dinu, Dmitry Khovratovich, Jean-Philippe Aumasson, and Samuel Neves",
          "start_line": 5,
          "end_line": 5
        }
      ],
      "authors": [],
      "emails": [],
      "urls": [
        {
          "url": "https://creativecommons.org/publicdomain/zero/1.0",
          "start_line": 11,
          "end_line": 11
        },
        {
          "url": "https://www.apache.org/licenses/LICENSE-2.0",
          "start_line": 12,
          "end_line": 12
        }
      ],
      "files_count": 0,
      "dirs_count": 0,
      "size_count": 0,
      "scan_errors": []
    },

PS: I've recently made some improvements to how askalono is packaged - notably, it now ships with up-to-date matching data for SPDX licenses. But that still doesn't fix the other issues (no support for exceptions, and only being able to scan actual license texts in separate files, not license headers in source code).
Metadata
None
None
None
None
None
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.