golist

#7 runtime panic, nil dereference when running %gochecks
Closed: Fixed by qulogic. Opened by nim.

Closed: Fixed
Reopen Issue

Issue migrated from https://github.com/gofed/symbols-extractor/issues/157

This is from the mock build log:

+ go-rpm-integration check -i github.com/syncthing/notify -p /builddir/build/BUILDROOT/golang-github-syncthing-notify-0-0.5.20181011git116c45b.fc30.x86_64 -g /usr/share/gocode -b /builddir/build/BUILD/notify-116c45bb5ad48777321e4984d1320d56889b6097/_build -r '.*example.*'
Testing: github.com/syncthing/notify
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
    panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x5ad9fd]
goroutine 1 [running]:
github.com/gofed/symbols-extractor/vendor/github.com/urfave/cli.HandleAction.func1(0xc000145a30)
    /builddir/build/BUILD/symbols-extractor-9f4330a0f4437ca61ba92f9f30e34424c6742ad6/src/github.com/gofed/symbols-extractor/vendor/github.com/urfave/cli/app.go:472 +0x278
panic(0x920cc0, 0xd79220)
    /usr/lib/golang/src/runtime/panic.go:513 +0x1b9
github.com/gofed/symbols-extractor/pkg/util/internal/load.ImportPaths(0xc000310160, 0x1, 0x1, 0x0, 0x1, 0xc0001450c8)
    /builddir/build/BUILD/symbols-extractor-9f4330a0f4437ca61ba92f9f30e34424c6742ad6/src/github.com/gofed/symbols-extractor/pkg/util/internal/load/pkg.go:1888 +0x5d
github.com/gofed/symbols-extractor/pkg/util/internal/load.PackagesAndErrors(0xc000310160, 0x1, 0x1, 0xa3c700, 0xc0002a7480, 0x1c)
    /builddir/build/BUILD/symbols-extractor-9f4330a0f4437ca61ba92f9f30e34424c6742ad6/src/github.com/gofed/symbols-extractor/pkg/util/internal/load/pkg.go:1842 +0xa3
github.com/gofed/symbols-extractor/pkg/util.ListPackage(0xc0002d22f1, 0x1b, 0x1b, 0x0, 0xc000145290)
    /builddir/build/BUILD/symbols-extractor-9f4330a0f4437ca61ba92f9f30e34424c6742ad6/src/github.com/gofed/symbols-extractor/pkg/util/util.go:420 +0x8f
github.com/gofed/symbols-extractor/pkg/util.(*PackageInfoCollector).CollectPackageInfos.func1(0xc0002d22a0, 0x6d, 0xa3e6a0, 0xc0002bcc30, 0x0, 0x0, 0x0, 0xc000145328)
    /builddir/build/BUILD/symbols-extractor-9f4330a0f4437ca61ba92f9f30e34424c6742ad6/src/github.com/gofed/symbols-extractor/pkg/util/util.go:181 +0x6e8
path/filepath.walk(0xc0002d22a0, 0x6d, 0xa3e6a0, 0xc0002bcc30, 0xc0002a7460, 0x0, 0x2)
    /usr/lib/golang/src/path/filepath/path.go:362 +0xf6
path/filepath.Walk(0xc0002d22a0, 0x6d, 0xc0002a7460, 0x9a32e6, 0x1)
    /usr/lib/golang/src/path/filepath/path.go:404 +0x105
github.com/gofed/symbols-extractor/pkg/util.(*PackageInfoCollector).CollectPackageInfos(0xc000283ae0, 0x7ffc9eabf894, 0x1b, 0x7ffc9eabf894, 0x1b)
    /builddir/build/BUILD/symbols-extractor-9f4330a0f4437ca61ba92f9f30e34424c6742ad6/src/github.com/gofed/symbols-extractor/pkg/util/util.go:139 +0x158
main.main.func1(0xc0002b7a40, 0x0, 0x0)
    /builddir/build/BUILD/symbols-extractor-9f4330a0f4437ca61ba92f9f30e34424c6742ad6/src/github.com/gofed/symbols-extractor/cmd/golist/golist.go:73 +0x5d2
reflect.Value.call(0x905580, 0x9c8240, 0x13, 0x9a3ba5, 0x4, 0xc0001459d0, 0x1, 0x1, 0xc0000cc200, 0x8a7ade, ...)
    /usr/lib/golang/src/reflect/value.go:447 +0x449
reflect.Value.Call(0x905580, 0x9c8240, 0x13, 0xc0001459d0, 0x1, 0x1, 0x1, 0x9a32d8, 0x1)
    /usr/lib/golang/src/reflect/value.go:308 +0xa4
github.com/gofed/symbols-extractor/vendor/github.com/urfave/cli.HandleAction(0x905580, 0x9c8240, 0xc0002b7a40, 0x0, 0x0)
    /builddir/build/BUILD/symbols-extractor-9f4330a0f4437ca61ba92f9f30e34424c6742ad6/src/github.com/gofed/symbols-extractor/vendor/github.com/urfave/cli/app.go:481 +0x1fb
github.com/gofed/symbols-extractor/vendor/github.com/urfave/cli.(*App).Run(0xc00009ac00, 0xc0000ce000, 0x7, 0x7, 0x0, 0x0)
    /builddir/build/BUILD/symbols-extractor-9f4330a0f4437ca61ba92f9f30e34424c6742ad6/src/github.com/gofed/symbols-extractor/vendor/github.com/urfave/cli/app.go:240 +0x47f
main.main()
    /builddir/build/BUILD/symbols-extractor-9f4330a0f4437ca61ba92f9f30e34424c6742ad6/src/github.com/gofed/symbols-extractor/cmd/golist/golist.go:125 +0x791

Since this crash doesn't impact the package build itself, I'm going to ignore it. But it looks like this should be fixed ...

Here's a link to the affected koji build: https://koji.fedoraproject.org/koji/taskinfo?taskID=30252400
The traceback can be seen in the %check section of the build.log.

So is this or is this not a problem? Are tests actually being run or are they being ignored just like this panic?

The original reporter may not see the question since this is a migrated issue.

But even if tests were still being run (not sure about this), a panic is pretty bad behavior: packagers are supposed to care about how builds proceed, we don't want them to ignore errors in build logs because they may be harmless (so the tooling should not inject errors there)

From a bit of testing in mock, the problem seems to be about running in the source directory. If I'm in the source directory, then run (simplified from go-rpm-integration):
GOPATH="/builddir/build/BUILD/viper-1.3.1/_build${GOPATH+:${GOPATH}}" golist --provided --tests --package-path "github.com/spf13/viper" -r '.*example.*'
it crashes, but if I move up one directory, it produces github.com/spf13/viper without crashing.

Also, it looks like tests are not being run, since go-rpm-integration does for dir in $(... golist ...); do which ignores the subprocess failure.

A workaround I've found that appears to work is:

%check
builddir="$PWD/_build"
cd ..
%gochecks -b "$builddir"

That sucks since other projects expect to be in the source dir when run. So I guess the shell wrapper will need to be changed to run golist outside this dir and switch to it afterwards: (

I've filled the shell workaround as https://pagure.io/go-rpm-macros/issue/6 A golist fix would be even nicer, but I don't know who maintains that coderbase today (@jchaloup is awfully busy)

Note that even if I write the workaround soonish, the Fedora go-rpm-macros clean up is stuck on https://pagure.io/GoSIG/go-sig/issue/22 and I'm sick of all the unrealistic conditions @jcajka invents to delay the operation (for example, macro unit tests, when we even a core well supported language like python does not have those and no one know hows to do it, as was proven by a recent fedora-devel exchange).

So I'll write the fix, but someone else will have to convince @jcajka to let it be imported in Fedora.
Edited by nim

@nim I would much appreciate if you woudn't put words in to my mouth, thanks

@jcajka: how do you expect anyone to understand your position? Months over months of not accepting macro fixes, months over months of finding new reasons not to look at them, refusal to discuss it in SIG meetings, no activity in the pagure tickets that were opened at your request to follow things, etc

You complain you're asked to look at a fork. But it's only a fork because you don't take steps to review and merge the result (or merge some other better implementation). Every time I fix a new problem (like the one reported here) that takes the code further from what got into Fedora and makes you less willing to look at the result.

For readers that don't understand the latest exchange, the participants wrote their piece here https://pagure.io/GoSIG/go-sig/issue/3

The backtraces point to line 1888 here:

1886 func ImportPaths(args []string) []*search.Match {
1887         if ModInit(); cfg.ModulesEnabled {
1888>                return ModImportPaths(args)
1889         }
1890         return search.ImportPaths(args)
1891 }

Both cfg.ModulesEnabled and ModImportPaths are semi-global variables that are set to something in the ModInit call in internal/modload/init.go:

170         cfg.ModulesEnabled = true
171         load.ModBinDir = BinDir
172         load.ModLookup = Lookup
173         load.ModPackageModuleInfo = PackageModuleInfo
174         load.ModImportPaths = ImportPaths
175         load.ModPackageBuildInfo = PackageBuildInfo
176         load.ModInfoProg = ModInfoProg
177         load.ModImportFromFiles = ImportFromFiles
178         load.ModDirImportPath = DirImportPath
179 
180         search.SetModRoot(ModRoot)
181 }

I took a look in delve, and something seems up with the variables:

  • Great, only one copy:
(dlv) vars ModulesEnabled
cmd/go/internal/cfg.ModulesEnabled = true
  • Multiple versions, but at least they're in sync:
(dlv) vars ModInit
pagure.io/golist/pkg/util/internal/load.ModInit = pagure.io/golist/pkg/util/internal/modload.Init
cmd/go/internal/load.ModInit = pagure.io/golist/pkg/util/internal/modload.Init
  • Multiple versions, but not in sync:
(dlv) vars ModImportPaths
pagure.io/golist/pkg/util/internal/load.ModImportPaths = nil
cmd/go/internal/load.ModImportPaths = pagure.io/golist/pkg/util/internal/modload.ImportPaths

I don't exactly know why this occurs, but finding the root of this would obviously fix the problem.

Metadata Update from @qulogic:
- Issue assigned to qulogic

BTW:

  • thanks an awful lot for looking at golist code maintenance
  • if you can figure how to make golist process module info, and spit out the version constrains that only exists in those modules, that would be much nicer than inhibiting the module code path. Mid term, it would be more efficient for Fedora to process module info by default, and contribute module descriptors to projects that lack then, rather than having packagers deal with the complexity of not using module info when upstreams already migrated to modules, or (worse) having to deal with some Go components in module mode and others in legacy something else mode

That's more or less liked to issue #13. A lot of the un-necessary complexity in the macro code is due to reprocessing the golist output format into something that rpm can use. The macros would be much more manageable if golist had good rpm-compatible defaults, and if the macros needn't feed golist dozens of parameters, and then reprocess golist output in something rpm understands.

I think at least 25% of the macro code is just massaging golist output and massaging the arguments and variables needed for this output to be useful.

Edited by nim

Add a similar error with latest golang-github-sirupsen-logrus:

Testing in: /home/bob/packaging/golang/golang-github-Sirupsen-logrus/logrus-1.4.0/_build/src
      PATH: /home/bob/packaging/golang/golang-github-Sirupsen-logrus/logrus-1.4.0/_build/bin:/home/bob/.local/bin:/usr/local/bin:/usr/bin:/bin
    GOPATH: /home/bob/packaging/golang/golang-github-Sirupsen-logrus/logrus-1.4.0/_build:/usr/lib64/gocode
   command: go test -buildmode pie -compiler gc -ldflags "-X github.com/sirupsen/logrus/version=1.4.0 -extldflags '-Wl,-z,relro -Wl,--as-needed  -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld '"
   testing: github.com/sirupsen/logrus
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
        panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x564cf5030c9f]
goroutine 1 [running]:
panic(0x564cf54e7200, 0x564cf586dd40)
        /usr/lib/golang/src/runtime/panic.go:556 +0x2cf fp=0xc000136e20 sp=0xc000136d90 pc=0x564cf4ead2bf
github.com/urfave/cli.HandleAction.func1(0xc000137a30)
        /usr/lib64/gocode/src/github.com/urfave/cli/app.go:473 +0x27a fp=0xc000136ec8 sp=0xc000136e20 pc=0x564cf532d5aa
runtime.call32(0x0, 0x564cf5569f88, 0xc0000b8230, 0x800000008)
        /usr/lib/golang/src/runtime/asm_amd64.s:522 +0x3d fp=0xc000136ef8 sp=0xc000136ec8 pc=0x564cf4ed89fd
panic(0x564cf54e7200, 0x564cf586dd40)
        /usr/lib/golang/src/runtime/panic.go:513 +0x1bd fp=0xc000136f88 sp=0xc000136ef8 pc=0x564cf4ead1ad
runtime.panicmem()
        /usr/lib/golang/src/runtime/panic.go:82 +0x60 fp=0xc000136fa8 sp=0xc000136f88 pc=0x564cf4eac2c0
runtime.sigpanic()
        /usr/lib/golang/src/runtime/signal_unix.go:390 +0x186 fp=0xc000136ff8 sp=0xc000136fa8 pc=0x564cf4ec2736
pagure.io/golist/pkg/util/internal/load.ImportPaths(0xc00030c510, 0x1, 0x1, 0x0, 0x1, 0xc0001370f0)
        /builddir/build/BUILD/golist-0.9.0/_build/src/pagure.io/golist/pkg/util/internal/load/pkg.go:1888 +0x5f fp=0xc000137038 sp=0xc000136ff8 pc=0x564cf5030c9f
pagure.io/golist/pkg/util/internal/load.PackagesAndErrors(0xc00030c510, 0x1, 0x1, 0x564cf55743a0, 0xc0002bb640, 0xf)
        /builddir/build/BUILD/golist-0.9.0/_build/src/pagure.io/golist/pkg/util/internal/load/pkg.go:1842 +0xa5 fp=0xc0001371b0 sp=0xc000137038 pc=0x564cf5030545
pagure.io/golist/pkg/util.ListPackage(0xc0002ee2f1, 0x1a, 0x1a, 0x0, 0xc0001372b8)
        /builddir/build/BUILD/golist-0.9.0/_build/src/pagure.io/golist/pkg/util/util.go:420 +0x91 fp=0xc000137220 sp=0xc0001371b0 pc=0x564cf531c1a1
pagure.io/golist/pkg/util.(*PackageInfoCollector).CollectPackageInfos.func1(0xc0002ee2a0, 0x6c, 0x564cf5576180, 0xc000268f70, 0x0, 0x0, 0x0, 0xc000137350)
        /builddir/build/BUILD/golist-0.9.0/_build/src/pagure.io/golist/pkg/util/util.go:181 +0x6ea fp=0xc0001372c8 sp=0xc000137220 pc=0x564cf531cada
path/filepath.walk(0xc0002ee2a0, 0x6c, 0x564cf5576180, 0xc000268f70, 0xc0002bb620, 0x0, 0x2)
        /usr/lib/golang/src/path/filepath/path.go:362 +0xf8 fp=0xc0001373a0 sp=0xc0001372c8 pc=0x564cf4fa91b8
path/filepath.Walk(0xc0002ee2a0, 0x6c, 0xc0002bb620, 0x564cf533402e, 0x1)
        /usr/lib/golang/src/path/filepath/path.go:404 +0x107 fp=0xc000137400 sp=0xc0001373a0 pc=0x564cf4fa9627
pagure.io/golist/pkg/util.(*PackageInfoCollector).CollectPackageInfos(0xc000295db0, 0x7ffe0d6433f8, 0x1a, 0x7ffe0d6433f8, 0x1a)
        /builddir/build/BUILD/golist-0.9.0/_build/src/pagure.io/golist/pkg/util/util.go:139 +0x15a fp=0xc000137478 sp=0xc000137400 pc=0x564cf531890a
main.main.func1(0xc00031c500, 0x0, 0x0)
        /builddir/build/BUILD/golist-0.9.0/_build/src/pagure.io/golist/cmd/golist/golist.go:73 +0x5d4 fp=0xc000137610 sp=0xc000137478 pc=0x564cf53316e4
runtime.call32(0xc0002ec7b0, 0x564cf556a060, 0xc0002bb5c0, 0x800000018)
        /usr/lib/golang/src/runtime/asm_amd64.s:522 +0x3d fp=0xc000137640 sp=0xc000137610 pc=0x564cf4ed89fd
reflect.Value.call(0x564cf54cba60, 0x564cf556a060, 0x13, 0x564cf53348ed, 0x4, 0xc0001379f0, 0x1, 0x1, 0xc0000b8200, 0xc00031c500, ...)
        /usr/lib/golang/src/reflect/value.go:447 +0x44b fp=0xc000137908 sp=0xc000137640 pc=0x564cf4f0937b
reflect.Value.Call(0x564cf54cba60, 0x564cf556a060, 0x13, 0xc0001379f0, 0x1, 0x1, 0x2, 0x2, 0x1)
        /usr/lib/golang/src/reflect/value.go:308 +0xa6 fp=0xc000137970 sp=0xc000137908 pc=0x564cf4f08df6
github.com/urfave/cli.HandleAction(0x564cf54cba60, 0x564cf556a060, 0xc00031c500, 0x0, 0x0)
        /usr/lib64/gocode/src/github.com/urfave/cli/app.go:482 +0x1fa fp=0xc000137a18 sp=0xc000137970 pc=0x564cf5321f7a
github.com/urfave/cli.(*App).Run(0xc000088d80, 0xc0000ba000, 0x7, 0x7, 0x0, 0x0)
        /usr/lib64/gocode/src/github.com/urfave/cli/app.go:240 +0x481 fp=0xc000137bc8 sp=0xc000137a18 pc=0x564cf53200f1
main.main()
        /builddir/build/BUILD/golist-0.9.0/_build/src/pagure.io/golist/cmd/golist/golist.go:125 +0x793 fp=0xc000137f98 sp=0xc000137bc8 pc=0x564cf5330f33
runtime.main()
        /usr/lib/golang/src/runtime/proc.go:201 +0x20f fp=0xc000137fe0 sp=0xc000137f98 pc=0x564cf4eaf01f
runtime.goexit()
        /usr/lib/golang/src/runtime/asm_amd64.s:1333 +0x1 fp=0xc000137fe8 sp=0xc000137fe0 pc=0x564cf4eda761
Edited by eclipseo

If I initialize the variables to be non nil, 

    load.ModImportPaths = modload.ImportPaths
    load.ModBinDir = modload.BinDir
    load.ModLookup = modload.Lookup
    load.ModDirImportPath = modload.DirImportPath
    load.ModPackageModuleInfo = modload.PackageModuleInfo
    load.ModPackageBuildInfo = modload.PackageBuildInfo
    load.ModImportFromFiles = modload.ImportFromFiles

I then get a different error:
go: finding github.com/sirupsen/logrus/hooks latest panic: package github.com/sirupsen/logrus/hooks: unknown import path "github.com/sirupsen/logrus/hooks": cannot find module providing package github.com/sirupsen/logrus/hooks [recovered] panic: package github.com/sirupsen/logrus/hooks: unknown import path "github.com/sirupsen/logrus/hooks": cannot find module providing package github.com/sirupsen/logrus/hooks

It tries to d/l a subfolder as if it was a module. Works fine outside the source tree as mentioned before.

PR that works: https://pagure.io/golist/pull-request/19

Metadata Update from @qulogic:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

Fixed by #17.
Metadata

BUG

None
None
None
Related Pull Requests
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.