fm-orchestrator

#1183 A module built on a platform isn't available for use on another platform
Opened by dmoluguw. Modified

Open
Close Issue

I tried to place BuildRequires on javapackages-tools as shown in [1]. Though javapackages-tools is built on f28, it's supposed to run on ALL platforms as per the config [2].

When I try to trigger a koji build, I get the following error: 

$ fedpkg module-build 
Submitting the module build...
Could not execute module_build: The build failed with:
Cannot find any module builds for javapackages-tools:201801

When I tried a local mbs build for f30, I get the following error:

$ mbs-manager build_module_locally --file pki.yaml -s platform:f30
2019-03-25 16:51:08,895 - MainThread - urllib3.util.retry - DEBUG - Converted retries value: 3 -> Retry(total=3, connect=None, read=None, redirect=None, status=None)
2019-03-25 16:51:09,074 - MainThread - moksha.hub - WARNING - Cannot find qpid python module. Make sure you have python-qpid installed.
2019-03-25 16:51:09,251 - MainThread - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): mbs.fedoraproject.org:443
2019-03-25 16:51:10,038 - MainThread - urllib3.connectionpool - DEBUG - https://mbs.fedoraproject.org:443 "GET /module-build-service/1/module-builds/?name=platform&stream=f30&state=ready&verbose=True&order_desc_by=version&page=1&per_page=10 HTTP/1.1" 200 4209
2019-03-25 16:51:10,255 - MainThread - urllib3.connectionpool - DEBUG - https://mbs.fedoraproject.org:443 "GET /module-build-service/1/module-builds/?name=javapackages-tools&stream=201801&state=ready&verbose=True&order_desc_by=version&page=1&per_page=10&base_module_br=platform%3Af30%3A6%3A00000000 HTTP/1.1" 200 608
Traceback (most recent call last):
  File "/usr/bin/mbs-manager", line 11, in <module>
    load_entry_point('module-build-service==2.14.0', 'console_scripts', 'mbs-manager')()
  File "/usr/lib/python3.7/site-packages/module_build_service/manage.py", line 238, in manager_wrapper
    manager.run()
  File "/usr/lib/python3.7/site-packages/flask_script/__init__.py", line 417, in run
    result = self.handle(argv[0], argv[1:])
  File "/usr/lib/python3.7/site-packages/flask_script/__init__.py", line 386, in handle
    res = handle(*args, **config)
  File "/usr/lib/python3.7/site-packages/flask_script/commands.py", line 216, in __call__
    return self.run(*args, **kwargs)
  File "/usr/lib/python3.7/site-packages/module_build_service/manage.py", line 154, in build_module_locally
    username, handle, str(stream), skiptests, optional_params)
  File "/usr/lib/python3.7/site-packages/module_build_service/utils/submit.py", line 412, in submit_module_build_from_yaml
    return submit_module_build(username, None, mmd, optional_params)
  File "/usr/lib/python3.7/site-packages/module_build_service/utils/submit.py", line 512, in submit_module_build
    mmds = generate_expanded_mmds(db.session, mmd, raise_if_stream_ambigous, default_streams)
  File "/usr/lib/python3.7/site-packages/module_build_service/utils/mse.py", line 345, in generate_expanded_mmds
    current_mmd, default_streams, raise_if_stream_ambigous)
  File "/usr/lib/python3.7/site-packages/module_build_service/utils/mse.py", line 303, in get_mmds_required_by_module_recursively
    "Cannot find any module builds for %s" % (ns))
module_build_service.errors.UnprocessableEntity: Cannot find any module builds for javapackages-tools:201801

Nevertheless, building for f28 picks up the right module: 

$ mbs-manager build_module_locally --file pki.yaml -s platform:f28
2019-03-25 16:51:38,927 - MainThread - urllib3.util.retry - DEBUG - Converted retries value: 3 -> Retry(total=3, connect=None, read=None, redirect=None, status=None)
2019-03-25 16:51:39,103 - MainThread - moksha.hub - WARNING - Cannot find qpid python module. Make sure you have python-qpid installed.
2019-03-25 16:51:39,286 - MainThread - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): mbs.fedoraproject.org:443
2019-03-25 16:51:39,666 - MainThread - urllib3.connectionpool - DEBUG - https://mbs.fedoraproject.org:443 "GET /module-build-service/1/module-builds/?name=platform&stream=f28&state=ready&verbose=True&order_desc_by=version&page=1&per_page=10 HTTP/1.1" 200 2391
2019-03-25 16:51:41,608 - MainThread - urllib3.connectionpool - DEBUG - https://mbs.fedoraproject.org:443 "GET /module-build-service/1/module-builds/?name=javapackages-tools&stream=201801&state=ready&verbose=True&order_desc_by=version&page=1&per_page=10&base_module_br=platform%3Af28%3A4%3A00000000 HTTP/1.1" 200 1556374
2019-03-25 16:51:44,446 - MainThread - urllib3.connectionpool - DEBUG - https://mbs.fedoraproject.org:443 "GET /module-build-service/1/module-builds/?name=javapackages-tools&stream=201801&state=ready&verbose=True&order_desc_by=version&page=2&per_page=10&base_module_br=platform%3Af28%3A4%3A00000000 HTTP/1.1" 200 669483
2019-03-25 16:51:45,328 - MainThread - module_build_service - DEBUG - Adding module platform:f28:4:00000000 with requires: None
2019-03-25 16:51:45,329 - MainThread - module_build_service - DEBUG - Adding module javapackages-tools:201801:2820190319130429:819b5873 with requires: <Id #195 module(platform)>
2019-03-25 16:51:45,329 - MainThread - module_build_service - INFO - Starting resolving with following input modules: ['platform:f28:4:00000000', 'javapackages-tools:201801:2820190319130429:819b5873']
2019-03-25 16:51:45,330 - MainThread - module_build_service - DEBUG - Adding module pki:f28:20190325205139:0 with requires: <Id #-2147483643 ((module(javapackages-tools) with module(javapackages-tools:201801)) and (module(platform) with module(platform:f28)))>
2019-03-25 16:51:45,330 - MainThread - module_build_service - DEBUG - Testing pki:f28:20190325205139:0-20190325205139.src with combination: (<Solvable #3 javapackages-tools:201801:2820190319130429:819b5873-2820190319130429.x86_64>, <Solvable #2 platform:f28:4:00000000-4.x86_64>)
2019-03-25 16:51:45,330 - MainThread - module_build_service - DEBUG - Jobs:
2019-03-25 16:51:45,330 - MainThread - module_build_service - DEBUG -   - favor javapackages-tools:201801:2820190319130429:819b5873-2820190319130429.x86_64
2019-03-25 16:51:45,330 - MainThread - module_build_service - DEBUG -   - favor platform:f28:4:00000000-4.x86_64
2019-03-25 16:51:45,330 - MainThread - module_build_service - DEBUG -   - install pki:f28:20190325205139:0-20190325205139.src
2019-03-25 16:51:45,330 - MainThread - module_build_service - DEBUG - Transaction:
2019-03-25 16:51:45,330 - MainThread - module_build_service - DEBUG -   - pki:f28:20190325205139:0-20190325205139.src
2019-03-25 16:51:45,330 - MainThread - module_build_service - DEBUG -   - platform:f28:4:00000000-4.x86_64
2019-03-25 16:51:45,331 - MainThread - module_build_service - DEBUG -   - javapackages-tools:201801:2820190319130429:819b5873-2820190319130429.x86_64
2019-03-25 16:51:45,331 - MainThread - module_build_service - INFO - Resolving done, possible requires: {frozenset({'pki:f28:20190325205139:0:src', 'javapackages-tools:201801:2820190319130429:819b5873:x86_64', 'platform:f28:4:00000000:x86_64'})}
... <clipped>

In short, I'm trying to build my module for f30+. Any reasons why it doesn't detect the module?

[1] https://src.fedoraproject.org/modules/pki/blob/private-dmoluguw-10.6/f/pki.yaml
[2] https://src.fedoraproject.org/modules/javapackages-tools/blob/201801/f/javapackages-tools.yaml#_18

Note that module javapackages-tools, stream 201801 is available in MBS - eg. module build 3695. That module works on any platform.

@psabata confirmed that this is desired behavior but it's something that is not implemented.

It might have been working before https://pagure.io/fm-orchestrator/c/f2a236bc74486842af4309718a7e8353375eaac2

I remember this used to work in the past. I tried looking for an example in Fedora MBS instance, but the module build I had in mind was already removed from this MBS instance.

I guess I have the same problem. When I have a dep section of:

   dependencies:
        - buildrequires:
            javapackages-tools: [201801]
            platform: []
          requires:
            javapackages-tools: [201801]
            platform: []

Then local builds fail on all platforms except F28:

$ fedpkg module-build-local -s platform:f30
module_build_service.errors.UnprocessableEntity: Cannot find any module builds for javapackages-tools:201801
Could not execute module_build_local: Failed to execute command.

Workaround for building against javapackages-tools module:

--- /usr/lib/python3.7/site-packages/module_build_service/resolver/MBSResolver.py~  2019-04-17 12:08:26.316446269 +0200
+++ /usr/lib/python3.7/site-packages/module_build_service/resolver/MBSResolver.py   2019-04-17 12:08:51.147582892 +0200
@@ -182,6 +182,8 @@
         :return: List of modulemd metadata.
         """
         yaml = None
+        if name == 'javapackages-tools' and stream == '201801':
+            base_module_nsvc = 'platform:f28:4:00000000'
         modules = self._get_modules(name, stream, strict=False,
                                     base_module_br=base_module_nsvc)
         if not modules:

I guess I have the same problem. When I have a dep section of:
dependencies:
- buildrequires:
javapackages-tools: [201801]
platform: []
requires:
javapackages-tools: [201801]
platform: []

Then local builds fail on all platforms except F28:
$ fedpkg module-build-local -s platform:f30
module_build_service.errors.UnprocessableEntity: Cannot find any module builds for javapackages-tools:201801
Could not execute module_build_local: Failed to execute command.

Just a note: @mizdebsk confirmed that javapackages-tools module isn't available at runtime. So, you can't list it in the requires section.
Edited by dmoluguw

Correction: javapackages-tools can be required by other modules, as long as these modules are buildroot-only too.

How to solve this problem? It feels like mbs is not reliable. .

@youyifeng - what particular problem are you running into? It would be good to update this bug with a more current example.

This issue has been migrated to Fedora Forge:
https://forge.fedoraproject.org/koji/mbs/issues/1183

Please continue any further discussion there.
Metadata
None
None
None
None
None
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.