#1192 Rework the CI integration
Merged by pingou. Opened by pingou.
pagure_ci  into  master

Download 1192.patch
  • Rely on REDIS instead of fedmsg
  • Make it more generic so that we can add more type of CI servers in the future
  • Trigger build upon creating a new PR or adding a notification to existing one
  • Create the pagure-ci sub-package in the spec file
  • Add the pagure-ci systemd file

2 new commits added

  • Fix the messages sent to the pagure-ci service via redis
  • Fix setting the different CI services supported

2 new commits added

  • Also set-up redis if pagure-ci is configured
  • Notify pagure-ci only if redis is configured

1 new commit added

  • Add proper attribution of the code in lib_ci

3 new commits added

  • Disable pagure-ci by default
  • Log a little more info about where the build is triggered
  • Fix retrieving the name of the project from the JSON blob

1 new commit added

  • Fix invalid variable and module names and log when it finished to process

1 new commit added

  • Enable the jenkins CI service for the tests

1 new commit added

  • Undo specifying a config file to run the tests and just bail on jenkins

add readme :)

This is Jenkins token to verify it is coming from a legit source

So one more field in the db and one more field in the form for jenkins token needs to be there

repo?? this I guess is project.user.username

this will be session.commit

Only thing is jenkins token apart from that minor issues are there !

And that can be used to ensure only us are triggering the build on jenkins. Any reasons these two tokens should be different?

1 new commit added

  • Fix typos in lib_ci.py

:thumbsup: for me !

1 new commit added

  • Improve the README for pagure-ci

Why does this run as git? I don't think this needs any access to files, right? Then we should be able to get a new user for privilege separation.

We probably should remove this?

Should this be configurable?

Coordinate this PR with the private repo one.

Why do all this messing around, and not just posixpath.join(base, project.path)?

Is this really the url that we need to provide? How about external PRs, and shouldn't this be a full clone URL?

Why a global variable that's shared?
Perhaps it would be better to have this be an internal thing, and have a getter to get access to this info.

The fact that this exists sounds wrong to me.
You should not be able to find a project by just the token, as that decreases the requirement from two to a single factor.
Instead, just get the project by the project name, and then validate the ci token in constant time.

This all assumes that this project is public. Maybe tell people to create a new user for Pagure CI, and use that here so that the Jenkins stuff doesn't need to be public? (see: private repo branch)

s/No PR found corresponding/No corresponding PR found/

s/tiven/given/

This should not need to be called outside of pagure.lib since it's porivate.
Missing some sort of wrapping function?

private*

it runs as git to avoid having to create a new user just for this, I don't know if using nobody would work

for this file, yes

:thumbsup:

it actually is, I should take these from the config iirc

If it's not needed, a new user would be preferred, since this one gives it access to more data then it needs.

agreed, but in a different PR if possible as we could then adjust other services while at it

Fixed using urlparse.urljoin

I do not know how that will work w/ remote PR for the moment, we'll need to test this

But that's unrelated to the URL provided here which is the URL of the project on jenkins

Same approach as REDIS above, it allows us to set it once when the app is started and re-use it then later in the code

w/o having to specify it to each and every function that could potentially use it

As long as the private repo changes aren't in master this is something for the person working on private repo :)

I'm fine with making it a public method but that's imho outside the scope of this PR

I'm not sure what this would gain us, could you expand?

Sure, but how about a get_pagure_ci() function, which would then be the only function accessing the global variable?

What's the benefit?

That it would become significantly harder to find a working token, since you need a correct project-token combination rather than just a token.

No global variables used across the code, making it easier to swap it out with some other implementation at a later stage.

Okay.

hm ok, let me adjust the URL

Tbh, I would think that if an user activates CI on a private project, it would most likely be by their own will.

So either the jenkins instance is private as well, in which case no problem, of the jenkins instance is public and then there is not much we can do

How about a jenkins instance that has both public and private projects?

8 new commits added

  • cryptography is also a dep for pagure-ci
  • Change the endpoint called by jenkins to include the repo name and username
  • Remove un-used import
  • Fix typo in the docstring of the jinja_render filter
  • Fix wording in the exception message
  • Drop un-used variable and retrieve the redis information from the configuration
  • Use urlparse.urljoin to build the final url to call
  • Fix typo in the pagure-ci tests

:thumbsup:

Many thanks for the review :)

Pull-Request has been merged by pingou

fedmsg is no more needed for PagureCI I feel

Metadata