PR#1679: Make logging the commits into the DB its own process -

pagure

#1679 Make logging the commits into the DB its own process

Merged Dec 22, 2016 by pingou. Opened Dec 19, 2016 by pingou.

logcom into master

pingou commented Dec 19, 2016

Currently when doing a full git push on the linux tree, it takes 45 minutes before the command finishes.
This is of course problematic when doing git push over ssh.

We are here moving the logging the commits into the DB into its own process instead of in the git hook.
Now the git hook sends a notification that is received by the service that goes over these commits and
log them.
With this approach, logging all the commits still takes 45 minutes but the push command finishes much
earlier allowing the user to carry on.

jcline commented Dec 20, 2016

requests isn't stdlib so it should be grouped with trollius

jcline commented Dec 20, 2016

My preference is to call it _log or similar. It's a private object and its name should reflect that. This isn't a blocker for me, just a suggestion.

jcline commented Dec 20, 2016

I think this docblock needs updating :smile:

jcline commented Dec 20, 2016

Should this all be in a loop that gracefully handles Redis not being available? Does trollius offer a way to auto-reconnect?

jcline commented Dec 20, 2016

It's probably worth documenting what the expected message schema is in the docblock.

pingou commented Dec 21, 2016

indeed, thanks

pingou commented Dec 21, 2016

but it's a global variable, won't pep8 complain if it's not capitalized? Otherwise I don't mind renaming it but I remember when I first named it log pep8 was unhappy.

/me will test

pingou commented Dec 21, 2016

yup :)

pingou commented Dec 21, 2016

Good idea

pingou commented Dec 21, 2016

It is actually included:

INFO:trollius_redis:Connecting to redis
INFO:trollius_redis:Connecting to redis failed. Retrying in 0 seconds
INFO:trollius_redis:Connecting to redis
INFO:trollius_redis:Connecting to redis failed. Retrying in 1 seconds
INFO:trollius_redis:Connecting to redis
....

2 new commits added

Rename the LOG variable to _log, it's a private variable to that module
Adjust pagure_logcom server based on feedback from the review

Dec 21, 2016

jcline commented Dec 21, 2016

:thumbsup:

pingou commented Dec 22, 2016

Thanks for the review :)

rebased

Dec 22, 2016

Pull-Request has been merged by pingou

Dec 22, 2016

Metadata

Assignee

None

Tags

No Tags

pagure

#1679 Make logging the commits into the DB its own process Merged Dec 22, 2016 by pingou. Opened Dec 19, 2016 by pingou. logcom into master

Metadata

#1679 Make logging the commits into the DB its own process

Merged Dec 22, 2016 by pingou. Opened Dec 19, 2016 by pingou.

logcom into master