#1714 use BulkInsertProcessor for hub mass inserts
Merged by mikem. Opened by tkopecek.
tkopecek/koji issue1712  into  master

Download 1714.patch

Fixes: https://pagure.io/koji/issue/1712

is this a debug print?

rebased onto b1ac83aaf445b177fb0aa05e3cfa203c551051c5

is this a debug print?

Yes, fixed.

:thumbsup:

The part in import_components() is still calling set() for the bulk operation.

I don't think set_record is quite the right name for this. Seems like add_record (or even just add) would be more accurate.

1 new commit added

  • rename set_record to add_record

renamed

On Friday we were talking about possibly chunking this for large numbers of inserts. I just took a stab at this here. If we still think this is worthwhile we can pull this into the PR, or possibly we can split it off into a separate one.

https://github.com/mikem23/koji-playground/commits/pagure/pr/1714

1 new commit added

  • support batch operation in BulkInsertProcessor

Yep, I pulled it into this PR.

Oops, there's a typo in my patch. Should be self._one_insert() both places in execute().

@@ -8386,7 +8386,7 @@ class BulkInsertProcessor(object):
     def execute(self):
         if not self.batch:
-            self.__one_insert(self.data)
+            self._one_insert(self.data)
         else:
             for i in range(0, len(self.data), self.batch):
                 data = self.data[i:i+self.batch]

Fixed with added unit test at same location

https://github.com/mikem23/koji-playground/commits/pagure/pr/1714

1 new commit added

  • fix typo and extend unit test to cover where it occurred

Commit aaaabcc9 fixes this pull-request

Pull-Request has been merged by mikem

Metadata Update from @jcupova:
- Pull-request tagged with: no_qe

Metadata