The current BDB doc (for the latest release), says in "Berkeley DB Transactional Data Store locking conventions":

"Under non-transaction operations, the access methods do not normally hold locks across calls to the Berkeley DB interfaces. The one exception to this rule is when cursors are used. Because cursors maintain a position in a file, they must hold locks across calls; in fact, they will hold a lock until the cursor is closed."

And in the chapter on non-transactional locking, the behaviour for cursors is described like this:

"For this reason, cursors must retain read locks across cursor calls to make sure that the position is uniquely identifiable during a subsequent cursor call, and so that an operation using DB_CURRENT will always refer to the same record as a previous cursor call. These cursor locks cannot be released until the cursor is either repositioned and a new cursor lock established (for example, using the DB_NEXT or DB_SET flags), or the cursor is closed. As a result, application writers are encouraged to close cursors as soon as possible."

Replying to [comment:1 lkrispen]:

The current BDB doc (for the latest release), says in "Berkeley DB Transactional Data Store locking conventions":

"Under non-transaction operations, the access methods do not normally hold locks across calls to the Berkeley DB interfaces. The one exception to this rule is when cursors are used. Because cursors maintain a position in a file, they must hold locks across calls; in fact, they will hold a lock until the cursor is closed."

If you look at the code in clcache_load_buffer_bulk() you will see that it opens a cursor only so that it can use the DB_NEXT/DB_SET flags to position the reader to the right record. Once positioned, the cursor is immediately closed. The changelog reader does not keep the cursor open to read the next record.

And in the chapter on non-transactional locking, the behaviour for cursors is described like this:

"For this reason, cursors must retain read locks across cursor calls to make sure that the position is uniquely identifiable during a subsequent cursor call, and so that an operation using DB_CURRENT will always refer to the same record as a previous cursor call. These cursor locks cannot be released until the cursor is either repositioned and a new cursor lock established (for example, using the DB_NEXT or DB_SET flags), or the cursor is closed. As a result, application writers are encouraged to close cursors as soon as possible."

"close cursors as soon as possible" which is what the code in clcache_load_buffer_bulk() does.

yes, it looks like it is closing immediately, but with the us of DB_MULTIPLE_KEY the first call to c_get will iterate thru the changelog using the cursor to fill the buffer.

Replying to [comment:3 lkrispen]:

yes, it looks like it is closing immediately, but with the us of DB_MULTIPLE_KEY the first call to c_get will iterate thru the changelog using the cursor to fill the buffer.

Right. And that's why the changelog reader thread can have more than one page locked.

achieve this with mdb?

Could also achieve this with bdb using MVCC - but there is the entry/dn cache per transaction problem

389-ds-base is moving from Pagure to Github. This means that new issues and pull requests
will be accepted only in 389-ds-base's github repository.

This issue has been cloned to Github and is available here:
- https://github.com/389ds/389-ds-base/issues/754

If you want to receive further updates on the issue, please navigate to the github issue
and click on subscribe button.

Thank you for understanding. We apologize for all inconvenience.