389-ds-base

#49178 389 (setup-ds.pl) wants to allocate 40GiB of RAM
Closed: wontfix by mreynolds. Opened by ttorcz.

Closed: wontfix
Reopen Issue

Issue Description

I'm trying to create FreeIPA replica using ipa-replica-install. The commands fails, and log file shows:
2017-03-21T14:19:47Z DEBUG calling setup-ds.pl
2017-03-21T14:19:47Z DEBUG Starting external process
2017-03-21T14:19:47Z DEBUG args=/usr/sbin/setup-ds.pl --silent --logfile - -f /tmp/tmp1bwAYm
2017-03-21T14:20:02Z DEBUG Process finished, return code=1
2017-03-21T14:20:02Z DEBUG stdout=[17/03/21:15:20:02] - [Setup] Info Could not import LDIF file '/var/lib/dirsrv/boot.ldif'. Error: 3072. Output: importing data ...
[21/Mar/2017:15:20:01.646134308 +0100] WARNING: Import is running with nsslapd-db-private-import-mem on; No other process is allowed to access the database
[21/Mar/2017:15:20:01.927035173 +0100] check_and_set_import_cache: pagesize: 4096, pages: -1, procpages: 4593
[21/Mar/2017:15:20:02.129829116 +0100] Import allocates -859122368KB import cache.
[21/Mar/2017:15:20:02.203908584 +0100] Setting ncache to: 858993460 to keep each chunk below 4Gbytes
[21/Mar/2017:15:20:02.348597443 +0100] libdb: BDB0147 malloc: 41231686328: Cannot allocate memory
[21/Mar/2017:15:20:02.452740622 +0100] libdb: BDB1543 unable to create new master region array: Cannot allocate memory
[21/Mar/2017:15:20:02.526455796 +0100] Unable to open new DB_ENV for import/export! 12
[21/Mar/2017:15:20:02.613398853 +0100] dblayer_instance_start fail: Cannot allocate memory (12)

Could not import LDIF file '/var/lib/dirsrv/boot.ldif'. Error: 3072. Output: importing data ...
[21/Mar/2017:15:20:01.646134308 +0100] WARNING: Import is running with nsslapd-db-private-import-mem on; No other process is allowed to access the database
[21/Mar/2017:15:20:01.927035173 +0100] check_and_set_import_cache: pagesize: 4096, pages: -1, procpages: 4593
[21/Mar/2017:15:20:02.129829116 +0100] Import allocates -859122368KB import cache.
[21/Mar/2017:15:20:02.203908584 +0100] Setting ncache to: 858993460 to keep each chunk below 4Gbytes
[21/Mar/2017:15:20:02.348597443 +0100] libdb: BDB0147 malloc: 41231686328: Cannot allocate memory
[21/Mar/2017:15:20:02.452740622 +0100] libdb: BDB1543 unable to create new master region array: Cannot allocate memory
[21/Mar/2017:15:20:02.526455796 +0100] Unable to open new DB_ENV for import/export! 12
[21/Mar/2017:15:20:02.613398853 +0100] dblayer_instance_start fail: Cannot allocate memory (12)

Package Version and Platform

389-ds-base-1.3.5.15-1.fc25.x86_64

```# free -h
total used free shared buff/cache available

Mem: 8388607T 223M 8388607T 307M 59M 8388607T

Swap: 15G 7.4G 8.6G```

Steps to reproduce

  1. Create libvirt-lxc container with 2GB of RAM.
  2. Install base fedora 25
  3. ipa-replica-install --mkhomedir --setup-ca --setup-dns --auto-forwarders

Actual results

2017-03-21T14:20:02Z DEBUG The ipa-replica-install command failed, exception: RuntimeError: failed to create ds instance Command '/usr/sbin/setup-ds.pl --silent --logfile - -f /tmp/tmp1bwAYm' returned non-zero exit status 1 2017-03-21T14:20:03Z ERROR failed to create ds instance Command '/usr/sbin/setup-ds.pl --silent --logfile - -f /tmp/tmp1bwAYm' returned non-zero exit status 1
2017-03-21T14:20:03Z ERROR The ipa-replica-install command failed. See /var/log/ipareplica-install.log for more information

Expected results

Working replica.

Likely caused by https://pagure.io/389-ds-base/issue/48864 which affects the import autotuning during setup-ds.pl import.

Metadata Update from @firstyear:
- Custom field reviewstatus adjusted to new
- Custom field type adjusted to defect

Metadata Update from @firstyear:
- Issue assigned to firstyear
- Issue set to the milestone: 1.3.7 backlog

Metadata Update from @firstyear:
- Issue tagged with: Complex, Performance

Metadata Update from @firstyear:
- Custom field reviewstatus reset (from new)
- Issue close_status updated to: fixed
- Issue status updated to: Closed (was: Open)

Metadata Update from @ttorcz:
- Issue status updated to: Open (was: Closed)

Reopening, as it wasn't fixed. I've tried to create replica in libvirt-lxc container today again, with 389-ds-base-1.3.7.7-1.fc27.x86_64.
The result is

[26/Nov/2017:13:21:30.424392946 +0000] - INFO - check_and_set_import_cache - pagesize: 4096, available bytes 9223372021559820288, process usage 24150016 
[26/Nov/2017:13:21:30.463022012 +0000] - INFO - check_and_set_import_cache - Import allocates 144115182101263KB import cache.
[26/Nov/2017:13:21:30.502810880 +0000] - NOTICE - dblayer_select_ncache - Setting ncache to: 34359737 to keep each chunk below 4Gbytes
tcmalloc: large alloc 1649270784 bytes == 0x555949b6e000 @  0x7fc1658719ad 0x7fc165891413 0x7fc15cb9d485 0x7fc15cb67f90 0x7fc15cb713bd 0x7fc15cb6b610 0x7fc15cb6bf32 0x7fc1580f7b77 0x7fc15813f9aa 0x555947949e8b 0x7fc16375703a 0x55594794b00a
tcmalloc: large alloc 584613150720 bytes == (nil) @  0x7fc1658719ad 0x7fc165891413 0x7fc15cb9d485 0x7fc15cb67f90 0x7fc15cb29149 0x7fc15cb6b6a8 0x7fc15cb6bf32 0x7fc1580f7b77 0x7fc15813f9aa 0x555947949e8b 0x7fc16375703a 0x55594794b00a
[26/Nov/2017:13:21:31.548291806 +0000] - ERR - libdb - BDB0147 malloc: 584613149444: Cannot allocate memory
[26/Nov/2017:13:21:31.676567865 +0000] - ERR - libdb - BDB2014 Unable to allocate memory for mutexes from the region
[26/Nov/2017:13:21:31.744285973 +0000] - ERR - libdb - BDB0061 PANIC: Cannot allocate memory

Something is very odd with your environment. I'll need the following log output:

cat /var/log/slapd-INSTANCE-NAME/errors | grep spal

Your partially installed IPA instance (before you run ipa-server-install -u) will work for this. Then send me the log from /tmp/spal.log.

We try to determine memory limits from a number of places. These are:

  • rlimit
  • cgroup
  • /proc/meminfo

The only time we've seen this kind of behaviour is when your /proc/meminfo does not provite MemAvailable, and we resolved that. This looks more like your /proc/meminfo may not exist at all, and we fall back to cgroup, which has a very large available (9223372021559820288).

So in addition, I want to see your lxc:

  • /proc/meminfo
  • /sys/fs/cgroup/memory/memory.soft_limit_in_bytes
  • /sys/fs/cgroup/memory/memory.limit_in_bytes
  • /sys/fs/cgroup/memory/memory.usage_in_bytes

Thanks,

Metadata Update from @firstyear:
- Custom field component adjusted to None
- Custom field origin adjusted to None
- Custom field reviewstatus adjusted to None
- Custom field version adjusted to None

No 'spal' in errors:

# cat /var/log/dirsrv/slapd-PIPEBREAKER-PL/errors | grep -i spal
#

/tmp/spal.log does not exist. If it were here previously, it was deleted on reboot.

/proc/meminfo:

MemTotal:       9007199254740991 kB
MemFree:        9007199254589103 kB
MemAvailable:   9007199254589103 kB
Buffers:               0 kB
Cached:           113196 kB
SwapCached:       105524 kB
Active:            39236 kB
Inactive:          92728 kB
Active(anon):      17272 kB
Inactive(anon):     1612 kB
Active(file):      21964 kB
Inactive(file):    91116 kB
Unevictable:           0 kB
Mlocked:             944 kB
SwapTotal:      16777212 kB
SwapFree:       14824956 kB
Dirty:             14200 kB
Writeback:            12 kB
AnonPages:       6222176 kB
Mapped:           618516 kB
Shmem:            370032 kB
Slab:                  0 kB
SReclaimable:          0 kB
SUnreclaim:            0 kB
KernelStack:       37456 kB
PageTables:       124448 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    24912936 kB
Committed_AS:   20983636 kB
VmallocTotal:   34359738367 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
HardwareCorrupted:     0 kB
AnonHugePages:   s:        0 kB
ShmemPmdMapped:        0 kB
CmaTotal:              0 kB
CmaFree:               0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:     2600024 kB
DirectMap2M:    14016512 kB

cgroup files:

# cat /sys/fs/cgroup/memory/memory.soft_limit_in_bytes
9223372036854771712
# cat /sys/fs/cgroup/memory/memory.limit_in_bytes
9223372036854771712
# cat /sys/fs/cgroup/memory/memory.usage_in_bytes
14551904256

I don't think this is an issue anymore. The perl tools are also deprecated. Closing ticket, but if the problem happens with dscreate please reopen this ticket

Metadata Update from @mreynolds:
- Issue close_status updated to: wontfix
- Issue status updated to: Closed (was: Open)

389-ds-base is moving from Pagure to Github. This means that new issues and pull requests
will be accepted only in 389-ds-base's github repository.

This issue has been cloned to Github and is available here:
- https://github.com/389ds/389-ds-base/issues/2237

If you want to receive further updates on the issue, please navigate to the github issue
and click on subscribe button.

Thank you for understanding. We apologize for all inconvenience.
Metadata

Complex Performance

None
None
None
None
None
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.