Re: Cursor read fail at exactly 40,000 record point

Michael,

I was getting no errors from the database (even with verbose message).
I have just tracked down the problem to:

(1) The database cache was set to 1 GB on a machine with 8 GB of RAM.
(2) It would appear that Windows limits the per address space storage
to 1 GB.
(3) Each record is 300 bytes.
(4) I use a custom array function that automatically grows (via
realloc). In this case it was set to grow by 10,000 records each time.

It would appear that the database was monopolizing most of the
available address space memory. When the array grew and the record set
got large the realloc was requiring a lot of memory for both the
source and destination.

The array function was returning a failure after the array failed to
grow but I was not catching this.

So in the end it was not the database environment as such. It was the
database environment sucking up the RAM and leaving insufficient
contiguous memory for the rest of the application.

A work around for the problem was to reduce the cache size. I will
also fix the code to read through the record set twice, once to count
the records and a second time to populate the array with a single
malloc. It would be nice if you could count the keys between a key
range without having to read every record!

Thanks for taking an interest.

Gerard