Hi,
I use Berkeley-DB to manage more than 250 dbs and every db has the
same structure (16 bytes key and 8 bytes data) and almost the same
acount of records (about 500,000). These dbs are opened in the same
environment with transaction enabled. When initialization, I have 4
threads to retrieve data from them through cursor simultaneously but
no two thread will operate on the same db (thread 1 reads db 0,
4, ...; thread 2 reads db 1, 5, ...; etc.). After a while, the cursor
read process becomes very slow, it need more than 20 seconds to finish
the read from a db.
My cache size is 4G and I use the default page size which is 4K.
Following is part of my DB_CONFIG:
set_cachesize 4 0 4
mutex_set_max 1572864
set_lk_max_lockers 4096
set_lk_max_objects 2048
set_lk_max_locks 8388608
I also used "valgrind --tool=callgrind" to profile my program. I got
the following result (I'm not familiar with valgrind so I don't know
how to can get a better profiling result):
1,741,812,564 ???:__memp_get_bucket [libdb-4.6.so]
1,727,316,262 ???:__db_tas_mutex_lock [libdb-4.6.so]
959,780,404 ???:__memp_fget [libdb-4.6.so]
888,693,549 ???:__env_alloc [libdb-4.6.so]
882,790,067 ???:__dbc_get [libdb-4.6.so]
798,835,644 ???:__db_tas_mutex_unlock [libdb-4.6.so]
722,079,676 ???:__os_id [libdb-4.6.so]
685,849,568 ???:__ham_func4 [libdb-4.6.so]
681,554,232 ???:__db_retcopy [libdb-4.6.so]
659,823,410 ???:__db_cursor_int [libdb-4.6.so]
526,604,190 ???:__memp_fput [libdb-4.6.so]
505,002,137 // my function
492,475,941 ???:__lock_get_internal [libdb-4.6.so]
455,045,660 ???:__db_ret [libdb-4.6.so]
451,725,119 ???:__db_lget [libdb-4.6.so]
449,867,405 ???:__bamc_next [libdb-4.6.so]
448,777,043 ???:__lock_getobj [libdb-4.6.so]
400,442,895 ???:__dbc_close [libdb-4.6.so]
359,487,207 ???:__dbc_get_arg [libdb-4.6.so]
348,992,625 // ...........
332,184,228 ???:__dbc_idup [libdb-4.6.so]
327,633,228 ???:__bamc_get [libdb-4.6.so]
323,084,790 ???:__lock_put [libdb-4.6.so]
313,985,666 ???:memcpy [/lib/libc-2.3.6.so]
295,780,327 ???:__dbc_get_pp [libdb-4.6.so]
286,679,279 ???:__dbc_cleanup [libdb-4.6.so]
285,409,596 ???:__lock_open [libdb-4.6.so]
284,264,251 ???:__logc_get_int [libdb-4.6.so]
277,579,162 ???:__bamc_close [libdb-4.6.so]
266,029,764 ???:pthread_self [/lib/libpthread-2.3.6.so]
245,727,648 ???:__lock_get [libdb-4.6.so]
213,873,198 ???:__bamc_refresh [libdb-4.6.so]
180,765,978 ???:__lock_put_internal [libdb-4.6.so]
178,388,632 // ...........
133,158,636 // ...........
128,302,983 // ...........
118,312,090 ???:__dbt_userfree [libdb-4.6.so]
109,718,273 ???:0x000000000006C270 [/lib/libc-2.3.6.so]
107,838,186 ???:__db_check_chksum [libdb-4.6.so]
101,720,408 ???:__db_pthread_mutex_init [libdb-4.6.so]
Someone can give me some advice? Thanks a lot!
|
