Talk About Network

Google


Register and Login
Nick
Password
Register create new account Sign up is FREE and you can post replies, new topics, bookmark posts and more!
Recover lost password


Data Bases > Pgsql Performance > Re: Very poor p...
Latest [ Topics | Posts ] Archive Post A New Topic Post a Reply
<< Topic < Post Post 6 of 8 Topic 4030 of 4424
Post > Topic >>

Re: Very poor performance loading 100M of sql data using

by gsmith@[EMAIL PROTECTED] (Greg Smith) Apr 28, 2008 at 02:16 PM

On Mon, 28 Apr 2008, John Rouillard wrote:

>    2008-04-21 11:36:43 UTC @[EMAIL PROTECTED]
(2761)i: LOG: checkpoints ... (27 seconds
apart)
> so I changed:
>   checkpoint_segments = 30
>   checkpoint_warning = 150

That's good, but you might go higher than 30 for a bulk loading operation 
like this, particularly on 8.1 where checkpoints are no fun.  Using 100 is

not unreasonable.

> shared_buffers = 3000
> I don't see any indication in the docs that increasing shared memory
> would help speed up a copy operation.

The index blocks use buffer space, and what ends up happening if there's 
not enough memory is they are written out more than they need to be (and 
with your I/O hardware you need to avoid writes unless absolutely 
necessary).  Theoretically the OS is caching around that situation but 
better to avoid it.  You didn't say how much RAM you have, but you should 
start by a factor of 10 increase to 30,000 and see if that helps; if so, 
try making it large enough to use 1/4 of total server memory.  3000 is 
only giving the server 24MB of RAM to work with, and it's unfair to expect

it to work well in that situation.

While not relevant to this exercise you'll need to set 
effective_cache_size to a useful value one day as well.

--
* Greg Smith gsmith@[EMAIL PROTECTED]
 http://www.gregsmith.com
Baltimore, MD

-- 
Sent via pgsql-performance mailing list (pgsql-performance@[EMAIL PROTECTED]
)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
 




 8 Posts in Topic:
Very poor performance loading 100M of sql data using copy
rouilj@[EMAIL PROTECTED]   2008-04-28 17:24:31 
Re: Very poor performance loading 100M of sql data using copy
heikki@[EMAIL PROTECTED]   2008-04-28 18:53:09 
Re: Very poor performance loading 100M of sql data using copy
rouilj@[EMAIL PROTECTED]   2008-04-28 18:00:53 
Re: Very poor performance loading 100M of sql data using
pgsql@[EMAIL PROTECTED]   2008-04-29 05:19:59 
Re: Very poor performance loading 100M of sql data using copy
rouilj@[EMAIL PROTECTED]   2008-04-29 15:04:32 
Re: Very poor performance loading 100M of sql data using
gsmith@[EMAIL PROTECTED]   2008-04-28 14:16:02 
Re: Very poor performance loading 100M of sql data using copy
rouilj@[EMAIL PROTECTED]   2008-04-29 15:16:22 
Re: Very poor performance loading 100M of sql data using
gsmith@[EMAIL PROTECTED]   2008-04-29 11:58:00 

Post A Reply:
  Go here to Signup

AddThis Feed Button


About - Advertising - Contact - Frequently Asked Questions - Privacy Policy - Terms of Use - Signup

Contact
tan12V112 Mon Dec 1 9:03:30 CST 2008.