------=_Part_20896_8119978.1208898335253
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline
Hi all,
I=B4m running a copy for a 37G CSV and receiving the following error:
"invalid string enlargement request size 65536"
The file has about 70 million lines with 101 columns, all them varchar.
When I run the command with the whole file i receive the error after
loadin=
g
about 29million lines. So i=B4ve spllited the file in 10 million lines
with
split:
split --lines=3D10000000
And running the copy i receive the error on the 5th file:
psql:/srv/www/htdocs/im****t/script_q2.sql:122: ERROR: invalid string
enlargement request size 65536
CONTEXT: COPY temp_q2, line 3509639: ""000000009367276";"4";"DANIEL DO
CARMO BARROS";"31-Jan-1986";"M";"1";"10";"3162906";"GILSON TEIXEIRA..."
Any clues?
My postgresql version is 8.2.4 the server is running suse linux with 1.5GB
Sensitive changes in postgresql.conf are:
shared_buffers =3D 512MB
temp_buffers =3D 256MB
checkpoint_segments =3D 60
I=B4d also like to know if there=B4s any way to optimize huge data load in
operations like these.
Regards
Adonias Malosso
------=_Part_20896_8119978.1208898335253
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline
Hi all,<br><br>I=B4m running a copy for a 37G CSV and receiving the
followi=
ng error:<br><br>"invalid string enlargement request size
65536"<=
br><br>The file has about 70 million lines with 101 columns, all them
varch=
ar.<br>
<br>When I run the command with the whole file i receive the error after
lo=
ading<br>about 29million lines. So i=B4ve spllited the file in 10 million
l=
ines with split:<br><br>split --lines=3D10000000<br><br>And running the
cop=
y i receive the error on the 5th file:<br>
<br>psql:/srv/www/htdocs/im****t/script_q2.sql:122: ERROR: invalid
str=
ing enlargement request size 65536<br>CONTEXT: COPY temp_q2, line
350=
9639: ""000000009367276";"4";"DANIEL DO
CARMO=
BARROS";"31-Jan-1986";"M";"1";"10&=
quot;;"3162906";"GILSON TEIXEIRA..."<br>
<br>Any clues?<br><br>My postgresql version is 8.2.4 the server is running
=
suse linux with 1.5GB <br>Sensitive changes in postgresql.conf
are:<br><br>=
shared_buffers =3D 512MB<br>temp_buffers =3D 256MB<br>checkpoint_segments
=
=3D 60<br>
<br>I=B4d also like to know if there=B4s any way to optimize huge data
load=
in operations like these.<br><br>Regards<br><br>Adonias Malosso<br>
------=_Part_20896_8119978.1208898335253--
