PostgreSQL Bugs - BUG #2371: database crashes with semctl failed error

Author

BUG #2371: database crashes with semctl failed error

Brock Peabody

2006-04-04, 8:29 pm

The following bug has been logged online:

Bug reference: 2371
Logged by: Brock Peabody
Email address: brock. peabody@npcinternati
onal.com
PostgreSQL version: 8.1
Operating system: Windows Server 2003
Description: database crashes with semctl failed error
Details:

The full text of the error message is:

FATAL: semctl(167894456, 4, SETVAL, 0) failed: A non-blocking socket
operation could not be completed immediately.

I have a program that inserts/updates/deletes a large number of records to a
large database (~100 GB). The program works, but after it runs for a few
hours it starts getting this error message. Once this happens the database
is unusable to all clients until it is restarted.

I found this documented bug:

http://archives.postgresql.org/pgsq...02/msg00185.php

but I'm not sure if it is related to my problem or not.

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

Qingqing Zhou

2006-04-05, 9:34 am

""Brock Peabody"" <brock. peabody@npcinternati
onal.com> wrote

>
> FATAL: semctl(167894456, 4, SETVAL, 0) failed: A non-blocking socket
> operation could not be completed immediately.
>

Can you reliablly reproduce the problem? If so, we may come up with a
testing patch to it. We encounter similar problems before but it is hard to
reproduce.

Magnus? As Bruce suggested, we can plug in a check-EINTR-loop here in
semctl():

/* Quickly lock/unlock the semaphore (if we can) */
if (semop(semId, &sops, 1) < 0)
return -1;

Regards,
Qingqing

Brock Peabody

2006-04-05, 9:34 am

> -----Original Message-----
> From: pgsql-bugs-owner@postgresql.org [mailto:pgsql-bugs-
> owner@postgresql.org] On Behalf Of Qingqing Zhou
> Sent: Wednesday, April 05, 2006 6:33 AM
> To: pgsql-bugs@postgresql.org
> Subject: Re: [BUGS] BUG #2371: database crashes with semctl failed
error

>
>
> ""Brock Peabody"" <brock. peabody@npcinternati
onal.com> wrote
socket[color=darkred
]
>
> Can you reliablly reproduce the problem? If so, we may come up with a
> testing patch to it. We encounter similar problems before but it is
hard
> to
> reproduce.
>
> Magnus? As Bruce suggested, we can plug in a check-EINTR-loop here in
> semctl():
>
> /* Quickly lock/unlock the semaphore (if we can) */
> if (semop(semId, &sops, 1) < 0)
> return -1;
>
> Regards,
> Qingqing
>
>
>
> ---------------------------(end of
broadcast)---------------------------
> TIP 2: Don't 'kill -9' the postmaster

---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

Brock Peabody

2006-04-05, 9:34 am

> -----Original Message-----
> From: pgsql-bugs-owner@postgresql.org [mailto:pgsql-bugs-
> owner@postgresql.org] On Behalf Of Qingqing Zhou
> Sent: Wednesday, April 05, 2006 6:33 AM
> To: pgsql-bugs@postgresql.org
> Subject: Re: [BUGS] BUG #2371: database crashes with semctl failed
> error

> Can you reliablly reproduce the problem?

I can here :). I'm trying to figure out a way for someone to repeat it
outside my environment but I'm afraid it's got something to do with
timing. I have 50 threads that are collecting data. If I give each one
its own connection to the database the problem happens quickly. If they
all share one connection the problem does not happen.

> If so, we may come up with a
> testing patch to it. We encounter similar problems before but it is
> hard to reproduce.

Do you think this is a Windows only problem?

Thanks,
Brock

---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql
.org so that your
message can get through to the mailing list cleanly

Tom Lane

2006-04-05, 11:35 am

"Brock Peabody" <brock. peabody@npcinternati
onal.com> writes:
[color=darkred]
> I can here :). I'm trying to figure out a way for someone to repeat it
> outside my environment but I'm afraid it's got something to do with
> timing. I have 50 threads that are collecting data. If I give each one
> its own connection to the database the problem happens quickly. If they
> all share one connection the problem does not happen.

Perhaps you could whittle down your app into a testbed that just sends
dummy data with about the same timing as the real app?

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

Brock Peabody

2006-04-05, 11:35 am

> On Behalf Of Tom Lane

> Perhaps you could whittle down your app into a testbed that just sends
> dummy data with about the same timing as the real app?

I think I'm starting to get a better understanding of problem. It looks
like one of the threads is trying to insert a pathological (~1,800,000)
number of records in one transaction in a table while the other threads
are also reading from and writing to that table. I'll try to get
something simple to reproduce this behavior.

---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

Qingqing Zhou

2006-04-06, 3:28 am

""Brock Peabody"" <brock. peabody@npcinternati
onal.com> wrote

>
> Do you think this is a Windows only problem?
>

I am afraid so. We have received 3 reports of this (or quite similar)
problem, all in 8.1/windows. I just noticed that yours is actually an EAGAIN
error, so the loop patch in semctl() doesn't work I guess :-(. If you can
find an easier way to reproduce it outside your environment, that's very
sweet.

Regards,
Qingqing

Sponsored Links