|
Home > Archive > Slony1 PostgreSQL Replication > February 2006 > MERGE_SET causing duplicate replications
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
MERGE_SET causing duplicate replications
|
|
| Michael Crozier 2006-02-02, 3:25 am |
|
Regarding the problem I described earlier today, there is an additional
detail. I did not notice that a MERGE_SET event immediately proceeded all
three occurrances of the re-replicated sl_log_1 entries.
Where event N was a MERGE_SET, the processing of event N+1 would re-fetch some
of the previously processed sl_log_1 rows.
I've added and merged 199 tables incrementally over three days and three of
them appear to have caused this problem. All indications are that there was
only overlap, no dropped sl_log entries, but I have no detailed explaination
of why. It is certainly not the xid/btree problem, as there have only been
~200,000,000 transactions, not nearly the required 2G that Hannu indicated.
I have SQL and slon logging for all three occurances and will gladly provide
the important parts if it will help uncover the cause of the problem.
Regards,
Michael
| |
| Michael Crozier 2006-02-03, 8:24 pm |
|
For posterity and future victims, I'll note that this problem did result in
significant data loss. I'll need to try to re-subscribe my node with yet
more manual babysitting on my part to try and avoid the bug.
> Regarding the problem I described earlier today, there is an additional
> detail. I did not notice that a MERGE_SET event immediately proceeded all
> three occurrances of the re-replicated sl_log_1 entries.
>
> Where event N was a MERGE_SET, the processing of event N+1 would re-fetch
> some of the previously processed sl_log_1 rows.
>
> I've added and merged 199 tables incrementally over three days and three of
> them appear to have caused this problem. All indications are that there
> was only overlap, no dropped sl_log entries, but I have no detailed
> explaination of why. It is certainly not the xid/btree problem, as there
> have only been ~200,000,000 transactions, not nearly the required 2G that
> Hannu indicated.
>
> I have SQL and slon logging for all three occurances and will gladly
> provide the important parts if it will help uncover the cause of the
> problem.
>
> Regards,
>
> Michael
> ____________________
____________________
_______
> Slony1-general mailing list
> Slony1-general- AuKwsB3Fm+ugFIWk8tvy
RWD2FQJk+8+b@public.gmane.org
> http://gborg.postgresql.org/mailman.../slony1-general
--
Conducive Technology Corporation
Taking air cargo information
to the next level
http://www.conducivetech.com
http://www.pathfinder-web.com
Michael Crozier crozierm- 19LDBNnCZmbFzinHIz5S
+QC/G2K4zDHf@public.gmane.org
Voice: 503.445.4233
Fax: 503.274.0939
|
|
|
|
|