Slony1 PostgreSQL Replication - question about sl_log

Author

question about sl_log_1 data lifetime

Hannu Krosing

2005-08-30, 11:28 am

Hi

When I have three nodes with a set from A being replicated so

A --> B --> C

When are log records deleted from sl_log_1 on node A ?

Is it when they are copied to B or when they are copied to C ?

I have had times when node C is slow, and it seems the the records are
not deleted from sl_log_1 on A in this case. I this true ?

--
Hannu Krosing <hannu-7C/ iILuz2RdeoWH0uzbU5w@
public.gmane.org>

Darcy Buskermolen

2005-08-30, 11:28 am

On Tuesday 30 August 2005 09:14, Hannu Krosing wrote:
> Hi
>
> When I have three nodes with a set from A being replicated so
>
> A --> B --> C
>
> When are log records deleted from sl_log_1 on node A ?
>
> Is it when they are copied to B or when they are copied to C ?
>
> I have had times when node C is slow, and it seems the the records are
> not deleted from sl_log_1 on A in this case. I this true ?

yes this is correct. this is done so that when B fails you can tell C it's
new origin of A and you don;t loose any data in the process.

--
Darcy Buskermolen
Wavefire Technologies Corp.

http://www.wavefire.com
ph: 250.717.0200
fx: 250.763.1759

Christopher Browne

2005-08-30, 1:26 pm

Hannu Krosing wrote:

>Hi
>
>When I have three nodes with a set from A being replicated so
>
>A --> B --> C
>
>When are log records deleted from sl_log_1 on node A ?
>
>Is it when they are copied to B or when they are copied to C ?
>
>I have had times when node C is slow, and it seems the the records are
>not deleted from sl_log_1 on A in this case. I this true ?
>
>
The log records are not purged until they have been received by *ALL*
subscribers.

Otherwise, you would run into the problem that a subscriber could be
left orphaned/stranded in the case where one of the nodes in the chain
fell over.

Suppose the log records were purged on A because they had made it to B,
but they had NOT made it to C...

If B falls over, losing all its data, then there is no way for the data
to get to C, and thus the loss of one subscriber actually leads to the
loss of TWO subscribers.

Slony-I don't do that...

Hannu Krosing

2005-08-31, 7:25 am

On T, 2005-08-30 at 09:25 -0700, Darcy Buskermolen wrote:
> On Tuesday 30 August 2005 09:14, Hannu Krosing wrote:
>
> yes this is correct. this is done so that when B fails you can tell C it's
> new origin of A and you don;t loose any data in the process.

Makes sense. Thanks!

The problem is, that I installed B with sole purpose of offloading
expensive queries from A. Unfortunately slon generates queries which do
a seqscan over sl_log_1 if more than one set originate from A (actually
it is even worse - the queries do and *indexscan* using the index'es
first column, which is constant in my case), and it starts to take tens
of seconds once the sl_log_1 table has more than a few hundred thousand
rows, keeping the cpu usage at constant 100%.

Ok, I just have to fix the way slon generates the queries for several
sets. I took a brief look and it seems possible to do by rearranging
code and queries around line 3800 in remote_worker.c. ( The syncs are
done per node not per set, but they are used as if they were done per
set ).

And in the meantime use one cluster per set :P so that the query can use
the index.

--
Hannu Krosing <hannu-7C/ iILuz2RdeoWH0uzbU5w@
public.gmane.org>

elein

2005-08-31, 8:25 pm

On Tue, Aug 30, 2005 at 09:25:07AM -0700, Darcy Buskermolen wrote:
> On Tuesday 30 August 2005 09:14, Hannu Krosing wrote:
>
> yes this is correct. this is done so that when B fails you can tell C it's
> new origin of A and you don;t loose any data in the process.

Shouldn't failover do that resub automatically? If not it should.
I think this is the problem with my failover bug. C still references
A in sl_setsync and sl_nodes, etc.

--elein

>
> --
> Darcy Buskermolen
> Wavefire Technologies Corp.
>
> http://www.wavefire.com
> ph: 250.717.0200
> fx: 250.763.1759
> ____________________
____________________
_______
> Slony1-general mailing list
> Slony1-general- AuKwsB3Fm+ugFIWk8tvy
RWD2FQJk+8+b@public.gmane.org
> http://gborg.postgresql.org/mailman.../slony1-general
>

Sponsored Links