Slony1 PostgreSQL Replication - Slon daemon memory usage

Author

Slon daemon memory usage

Ian Burrell

2005-07-19, 1:24 pm

We have a three node slony cluster. Two of the slon daemons for the
slave nodes have large memory usage: virtual size of 2991 MB and 2214
MB. Luckily, the machine has 16 GB of RAM. These are 32-bit processes
and only have 3 GB of virtual memory per process so they are getting
close to running out of their own memory.

Here is the ps listing:

USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
imb 23278 0.0 0.0 2168 756 ? S Jul16 0:00 slon -s
1000 -d2 -g 80 vodslony host=3Dvoddb1 dbname=3Dvodlive user=3Dslony
port=3D5432
imb 23280 0.0 0.0 85812 1548 ? S Jul16 1:46 slon -s
1000 -d2 -g 80 vodslony host=3Dvoddb1 dbname=3Dvodlive user=3Dslony
port=3D5432
imb 23362 0.0 0.0 2196 812 ? S Jul16 0:00 slon -s
1000 -d2 -g 80 vodslony host=3Dvoddb4 dbname=3Dvodlive_sla
ve2 user=3Dslony
port=3D5432
imb 23370 0.4 13.6 3063396 2191596 ? S Jul16 16:41 slon -s
1000 -d2 -g 80 vodslony host=3Dvoddb4 dbname=3Dvodlive_sla
ve2 user=3Dslony
port=3D5432
imb 17706 0.0 0.0 2188 828 ? S Jul18 0:00 slon -s
1000 -d2 -g 80 vodslony host=3Dvoddb3 dbname=3Dvodlive_sla
ve1 user=3Dslony
port=3D5432
imb 17714 0.1 9.7 2268024 1565852 ? S Jul18 2:06 slon -s
1000 -d2 -g 80 vodslony host=3Dvoddb3 dbname=3Dvodlive_sla
ve1 user=3Dslony
port=3D5432

Is there a known cause of the memory usage? Is there a memory leak?=20
Is there anything we can do debug this?

- Ian

Jan Wieck

2005-07-20, 8:25 pm

On 7/19/2005 2:55 PM, Ian Burrell wrote:

> We have a three node slony cluster. Two of the slon daemons for the
> slave nodes have large memory usage: virtual size of 2991 MB and 2214
> MB. Luckily, the machine has 16 GB of RAM. These are 32-bit processes
> and only have 3 GB of virtual memory per process so they are getting
> close to running out of their own memory.

This is very unusual. I have not seen slon processes over 30MB size thus
far.

What are the exact version numbers of slon and libpq? Is there anything
special about the database (huge tuple sizes for example)?

Jan

>
> Here is the ps listing:
>
> USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
> imb 23278 0.0 0.0 2168 756 ? S Jul16 0:00 slon -s
> 1000 -d2 -g 80 vodslony host=voddb1 dbname=vodlive user=slony
> port=5432
> imb 23280 0.0 0.0 85812 1548 ? S Jul16 1:46 slon -s
> 1000 -d2 -g 80 vodslony host=voddb1 dbname=vodlive user=slony
> port=5432
> imb 23362 0.0 0.0 2196 812 ? S Jul16 0:00 slon -s
> 1000 -d2 -g 80 vodslony host=voddb4 dbname=vodlive_slave
2 user=slony
> port=5432
> imb 23370 0.4 13.6 3063396 2191596 ? S Jul16 16:41 slon -s
> 1000 -d2 -g 80 vodslony host=voddb4 dbname=vodlive_slave
2 user=slony
> port=5432
> imb 17706 0.0 0.0 2188 828 ? S Jul18 0:00 slon -s
> 1000 -d2 -g 80 vodslony host=voddb3 dbname=vodlive_slave
1 user=slony
> port=5432
> imb 17714 0.1 9.7 2268024 1565852 ? S Jul18 2:06 slon -s
> 1000 -d2 -g 80 vodslony host=voddb3 dbname=vodlive_slave
1 user=slony
> port=5432
>
> Is there a known cause of the memory usage? Is there a memory leak?
> Is there anything we can do debug this?
>
> - Ian
> ____________________
____________________
_______
> Slony1-general mailing list
> Slony1-general- AuKwsB3Fm+ugFIWk8tvy
RWD2FQJk+8+b@public.gmane.org
> http://gborg.postgresql.org/mailman.../slony1-general

--
#===================
====================
====================
===========#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#===================
====================
=========== JanWieck- bwPqjjyvM7QAvxtiuMwx
3w@public.gmane.org #

Christopher Browne

2005-07-20, 8:25 pm

Jan Wieck <JanWieck- bwPqjjyvM7QAvxtiuMwx
3w@public.gmane.org> writes:

> On 7/19/2005 2:55 PM, Ian Burrell wrote:
>
>
> This is very unusual. I have not seen slon processes over 30MB size
> thus far.
>
> What are the exact version numbers of slon and libpq? Is there
> anything special about the database (huge tuple sizes for example)?

Rod Taylor indicated that he ran into this sort of problem with his
"Slony-I with PostgreSQL 7.2" frankenstein system; this was the case
when catching up after replicating a Truly Enormous Set.

It would be quite interesting to see what it is that is occupying all
that memory. Big processes surprise me too...

Are those subscribers way behind, such that the groups of SYNCs being
processed are rather large?
--
"cbbrowne","@","ca.afilias.info"
<http://dev6.int.libertyrms.com/>
Christopher Browne
(416) 673-4124 (land)

Rod Taylor

2005-07-21, 9:24 am

> Rod Taylor indicated that he ran into this sort of problem with his
> "Slony-I with PostgreSQL 7.2" frankenstein system; this was the case
> when catching up after replicating a Truly Enormous Set.
>
> It would be quite interesting to see what it is that is occupying all
> that memory. Big processes surprise me too...

I figured it out when doing the 7.4 to 8.0 upgrade with standard Slony
as it happened there too. I thought I sent an email on the subject, but
perhaps not.

If a table with 1 million entries has an average tuple size of 6kb but
some tuples (say 1000) are as large as 30MB and you have 100 slots in
Slony (fetch 100), over many iterations a 30MB entry is bound to be in
each of those 100 slots and each slot ends up being 30MB in size.

30MB * 100 = 3GB

I have yet to put in any effort to determine whether this is a libpq
problem or a Slony problem.
--

Sponsored Links