Slony1 PostgreSQL Replication - [PATCH] Fix UTF-8 string truncation bug

Author

[PATCH] Fix UTF-8 string truncation bug

Ian Burrell

2005-06-30, 11:24 am

I didn't see any response to the bug I found with slon truncating
strings with UTF-8 characters in them. The truncated strings were
being written to the sl_log_1 table. This was causing the SYNC to
fail on the slave because of constraints violations. It was also
corrupting data. The slon_quote_literal shortens the copied string by
one byte for every multi-byte character.

- Ian

--- src/backend/slony1_funcs.c 9 Jun 2005 15:02:15 -0000 1.33
+++ src/backend/slony1_funcs.c 28 Jun 2005 18:46:17 -0000
@@ -1104,7 +1104,7 @@
cp2 =3D result;

*cp2++ =3D '\'';
- while (len-- > 0)
+ while (len > 0)
{
if ((wl =3D pg_mblen((unsigned char *)cp1)) !=3D 1)
{
@@ -1120,6 +1120,8 @@
if (*cp1 =3D=3D '\\')
*cp2++ =3D '\\';
*cp2++ =3D *cp1++;
+
+ len--;
}

*cp2++ =3D '\'';

Christopher Browne

2005-06-30, 1:29 pm

Ian Burrell wrote:

>I didn't see any response to the bug I found with slon truncating
>strings with UTF-8 characters in them. The truncated strings were
>being written to the sl_log_1 table. This was causing the SYNC to
>fail on the slave because of constraints violations. It was also
>corrupting data. The slon_quote_literal shortens the copied string by
>one byte for every multi-byte character.
>
>
>
I'm trying this out; it looks as though it's not causing any regressions
for existing tests, which is a good thing.

I don't get much call to set up queries involving UTF-8; would it be
possible for you to give a brief sample of what is breaking so that we
can set up a test for it?

It would suffice to have a little script that sets up a database (do you
need to specify locale at DB create time? not sure...), creates a table,
and then inserts some (achy-breaky) data into it.

The change LOOKS good; I just want to make sure that we have a proper
test case to verify that it's working properly.

I just don't want make up a useless example that doesn't test it...

Sponsored Links