Home > Archive > PostgreSQL Discussion > May 2005 > Re: Modulus operator returns negative values / numeric division rounds up sometimes









You are viewing an archived Text-only version of the thread. To view this thread in it's original format and/or if you want to reply to this thread please [click here]

 

Author Re: Modulus operator returns negative values / numeric division rounds up sometimes
Tom Lane

2005-05-26, 3:23 am

Paul Tillotson <pntil@shentel.net> writes:
> I don't think anyone wants to defend the negative modulus as such, but to fix it, we have to do one of these:


> (1) Keep rounding division, but rewrite the numeric modulus operator to use a form of division that always rounds towards zero.


> or


> (2) Give up rounding division in favor of truncating towards zero.


or (3) increase the calculation precision (rscale), as suggested by
Alvaro's message.

Possibly that cannot work, but I haven't seen a proof.

> It looks like the "bug" can be easily fixed by changing the end of div_var where it says
> round_var(result, rscale);
> to
> trunc_var(result, scale);


I cannot believe that that won't create problems at least as bad as it
solves. Have you even tried the regression tests on this?

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql
.org so that your
message can get through to the mailing list cleanly

Sponsored Links





Also available: Server administration forum archive | Web Design forum archive | Software forum archive | Hardware reviews archive | Programming forum archive

Copyright 2008 droptable.com