On Wed, May 14, 2008 at 02:08:47PM -0400, Justin wrote:
> Sam Mason wrote:
> >What does foxpro use for storing numbers? or is it just that you never
> >pushed it hard enough for the abstractions to show through.
>
> I know i pushed it. Foxpro for the most has only 4 basic data types
> Numeric (similar to Posgresql numeric), Boolean, Date, Text aka
> (string) The foxpro tables sup****ted far more data types but when every
> it was dumped to variable it acted like one of the 4.
I really meant how much did you check the results, or did you accept
that they were correct?
> Foxpro did not suffer floating point math errors. I normally used 8 to
> 10 points precision. Foxpro was limited to 15 points of precision
> period. No more and no less, once you hit that was it.
15 places seems very similar to what a 64bit IEEE floating point number
will give you, i.e. a double in C/C++.
> My problem is we calculate resistance of parts in a Foxpro app that we
> want to move because we want to bring all the custom apps into one
> framework and single database.
>
> Take this calculation (0.05/30000* 1.0025) which is used to calculate
> parts resistance and Tolerance. (its Ohms Law) The value returned from
> C++ = .0000016708 which is wrong
> it should be .00000167418. We just shrank the tolerance on the part we
> make
Why are you so sure about the FoxPro result? I've just checked a few
calculators and get results consistent with your C++ version.
Justin C: 0.0000016708
J FoxPro: 0.00000167418
My C: 0.000001670833
bc[1]: 0.0000016708333333333333333333333333333332
PG[2]: 0.0000016708333333333333336675
Google[3]: 0.00000167083333 (actually gives 1.67083333e-6)
Both bc and Postgres use their own code (i.e. not the CPU's FPU) to do
the math, and as they all agree I'm thinking FoxPro is incorrect! Next
I tried doing it accurately (in Haskell if it makes any difference) and
get an answer of 401/240000000 out, which would agree with everything
but FoxPro. If I calculate the ratio back out for FoxPro I get
401/239520242 which is a little way out.
> The Do***entation from MS says 15 points of precision but the result say
> otherwise.
The docs for what? FoxPro or their C compiler?
If you mean FoxPro, I think this is another case of MS screwing up.
> I'm glad You and others are taking the time to explain to me
> the odd results before i get into redoing that application.
Welcome to the PG community, lots of people to get interested in lots of
things!
> Why oh Why did MS kill Foxpro. :'( I understood it, knew its quirks
> and it worked very well with Postgresql
Are you sure you want to stay with it if its answers are wrong?
Sam
[1] http://www.gnu.org/software/bc/manual/html_mono/bc.html
[2]
http://doxygen.postgresql.org/backend_2utils_2adt_2numeric_8c-source.html
[3] http://www.google.com/search?q=0.05/30000*1.0025
--
Sent via pgsql-general mailing list (pgsql-general@[EMAIL PROTECTED]
)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general
|
