realbasic-nug
[Top] [All Lists]

Re: Different behavior of Str(Double)? / Re: Floating point overflow

To: REALbasic NUG <realbasic-nug@lists.realsoftware.com>
Subject: Re: Different behavior of Str(Double)? / Re: Floating point overflow
From: Joe Strout <joe@inspiringapps.com>
Date: Mon, 29 Sep 2008 08:27:53 -0600
Authentication-results: mx.google.com; spf=neutral (google.com: 74.124.194.228 is neither permitted nor denied by best guess record for domain of realbasic-nug-bounces@lists.realsoftware.com) smtp.mail=realbasic-nug-bounces@lists.realsoftware.com
Delivered-to: listarchive@realsoftware.com
In-reply-to: <D8B903FF324D454BA0300ABE6555093D1757A4@BASEL.rothsoft.local>
References: <mailman.3643.1222365006.22874.realbasic-nug@lists.realsoftware.com> <D8B903FF324D454BA0300ABE6555093D1757A4@BASEL.rothsoft.local>
Reply-to: REALbasic NUG <realbasic-nug@lists.realsoftware.com>
Sender: realbasic-nug-bounces@lists.realsoftware.com
On Sep 29, 2008, at 2:02 AM, LehrerOffice - Jürg Räss wrote:

RB 2008r3.1 (and prior) return: 3.141593
RB 2008r4 returns:      3.14159265358979

The Language Reference says:
"Use the Format function when you want to convert numeric values into formatted strings such as dates, times, currency, etc. For real numbers, Str returns 7 significant digits."

Good hunting! You've clearly found either a bug in Str, or a failure to document an intentional change. (Hopefully it's the former.)

- are both results 'correct'?
 no: because the LR says so and it used to be like that? (-> bug)

That'd be my opinion.

yes: because with 15 significant digits, the double gets represented more accurate in a string? (-> feature)

That's what you want, you could use the Variant->String conversion, or use Format with a spec like "-0.#". But as you point out, if you want a more "natural" representation with a reasonable number of significant digits, it's harder to get that yourself.

- what influence might this different behavior have in various apps (think of yours, too)...?

Probably not a lot, since Str with floats should mainly be used for debugging (you'd normally use CStr for anything shown to the user). Still, it makes debugging less pleasant.

- which behavior will the next release have?

Who can guess?

- what should i do regarding our 'reference files' (or more general: doubles represented in a string)?

That's the tricky bit, isn't it?

Best,
- Joe


--
Joe Strout
Inspiring Applications, Inc.
http://www.InspiringApps.com





_______________________________________________
Unsubscribe or switch delivery mode:
<http://www.realsoftware.com/support/listmanager/>

Search the archives:
<http://support.realsoftware.com/listarchives/lists.html>


<Prev in Thread] Current Thread [Next in Thread>