realbasic-nug
[Top] [All Lists]

Different behavior of Str(Double)? / Re: Floating point overflow

To: <realbasic-nug@lists.realsoftware.com>
Subject: Different behavior of Str(Double)? / Re: Floating point overflow
From: LehrerOffice - Jürg Räss <juerg.raess@lehreroffice.ch>
Date: Mon, 29 Sep 2008 10:02:19 +0200
Authentication-results: mx.google.com; spf=neutral (google.com: 74.124.194.228 is neither permitted nor denied by best guess record for domain of realbasic-nug-bounces@lists.realsoftware.com) smtp.mail=realbasic-nug-bounces@lists.realsoftware.com
Delivered-to: listarchive@realsoftware.com
In-reply-to: <mailman.3643.1222365006.22874.realbasic-nug@lists.realsoftware.com>
References: <mailman.3643.1222365006.22874.realbasic-nug@lists.realsoftware.com>
Reply-to: REALbasic NUG <realbasic-nug@lists.realsoftware.com>
Sender: realbasic-nug-bounces@lists.realsoftware.com
Thread-index: AckfmoOxtlNv75AERuCLzY+EgGzr0wCaylcA
Thread-topic: Different behavior of Str(Double)? / Re: Floating point overflow
>msgbox format(v,"#")
>
>the conversion to a string seems to be at issue, but not the actual overflow 
>itself

What we found might be related... but to be honest, i just had a quick look 
into the NUG the last few weeks as we were busy with quite some other things :-)

to test our software, we run some automated tests. in order to test if certain 
actions result into the same data (when making a change to the code or using a 
new RB-release), we write the data-structure to a textfile, which we can 
compare to a 'reference file' that is 'known to be correct'.

here we noticed a difference:
  Const Pi=3.14159265358979323846264338327950
  MsgBox Str(pi)

RB 2008r3.1 (and prior) return: 3.141593
RB 2008r4 returns:      3.14159265358979
  
The Language Reference says:
"Use the Format function when you want to convert numeric values into formatted 
strings such as dates, times, currency, etc. For real numbers, Str returns 7 
significant digits."

So far i just can say:
- RB 2008r3.1 and RB2008r4 have a different behavior of Str(Double)
- According to the LanguageReference, the behavior of 2008r3.1 is what we 
expect to get returned
  (because it used to be like that, and because the LR says so)

I'm not quite sure what to think of this - and what it might involve...
Some questions are:
- are both results 'correct'?
  no: because the LR says so and it used to be like that? (-> bug)
  yes: because with 15 significant digits, the double gets represented more 
accurate in a string? (-> feature)
- what influence might this different behavior have in various apps (think of 
yours, too)...?
- which behavior will the next release have?
  - stay like in 2008r4?
  - return 7 significant digits again?
- what should i do regarding our 'reference files' (or more general: doubles 
represented in a string)?
  - update them to more digits?
  - convert the new output back to 7 digits? this would take quite some time to 
find the 'right places'...


Juerg

______________________________________

Roth Soft
Kellhofstrasse
CH-8476 Stammheim

E-Mail:   juerg.raess@lehreroffice.ch
Internet:  http://www.lehreroffice.ch
______________________________________

_______________________________________________
Unsubscribe or switch delivery mode:
<http://www.realsoftware.com/support/listmanager/>

Search the archives:
<http://support.realsoftware.com/listarchives/lists.html>


<Prev in Thread] Current Thread [Next in Thread>