On Sep 30, 2008, at 12:10 PM, Joe Strout wrote:
2) format the number with the precision you want
tmp = format(number,",#.0000000") // exactly 7 digits after
the decimal
That gives you 7 decimal places, not 7 significant digits. Getting
a result with a desired number of significant digits is harder.
(Off the top of my head, I suppose I might convert it to a string
in exponential notation, then shift the decimal point manually...)
Forgive me for being pedantic here but this is something many (not
you Joe) seem to miss about significant figures
Significant figures really are about how to express (and read) the
accurracy of a number from a physical measurement.
Lets say you take some measurement and the result ( A in arbitary
units) is: 1.223345e-3 = 0.001223345
Lets say you measure something else and get result B : 1.223345e-2 =
0.01223345
If you are reporting values to 3 significant figures some would
always report these values as:
1.22e-3 and 1.22e-2
But it his right? It depends on the capability of the instrument. If
it can only measure accurately to 0.0001 units then the correct
results (numerically) would be expressed as:
A = 0.0012 = 1.2e-3
B= 0.0122 = 1.22e-2
And there is no built-in way to to do that for printing OR rounding...
Also In the real world most people want all results to the same power
and specifications are often written as something like:
1.0e-3 to 15.0e-1
and people want to see the results always expressed to the same power
with the correct number significant digits for the measurement.
So in that case A = 1.2e-3 And B = 12.2e-3 would be the desired
output
To deal with this I wrote rounding and printing functions that take a
modified format spec.
For this case my format spec would be "-0.0e-3"
I parse out the power of 10 and go from there for rounding or
printing functions. For when i need to deal with a lot of numbers
with same fromat I created a FormatClass to do rounding and printing
so I don't need to keep parsing the same format string every time.
It would be REALLY nice for RB to do be able handle formats and
rounding like this (my old TI-59 calculator could do it for powers
that were mutiples of 3) ... but I doubt a feature request would
get anywhere because so few worry about that level of detail in
handling scientific data.
Forgive me for boring everyone, but sig fig issues have been the
bane of my career! ( A slight exaggeration ;) )
-Karen
_______________________________________________
Unsubscribe or switch delivery mode:
<http://www.realsoftware.com/support/listmanager/>
Search the archives:
<http://support.realsoftware.com/listarchives/lists.html>
|