Length Units

[bdining]

Length Units

When I type (units 1.0 :inch)
Then type (DISPLAY (sd-inq-length-units))
Feedback is (INCH 1.0000000000000002)

This is causing a problem for me in some of my programs.
Granted the error is at the 16th decimal place but when I measure an item that should be .50 for example but it displays .4999999999999998. This causes concern for my users.

Could someone please explain and advise how I could fix this problem.

Current OSD versions:
OSD_Modeling_11.60.2.0
OSD_Modeling_11.60.0.13

Bill

[Markus]

Hello Bill,

are you using DISPLAY to output the measured value of 0.50 in your own custom code?

The number you see when you use DISPLAY is the exact representation of that number using a 64-bit LISP data structure. Therefore it is normal that you'll get some sort of flickering in a 10E-16 digit range which causes this ugly display.
Therefore we usually round numbers to e.g. 12 digits when we output those numbers. Internally these numbers are still exact, i.e. if you apply a rounded measured value, the exact number is taken and not the displayed rounded number.

To not confuse your users, I'd always use sd-num-to-string with e.g. 12 digits before outputting any number.


Hope this helps.

Best regards,
Markus

[bdining]

Hello Markus,

My programs are not using a display call.
The users will use a measure length and this dirty number is shown in the calculator.
If I shift the geo. over slightly and run the same program the number is displayed correctly when using edge length.

What would be the best way for me to get this to calculate correctly?

I have looked for a rounding function in my LISP books and all I could find is a truncate command, which will not clean up my calculations.

Is there a rounding function for LISP?

Thanks
Bill

[Andy Poulsen]

Hi Bill,

I'm not sure if this will work for you, but as Markus mentioned there is a function provided by CoCreate in the integration kit called sd-num-to-string that may do what you are looking for:

Description (from the docs): Converts number to a string with a maximum number of digits after the decimal point. If the input number contains more digits after the decimal point as specified in digits, the resulting string will be rounded.

Note that this is only useful for the final use (i.e. display) of the number, and that the result is a string, so it may not work correctly even if you convert the result back to a number.

For example, if I type the following into the OSD command line:

display pi -> 3.1415926535897931
display (oli::sd-num-to-string pi 6) -> 3.141593

so far, so good! but if we now try to use the result:

display (1+ (read-from-string (oli::sd-num-to-string pi 6))) -> 4.1415930000000003

so as you can see, this isn't the perfect solution, either... though if there's a way to use strings in what you present to the user, you may be able to make this work using a combination of sd-num-to-string and read-from-string.

Unfortunately, if you're using the calculator, I don't think you can change that behavior. (unless there is a way to specify the display format -- that would be nice!)

Please let me know if this helps. Good luck!

andy

[Thom Ivancso]

Hello Bill,

Try using this little piece of code to round your numbers to.

(defun rounded (number precision)
(* precision (round number precision))
)


You can also use this piece of code to control the output of the calculator

(setq ui::*m-calc-output-precision* 5)


Cheers
Thom

[bdining]

(DISPLAY (SETF N2 (ROUNDED 3.2499999999999998 .00001)))
RESULT 3.2500000000000004

(DISPLAY (SETF N1 (ROUNDED 3.2499999999999998 .0001)))
RESULT 3.25

(DISPLAY (+ N1 N2))
RESULT 6.5

(DISPLAY (+ N2 N1))
RESULT 6.5

(DISPLAY (+ N2 N2))
RESULT 6.5000000000000009

Further testing with different precision settings result in a
3.25.
The only ones that cause an incorrect feedback are at the 5th & 15 decimal place.

Why does a precision at the 5th or 15th cause dirty number?

Thanks
Bill

[bdining]

Thom,

Additional testing results in dirty numbers.
(DISPLAY (ROUNDED 3.2612345678 .0000001))
result 3.2612345999999999

If the number is not clean to begin with I am unable to round it to a set decimal place before applying a math function to it.

The resulting math is a dirty number.

Thanks
Bill

[Thom Ivancso]

Hello Bill,

How's every little thing in Lisle? I wish I could shed more light on this for you, I think this needs to go to the lab at CoCreate to get an answer. Rounding as you have seen is not going to fix this issue with dirty numbers. This seems to be something imbedded into the core code itself.


Cheers
Thom

[Andy Poulsen]

Hi Bill,

Where does the problem show up? Are you putting the numbers into a dialog? If so, will the suggestions I mentioned above, combined with Thom's calculator output suggestion make any difference? There may still be a way to work around this...

(and thanks, Thom, for the calculator info -- I think I knew that once, but had totally forgotten about it!)

Good luck!

andy