PDA

View Full Version : ME10 - Macro bug


Lim Chee Beng
11-14-2002, 04:02 PM
When usually I compare 2 points in the ME10 macro program statement as.....
IF (P1<>(PNT_XY 2.46 4.13))
.....I found sometimes P1 can be intepreted as long decimal point of "2.46000001,4.13" (can't remember exactly how many zeros) even though it was originally assigned as "2.46,4.13". Consequently the result of comparison is always inconsistent and unpredictable.

The purpose of the comparison is to ensure P1 will be always exactly at location of "2.46,4.13". I also face the same problem when trying to verify 2 variable points are at the same location.

To prevent the problem, sometimes I'm forced to re-program as....
IF ((ROUND (P1*10000)/10000)<>(PNT_XY 2.46 4.13))
.....to truncate the tiny, meaningless and annoying decimal value.

Similar problem happens for floating point variable, I always have to re-program as.....
LET fpv (ROUND (fpv*1000000)/1000000)
.....to truncate the tiny decimal value of fpv variable.

Is there any simple programmatic way or internal ME10 configuration to fix the problem?

:confused:

Michael Kahle
11-15-2002, 12:27 AM
I usally use something like
(ABS (Pkt1 - Pkt2)) < 1E-6

Regards,
Michael

ChrisE
11-15-2002, 05:31 AM
Hi guys,

I have seen such code quite often where people try to compare floating point numbers using "=" (or something similar)

With older versions of ME10, this could have worked.
!!! BUT THIS IS QUITE DANGEROUS !!!

Floating point numbers (or points) should always be compared by subtracting them from each other, and checking the positive result against some epsilon value
Example:

IF ((ABS (X2-X1)) < 1E-6)
DISPLAY "X2 equals (closely) X1"
END_IF

Chris