Sign in to follow this  
Followers 0
4b0082

Trouble Comparing Two Variables

6 posts in this topic

#1 ·  Posted (edited)

For some reason I can't compare $rTotal and $Total properly. Both have a return value of "0.57" and are set as double variables, but when I compare the two I still get my error message. I've tried converting the variables with Number() but that didn't seem to change anything.
 

Local $Total = .57

Local $SubTotal = Round($Total/1.15, 2)
Local $rFee = FloorEx($SubTotal * 0.1, 2) + FloorEx($SubTotal * 0.05, 2)
Local $rTotal = $SubTotal + $rFee

If $rTotal <> $Total Then
    MsgBox(0, "Broken", VarGetType($Total))
EndIf

Func FloorEx($fNumber, $iDec = 3)
    Return StringFormat("%." & $iDec & "f", Floor($fNumber * 10 ^ $iDec) / 10 ^ $iDec)
EndFunc
Edited by 4b0082

Share this post


Link to post
Share on other sites



Function FloorEx doesn't make much sense to me. You return a formatted string and then use it in a numeric operation forcing autoit to convert the string into a number.

Shouldn't you first do all the calculation and then format the result?


My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2017-04-18 - Version 1.4.8.0) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX (NEW 2017-02-27 - Version 1.3.1.0) - Download - General Help & Support - Example Scripts - Wiki
ExcelChart (2015-04-01 - Version 0.4.0.0) - Download - General Help & Support - Example Scripts
Excel - Example Scripts - Wiki
Word - Wiki
PowerPoint (2015-06-06 - Version 0.0.5.0) - Download - General Help & Support

Tutorials:
ADO - Wiki

 

Share this post


Link to post
Share on other sites

I needed a way to round down and that's a snippet of code I found through the forums that worked. I'm not sure of another way to do it.

Share this post


Link to post
Share on other sites

4b0082,

If you convert the 2 values to Hex then you will see that they are slightly different:

$rTotal: 3FE23D70A3D70A3E
$Total : 3FE23D70A3D70A3D
This is a result of the way in which computers store fractional number (Google "floating point arithmetic" to learn more) and not an AutoIt problem. The normal way to get over this is to use integer values - multiply the value by a large enough factor to get it as an integer for calculations and then divide by that factor when you need the actual value. ;)

M23


Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

#5 ·  Posted (edited)

I tried multiplying the results by 100 then comparing them but I'm still getting double as my variable and it's still processing the error.

Edit: Nevermind! I forgot to wrap Int() around the new variable. It works now, thanks!

Edited by 4b0082

Share this post


Link to post
Share on other sites

4b0082,

It works for me: ;)

$iFactor = 100

Local $Total = .57

Local $SubTotal = Round($Total/1.15, 2)
Local $rFee = Number(FloorEx($SubTotal * 0.1, 2)) + Number(FloorEx($SubTotal * 0.05, 2))
Local $rTotal = $SubTotal + $rFee

ConsoleWrite($rTotal & " - " & $Total & @CRLF)
ConsoleWrite(Hex($rTotal) & @CRLF & Hex($Total) & @CRLF)

If Int ($rTotal * $iFactor) <> Int ($Total * $iFactor) Then ; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
    MsgBox(0, "Broken", Hex($rTotal) & @CRLF & Hex($Total))
Else
    MsgBox(0, "Works", ";)")
EndIf

Func FloorEx($fNumber, $iDec = 3)
    Return StringFormat("%." & $iDec & "f", Floor($fNumber * 10 ^ $iDec) / 10 ^ $iDec)
EndFunc
M23
1 person likes this

Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0