Salvodelli Posted August 4, 2021 Share Posted August 4, 2021 Just updated to the newest release on macOS Big Sur 11.5.1 and I'm getting rounding errors for certain simple arithmetic expressions in Alfred. I don't know if this is a new issue or not, since I've only observed it with the example below. Reproduction steps: Invoke Alfred. Type "175203.52-174969.93" Result is "233.589999999996", it should be "233.59" Screen shot attached. Link to comment Share on other sites More sharing options...
deanishe Posted August 4, 2021 Share Posted August 4, 2021 1 hour ago, Salvodelli said: it should be "233.59" It's not an error or a bug. You've just found a fraction that can't be represented accurately as a binary float, in much the same way that base 10 can't accurately represent 10/3. Link to comment Share on other sites More sharing options...
Salvodelli Posted August 4, 2021 Author Share Posted August 4, 2021 (edited) I don't understand. There are no fractions or division involved here. I'm just subtracting one number from another. For example, a similar operation "2.52-1.93" yields the expected result, .59. Edited August 4, 2021 by Salvodelli Link to comment Share on other sites More sharing options...
deanishe Posted August 4, 2021 Share Posted August 4, 2021 Just now, Salvodelli said: There are no fractions or division involved here. Decimal fractions are fractions, too. It has nothing to do with division, only that there are some fractions that can be represented accurately in decimal, but not in binary. 3 minutes ago, Salvodelli said: For example, a similar operation "2.52-1.93" yields the expected result, .59. So does 10.52 - 9.93. But 100.52 - 99.93 doesn’t because your processor doesn’t have enough bits for a sufficiently-accurate representation. This is absolutely the expected result. Every piece of software will give you the same answer because it's fundamental to the way computers work. If you need bang-on accurate decimal arithmetic, you will have to find specialised software designed to perform decimal arithmetic, not the standard binary arithmetic. Link to comment Share on other sites More sharing options...
Andrew Posted August 4, 2021 Share Posted August 4, 2021 This is actually a 4.5 regression in some rounding I'm doing with the calculator code. I'm going to revert the code for a 4.5.1 pre-release build soon. As dean explains, the computer actually calculates the result you're seeing, and Alfred has to round off the result to remove the floating point error. Link to comment Share on other sites More sharing options...
Andrew Posted August 4, 2021 Share Posted August 4, 2021 @Salvodelli could you please update to the 4.5.1 pre-release and let me know Link to comment Share on other sites More sharing options...
Salvodelli Posted August 5, 2021 Author Share Posted August 5, 2021 @Andrew Thanks, I can confirm that 4.5.1 [1254] no longer has this issue. Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now