This is your resource to discuss support topics with your peers, and learn from each other.

turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

- BlackBerry® 10 Devices
- BlackBerry® Leap
- BlackBerry® Classic
- BlackBerry® Passport
- BlackBerry® Z30
- BlackBerry® Z10
- BlackBerry® Z3
- BlackBerry® Q10
- BlackBerry® Q5
- Porsche Design P’9982 smartphone from BlackBerry®
- Porsche Design P’9983 smartphone from BlackBerry®
- BlackBerry® 10 Functions and Features
- BlackBerry® 10 Accessories
- Device Switch Help

- Developer Support
- :
- Application Platforms
- :
- Java Development
- :
- Re: Eliminating double imprecision

inside custom component

Topic Options

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic to the Top
- Bookmark
- Subscribe
- Printer Friendly Page

My Device: Not Specified

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

11-17-2009 05:25 PM - edited 11-18-2009 02:58 PM

I have to eliminate double imprecision with real number math in my application but I can't figure out how using the BB API. I tried writing a method that checks for series of 9's and 0's and rounding the previous digit upwards or eliminating the trailing digits, but my method caused too many problems.

Edit: let me give an example of the error:

I am expecting this result: 0.0002

But through after some double math to retain the result the program returns 0.0002000001

Any ideas on how I can eliminate double imprecision?

Thanks,

Scott

Solved! Go to Solution.

My Device: Not Specified

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

11-18-2009 02:51 PM

Bumping

My Device: Not Specified

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

11-18-2009 02:58 PM

My first advice: don't use double (especially for currency). Use longs and "assume" the decimal position (fixed point math).

My 2nd suggtestion would be to use the MathUtilities.round() method, but it wasn't intriduced into the RIM api until 4.6.

This means you will have to "invent" the round() function for your builds lower than 4.6

My Device: Not Specified

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

11-18-2009 03:13 PM

Thanks Rex,

I tried casting to long and then using the formulas my application needs. The problem I am having after that is that I don't know where the decimal place is going to be. For this application the numbers being worked with can be anywhere between 1 billionth to 1 billion.

When I casted to long, I also multiplied each quantity by something like 1 billion to eliminate any decimals, ran the quantities through the formula, and then divided by 1 billion before displaying. It worked fine for most of the formulas, but it failed for other ones because I assume the number was too large to be a long.

I think what I am going to have to do is to handle cases depending on the size of the numbers. I think this is the no-man's land of coding. Tell my mom I love her if I don't make it back.

Thanks!

Scott

My Device: Not Specified

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

11-18-2009 03:46 PM

I feel your pain.

My Device: Storm

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

03-08-2010 10:43 PM - edited 03-08-2010 10:47 PM

Scott

Here's something that might work for you. I've been writing it as a learning exercise for myself (plus rounding errors irritate the *edited* out of me). It can handle very large positive doubles with (from what my testing tells me) perfect accuracy. It cannot handle exponential or negative numbers.

...yet

Give it a try and let me know how it turns out.

public static double decimalRound(double number, int roundTo, boolean roundUp) {

// approx range 7 to the left - 10 to the right, otherwise

// automatic formatting in exponential notation will occur

// number - non-scientific formatted double

// roundTo - how many digits you want to keep

// roundUp (true) - round to xx digits

// roundUp (false) - trim to xx digits

// setup local variables

String s = String.valueOf(number);

int rounder = 0;

if (roundUp) rounder = 5;

int dSave = roundTo+1;

int dIndex = s.indexOf(".");

// s.length()-dIndex-1 is the number of digits after the decimal

// s.indexOf("E") tests for the presence of a exponential number

// all numbers that cannot or should not be rounded are returned

if (roundTo < 0 || roundTo >= s.length()-dIndex-1 || s.indexOf("E") > 0) {

return number;

}

// remove the decimal and extra digits

s = s.substring(0, dIndex) + s.substring(dIndex+1, dIndex+1+dSave);

// do not use int or data loss will occur due to truncation

// rounder=5 will round the number up, but rounder=0 will not

// this is useful if you just want to trim the number to xx digits

s = Long.toString(Long.parseLong(s)+rounder);

s = s.substring(0, s.length()-dSave) + "." + s.substring(s.length()-dSave, s.length()-1);

// convert modified string back to double and return

number = Double.parseDouble(s);

return number;

}

My Device: Not Specified

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

03-08-2010 10:54 PM

Correct me if I'm wrong, but this is a symptom of using IEEE 754 floating point representation and you can't really "fix" the precision problems here in the general case.

The only way to do arbitrary precision mathematics accurately and correctly in the general case here is to use bignum-style number representation and this requires extra libraries.

The only other thing I can think of is that the operations you are performing aren't written as efficiently as possible to preserve precision, but this is a guess; you could copy what exactly you're trying to do here...

My Device: Storm

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

03-08-2010 11:14 PM

You're right. Nothing can fix the floating point accuracy. As I understand it, integer math does a better job. I should have pulled out just the part that was pertinent. And sorry for the newbie mistake. Using this method or parts of it to round out the error may still be a viable option depending on how many decimal places are needed for the task at hand.

Bob

Highlighted
## Re: Eliminating double imprecision

Options

My Device: Not Specified

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

03-08-2010 11:25 PM

Yes, if you're only ever needing k digits of precision, you can truncate your floating point numbers fine, but if your precision needs vary at any point, you have to make do one way or another.

My Device: Not Specified

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Highlight
- Email to a Friend
- Report Inappropriate Content

07-09-2011 11:52 AM

Thanks for all the responses. I forgot about this thread but remembered it when the problem came up again.

I solved the problem by using integer math instead of double math, and I use indexOf(".") to format the integers. It was actually a relatively easy solution.

Thanks,

Scott

Copyright © 2015 BlackBerry, unless otherwise noted.