Life's little surprises
#4
Quote:The INT function in programming means to take the integer of the number, chopping off the decimals. Since the result is not precisely 1, but something just less than 1, all the decimals get chopped off and we are left with zero.

Quite correct.

Quote:Now, something I discovered in C++ is that I can get the right answer if I program the equation differently.

int((6.3-6.2)*10) gives the answer 0, whereas
int((6.3f-6.2f)*10) gives the expected answer of 1.

That extra 'f' in there tells the compiler to treat the number as an 80bit extended floating point literal instead of the usual standard 64bit double precision floating point. Basically, I'm just using more decimals of precision even though it's still not exactly precise, so that when the answer is rounded off to 'only' 64bit, it rounds off to the exact value.

Since I'm programming in TrueBasic I don't have that "f" option so I did it this way;

fmag = 62
mag = 6.3
imag = mag*10
n = int(imag-fmag)

So n = 1

Quote:When you learn to program computers, you have to learn numbers and counting all over again.

Oh yes!

Roger




Reply


Messages In This Thread
Life's little surprises - by Roger Hunter - 12-30-2013, 04:26 PM
RE: Life's little surprises - by Skywise - 12-30-2013, 10:34 PM
RE: Life's little surprises - by Skywise - 12-31-2013, 12:40 AM
RE: Life's little surprises - by Roger Hunter - 12-31-2013, 03:08 AM
RE: Life's little surprises - by Skywise - 12-31-2013, 04:09 AM
RE: Life's little surprises - by Roger Hunter - 12-31-2013, 04:55 AM
RE: Life's little surprises - by Skywise - 12-31-2013, 05:37 AM
RE: Life's little surprises - by Roger Hunter - 12-31-2013, 03:17 PM

Forum Jump:


Users browsing this thread: 3 Guest(s)