When I write math in LaTeX I often need to perform simple arithmetic on numbers in my LaTeX source, like 515.1544 + 454 = ???. I usually copy-paste the LaTeX code into Google to get the result, but I still have to manually change the syntax, e.g. \frac{154,7}{25} - (289 - \frac{1337}{42}) must be changed to 154,7/25 - (289 - 1...
(Also posted on the MSDN forum - but that doesn't get much traffic, as far as I can see.) I've been trying to provide an example of Assert and Assume. Here's the code I've got: public static int RollDice(Random rng) { Contract.Ensures(Contract.Result<int>() >= 2 && Contract.Result<int>() <= 12); if (rng ==...
why is 24 * 60 * 60 * 1000 * 1000 divided by 24 * 60 * 60 * 1000 not equal to 1000 in Java? ...
When I try to get the sum of a column from a table I get the error 'Arithmetic overflow error converting expression to data type int' because the resulting number is to big for an INT. So I tried to CAST to a BIGINT using SELECT CAST(SUM(columnname) AS BIGINT) FROM tablename but I get the same error. Any ideas what i'm doing wrong? ...
I have some code where, if a user has referred X number of people, he will get X number of credits. For example, referring 2 people = 1 credit. 4 people = 2 credits, and so on. However where this gets tricky is, the numbers can be changed so he gets 1 credit per person, or 1 credit per 3 people, 1 credit for 5 people, etc. If he gets ...
I'm trying to understand a paper on lossless compression of floating point numbers and get stuck on one particular step where the authors map a signed integer from a certain range into a range of half the size, losing information that I would think is required. I have a feeling that the authors are using some standard technique which is ...
I'm storing bit patterns of unsigned 64-bit numbers in a long variable and want to calculate the distance between two of them on the unsigned range. Because Java interprets long as a two's complement signed integer, I can't just do a - b, as the following example shows: // on the unsigned range, these numbers would be adjacent long a = ...
With a back-of-the-envelope calculation you can mentally obtain an answer to a question, within an order of magnitude of its actual value. Some shortcuts can simplify these calculations. In computing I've found the rule 103n is-close-to 210n invaluable. For instance, through this I can say that I can binary search through a billion ...
For a computer working with a 64 bit processor, the largest number that it can handle would be 2^64 = 18446744073709551616. How does programming languages, say Java or be it C, C++ handle arithmetic of numbers higher than this value. Any register cannot hold it as a single piece. How was this issue tackled? ...
Currently, generics in C# do not allow any sane way to perform arithmetic. There are awkward workarounds available, but none of them are very neat and all of them reduce performance. According to this interview, an interface with arithmetic types is not possible to implement, and so one such workaround is suggested. But what you coul...
Why this code 7.30 - 7.20 in ruby returns 0.0999999999999996, not 0.10? But if i'll write 7.30 - 7.16, for example, everything will be ok, i'll get 0.14. What the problem, and how can i solve it? ...
Howdy folks, I would like my C function to efficiently compute the high 64 bits of the product of two 64 bit signed ints. I know how to do this in x86-64 assembly, with imulq and pulling the result out of %rdx. But I'm at a loss for how to write this in C at all, let alone coax the compiler to do it efficiently. Does anyone have any ...
Is there a way to use poiter arithmetic on a large malloc block, so you can assign multiple structs or primitive data types to that area already allocated? I'm writing something like this but it isnt working (trying to assign 200 structs to a 15000byte malloc area): char *primDataPtr = NULL; typedef struct Metadata METADATA; struct M...
My integers in Ruby (MRI) refuse to overflow. I've noticed the class change from fixnum to bignum but I'm wondering how this is modeled and what sort of process ruby uses to perform arithmetic on these massive integers. I've seen this behaviour in SCHEME as well as other environments. I ask because I'd like to implement something simil...
When attemptiing to solve the below assignment : Using the Arithmetic operators ( +,-,*,/) rearrange four fives to equal the number 1 to 10. Example : 5/5+5-5 =1 ,5/5+5/5=2 I tried in C# without using Linq (I don't know how to proceed further) public void GetDetails() { char[] sym = new char[] { '+', '-', '/', '*' }; ...
According to the C / C++ standard (see this link), the >> operator in C and C++ is not necessarily an arithmetic shift for signed numbers. It is up to the compiler implementation whether 0's (logical) or the sign bit (arithmetic) are shifted in as bits are shifted to the right. Will this code function to ASSERT (fail) at compile time f...
If you're optimizing for an architecture on which branching is expensive (say the PS3's cell processor), it can be important to be able to determine whether or not you can express a given algorithm without using branches or at least using fewer branches. One pattern that I see a lot in unoptimized code is a bunch of if's used to tweak a...
I want to handle the special case where multiplying two numbers together causes an overflow. The code looks something like this: int a = 20; long b = 30; // if a or b are big enough, this result will silently overflow long c = a * b; That's a simplified version - in the real app, a and b are sourced elsewhere at runtime. What I want ...
var price = $('#addprice').val(); var pass = $('#pass').val(); var total = $('#totalprice').attr('value') var left = $('#leftquota').attr('value') var balance = $('#balance').attr('value') var tprice = total + price; // total price var bprice = balance + price; // balance price var unitprice = bprice / left; ...
According to here, arithmetic-shift will bit-shift left and right. The right shift preserves the sign. Is there any unsigned right-shift operator, which fills the vacated bits with zero instead of the sign bit? ...