Here is an example to explain what I'm on about: c = Decimal(10) / Decimal(3) c = Decimal(10) / Decimal(2) If I do this, then print c, the inexact and rounded flags are raised. Even though the result is accurate. Shouldn't flags therefore be attributes of numbers rather than the context? This problem is especially apparent when I prog...
Hi, Today, I have started writing a query which is as given below: SELECT IP1.InstId,CAST(IP2.Quote AS DECIMAL(10,2)) AS 'CurrentPrice',IP1.Quote,(IP2.Quote-IP1.Quote) AS 'Change' FROM InstrumentPrices IP1 INNER JOIN InstrumentPrices IP2 ON IP1.InstId=IP2.InstId INNER JOIN Instruments I ON I.Id=IP2.InstId INNER JOIN Games G ON G.Id=I...
I am trying to understand floating point arithmetic better and have seen a few links to 'What Every Computer Scientist Should Know About Floating Point Arithmetic' I still don't understand how a number like 0.1 or 0.5 is stored in floats and as decimals Can someone please explain how it is laid out is memory I know about the float be...
So if i run a = b / c and get the result 1.2234 How do i separate it so that I have a = 1 b = 0.2234 ...
What can I use as a decimal type in JavaScript? It's not supported (0.1 + 0.2 !== 0.3), and I need it for representing exact values in a banking/financial application. See The State and Future of JavaScript for an good read and the dirty details behind JavaScript and its (lack of) support for decimal arithmetic. By "decimal", I mean eit...
Hi all, How can I round a decimal number in Javascript to the nearest 10? My math is pretty rubbish today, it could be the 2 hour sleep :/ Some sample cases $2823.66 = $2820 $142.11 = $140 $9.49 = $10 I understand I probably need a combination of Math.round/floor but I can't seem to get expected result. Any help/pointers appreciat...
I want to format the value in the getter and return a formated currency value. Is this possible or do i need to declare the property as a string and then use string.format. ...
Hey, A boolean array can be considered to be a binary number e.g. boolean[] four = {true, false, false}; //100 I would like to convert such an array to its decimal equivalent e.g. int decimal = convertBooleanArrayToDecimal(four); //decimal == 4 How can I do so? Cheers, Pete ...
I was playing with decimal today. I noticed this: Decimal.MaxValue 79228162514264337593543950335 Decimal.MaxValue - 0.5m 79228162514264337593543950334 The following code prints true. static void Main(string[] args) { decimal d = Decimal.MaxValue - 0.5M; var b = d % 1 == 0; Console.WriteLi...
I have class representing a unit of measure with a Decimal as the numeric. Just to simplify things for this example, let's say I'm representing centimeters. I'd like to implement a ToString() method for this class. The thing is, if the numeric value is a whole number I'd like to just display the decimal as an integer. Otherwise, I'd ...
I try to generate a trigger, in the trigger statement I have to set a column of type decimal(17,3) with the actual timestamp or the seconds of unix timestamp, but could not find a solution. ...
Hi All I am trying to format a decimal A into a fraction B + C/D, where certain limit is imposed on D, say D could be one among [2...9] or [2...19] etc. BCD are integers The goal is to get the formatted fraction as close to the decimal as possible. Is there an existing algorithm/theory on this? Or is there any API I can call on Mac SDK?...
I have an Access 2003 front-end that has data stored in a SQL Server backend database. I want to store monetary values (i.e. $1,3456.25) but I am not sure which datatype to use when going between SQL Server and Access. On the SQL Server side I figured Decimal (18,2) would be good to use as I only need a scale of two (two places to the ri...
How to display a decimal value in the gridview. Say if user enters a value 0.5, it got inserted into DB as 0.5, but while diplaying it back on the the front end , it is displayed as 0.50. How to avoid this scenario. And also the user should be able to enter 5 places after the decimal point. Like 0.12345 is acceptable and 0.123456 is no...
I have two numbers that I need to round to 4 decimal places in JSTL. One is positive: 40.7615811 40.7616 and the other is negative: -73.9940841 -73.9941 Both need to be rounded to 4 decimal places, as on the second line of each example. Is there a clean way to do this in JSTL or do I need to do this in a controller class or JSP s...
I am attempting to cast a string gotten from a local database into decimal, but resulted in a "Cannot implicitly convert Type 'GlobalCurrencyConverter.CurrencyRateDataSet.rateDataTable' to decimal". Below is my code, protected decimal calRate_Click(object sender, EventArgs e) { CurrencyRateDataSetTableAdapters.rateTableAdapter rat...
What is the best way to format the following number that is given to me as a String? String number = "1000500000.574" //assume my value will always be a String I want this to be a String with the value: 1,000,500,000.57 How can I format it as such? ...
how do i set up a Flash CS5 Slider component to trace decimal values, from 0.00 to 10.00 when the slider is dragged? i can't seem to find where i need to set these options to permit decimals. is it possible to set this in ActionScript? ...
Update OK, after some investigation, and thanks in big part to the helpful answers provided by Jon and Hans, this is what I was able to put together. So far I think it seems to work well. I wouldn't bet my life on its total correctness, of course. public static int GetSignificantDigitCount(this decimal value) { /* So, the decimal t...
Hi how to convert Decimal to binary and viceversa i am working on solaris10 platform can anybody help me with a command Decimal to Binary 4000000002-100000000000000000000000000010 Binary to decimal 100000000000000000000000000010-4000000002 ...