What you've written will work in the F# standalone CTP.
This error occurs in VS2010, because the BigInteger type has been moved from the F# library to the .Net4.0 core library. I'm not sure whether this issue has something to do with having both the F# CTP and the VS2010 beta installed.
Until a better solution comes along, you could roll your own conversion like this:
let numToDouble (n:bignum) = double n.Numerator / double n.Denominator
To convert a bignum to an integer you could then think of something like this:
let numToInt (n:bignum) = int n.Numerator / int n.Denominator
But this is rather dangerous: it'll overflow quite easily. A better version of numToInt would be to convert to a double first and then convert to int:
let numToInt = int << numToDouble
Still, both conversions aren't ideal for numerators/denominators of more than 308 digits, which will still overflow a double, while the fraction itself could be small.
ex: 11^300 / 13^280 ~= 3.26337, but
> numToDouble (pown 11N 300 / pown 13N 280);;
val it : float = nan