How can I create a std::vector with 64 bit indexes?

Question

I want to create a big std::vector so operator[] should receive long long rather than unsigned int, I tried writing my own allocator:

template <typename T>
struct allocator64 : std::allocator<T> {
    typedef long long difference_type;
    typedef unsigned long long size_type;
};

But when I try the following:

long long n = 5;
std::vector<int, allocator64<int> > vec(n);
vec[n-1] = 2;

I get the following warning for the second and third line:

warning C4244: 'argument' : conversion from '__int64' to 'unsigned int', possible loss of data

What am I missing? I thought the type for operator[] and for the size constructor should come from allocator::size_type.

I'm using VS9 (2008).

Answer 1

Do you really need to store more than numeric_limits<unsigned int>::max() entries in the vector? O.o

If not then just cast n-1 to int explicitly: int(n-1)

Anyway, the argument is of type vector::size_type (and not allocator::size_type, as far as I know) which is usually a typedef to size_t (but needs not be); it may be 4 bytes or 8 or other amounts; it is implementation defined.

Also see MSDN

Answer 2

The implementation of vector provided with your compiler already uses the largest integer type that it usefully can[*], usually size_t. Suppose you're on a machine with a 32bit address space, and you say you want a vector with 2^33 elements. There's no way that can be allocated, and changing the parameter type of operator[] would not give your computer the ability to allocate bigger blocks of memory.

So, make sure that you're compiling your code for 64bit Windows, and all should be well.

[*] That's not guaranteed by the standard. But compiler-writers don't actually enjoy putting in limits for no reason, even if it sometimes looks that way.

Answer 3

Maybe the STXXL library can help:

STXXL provides an STL replacement using an abstraction layer to storage devices to allow for the optimal layout of data structures. This allows for multi-terabyte datasets to be held and manipulated in standard C++ data structures, whilst abstracting the complexity of managing this behaviour efficiently. STXXL utilises multi-disk I/O to speed up I/O bound calculations. STXXL has been developed at the University of Karlsruhe.