I'm trying to create a "sparse" vector class in C++, like so: template<typename V, V Default> class SparseVector { ... } Internally, it will be represented by an std::map<int, V> (where V is the type of value stored). If an element is not present in the map, we will pretend that it is equal to the value Default from the template a...
I'm trying to do a sparse checkout of a folder containing externals, but none of the externals are being checked out. This issue seems to indicate that this behavior may be by design, or at least that it isn't clear what the behavior should be. From my point of view, the obvious behavior is that externals are treated just as any other di...
Does anyone have experience with Sparse? I seem unable to find any documentation, so the warnings and errors it produces are unclear to me. I tried checking the mailing list and man page but there really isn't much in either. For instance, I use INT_MAX in one of my files. This generates an error (undefined identifier) even though I #in...
I have two twins CentOS 5.4 servers with VMware Server installed on each. What is the most reliable and fast method for copying virtual machines files from one server to the other, assuming that I always use sparse file for my vmware virtual machines? The vm's files are a pain to copy since they are very large (50 GB) but since they ...
I'm building a recommendation engine using collaborative filtering. For similarity scores, I use a Pearson correlation. This is great most of the time, but sometimes I have users that only share a 1 or 2 fields. For example: User 1{ a: 4 b: 2 } User 2{ a: 4 b: 3 } Since this is only 2 data points, a Pearson correlation would always...
Consider a sparse file with 1s written to a portion of the file. I want to reclaim the actual space on disk for these 1s as I no longer need that portion of the sparse file. The portion of the file containing these 1s should become a "hole" as it was before the 1s were themselves written. To do this, I cleared the region to 0s. This d...
I'm trying to solve a Poisson equation on a rectangular domain which ends up being a linear problem like Ax=b but since I know the boundary conditions, there are nodes where I have the solution values. I guess my question is... How can I solve the sparse system Ax=b if I know what some of the coordinates of x are and the undetermin...
The following code runs too slowly even though everything seems to be vectorized. from numpy import * from scipy.sparse import * n = 100000; i = xrange(n); j = xrange(n); data = ones(n); A=csr_matrix((data,(i,j))); x = A[i,j] The problem seems to be that the indexing operation is implemented as a python function, and invoking A[i,...
We are evaluating Tableau and noticed that it doesn't appear to recognize sparse columns in our SQL Server 2008 tables. Is this possible or are there any common workarounds? ...
I have two sparse matrices, m1 and m2: > m1 <- Matrix(data=0,nrow=2, ncol=1, sparse=TRUE, dimnames=list(c("b","d"),NULL)) > m2 <- Matrix(data=0,nrow=2, ncol=1, sparse=TRUE, dimnames=list(c("a","b"),NULL)) > m1["b",1]<- 4 > m2["a",1]<- 5 > m1 2 x 1 sparse Matrix of class "dgCMatrix" b 4 d . > m2 2 x 1 sparse Matrix of class "dgCMatrix" ...
How to get sparse block size and check if data is present at the given offset in sparse file in reiserfs/ext3 in Linux? I want to use it to implement simple copy-on-write block device using FUSE. Or I should better keep a bitmap in a separate file? ...
This great SO answer points to a good sparse solver for Ax=b, but I've got constraints on x such that each element in x is >=0 an <=N. Also, A is huge (around 2e6x2e6) but very sparse with <=4 elements per row. Any ideas/recommendations? I'm looking for something like MATLAB's lsqlin but with huge sparse matrices. I'm essentially try...
I have to create an image with very large resolution, but the image is relatively "sparse", only some areas in the image need to draw. For example with following code /* this take 5GB memory */ final BufferedImage img = new BufferedImage( 36000, 36000, BufferedImage.TYPE_INT_ARGB); /* draw something */ Graphics g = img.getG...
Hello all, I have a sparse matrix that represents a 3D rectangular space. Along some of the boundaries, I know what the value is going to be (it's a constant). The other boundaries may be reflective, differential, etc. Should I just set the problem up as if all the boundaries were say, differential, and then go back and set the nodes ...
I think this is a pretty common issue, but I don't know what the process is called, so I'll describe it with an example. The concept is that I want to join a sparse dataset to a complete series, such as the days of the week, months of the year, or any ordered set (for example, for ranking). Empty positions in the sparse data will show as...
i'd like to do large-scale regression (linear/logistic) in R with many (e.g. 100k) features, where each example is relatively sparse in the feature space---e.g., ~1k non-zero features per example. it seems like the SparseM package slm should do this, but i'm having difficulty converting from the sparseMatrix format to a slm-friendly for...
I'm creating sparse files in python as follows: >>> f = open('testfile', 'ab') >>> f.truncate(1024000) >>> f.close() when the file is done, it takes up 0 disk space, but its inode size is set to my truncated value (1000K): igor47@piglet:~/test$ ls -lh testfile -rw-r--r-- 1 igor47 igor47 1000K 2010-07-09 04:02 testfile igor47@piglet:...
Hi All, I have a DataTable which stores very sparse data, something like: P1 P2 P3 P4 P5 ... J1 1 1 J2 1 1 J3 1 . . . The number of rows and columns might reach over 10^8. How can I store this data in more efficient way? Thanks. ...
I need to compile a binary file in pieces with pieces arriving in random order (yes, its a P2P project) def write(filename, offset, data) file.open(filename, "ab") file.seek(offset) file.write(data) file.close() Say I have a 32KB write(f, o, d) at offset 1MB into file and then another 32KB write(f, o, d) at offset ...
I am migrating a large quantity of mostly empty tables into SQL Server 2008. The tables are vertical partitions of one big logical table. Problem is this logical table has more than 1024 columns. Given that most of the fields are null, I plan to use a sparse table. For all of my tables so far I have been using SELECT...INTO, which ha...