space-efficiency

Ideal options for archiving flat files

We receive multiple thousands of flat files per week currently, and I have a system that runs reports on these and exports them to PDF for our people to process and reference. I currently bulk load these into a database, make sure all fields/formatting is valid, export them, and truncate the tables on the next run. What I'm wondering...

Algorithms for Big O Analysis

What all algorithms do you people find having amazing (tough, strange) complexity analysis in terms of both - Resulting O notation and uniqueness in way they are analyzed? ...

more efficient way to pickle a string

The pickle module seems to use string escape characters when pickling; this becomes inefficient e.g. on numpy arrays. Consider the following z = numpy.zeros(1000, numpy.uint8) len(z.dumps()) len(cPickle.dumps(z.dumps())) The lengths are 1133 characters and 4249 characters respectively. z.dumps() reveals something like "\x00\x00" (act...

Best shortcut management tool for development machine?

As a developer I had a slew of shortcuts and applications used daily. Visual Studio 6, 2003, 2005, 2008, SQL Client tools, WinMerge, Notepad++, Synergy, VMware (and lots and lots of VMs, multiple browsers, and on top of all that about 20 shortcuts to important directories, servers, test environments etc... The Vista start menu is a lif...

Space efficiency of algorithms...

It seems like none of the algorithm textbooks mentions about space efficiency as much, so I don't really understand when I encounter questions asking for an algorithm that requires only constant memory. What would be an example of a few examples of algorithms that uses constant memory and algorithms that doesn't use constant memory? ...

What is Java's lightest weight non-concurrent implementation of Iterable?

I need a class that implements Iterable, and does not need to be safe for concurrent usage. Of the various options, such as LinkedList, HashSet, ArrayList etc, which is the lightest-weight? To clarify the use-case, I need to be able to add a number of objects to the Iterable (typically 3 or 4), and then something else needs to iterate ...

Efficiently finding the ranks of elements in an array?

How does one find the rank of each element of an array, averaging in the case of ties, efficiently? For example: float[] rank(T)(T[] input) { // Implementation } auto foo = rank([3,6,4,2,2]); // foo == [3, 5, 4, 1.5, 1.5] The only way I can think of doing it requires allocating 3 arrays: A duplicate of the input array because...

Most compact way to encode a sequence of random variable length binary codes?

Let's say you have a List<List<Boolean>> and you want to encode that into binary form in the most compact way possible. I don't care about read or write performance. I just want to use the minimal amount of space. Also, the example is in Java, but we are not limited to the Java system. The length of each "List" is unbounded. Therefo...