views:

1078

answers:

12

How can i quickly shift all the items in an array one to the left, padding the end with null?

[0,1,2,3,4,5,6] would become [0,2,3,4,5,6,null]

Edit: I said quickly but I guess I meant efficiently. I need to do this without creating a List or some other data structure. This is something I need to do several hundred thousand times in as short amount of time as possible.

+1  A: 

You might do it like this:

var items = new int?[] { 0, 1, 2, 3, 4, 5, 6 };  // Your array
var itemList = new List<int?>(items);  // Put the items in a List<>
itemList.RemoveAt(1); // Remove the item at index 1
itemList.Add(null); // Add a null to the end of the list
items = itemList.ToArray(); // Turn the list back into an array

Of course, it would be more efficient to get rid of the array entirely and just use a List<>. You could then forget the first line and last line and do it like this:

var itemList = new List<int?> { 0, 1, 2, 3, 4, 5, 6 };
itemList.RemoveAt(1); // Remove the item at index 1
itemList.Add(null); // Add a null to the end of the list
Aaron
hey! did managed programming completely stole ability to think about performance?
Andrey
You're absolutely right about this not being the most efficient way. If this operation is going to happen a lot, then this isn't the right answer. And like I said, going from array to List and back to array isn't efficient, so I would skip the array and just use a List to begin with. If this is an operation that only happens on occasion, then this will work just fine and it will be easy to understand. Plus this way can handle changing List lengths.
Aaron
+1  A: 

just do:

int?[] newAr = new int?[oldAr.Length];
for (int i = 1; i < oldAr.Length; i++)
   newAr[i - 1] = oldAr[i];
Andrey
BE MORE DECLARATIVE!
Jason Punyon
Also the final output here ends in a zero, not a null.
Jason Punyon
yeah, add null into end. "BE MORE DECLARATIVE! " you mean use declarative style? you should put it everywhere. simple operations have to be done in simple way.
Andrey
You can't null an int in C#.
Jason Punyon
right right, i need be more carefull
Andrey
@Jason Punyon, you can at least assign the default value to it.
Dykam
@Andrey, just switch your int[] to int?[] and problem solved
Stan R.
That's `int?[] newAr`, not `int newAr?[]`. Also, `newAr` as a variable name is really ugly, would it kill you to add `ray`?
Trillian
What does the '?' in `int newAr?[]` actually mean?
TP
@Jaelebi, int is a value type so it doesnt accept "null" as a value, int[] is an array of int's. int? is a Nullable type it allows null as a value and int?[] is an array of Nullable int's. Here's a link (http://msdn.microsoft.com/en-us/library/b3h38hb0.aspx)
Stan R.
@Jaelebi: It doesn't mean anything in that code because that code doesn't actually compile. int? double? DateTime? are synonyms for System.Nullable<int>, System.Nullable<double> and System.Nullable<DateTime>.
Jason Punyon
That is some nasty C#..
CaptainCasey
@Trillian +1 to adding `ray`
bendewey
A: 

Incorrect and slightly amusing answer (thanks, i'll be here all night !)

int?[] test = new int?[] {0,1,2,3,4,5,6 };

        int?[] t = new int?[test.Length];
        t = test.Skip(1).ToArray();
        t[t.Length - 1] = null; 

In the spirit of still using Skip (dont ask me, i know worst usage of LINQ extension methods ever), the only way I thought of rewriting it would be

        int?[] test = new int?[] { 0, 1, 2, 3, 4, 5, 6 };

        int?[] t = new int?[test.Length];
        Array.Copy(test.Skip(1).ToArray(), t, t.Length - 1);

But it's in NO WAY faster than the other options.

Stan R.
ToArray is too expensive
Dested
I read that as: `ToArray` is `ToExpensive`.
Steven Sudit
This actually turns the last element you want to keep into a null...so I'm not profiling it in my answer.
Jason Punyon
@Jason, what exactly are you talking about? The result i get is [1,2,3,4,5,6,null]
Stan R.
@Stan R.: The code posted results in [1,2,3,4,5,null]. You can immediately see that length isn't preserved. test.Skip(1).ToArray() will have a length less than test.
Jason Punyon
@Jason, ahhh i see..ToArray actually creates a whole new array. Wow, def need to be more careful with this kind of stuff. Thanks for pointing that out. I will rewrite my answer, then you can profile it :)
Stan R.
@Jason, yes, i know..my mistake.
Stan R.
@Jason, i rewrote the answer, yes I know its stupid/redundant, but I wanted to stick with my original Skip idea and bathe in all of my shame. I am going to go stand in a corner now.
Stan R.
@Stan R. - even though you have already confessed to the answer being "stupid/redundant", I feel compelled to point out that you are allocating an array that never gets used at all, then creating an array that's too short, and then overwriting the last useful piece of data with null. Enjoy your shame bath! :-)
Jeffrey L Whitledge
@Jeffrey, i assume you're talking about the 1st answer, the 2nd answer works, it copies the array created into the destination array...the 1st answer..well..lets just say I haven't had breakfast yet and its 1:42pm
Stan R.
@Stan R. - Ah, yes, I missed that part. Sorry. The second answer is very nice. :-)
Jeffrey L Whitledge
+5  A: 

Couldn't you use a System.Collections.Generic.Queue instead of an array ?

I feel like you need to perform actions on your value the discard it, thus using a queue seems to be more appropriate :

// dummy initialization
        System.Collections.Generic.Queue<int> queue = new Queue<int>();
        for (int i = 0; i < 7; ++i ) { queue.Enqueue(i); }// add each element at the end of the container

        // working thread
        if (queue.Count > 0)
            doSomething(queue.Dequeue());// removes the last element of the container and calls doSomething on it
Seb
Can I? Can you elaborate?
Dested
+31  A: 

The quickest way to do this is to use Array.Copy, which in the final implementation uses a bulk memory transfer operation (similar to memcpy):

var oldArray = new int?[] { 1, 2, 3, 4, 5, 6 };
var newArray = new int?[oldArray.Length];
Array.Copy(oldArray, 1, newArray, 0, oldArray.Length - 1);
// newArray is now { 2, 3, 4, 5, 6, null }

Edited: according to the documentation:

If sourceArray and destinationArray overlap, this method behaves as if the original values of sourceArray were preserved in a temporary location before destinationArray is overwritten.

So if you don't want to allocate a new array, you can pass in the original array for both source and destination--although I imagine the tradeoff will be a somewhat slower performance since the values go through a temporary holding position.

I suppose, as in any investigation of this kind, you should do some quick benchmarking.

Ben M
the two arrays in the `Copy` method may overlap, so you don't need to create another array. `Array.Copy(oldArray, 1, oldArray, 0, oldArray.Length - 1)` together with setting the last element to `null` will also work.
MartinStettner
I havent ran the tests yet but I think the for loop approach will end up being faster int eh log run. Thank you for the response though.
Dested
I think both would be fast enough for you...
Stan R.
If you haven't "ran the tests", obviously you don't care that much about performance! You might as well use ToList, Remove, ToArray.
Iain Galloway
@Dested: unless the `for` loop compiles down into the same lean loop that a traditional memcpy uses, that won't be the case. See, for example, http://waldev.blogspot.com/2008/05/efficiently-copying-items-from-one.html
Ben M
I haven't looked at the code, but the docs only say that it behaves "as if" the original values were preserved in a temporary location. I would expect this behavior to be simulated by either starting at the left and incrementing or starting at the right and decrementing, depending on the way the overlap occurs. I would be surprised if there actually is a temporary location. (ConstrainedCopy, on the other hand, probably does use a temp location.)
Jeffrey L Whitledge
@Ben M - optimizers can be pretty amazing sometimes. The only way to know for sure is to benchmark it, but the for loop stands a good chance of compiling to the same lean loop, and it can avoid all the checks and planning that the Array.Copy method must do on each invocation. Copy is, after all, an extremely flexible method.
Jeffrey L Whitledge
Allocating a new (temporary) array takes much longer than an overlapped copy: In the latter case the direction of the loop is simply chosen so that it works...
MartinStettner
I recall performance tests between memcpy and a for loop for copying a string. Strangely the for loop was quicker than the bulk memory copy - something to do with the x86 instruction set... see http://bytes.com/topic/c/answers/212738-comparative-performance-str-functions-vs-mem-functions often what you think should be obvious, isn't.
gbjbaanb
And that's why you profile! See Jason's answer.
Iain Galloway
+4  A: 

Use the Array.Copy() method as in

int?[] myArray = new int?[]{0,1,2,3,4};
Array.Copy(myArray, 1, myArray, 0, myArray.Length - 1);
myArray[myArray.Length - 1] = null

The Array.Copy is probably the way, Microsoft wanted us to copy array elements...

MartinStettner
+30  A: 

Here's my test harness...

var source = Enumerable.Range(1, 100).Cast<int?>().ToArray();
var destination = new int?[source.Length];

var s = new Stopwatch();
s.Start();
for (int i = 0; i < 1000000;i++)
{
    Array.Copy(source, 1, destination, 0, source.Length - 1);
}
s.Stop();
Console.WriteLine(s.Elapsed);

Here are the performance results for 1 million iterations of each solution (8 Core Intel Xeon E5450 @ 3.00GHz)

                       100 elements  10000 elements
For Loop                     0.390s         31.839s 
Array.Copy()                 0.177s         12.496s
Aaron 1                      3.789s         84.082s
Array.ConstrainedCopy()      0.197s         17.658s

Make the choice for yourself :)

Jason Punyon
obviously not !!!! :)
Stan R.
Try it again with the optimizations turned on. :-)
Jeffrey L Whitledge
@Jeffrey L. Whitledge: This was all done compiled in release mode and running outside the debugger. Are there further optimizations to turn on?
Jason Punyon
might as well benchmark the rest of our code! you know you want to ;)
Stan R.
Nope. I just tried it myself, and got the same results. I guess `Array.Copy` wins. Dang, that's one fast method.
Jeffrey L Whitledge
def +1 for doing all this work, thanks.
Stan R.
Array.Copy is probably faster because it can do things you can't do in C#. It could very well be just playing with the page table and setting the flags to force copy on write.
Robert Davis
I'd give you +100 if I could. There are only two ways to approach a performance "problem" like this one. A) Profile and pick the best, or B) Decide that you don't really care about a difference of a few hundred ms under realistic conditions. Write your code for readability first, then profile if and only if you need to!
Iain Galloway
@Robert Davis, you can do anything in unsafe mode, as long as you are comfortable cleaning up after yourself.
NickLarsen
+4  A: 

If it absolutely has to be in an array, then I would recommend the most obvious code possible.

for (int index = startIndex; index + 1 < values.Length; index++)
     values[index] = values[index + 1];
values[values.Length - 1] = null;

This gives the optimizer the most opportunities to find the best way on whatever target platform the program is installed on.

EDIT:

I just borrowed Jason Punyon's test code, and I'm afraid he's right. Array.Copy wins!

    var source = Enumerable.Range(1, 100).Cast<int?>().ToArray();
    int indexToRemove = 4;

    var s = new Stopwatch();
    s.Start();
    for (int i = 0; i < 1000000; i++)
    {
        Array.Copy(source, indexToRemove + 1, source, indexToRemove, source.Length - indexToRemove - 1);
        //for (int index = indexToRemove; index + 1 < source.Length; index++)
        //    source[index] = source[index + 1]; 
    }
    s.Stop();
    Console.WriteLine(s.Elapsed);

Array.Copy takes between 103 and 150 ms on my machine.

for loop takes between 269 and 338 ms on my machine.

Jeffrey L Whitledge
A: 

Array copying is an O(n) operation and creates a new array. While array copying can certainly be done quickly and efficiently, the problem you've stated can actually be solved in an entirely different way without (as you've requested) creating a new array/data structure and only creating one small wrapping object instance per array:

using System;
using System.Text;

public class ArrayReindexer
{
    private Array reindexed;
    private int location, offset;

    public ArrayReindexer( Array source )
    {
        reindexed = source;
    }

    public object this[int index]
    {
        get
        {
            if (offset > 0 && index >= location)
            {
                int adjustedIndex = index + offset;
                return adjustedIndex >= reindexed.Length ? "null" : reindexed.GetValue( adjustedIndex );
            }

            return reindexed.GetValue( index );
        }
    }

    public void Reindex( int position, int shiftAmount )
    {
        location = position;
        offset = shiftAmount;
    }

    public override string ToString()
    {
        StringBuilder output = new StringBuilder( "[ " );
        for (int i = 0; i < reindexed.Length; ++i)
        {
            output.Append( this[i] );
            if (i == reindexed.Length - 1)
            {
                output.Append( " ]" );
            }
            else
            {
                output.Append( ", " );
            }
        }

        return output.ToString();
    }
}

By wrapping and controlling access to the array in this manner, we can now demonstrate how the problem was solved with an O(1) method call...

ArrayReindexer original = new ArrayReindexer( SourceArray );
Console.WriteLine( "   Base array: {0}", original.ToString() );
ArrayReindexer reindexed = new ArrayReindexer( SourceArray );
reindexed.Reindex( 1, 1 );
Console.WriteLine( "Shifted array: {0}", reindexed.ToString() );

Will produce the output:

Base array: [ 0, 1, 2, 3, 4, 5, 6 ]
Shifted array: [ 0, 2, 3, 4, 5, 6, null ]

I'm willing to bet that there will be a reason that such a solution won't work for you, but I believe this does match your initial stated requirements. 8 )

It's often helpful to think about all the different kinds of solutions to a problem before implementing a specific one, and perhaps that might be the most important thing that this example can demonstrate.

Hope this helps!

Task
+1  A: 

You can use the same array as source and destination for fast in-place copy:

static void Main(string[] args)
        {
            int[] array = {0, 1, 2, 3, 4, 5, 6, 7};
            Array.ConstrainedCopy(array, 1, array, 0, array.Length - 1);
            array[array.Length - 1] = 0;
        }
RocketSurgeon
+1  A: 

Here is my solution, similar to Task's in that it is a simple Array wrapper and that it takes O(1) time to shift the array to the left.

public class ShiftyArray<T>
{
    private readonly T[] array;
    private int front;

    public ShiftyArray(T[] array)
    {
        this.array = array;
        front = 0;
    }

    public void ShiftLeft()
    {
        array[front++] = default(T);
        if(front > array.Length - 1)
        {
            front = 0;
        }
    }

    public void ShiftLeft(int count)
    {
        for(int i = 0; i < count; i++)
        {
            ShiftLeft();
        }
    }

    public T this[int index]
    {
        get
        {
            if(index > array.Length - 1)
            {
                throw new IndexOutOfRangeException();
            }

            return array[(front + index) % array.Length];
        }
    }

    public int Length { get { return array.Length; } }
}

Running it through Jason Punyon's test code...

int?[] intData = Enumerable.Range(1, 100).Cast<int?>().ToArray();
ShiftyArray<int?> array = new ShiftyArray<int?>(intData);

Stopwatch watch = new Stopwatch();
watch.Start();

for(int i = 0; i < 1000000; i++)
{
    array.ShiftLeft();
}

watch.Stop();

Console.WriteLine(watch.ElapsedMilliseconds);

Takes ~29ms, regardless of the array size.

tdaines
+2  A: 

Can't you

  • allocate the array with an extra 1000 elements

  • have an integer variable int base = 0

  • instead of accessing a[i] access a[base+i]

  • to do your shift, just say base++

Then after you've done this 1000 times, copy it down and start over.
That way, you only do the copy once per 1000 shifts.


Old joke:
Q: How many IBM 360s does it take to shift a register by 1 bit?
A: 33. 32 to hold the bits in place, and 1 to move the register. (or some such...)

Mike Dunlavey
I actually ended up doing something like this.
Dested
That's basically the "use a queue with start and end pointers" suggested by several others. But good clear explanation.
Ben Voigt