Finding position of each word in a sub-array of a multidimensional array

Question

I have an array:

tokens = [["hello","world"],["hello","ruby"]]
all_tokens = tokens.flatten.uniq # all_tokens=["hello","world","ruby"]

Now I need to create two arrays corresponding to all_tokens, where the first array will contain the position of each word in sub-array of tokens. I.E Output:

[[0,0],[1],[1]] # (w.r.t all_tokens)

To make it clear it reads, The index of "hello" is 0 and 0 in the 2 sub-arrays of tokens.

And second array contains index of each word w.r.t tokens.I.E Output:

[[0,1],[0],[1]]

To make it clear it reads,the index of hello 0,1. I.E "hello" is in index 0 and 1 of tokens array.

Cheers!

Answer 1

Your approach sounds difficult to maintain. If you stay on your current path, you'll end up with your tokens array-of-arrays, an array of unique tokens (all_tokens), and then two additional array-of-arrays to keep track of the positions of the unique tokens within the original tokens structure.

An alternative approach is to start with the most natural way to store the unique tokens: a hash. Within that hash you can also store the positional information. That way, all of the information travels together.

There might be a slicker way to achieve this, but here's a simple implementation:

tokens = [["hello","world"],["hello","ruby"]]

token_info     = {}
ordered_tokens = []

tokens.each_with_index do |group, i|
    group.each_with_index do |t, j|
        unless token_info.has_key?(t)
            token_info[t] = {:i => [], :j => []}
            ordered_tokens.push(t)
        end
        token_info[t][:i].push(i)
        token_info[t][:j].push(j)
    end
end

ordered_tokens.each do |t|
    p t, token_info[t]
end

Answer 2

I agree with FM, but this will create your first array:

tokens = [["hello","world"],["hello","ruby"]]
all_tokens = tokens.flatten.uniq

sublist_indices = all_tokens.collect do |token|
  tokens.inject([]) do |indices, list|
    indices += list.each_with_index.select {|pair| pair[0] == token}.map {|pair| pair[1]}
  end
end  # => [[0, 0], [1], [1]]

Remainder left as an exercise.