converting IP ranges into discrete ip numbers using bash/awk/sed

Question

I am processing IP source/destination/port lists created as acl requests The request looks some thing like this:

source IP    destination IP  Port
76.211.12.9  10.112.12.232   1521

The Source and destination IP's have three distinct formats

1. x.x.x.x,y,z
2. x.x.x.x-z
3. x.x.x.x,y.y.y.y,z,z,z,z

I want to create output

1. x.x.x.x
   x.x.x.y
   x.x.x.z
2. x.x.x.x
   x.x.x.y
   x.x.x.z
3. x.x.x.x
   y.y.y.y
   z.z.z.z

using bash, sed ,awk how can I accomplish this? in my example:

76.211.12.9,10,11 10.112.12.232 1521
76.211.12.9-11 10.112.12.232 1521

Both outputs would look like this:

76.211.12.9 10.112.12.232 1521
76.211.12.10 10.112.12.232 1521
76.211.12.11 10.112.12.232 1521

Answer 1

BEGIN { DEBUG = 0 }

function setup(first_split, second_split) {
  src = $1; dst = $2; port = $3
  j = split(src, src_a, first_split)
  k = split(src_a[4], src_a_2, second_split)
  if(DEBUG)
    print "<" first_split second_split ">", j, k, "\n" src
}

/^[0-9]*\.[0-9]*\.[0-9]*\.[0-9]*,[0-9]*\.[0-9]*\.[0-9]*\.[0-9]*/ {
  setup(",", ",")
  for(i = 1; i <= j; ++i)
    print src_a[i], dst, port
  next
}

/^[0-9]*\.[0-9]*\.[0-9]*\.[0-9]*,[0-9][0-9]*[ ,]/ {
  setup(".", ",")
  for(i = 1; i <= k; ++i)
    print src_a[1] "." src_a[2]"." src_a[3] "." src_a_2[i], dst, port
  next
}

/^[0-9]*\.[0-9]*\.[0-9]*\.[0-9]*-/ {
  setup(".", "-")
  for(i = src_a_2[1]; i <= src_a_2[2]; ++i)
    print src_a[1] "." src_a[2] "." src_a[3] "." i, dst, port
  next
}

{ print }

My test file:

76.211.77.7 10.112.12.232 1521
76.211.77.8 10.112.12.232 1521
76.211.77.9,10,11 10.112.12.232 1521
76.211.77.12-13 10.112.12.232 1521
76.211.77.14,76.211.77.15,76.211.77.16 10.112.12.232 1521

This script will work in the One True Awk and also with gawk.

Answer 2

In order to generalize the awk solution from my other answer to all three fields, I think we would end up using awk simply as a general purpose programming language.

Now, awk is wonderful, and in the mid-1970's when Unix appeared, well, it was a collective software Renaissance. But advanced languages have joined the Unix software tools club as well, and really, I would rather write it in Ruby...

Ruby this time, see my other answer for an Awk approach...

class IPMacro
  private_class_method :new
  def self.get_base a
    @@root = a[0].split('.').tap { |x| @@last = x[3] || x[0] }.take(3).join('.')
  end
  def self.expand_last_comma a
    self.get_base a
    [a[0], *(a.drop(1).map { |e| [@@root, e].join('.') })]
  end
  def self.expand_last_dash a
    self.get_base a
    @@root = @@root['.'] ? [@@root] : []
    [*(@@last..a[1]).to_a.map do |e|
        (@@root +  [String(e)]).join '.'
      end
    ]
  end
  def self.expand f
    a = f.split ','
    if a[1]
      return self.expand_last_comma a unless a[1]['.'] || !a[0]['.']
    else
      a = f.split '-'
      return self.expand_last_dash a if a[1]
    end
    return [*a]
  end
  def self.run
    $<.each_line do |line|
      a = line.split(' ').map do |f|
        self.expand f
      end
      a[0].each do |a1|
        a[1].each do |a2|
          a[2].each do |a3|
            puts '%s %s %s' % [a1, a2, a3]
          end
        end
      end
    end
  end
  self
end.run

My test case this time was...

76.211.77.5 10.112.12.227 1400,1401,1402
76.211.77.6 10.112.12.228-231 1510-1515
76.211.77.7 10.112.12.232 1521
76.211.77.8 10.112.12.232 1521
76.211.77.9,10,11 10.112.12.232 1521
76.211.77.12-13 10.112.12.232 1521
76.211.77.14,76.211.77.15,76.211.77.16 10.112.12.232 1521

Answer 3

Expletive deleted dude! This is some sweet code. Very elegant.. It's exactly what I needed. I ran it against my 340 lines of data and generated over 7500 individual src/dest/port pairs.

Thanks