"Unix shell"-alike script under Windows

Question

Hello everybody.
I need some help from the shell-script gurus out there.

I have a .txt file (log) that traces the IP addresses of clients on several lines, in a format similar to this one:

Line1 - Client IP [192.168.0.1] Other data
Line2 - Client IP [192.168.0.2] Other data
Line3 - Client IP [192.168.0.3] Other data
Line4 - Client IP [192.168.0.2] Other data
Line5 - Client IP [192.168.0.1] Other data
...

I need to create script that:

• extracts the IP addresses from this file
• groups the IP addresses (the same IP address is reported only once)
• outputs a file with the resulting IP addresses

For the previous example, the resulting file would be:

192.168.0.1
192.168.0.2
192.168.0.3

I am on the Windows OS, but I can use tools like Cygwin or Unix Tools (that provide Unix-like commands as grep, sort, etc. under Windows).

A solution without scripting could be good as well.

Thanks in advance for your help.

Answer 1

 cat yourfile.txt | sed 's/*\[//g' | sed 's/\]*//g' | sort | uniq > newfile.txt

Escapes may be unnecessary for the brackets. I don't remember. These tools should all be available on Cygwin.

Answer 2

Here is a short sed script that extracts the part between square brackets, then sort -u removes duplicates:

sed -e 's/^.*\[\(.*\)\].*$/\1/g' < inputfile | sort -u

Answer 3

In PowerShell:

The Verbose Way -

$regex = '(?<IPAddress>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})'
get-content log.txt | where-object {if ($_ -match $regex){$matches.ipaddress}} | group-object -noelement

Shorter version

gc log.txt | % {if ($_ -match $regex){$matches.ipaddress}} | group -n

Answer 4

It's pretty hard to beat those sed scripts for conciseness. Okay, readability is a problem ...

You could do a rather more verbose, and perhaps more readable version in VBScript using the Scripting.FileSystemObject for file access, using VBScript's regular expressions and the Dictionary object, as below.

Option Explicit

Dim oFSO
Dim oRgx
Dim oMatch
Dim oMatches
Dim oStream
Dim sLine
Dim oDict
Dim sIP
Dim aKeys
Dim sKey

Set oFSO     = CreateObject( "Scripting.FileSystemObject" )
Set oDict    = CreateObject( "Scripting.Dictionary" )

Set oStream  = oFSO.OpenTextFile( "log.txt", 1, False )

Set oRgx     = new regexp
oRgx.Pattern = "\[(.+?)\]"
oRgx.Global  = True

Do While Not oStream.AtEndOfStream
  sLine        = oStream.ReadLine
  Set oMatches = oRgx.Execute(sLine)

  For Each omatch in omatches
    sIP         = oMatch.SubMatches(0)

    If Not oDict.Exists( sIP ) Then
      oDict.Add sIp,1
    End If

  Next

Loop

aKeys = oDict.Keys

For Each sKey in aKeys
  wscript.echo sKey
Next

Answer 5

If you can use Cygwin, there is little point to needing to worry about a Windows scripting solution.