views:

123

answers:

6

I'm looking for a good and fast way to manage IP addresses and ports in a file. Sort of a DB Table that has 2 columns: IP and Port, but in a file, without using a DB.

It has to support adding, deleting and updating. I don't care from concurrency.

Thanks.

A: 

The easiest way is probably to create a small class that contains your IP and port

class IpAddress
{
    public string IP;
    public int port;
}

and then create a list<IpAddress> of them. You can then use XML Serialization and Deserialization to read to and write from a file your list.

Robert Harvey
In your example, I need first to load the whole data to the memory, and then insert/update/delete in-memory, and then rewrite the whole file. I need a solution that won't require loading the whole data to the memory or rewriting the whole file, I need solution that will just make the specific insert/update/delete operation.
TTT
How many IP's are we talking about? Even if it's thousands, the file is still going to be relatively small.
Robert Harvey
If you really want to add and remove from the middle of the file, consider SQLite. It's a lightweight, single-file database with a very small DLL that runs it. A C# wrapper is available.
Robert Harvey
A: 

The .NET BCL does not offer what you are looking for as you want to query against a file without loading it into memory first and support add/remove. So you'd either have to roll your own embedded database or you could simply use something like SQLite http://www.sqlite.org/

Tom Frey
A: 

IP and Port is a one to many relationship. I would consider something like this

\t192.168.1.1\r\n25\r\n26\r\n\t192.168.1.2\r\n2\r\n80\r\n110

where \t is a tab and \r\n is a carriage return followed by a newline

So when you parse, if you hit a tab character, you know everything that's in that line from there to the newline is an IP address, then everything in between the next newlines is a port number for that IP address until you hit a tab, in which case you're on a new IP address. That's simple and fast but not as human readable.

Jeff Tucker
wouldn't it be much more human readable to do the tabs on the port lines?
Moose
Yes, it probably would be. Good call.
Jeff Tucker
A: 

this has nothing to do with IP and ports.. the problem is that, as far as i know, windows does not allow to INSERT or remove bytes in/from the middle of a file..

Itay
+2  A: 

Below, some come to complete your task. I tried to go strictly to the point, so maybe something is missing.

I'd to create a "Record" class, to keep ip/port pairs

class Record : IPEndPoint, IComparable<Record>
{
    internal long Offset { get; set; }
    public bool Deleted  { get; internal set; }

    public Record() : base(0, 0)
    { 
        Offset = -1;
        Deleted = false;
    }

    public int CompareTo(Record other)
    {
        if (this.Address == other.Address && this.Address == other.Address )
            return 0;
        else if (this.Address == other.Address)
            return this.Port.CompareTo(other.Port);
        else
            return 
              BitConverter.ToInt32(this.Address.GetAddressBytes(), 0).CompareTo(
              BitConverter.ToInt32(other.Address.GetAddressBytes(), 0));
    }
}

class RecordComparer : IComparer<Record>
{
    public int Compare(Record x, Record y)
    {
        return x.CompareTo(y);
    }
}

...And a "DatabaseFile" class to manage datafile interaction.

class DatabaseFile : IDisposable
{
    private FileStream file;
    private static int RecordSize = 7;
    private static byte[] Deleted = new byte[] { 42 };
    private static byte[] Undeleted = new byte[] { 32 };
    public DatabaseFile(string filename)
    {
        file = new FileStream(filename, 
            FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.None);
    }

    public IEnumerable<Record> Locate(Predicate<Record> record)
    {
        file.Seek(0, SeekOrigin.Begin);
        while (file.Position < file.Length)
        {
            long offset = file.Position;
            byte[] buffer = new byte[DatabaseFile.RecordSize];
            file.Read(buffer, 0, DatabaseFile.RecordSize);
            Record current = Build(offset, buffer);
            if (record.Invoke(current))
                yield return current;
        }
    }

    public void Append(Record record)
    {
        // should I look for duplicated values? i dunno
        file.Seek(0, SeekOrigin.End);
        record.Deleted = false;
        record.Offset = file.Position;
        Write(record);
    }

    public void Delete(Record record)
    {
        if (record.Offset == -1) return;
        file.Seek(record.Offset, SeekOrigin.Begin);
        record.Deleted = true;
        Write(record);
    }

    public void Update(Record record)
    {
        if (record.Offset == -1)
        {
            Append(record);
        }
        else
        {
            file.Seek(record.Offset, SeekOrigin.Begin);
            Write(record);
        }
    }

    private void Write(Record record)
    {
        file.Write(GetBytes(record), 0, DatabaseFile.RecordSize);
    }

    private Record Build(long offset, byte[] data)
    {
        byte[] ipAddress = new byte[4];
        Array.Copy(data, 1, ipAddress, 0, ipAddress.Length);
        return new Record
        {
            Offset = offset,
            Deleted = (data[0] == DatabaseFile.Deleted[0]),
            Address = new IPAddress(ipAddress), 
            Port = BitConverter.ToInt16(data, 5)
        };
    }

    private byte[] GetBytes(Record record)
    {
        byte[] returnValue = new byte[DatabaseFile.RecordSize];
        Array.Copy(
            record.Deleted ? DatabaseFile.Deleted : DatabaseFile.Undeleted, 0, 
            returnValue, 0, 1);
        Array.Copy(record.Address.GetAddressBytes(), 0, 
            returnValue, 1, 4);
        Array.Copy(BitConverter.GetBytes(record.Port), 0, 
            returnValue, 5, 2);
        return returnValue;
    }

    public void Pack()
    {
        long freeBytes = 0;
        byte[] buffer = new byte[RecordSize];
        Queue<long> deletes = new Queue<long>();

        file.Seek(0, SeekOrigin.Begin);
        while (file.Position < file.Length)
        {
            long offset = file.Position;
            file.Read(buffer, 0, RecordSize);
            if (buffer[0] == Deleted[0])
            {
                deletes.Enqueue(offset);
                freeBytes += RecordSize;
            }
            else
            {
                if (deletes.Count > 0)
                {
                    deletes.Enqueue(offset);
                    file.Seek(deletes.Dequeue(), SeekOrigin.Begin);
                    file.Write(buffer, 0, RecordSize);
                    file.Seek(offset + RecordSize, SeekOrigin.Begin);
                }
            }
        }
        file.SetLength(file.Length - freeBytes);
    }

    public void Sort()
    {
        int offset = -RecordSize; // lazy method
        List<Record> records = this.Locate(r => true).ToList();
        records.Sort(new RecordComparer());
        foreach (Record record in records)
        {
            record.Offset = offset += RecordSize;
            Update(record);
        }
    }

    public void Dispose()
    {
        if (file != null)
            file.Close();
    }
}

Below, a working example:

static void Main(string[] args)
{
    List<IPEndPoint> endPoints = new List<IPEndPoint>(
        new IPEndPoint[]{
            new IPEndPoint(IPAddress.Parse("127.0.0.1"), 80),
            new IPEndPoint(IPAddress.Parse("69.59.196.211"), 80),
            new IPEndPoint(IPAddress.Parse("74.125.45.100"), 80)
        });
    using (DatabaseFile dbf = new DatabaseFile("iptable.txt"))
    {
        foreach (IPEndPoint endPoint in endPoints)
            dbf.Append(new Record { 
                Address = endPoint.Address, 
                Port = endPoint.Port });

        Record stackOverflow = dbf.Locate(r => 
            Dns.GetHostEntry(r.Address)
                .HostName.Equals("stackoverflow.com")).FirstOrDefault();
        if (stackOverflow != null)
            dbf.Delete(stackOverflow);

        Record google = dbf.Locate(r =>
            r.Address.ToString() == "74.125.45.100").First();
        google.Port = 443;
        dbf.Update(google);

        foreach(Record http in dbf.Locate(r => 
            !r.Deleted && r.Port == 80))
            Console.WriteLine(http.ToString());
    }
    Console.ReadLine();
}

dBase III, I miss you.

Well, that was fun, thank you!

EDIT 1: Added Pack() and lazy Sort() code;

EDIT 2: Added missing IComparable/IComparer implementation

Rubens Farias
Wow, thank you so much! It's EXACTLY what I needed! Thanks!!!
TTT
Nice to hear that, Alon, ty; for curiosity's sake: where are you planning to use this?
Rubens Farias
+1  A: 

I personally will go for

192.100.10.1:500:20-21

192.100.10.2:27015-27016:80

Where the first is the Ip and every thing after the : is a port, We can also represent a range by - and if we want to be very crazy about it we can introduce a u which will represent the port type UDP or TCP for example:

192.100.10.2:27015-27016:80:90u

And explode() would work for the above quite easily.

When talking about Inserting Deleting and updating.. We can simply create a class structure such as

struct port{
   int portnum;
   char type;

   port(int portnum = 0, char type = 't'){
       this.portnum = portnum; this.type = type;
   }
}

class Ip{

    public:
    string Ip_str;
    list <port> prt;
}

And then you can have the main to look like

int main(){

    list<Ip> Ips;

    //Read the list from file and update the list.

    //Sort delete update the list

    //Rewrite the list back into file in the order mentioned obove

    return 0;
}
Shahmir Javaid
+1: for range/protocol idea; I also agree with Robert Harvey about small memory usage, but Alon said to avoid working with addresses in memory.
Rubens Farias