Read large xml file and extract it's data into a form application in C#.Net project

Question

I want to read a xml file data and extract information from it to show in a form in a C#.Net vs2008 env. I can't read this data into a list in Memory, but I want to have a .exe file from project that can run in another computer System. So I think, I can't use database for saving and retrieving data! Please help me to solve my problem!

Answer 1

Recently microsoft provide a synidcation class in WCF. you can use it for doing this task

Answer 2

Use System.Xml.XmlReader to read the XML file. This is a forward only reader which only loads a bit of XML at a time.

Combine this with an iterator method which produces an IEnumerable, such as this example, where MyXmlData is a class representing what is held in the XML file and a class that your forms can work with:

public IEnumerable<MyXmlData> QueryFile(String xmlFile)
{
    using (var reader = XmlReader.Create(xmlFile))
    {
        // These are the variables you want to read for each 'object' in the
        // xml file.
        var prop1 = String.Empty;
        var prop2 = 0;
        var prop3 = DateTime.Today;

        while (reader.Read())
        {
            // Here you'll have to read an xml node at a time.
            // As you read, assign appropriate values to the variables
            // declared above.

            if (/* Have we finished reading an item? */)
            {
                // Once you've completed reading xml representing a single
                // MyXmlData object, return it using yield return.
                yield return new MyXmlData(prop1, prop2, prop3);
            }
        }
    }
}

The value returned from that method is a sequence of MyXmlData objects where each one will be created on demand as the file is read, one at a time. This will greatly reduce memory requirements for a large XML file.

You can then query your MyXmlData objects using Linq functions. For example, you can emulate paging by using the Take and Skip methods.

// Third page - 50 objects per page.
QueryFile(@"x.xml").Skip(100).Take(50);

Answer 3

You should look into VTD-XML, it is the most efficient xml parser in terms of memory usage without losing random access and xpath..

http://vtd-xml.sf.net