views:

1227

answers:

3

I am trying to figure out how to read/write lock a CSV or XLS file when I read it as a Database via Jet OLEDB.

The following code will open a CSV as a DB and load it into a DataTable object:

        private DataTable OpenCSVasDB(string fullFileName)
        {
           string file = Path.GetFileName(fullFileName);
           string dir = Path.GetDirectoryName(fullFileName);
           string cStr = "Provider=Microsoft.Jet.OLEDB.4.0;"
               + "Data Source=\"" + dir + "\\\";"
               + "Extended Properties=\"text;HDR=Yes;FMT=Delimited\";";
           string sqlStr = "SELECT * FROM [" + file + "]";
           OleDbDataAdapter da;
           DataTable dt = new DataTable();
           try
           {
               da = new OleDbDataAdapter(sqlStr, cStr);
               da.Fill(dt);
           }
           catch { dt = null; }
        }

What I want to make sure of is that while I have the CSV or XLS file open, that I have a Read/Write LOCK on the Table (aka. the file), so that any other application that comes along and tries to read/write to this file has to wait its turn.

Does this happen automatically? If not, what do I need to do to make sure this does happen?

Btw, I'm working in C#/.NET 2.0, if that makes any difference...

Update: So, I'm clarifying my requirements now:

  • XLS file (because I need SELECT and UPDATE functionality) [CSV can only SELECT and INSERT]
  • LOCK the XLS file while the DB is Open. (can't have multiple threads and/or processes stepping on each other's changes...)
  • Read into DataTable object (for ease of working)
A: 

UPDATE: The following does not appear to lock my DB as I had hoped...

After much more digging, I found this page:

ADO Provider Properties and Settings

It says:

Jet OLEDB:Database Locking Mode

A Long value (read/write) that specifies the mode used when locking the database to read or modify records.

The Jet OLEDB:Database Locking Mode property can be set to any of the following values:

Page-level Locking 0

Row-level Locking 1

Note A database can only be open in one mode at a time. The first user to open the database determines the locking mode to be used while the database is open.

So I assume that my code would get changed to:

       string cStr = "Provider=Microsoft.Jet.OLEDB.4.0;"
           + "Data Source=\"" + dir + "\\\";"
           + "Extended Properties=\"text;HDR=Yes;FMT=Delimited\";"
           + "Jet OLEDB:Database Locking Mode=0";

Which should give me Page-level locking. If I wanted Row-level locking, I'd switch the value to 1.

Unfortunately, this doesn't actually appear to do any table/row/page locking when opening a CSV file as a DB.

Pretzel
+3  A: 

OLEDB's Jet driver locks flat files while there's an open OleDbDataReader to them. To verify this, look at the VerifyFileLockedByOleDB method in the code sample below. Note that having an open OleDbConnection is not enough-- you have to have an open Reader.

That said, your code posted above does not keep an open connection, since it uses OleDbDataAdapter.Fill() to quickly connect to the data source, suck out all the data, and then disconnect. The reader is never left open. The file is only locked for the (short) time that Fill() is running.

Furthermore, even if you open the reader yourself and pass it into DataTable.Load(), that method will close your DataReader for you once it's done, meaning that the file gets unlocked.

So if you really want to keep the file locked and still use a DataTable, you'll need to manually populate the datatable (schema and rows!) from an IDataReader, instead of relying on DataAdapter.Fill() or DataTable.Load().

Anyway, here's a code sample which shows:

  • your original code
  • an example which won't work because DataTable.Load() will close the DataReader and unlock the file
  • an alternate approach which will keep the file locked while you're working with the data, via operating at the row level using DataReader rather than using a DataTable

UPDATE: looks like keeping a DataReader open will prevent the same process from opening the file, but another process (e.g. Excel) can open (and write to!) the file. Go figure. Anyway, at this point I'd suggest, if you really want to keep the file locked, consider using something else besides OLEDB where you have more fine-grained control over how (adn when!) the file is opened and closed. I'd suggest the CSV reader fromhttp://www.codeproject.com/KB/database/CsvReader.aspx, which is well-tested and fast, but will give you the source code so if you need to change file-locking/opening/closing, you can do so.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Data.OleDb;
using System.Data;
using System.IO;

namespace TextFileLocking
{
    class Program
    {
        private static DataTable OpenCSVasDB(string fullFileName)
        {
            string file = Path.GetFileName(fullFileName);
            string dir = Path.GetDirectoryName(fullFileName);
            string cStr = "Provider=Microsoft.Jet.OLEDB.4.0;"
                + "Data Source=\"" + dir + "\\\";"
                + "Extended Properties=\"text;HDR=Yes;FMT=Delimited\";";
            string sqlStr = "SELECT * FROM [" + file + "]";
            OleDbDataAdapter da;
            DataTable dt = new DataTable();
            try
            {
                da = new OleDbDataAdapter(sqlStr, cStr);
                da.Fill(dt);
            }
            catch { dt = null; }

            return dt;
        }
        private static DataTable OpenCSVasDBWithLockWontWork(string fullFileName, out OleDbDataReader reader)
        {
            string file = Path.GetFileName(fullFileName);
            string dir = Path.GetDirectoryName(fullFileName);
            string cStr = "Provider=Microsoft.Jet.OLEDB.4.0;"
                + "Data Source=\"" + dir + "\\\";"
                + "Extended Properties=\"text;HDR=Yes;FMT=Delimited\";";
            string sqlStr = "SELECT * FROM [" + file + "]";
            OleDbConnection openConnection = new OleDbConnection(cStr);
            reader = null;
            DataTable dt = new DataTable();
            try
            {
                openConnection.Open();
                OleDbCommand cmd = new OleDbCommand(sqlStr, openConnection);
                reader = cmd.ExecuteReader();
                dt.Load (reader);       // this will close the reader and unlock the file!
                return dt;  
            }
            catch 
            { 
                return null; 
            }
        }
        private static void OpenCSVasDBWithLock(string fullFileName, Action<IDataReader> dataRowProcessor)
        {
            string file = Path.GetFileName(fullFileName);
            string dir = Path.GetDirectoryName(fullFileName);
            string cStr = "Provider=Microsoft.Jet.OLEDB.4.0;"
                + "Data Source=\"" + dir + "\\\";"
                + "Extended Properties=\"text;HDR=Yes;FMT=Delimited\";";
            string sqlStr = "SELECT * FROM [" + file + "]";
            using (OleDbConnection conn = new OleDbConnection(cStr))
            {
                OleDbCommand cmd = new OleDbCommand(sqlStr, conn);
                conn.Open();
                using (OleDbDataReader reader = cmd.ExecuteReader())
                {
                    while (reader.Read())
                    {
                        dataRowProcessor(reader);
                    }
                }
            }
        }
        private static void VerifyFileLockedByOleDB(string fullFileName)
        {
            string file = Path.GetFileName(fullFileName);
            string dir = Path.GetDirectoryName(fullFileName);
            string cStr = "Provider=Microsoft.Jet.OLEDB.4.0;"
                + "Data Source=\"" + dir + "\\\";"
                + "Extended Properties=\"text;HDR=Yes;FMT=Delimited\";";
            string sqlStr = "SELECT * FROM [" + file + "]";
            using (OleDbConnection conn = new OleDbConnection(cStr))
            {
                OleDbCommand cmd = new OleDbCommand(sqlStr, conn);
                conn.Open();
                using (OleDbDataReader reader = cmd.ExecuteReader())
                {
                    File.OpenRead(fullFileName);   // should throw an exception

                    while (reader.Read())
                    {
                        File.OpenRead(fullFileName);   // should throw an exception

                        StringBuilder b = new StringBuilder();
                        for (int i = 0; i < reader.FieldCount; i++)
                        {
                            b.Append(reader.GetValue(i));
                            b.Append(",");
                        }
                        string line = b.ToString().Substring(0, b.Length - 1);
                        Console.WriteLine(line);
                    }
                }
            }
        }

        static void Main(string[] args)
        {
            string filename = Directory.GetCurrentDirectory() + "\\SomeText.CSV";
            try
            {
                VerifyFileLockedByOleDB(filename);
            }
            catch { }   // ignore exception due to locked file

            OpenCSVasDBWithLock(filename, delegate(IDataReader row)
            {
                StringBuilder b = new StringBuilder();
                for (int i = 0; i <row.FieldCount; i++)
                {
                    b.Append(row[i].ToString());
                    b.Append(",");
                }
                string line = b.ToString().Substring(0, b.Length - 1);
                Console.WriteLine(line);
            });

        }
    }
}
Justin Grant
Ok, so the OleDbDataAdapter.Fill() opens, locks, fills, releases lock, and closes in one motion. I wasn't understanding that before, but it makes sense now. How do I get that data into a DataTable object? (Incidentally, I've discovered that CSV file is not the way to go for my project because I need to UPDATE rows as well, and unfortunately, CSV files can only SELECT and INSERT.)
Pretzel
Hi Ranger - I expanded the answer above to cover your question about how you get a DataTable which still locks the connection for the full time you're processing the data.
Justin Grant
Ok, I'll take a look at this later and get back to you. Thanks for the help!
Pretzel
Ok, I tried pasting your OpenCSVasDBWithLock() method into my code and it does open/fill the DataTable, but it certainly doesn't lock the file. I was able to launch a 2nd copy of my application and open the same file. Similarly, I was able to go to the directory and read the file in via notepad.exe -- I stopped debugging of the first instance of my application and it claims that the OleDbConnection is still in an Open state. I'm not sure what to make of it...
Pretzel
hmmm, indeed you have discovered an oddity. I'm updating my answer now. See above in about 5 minutes.
Justin Grant
OK, look now. turns out that DataAdapter.Fill() and DataTable.Load() will both close the underlying DataReader and hence unlock the file-- with this provider, the locking happens on query, not on calling Open(). This makes sense since the filename is inside the query string, not the connection string.
Justin Grant
The same phenomenon happens with XLS files, though, and the filename is in the connection string rather than the query string. I think that the .NET code automatically closes the DB to prevent people from screwing up, but in my case, I want to actually keep the lock.Anyway, thanks for the help. I guess I'll have to figure out how this DataReader is supposed to work.Cheers.
Pretzel
I'm not familiar with the Action<> delegate. This appears to be a .NET 3 thing (which I haven't "graduated" to yet...) I guess I don't need to know what it is other than another method to handle the IDataReader interface type that's passed in. Right?
Pretzel
Oops, sorry -- Action is a simple delegate defined in .NET 3.0 which returns void and accepts a single template parameter. You can simply define it yourself. It's definition is this: public delegate void Action<T>(T obj);
Justin Grant
See my comments below in the answer below. There's still an issue with locking. At this rate, I may just abandon trying to use OleDB. It seems to be more trouble than it is worth. I guess what I'm having trouble believing is that I'm the only person who wants to have his DB locked while working on it. I guess most people just use real DBs and not CVS/XLS files.
Pretzel
Hmmm. THat is really odd-- if you try to open the file in the same process, it fails, but Excel can open it. At this point, I agree with you-- switching to another, simpler CSV reader (not OLEDB) will give you fine-grained control over when files are opened, closed, and locked. I edited my answer to include a reference to a popular open-source C# CSV reader, which is probably better than continuing to mess around with OLEDB here.
Justin Grant
Cool, man. Thanks for all your help.
Pretzel
no prob, happy to help!
Justin Grant
A: 

Ok, so that new function you wrote kinda works, but I still end up with a "Race condition" which then causes an exception to be thrown. So in this section of the code:

        using (OleDbConnection conn = new OleDbConnection(cStr))
        {
            OleDbCommand cmd = new OleDbCommand(sqlStr, conn);
            conn.Open();
            using (OleDbDataReader reader = cmd.ExecuteReader())
            {
                while (reader.Read())
                {
                    reader.GetString(0);  // breakpoint here
                }
            }
        }

I put a breakpoint on the line with the comment "breakpoint here" and then ran the program. I then located the CSV file in File Explorer and tried to open it with Excel. It causes Excel to wait for the file to be unlocked, which is good.

But here's the bad part. When I clear the breakpoint and then tell it to continue debugging, Excel sneaks in, grabs a lock on the file and causes an exception in my running code.

(The exception is: The Microsoft Jet database engine cannot open the file ''. It is already opened exclusively by another user, or you need permission to view its data.)

I guess I can always wrap that code in a try-catch block, but when the exception occurs, I won't know if it is a legitimate exception or one caused by this weird condition.

The exception seems to occur when the Reader is finished reading. (after it reads the last row, but still is in the "using (OleDbDataReader reader = cmd.ExecuteReader())" loop.

Pretzel