views:

81

answers:

4

I'm not sure if im delluded but what I would like to do is create a method that will return the results of a query, so that i can reuse the connection code. As i understand it, a query returns an object but how do i pass that object back? I want to send the query into the method as a string argument, and have it return the results so that I can use them. Here's what i have which was a stab in the dark, it obviously doesn't work. This example is me trying to populate a listbox with the results of a query; the sheet name is Employees and the field/column is name. The error i get is "Complex DataBinding accepts as a data source either an IList or an IListSource.". any ideas?

 public Form1()
        {
            InitializeComponent();
            openFileDialog1.ShowDialog();
            openedFile = openFileDialog1.FileName;

            lbxEmployeeNames.DataSource = Query("Select [name] FROM [Employees$]");


        }

        public object Query(string sql)
        {
            System.Data.OleDb.OleDbConnection MyConnection;
            System.Data.OleDb.OleDbCommand myCommand = new System.Data.OleDb.OleDbCommand();
            string connectionPath;

            //build connection string
            connectionPath = "provider=Microsoft.Jet.OLEDB.4.0;Data Source='" + openedFile + "';Extended Properties=Excel 8.0;";

            MyConnection = new System.Data.OleDb.OleDbConnection(connectionPath);
            MyConnection.Open();
            myCommand.Connection = MyConnection;

            myCommand.CommandText = sql;
            return myCommand.ExecuteNonQuery();


        }
+1  A: 

Try ExecuteReader instead. It returns an object which can then be read like a file to get the results:

OleDbDataReader myReader = myCommand.ExecuteReader(CommandBehavior.CloseConnection);
while(myReader.Read()) 
{
   Console.WriteLine(myReader.GetString(0));
}
Travis Christian
says "specified cast is not valid"
Sinaesthetic
cast of what? there's no casting in this code fragment
Travis Christian
yea i know lol. but thats what its sayin! im thinking its an implicit cast from type "oledbdatareader" to string...
Sinaesthetic
A: 

The huge problem with the code as you posted it is that there's no way to correctly parameterize the query. You have to do string concatenation before calling your function, and that leaves you open to sql injection attacks. You need a way in your code to allow query parameters to come in separate from the sql string.

Some other problems in your sample include not correctly closing the connection (it will be left hanging if your query throws an exception) and calling the wrong ADO method.

I've put a lot of work into getting this right, I think I have the ideal pattern for what you want pretty well nailed in my answer to another question here:
http://stackoverflow.com/questions/2862428/fastest-method-for-sql-server-inserts-updates-selects/2862490#2862490

Basically, when you call the ADO function to actually run the query, you get back a DbDataReader. I use an iterator block to turn that data reader into an IEnumerable<IDataRecord> that works nice with linq and other code, and an Action<SqlCommand> to encourage correct query parameterization. So you abstract out your connection code to a method like this:

private static IEnumerable<IDataRecord> Retrieve(string sql, Action<SqlCommand> addParameters)
{
    using (var cn = new SqlConnection(ConnectionString))
    using (var cmd = new SqlCommand(sql, cn))
    {
        addParameters(cmd);

        cn.Open();
        using (var rdr = cmd.ExecuteReader())
        {
            while (rdr.Read())
                yield return rdr;
            rdr.Close();
        }
    }
}

And use it in code for the actual queries like this:

public IEnumerable<IDataRecord> GetSomeDataById(int MyId)
{
    return Retrieve(
        "SELECT * FROM [MyTable] WHERE ID= @MyID", 
       cmd =>
       {
          cmd.Parameters.Add("@MyID", SqlDbType.Int).Value = MyId;
       }
     );
}

Note that this let's you correctly parameterize the query, will always correctly close and dispose of your connections objects, sets you up to do pipelining between each of the layers in a 3-tier or service architecture (makes it fast), and does so with the minimum of code overhead.

Joel Coehoorn
i was aware of the lack of closing statement. i sort of see where you're coming from with your example post, but its a bit convoluded for my skill level lol. I rewrote my method and used a datatable as an intermediate. it works, using the same way of passing the sql query as i posted, but to test it, i had it populate the box from inside the method. The only issue, is in order to reuse this code, i need to be able to return that datatable to what called it. any ideas?
Sinaesthetic
@Sina it's not just that you don't call .Close(). Even if you _do_ call .Close() you can still leave connections hanging open when you have the inevitable query that throws an exception. And you shouldn't be writing **any** database code until you take the time to understand sql injection and how to do query parameters. **It is NOT okay to build sql queries through string concatenation!**
Joel Coehoorn
If this is beyond your skill level, think of it as a chance to get better. You don't have to use all the code I posted, but for closing your queries and parameterization: you do **need** to get those two things down up front or your projects will end up either broken or hacked or both. I'll add another answer that takes you code and walks you through step by step how to fix the important parts.
Joel Coehoorn
im not rejecting the idea of learning something new, i just dont know enough about how you approached it to pick it apart. Im also confused where you're saying im building queries by concatenation, because im not O_o. The only concatenation i used was in building the connection string so that i could specify the file name/path through an open file dialogue. And at the moment, the query itself is all that I need. I did finally get this to work. I just made the return type as DataTable, dumped the query results into the data adapter, filled the table, then returned the table.
Sinaesthetic
DataTable employees = Query("Select [name] FROM [Employees$]"); for (int i = 0; i < employees.Rows.Count; i++) { //add any item in the query table under the column named "name" string data1 = employees.Rows[i]["name"].ToString(); lbxEmployeeNames.Items.Add(data1); }
Sinaesthetic
i wish i could post code in comments like you can in the original post. id show you what i did so we could see if there was anything dangerous in it
Sinaesthetic
update the OP with your new code and explain what you're seeing now
Travis Christian
@Sina - right now you may not need to concatenate anything. But what happens when you need to add a where clause with a condition that's determined by the user? For example, letting the user enter a name to filter by. Your code makes no provision for that, and you'll be forced to use string concatenation on your sql query. It's hard to understand how bad that is.
Joel Coehoorn
A: 

Hey! Try this, If you just want to display the names of all employees into a listBox, this should work. I just edited some lines from your code...

Form1()
{
    InitializeComponent();
    openFileDialog1.ShowDialog();
    openedFile = openFileDialog1.FileName;

    lbxEmployeeNames.DataSource = Query("Select [name] FROM [Employees$]");
    lbxEmployeeNames.DisplayMember = "name"; // The column you want to be displayed in your listBox.
}

// Return a DataTable instead of String.
public DataTable Query(string sql)
{
    System.Data.OleDb.OleDbConnection MyConnection;
    string connectionPath;

    //build connection string
    connectionPath = "provider=Microsoft.Jet.OLEDB.4.0;Data Source='" + openedFile + "';Extended Properties=Excel 8.0;";

    MyConnection = new System.Data.OleDb.OleDbConnection(connectionPath);
    MyConnection.Open();
    System.Data.OleDb.OleDbDataAdapter myDataAdapter = new System.Data.OleDb.OleDbDataAdapter(sql, MyConnection);
    DataTable dt = new DataTable();
    myDataAdapter.Fill(dt);
    return dt;
}
yonan2236
this was one of the first things i tried! lol, but when i did this, the list box populates with references to the object instead of the actual contents. What i forgot was the displaymember! thanks :)
Sinaesthetic
+8  A: 

There are two fundamental things that every programmer must do when sending sql code to a database: close the connections and parameterize the queries. For some reason, most tutorials available on the internet just gloss over them or even get it just plain wrong, perhaps because it's so second nature to anyone advanced enough to write the tutorial. My goal here is to show you how to do both in a way that makes it easier to get this right every time.

The first thing to do is realize that hiding the code away in one method is not enough. We actually want to build a separate class for this. By creating a separate class, we can make our actual connection method private inside that class, so that only other methods in the class can connect to the database. This way, we force all database code in the program to run through your gatekeeper method. Get the gatekeeper method right with regards to the two issues I talked about above, and your whole program will consistently get it right, too. So here's our start:

public class DataLayer
{
   private DbConnection GetConnection()
   {
        //This could also be a connection for OleDb, ODBC, Oracle, MySQL, 
        // or whatever kind of database you have.
        //We could also use this place (or the constructor) to load the 
        // connection string from an external source, like a
        // (possibly-encrypted) config file
        return new SqlConnection("connection string here");
   }
}

Now, whenever you need to write a database query you add a method to that class. If it is a large project, you might even break the class up into several smaller classes in a dll file. But the main thing is that the class the provides this raw database connection should be private to other code that talks to the database.

But this is just to get us started. The next thing to do is adjust it to automate closing the connection. To do this, we can add a "Query" method to the class:

private DataTable Query(string sql)
{
     var result = new DataTable();
     using (var connection = GetConnection())
     using (var command = new SqlCommand(sql, connection)
     {
         connection.Open();
         result.Load(command.ExecuteReader(CommandBehavior.CloseConnection));
     }
     return result;
}

There's already a lot in here and some of it might be unfamiliar, so let's stop for a moment and look at those items. I'll start with the var keyword, which is simple enough. var is just a shorthand way to declare a variable. It means that the type name for the variable declaration should be inferred by the compiler. The variable is still strongly-typed, because the type of the variable is known and fixed when the program is compiled, but you the programmer don't have to go to the trouble of typing out a long name like "SqlConnection" when a simple "var" will do. This also makes it easy to write code that might be re-used on one project that uses a Sql Server database and on another project that uses an Access database, as the "var" keyword works with both declarations.

The other keyword I want to highlight is using. This keyword is a powerful way to declare a variable in .Net and C#. The keyword creates a scope block underneath the variable declaration. At the end of the scope block, your variable is disposed. Note that there are three important parts two this. The first is that this really only applies to unmanaged resources like database connections. Memory is still collected in the usual way. The second is that the variable is disposed even if an exception is thrown. This makes the keyword suitable to use with time-sensitive or tightly-constrained resources like database connections, without a separate try/catch block nearby. The final piece is that the keywords make use of the IDisposable pattern in .Net. You don't need to know all about IDisposable right now. Just know that database connections implement (like inheritance) IDisposable, and so will work with a using block.

You don't have to use the using keyword in your code. But if you don't, the correct way to handle a connection looks like this:

SqlConnection connection;
try
{
   connection = new SqlConnection("connection string here");
   SqlCommand command = new SqlCommand("sql query here", connetion);

   connection.Open();
   SqlDataReader reader = command.ExecuteReader(); 
   //do something with the data reader here
}
finally
{
    connection.Close();
}

And that's even the simple version. You actually still need an additional check in the finally block to make sure your connection variable is valid. The using keyword is a much more concise way to express this, and it makes sure you get the pattern right each time. The main thing to take away here is that if you just call connection.Close(), with no protection to make sure the program actually reaches that line, you've failed.

There's one more important thing with this method. Notice that it's still private. This is because we still don't want code outside of the class to call this method. A good programmer will not put any sql code in the same class as his windows form or asp.net page. This is a simple case of separation of concerns. Keep presentation in one place and data access in another. So, as I mentioned early, each of your queries will go in it's own method. Let's build an example now. Here's a method for a simple query to get all your employee data:

public DataTable GetEmployeeData()
{
    return Query("SELECT * FROM Employees");
}

Wow, that was easy. We're really getting somewhere. Unfortunately, we're still missing one piece of the puzzle. You see, it's pretty rare to want to return an entire table. Typically, you'll want to filter that table in some way, and maybe join it with another table. So let's alter this query to return all the data for a fictional employee named "Fred":

public DataTable GetFredsEmployeeData()
{
     return Query("SELECT * FROM Employees WHERE Firstname='Fred'");
}

Still pretty easy, but that misses the spirit of what we're trying to accomplish. You don't want to build another method for every possible employee name. You want something more like this:

public DataTable GetEmployeeData(string FirstName)
{
    return Query("SELECT * FROM Employees WHERE FirstName='" + FirstName + "'");
}

Uh oh. Now we have a problem. There's that pesky string concatenation, just waiting for someone to come along and enter the text ';Drop table employees;-- (or worse) into your first name textbox. The correct way to handle this is using query parameters. But this is where it gets tricky, because several paragraphs back we built a query method that only accepts a finished sql string.

A lot of people want to write a method just like that Query method. I think just about every database programmer is tempted by that pattern at a certain point in there career, and unfortunately it's just plain wrong until you add a way to accept parameters. Fortunately, there are number of different way to correct this. For example, you could pass an array of SqlParameters, or even just an array of objects. Or a set of key/value pairs to use to create the parameters.

I've spent a lot of time working through the different options, and I've narrowed down what I think is the simplest, most effective, and (more importantly) more accurate and maintainable option for C#. Unfortunately, it does require that you understand the syntax for one more advanced language feature in C#: anonymous methods. What this feature allows you to do is define a function within another function, hold on to it with a variable, pass it to other functions, and call it at your leisure. Here's how we'll modify the Query() function to take advantage of this:

private DataTable Query(string sql, Action<SqlCommand> addParameters)
{
     var result = new DataTable();
     using (var connection = GetConnection())
     using (var command = new SqlCommand(sql, connection)
     {
         addParameters(command); //<--- the addParameters parameter is a function we can call

         connection.Open(); 
         result.Load(command.ExecuteReader(CommandBehavior.CloseConnection));
     }
     return result;
}

Note the new Action<SqlCommand> parameter. Don't mind the < > part. If you're not familiar with generics, you can just pretend it's part of the class name for now. Just now that the special Action type here allows you to pass one function (in this case, one that takes an SqlCommand as an argument) to another function. And here's how we'd use this in the GetEmployeeData() function:

public DataTable GetEmployeeData(string firstName)
{
    return Query("SELECT * FROM Employees WHERE FirstName= @Firstname", 
    (cmd) => 
    {
        cmd.Parameters.AddWithValue("@FirstName", firstName);
    });
}

Now the Query() function has two parameters instead of one, and the expression for that second parameter looks a little goofy. This uses something called a lambda expression to create our special Action<SqlCommand> argument. You don't need to know much about lambda expressions yet. Just know that the => token is an operator defined in C#, just like +, -, *, /, &&, or ||. It tells the compiler that you're building a new method right there on the spot. You can even have curly-braces ( { and } ) to denote where it starts and stops.

The important part of all this is that the the Query() function now has a way to connect the firstName argument passed to it's parent GetEmployeeData() function to the @FirstName expression in the sql string. This is done using features built into ADO.Net and your sql database engine. And it happens in a way that prevents any possibility for sql injection attacks.

I'll finish (finally!) with two short items. The first is the syntax for calling your new query method with no parameters:

public DataTable GetAllEmployees()
{
    return Query("SELECT * FROM Employees", (cmd) => {});
}

While we could also provide this as an overload of the original Query() function, in my own code I prefer not to do that, as I want to communicate to other developers that should be looking to parameterize their code, and not sneak around with string concatenation.

Secondly, the code outlined in this answer is still unfinished. There are some important weaknesses yet to address. For example, using a datatable rather than a datareader forces you to load the entire result set from every query into memory all at once. There are some things we can do to avoid that. We also haven't discussed inserts, updates, deletes, or alters.

In conclusion, I'm not saying this is the only way to do it. But do remember the two things you must do: close your connections, and parameterize your queries.

Joel Coehoorn
+1 This would certainly make a better tutorial than most out there :)
BoltClock
+1 - great explanation! Here, I'll add chapter 2 to your book: "It's good to know everything you read in chapter 1, but just use NHibernate instead." :)
Scott Whitlock
@Scott Whitlock - Ah, I see you've read my full answer to this other question, then: http://stackoverflow.com/questions/2862428/fastest-method-for-sql-server-inserts-updates-selects/2862490#2862490
Joel Coehoorn
+1 to that one too! :)
Scott Whitlock
very informative
yonan2236
+1 for a well-written article
Travis Christian
Thanks. This wasn't what i used, but im definitely going to use this. Thanks for this, best explanation ive seen yet.
Sinaesthetic
Are there some assembly references missing here? I'm trying to use some of this and its showing errors all over the place. Like with the scope block under the using statement. The compiler thinks im trying to initialize a collection. It could be that you're using something earlier than VS2010. Would you be willing to make available the actual class you've created for this, with comments so that we could pick it apart?
Sinaesthetic
This should work with vs2010. You need system.data, system.collections, system.collections.generic, and system.linq.
Joel Coehoorn
Also, some of it was written assuming sql server, where you look you might be connection to an excel sheet (OLE). That change any type prefixed with 'Sql' to use 'OleDb' for the prefix instead.
Joel Coehoorn