I have a "Cron Service" that i've had running on our production server for over a year and a half.
It's a regular windows service which every minute connects to the DB, checks on one table whether there's something to do, and if so, does it. This is a server for a website that's not used a lot, so most of the time the service is completely idle.
A couple months ago, this service started throwing the following exception and dying:
MESSAGE: Object reference not set to an instance of an object.
SOURCE: MySql.Data
TARGETSITE: MySql.Data.MySqlClient.Driver CheckoutConnection()
STACKTRACE: at MySql.Data.MySqlClient.MySqlPool.CheckoutConnection()
at MySql.Data.MySqlClient.MySqlPool.GetPooledConnection()
at MySql.Data.MySqlClient.MySqlPool.TryToGetDriver()
at MySql.Data.MySqlClient.MySqlPool.GetConnection()
at MySql.Data.MySqlClient.MySqlConnection.Open()
at DBA.Connect() in D:\xxx\DBA.vb:line 21
at CronService.TaskMailings.OnExecute() in D:\xxx\TaskMailings.vb:line 54
at CronService.CronTask.ThreadExecute() in D:\xxx\CronTask.vb:line 99
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart()
The exception is clearly coming from inside the MySQL Library. Also, my function that calls it is pretty simple:
Public Shared Function Connect() As MySqlConnection
Dim strConnString As String = WebConfigurationManager.AppSettings("ConnectionString")
Dim DBConn As New MySqlConnection(strConnString)
DBConn.Open()
Return DBConn
End Function
And the function where I'm calling DBA.Connect() (CronService.TaskMailings.OnExecute) is doing so with a "Using" statement, so the connection is getting closed, this shouldn't be a problem of connections being left open and getting exhausted (although it might, if there's a bug inside the MySQL library).
This is the connection string I'm using for the Service:
server=localhost;port=13306;uid=xxx;pwd=xxx;database=xxx;Pooling=True;charset=utf8;
Any ideas what might be going on here? The strangest part is that this has worked for well over a year before it started happening out of the blue, and load to the server hasn't changed enough to justify it being a load problem, I believe.
EDIT: Some of the times it dies, it gives this other error message:
MESSAGE: error connecting: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
SOURCE: MySql.Data
TARGETSITE: MySql.Data.MySqlClient.Driver GetConnection()
STACKTRACE: at MySql.Data.MySqlClient.MySqlPool.GetConnection()
at MySql.Data.MySqlClient.MySqlConnection.Open()
atDBA.Connect() in D:\xxx\DBA.vb:line 21
Although the error message is pretty clear, like I mentioned before, we are releasing all connections, so this shouldn't happen. Any ideas?