Increasing memory usage in sqlite3 ?

Question

I've written a console app which receives events via boost::interprocess memory and dumps the info into an sqlite3 database. While running the app I've noticed that, in the Windows task manager, the memory usage was cyclically increasing every... 30s-1min. This led me to believe that the problem lies within the main loop in which I execute my SQL. I've added some monitoring and apparently the sqlite3_memory_usage returns increasing results every couple loop iterations.

Can somebody tell me what am I doing wrong? Am I missing something I should de-allocate ?

Here are 2 strings I use to generate SQL

    const std::string sql_insert =
      "INSERT INTO EventsLog "
      "(Sec, uSec, DeviceId, PmuId, EventId, Error, Msg) "
      "VALUES (%ld, %ld, %ld, %d, %ld, %d, %Q)";

    const std::string sql_create =
      "CREATE TABLE IF NOT EXISTS EventsLog("
      "Id INTEGER PRIMARY KEY AUTOINCREMENT, "
      "Sec INTEGER NOT NULL, "
      "uSec INTEGER NOT NULL, "
      "DeviceId INTEGER NOT NULL, "
      "PmuId INTEGER NOT NULL, "
      "EventId INTEGER NOT NULL, "
      "Error INTEGER NOT NULL, "
      "Msg TEXT"
      ")";

In here, I generate the SQL INSERT command

std::string construct_sql_query
    (const ELMessageData & data)
    {
      std::string query = "";

      ptime jan1st1970 = ptime(date(1970,1,1));
      ptime now = boost::posix_time::microsec_clock::universal_time();
      time_duration delta = now - jan1st1970;
      TimeVal time((uint32)delta.total_seconds(),
                   (uint32)now.time_of_day().fractional_seconds());

      char * const sql = sqlite3_mprintf(sql_insert.c_str(),
                                         time.tv_sec,
                                         time.tv_usec,
                                         data.getDeviceId(),
                                         data.getPmuId(),
                                         data.getEventId(),
                                         (data.getIsError() ? 1 : 0),
                                          data.getExMsg().c_str());
      if(sql == NULL)
        post_event(EvIOError("Failed to create the SQL command",
                             "StLoggingEvents::_construct_sql_query"));

      query = std::string(sql);
      sqlite3_free(sql);

      return query;
    } // construct_sql_query

Here's the main loop in which I execute the INSERT commands

 while(true)
    {
      m_exchange_obj->wait(); // wait for the semaphore to be raised
  const std::string sql = construct_sql_query
    (m_exchange_obj->receive());

  char ** err = NULL;

  const int rc = sqlite3_exec(m_db_handle,
                              sql.c_str(),
                              NULL,
                              NULL,
                              err);
  sqlite3_free(err);

  if(rc != SQLITE_OK)
  {
    LERR_ << "Error while inserting into the database";
    LERR_ << "Last SQL Query : ";
    LERR_ << sql;
  }
  else
  {
    LDBG_ << "Event logged...";
    LDBG_ << "Sqlite3 memory usage : "
          << sqlite3_memory_used();
  }
}

Answer 1

How are you determining your memory usage? You may not have a real leak.

If you are on a windows system and using visual studio, compile in debug mode and use the memory debugging macros to find your leaks.

If you are on a unix based system, try valgrind / memcheck.

I think OS X's xcode also detects leaks too.

Answer 2

I second the suggestion of trying this under valgrind. You may also want to look at google's tcmalloc replacement... It can print pretty graphs showing you all your leaks... That said, I hope you get the answer for this... I plan on using SQLite in an upcoming project...