How can I UPDATE rows returned by a SELECT in a loop?

Question

I have a loop on the rows returned by an SQL SELECT statement, and, after some processing on a row's data, I sometimes want to UPDATE the row's value. The processing in the loop's body is non-trivial, and I can't write it in SQL. When I try to execute the UPDATE for the selected row I get an error (under Perl's DBD::SQLite::st execute failed: database table is locked). Is there a readable, efficient, and portable way to achieve what I'm trying to do? Failing that, is there a DBD or SQLite-specific way to do it?

Obviously, I can push the updates in separate data structure and execute them after the loop, but I'd hate the code's look after that.

If you're interested, here is the corresponding Perl code.

my $q = $dbh->prepare(q{
  SELECT id, confLoc FROM Confs WHERE confLocId ISNULL});
$q->execute or die;
my $u = $dbh->prepare(q{
  UPDATE Confs SET confLocId = ? WHERE id = ?});
while (my $r = $q->fetchrow_hashref) {
 next unless ($r->{confLoc} =~ m/something-hairy/);
 next unless ($locId = unique_name_state($1, $2));
 $u->execute($locId, $r->{id}) or die;
}

Answer 1

Temporarily enable AutoCommit:

sqlite> .header on
sqlite> select * from test;
field
one
two

#!/usr/bin/perl

use strict;
use warnings;

use DBI;

my $dbh = DBI->connect('dbi:SQLite:test.db', undef, undef,
    { RaiseError => 1, AutoCommit => 0}
);

test_select_with_update($dbh);

sub test_select_with_update {
    my ($dbh) = @_;
    local $dbh->{AutoCommit} = 1;
    my $q = $dbh->prepare(q{SELECT field FROM test});
    my $u = $dbh->prepare(q{UPDATE test SET field = ? WHERE field = ?});
    $q->execute or die;
    while ( my $r = $q->fetchrow_hashref ) {
        if ( (my $f = $r->{field}) eq 'one') {
            $u->execute('1', $f) or die;
        }
    }
}

After the code has been run:

sqlite> .header on
sqlite> select * from test;
field
1
two

Answer 2

More in answer to Zoidberg's comment but if your were able to switch to an ORM like Perl's DBIx::Class then you find that you could write something like this:

my $rs = $schema->resultset('Confs')->search({ confLocId => undef });

while ( my $data = $rs->next ) {
    next unless $data->confLoc =~ m/(something)-(hairy)/;
    if ( my $locId = unique_name_state( $1, $2 ) ) {
        $data->update({ confLocID => $locid });
    }
}

And if DBIx::Class doesn't grab your fancy there are a few others on CPAN like Fey::ORM and Rose::DB for example.

/I3az/

Answer 3

Your problem is that you're using the same database handler to perform an update while you're in a fetching loop.

So have another instance of your database handler to perform the updates:

my $dbh = DBI->connect(...);
my $dbhForUpdate = DBI->connect(...) ;

Then use dbhForUpdate in your loop:

while(my $row = $sth->fetch()){
   ...
   $dbhForUpdate->do(...) ;
}

Anyway, I wouldn't recommend doing this since there's good chances you run into concurrency issues at the database level.