views:

278

answers:

1

Lets say i have a simple array of x rows and y columns with corresponding values, What is the best method to do 3 things? How to insert, update a value at a specific row column? How to select a value for each row and column,

import sqlite3
con = sqlite3.connect('simple.db')
c = con.cursor()
c.execute('''create table simple (links text)''')
con.commit()

dic = {'x1':{'y1':1.0,'y2':0.0},'x2':{'y1':0.0,'y2':2.0,'y3':1.5},'x3':{'y2':2.0,'y3':1.5}}
ucols = {}
## my current thoughts are collect all row values and all column values from dic and populate table row and columns accordingly how to call by row and column i havn't figured out yet
##populate rows in first column
for row in dic:
    print row
    c.execute("""insert into simple ('links') values ('%s')"""%row)
con.commit()

##unique columns
for row in dic:
    print row
    for col in dic[row]:
        print col
        ucols[col]=dic[row][col]

##populate columns    
for col in ucols:
    print col
    c.execute("alter table simple add column '%s' 'float'" % col)
con.commit()

#functions needed
##insert values into sql by row x and column y?how to do this  e.g. x1 and y2 should put in 0.0
##I tried as follows didn't work
for row in dic:
    for col in dic[row]:
        val =dic[row][col]
        c.execute("""update simple SET '%s' = '%f' WHERE 'links'='%s'"""%(col,val,row))
con.commit()

##update value at a specific row x and column y?


## select a value at a specific row x and column y?
+1  A: 

So you have a dictionary of dictionaries, that you want to convert into a SQL table.

Steps I'd take

  1. Find the columns you'll need.
  2. Create the table schema.
  3. Loop through each row.
    1. Compile the set of values for each column.
    2. Insert it.

So:

import sqlite3
con = sqlite3.connect('simple.db')
c = con.cursor()

dic = {
    'x1':{'y1':1.0,'y2':0.0},
    'x2':{'y1':0.0,'y2':2.0,'y3':1.5},
    'x3':{'y2':2.0,'y3':1.5}
    }

# 1. Find the unique column names.
columns = set()
for _, cols in dic.items():
    for key, _ in cols.items():
       columns.add(key)

# 2. Create the schema.
col_defs = [
    # Start with the column for our key name
    '"row_name" VARCHAR(2) NOT NULL PRIMARY KEY'
    ]
for column in columns:
    col_defs.append('"%s" REAL NULL' % column)
schema = "CREATE TABLE simple (%s);" % ",".join(col_defs)
c.execute(schema)

# 3. Loop through each row
for row_name, cols in dic.items():

    # Compile the data we have for this row.
    col_names = cols.keys()
    col_values = [str(val) for val in cols.values()]

    # Insert it.
    sql = 'INSERT INTO simple ("row_name", "%s") VALUES ("%s", "%s");' % (
        '","'.join(col_names),
        row_name,
        '","'.join(col_values)
        )
    c.execute(sql)

Then your other questions are pretty simple:

## update value at a specific row x and column y?
def set_cell(connection, x_name, y_name, value):
    sql = 'UPDATE simple SET %s="%s" WHERE row_name="%s"' % (
        y_name, value, x_name
        )
    connection.execute(sql)

## select a value at a specific row x and column y?
def get_cell(connection, x_name, y_name):
    sql = 'SELECT %s FROM simple WHERE row_name="%s"' % (
        y_name, x_name
        )
    # Return the first row of results (there should be only one)
    # and the first column from that row
    return list(connection.execute(sql))[0][0]
Ian
thanks alot for your detailed answer, nice to see great code!
Cool, thanks. If it was useful, would you mind accepting my answer?
Ian
Oh ya didn't see it, i tried upvoting but don't have the rep for it yet, lol, one other question I am grappling with, should i put this as another answer? If i change the column values in the dict to lets say 'y3' to 'some random string' I can't get the code to work. I tried changing several things instead of using real I used text, but when I call it says the column doesn't exist any thoughts?
never mind :), just seen the sql = 'SELECT %s needs to be "%s" for a string, pesky sql and its touchy syntax lol
I've run into a problem, with the schema approach of populating the columns if there are a lot of columns in the dictionary, I get this error c.execute(schema)OperationalError: too many columns on dic any thought on how to get around this?
This generally isn't a good approach if (a) you have lots of data, or (b) you need to change columns at all.(a) is just unsolvable, sqlite has a hard limit on the number of columns, and (b) is a mess. You need to issue an sql alter column statement.Much better would be the approach in the code from my first answer. Use a table with three columns:CREATE TABLE simple ( "x_key" varchar(16), "y_key" varchar(16), "value" real null, unique(x_key, y_key));(b) is still tricky, but much less so. Ask a new question for it, if you like.
Ian