Python and SQLite: insert into table
Asked Answered
P

8

67

Having the following table schema:

tablename(
    name varchar[100], 
    age int, 
    sex char[1]
)

Having a list that has 3 rows each representing a table row:

row1 = [laks,444,M]
row2 = [kam,445,M]
row3 = [kam,445,M]

Or should I use something other than list?

Here is the actual code part:

    for record in self.server:
        print "--->",record
        t=record
        self.cursor.execute("insert into server(server) values (?)",(t[0],));
        self.cursor.execute("insert into server(id) values (?)",(t[1],))
        self.cursor.execute("insert into server(status) values (?)",(t[2],));

Inserting the three fields separately works, but using a single line like these 2 example doesn't work.

self.cursor.execute("insert into server(server,c_id,status) values (?,?,?)",(t[0],),(t[1],),(t[2],))
self.cursor.execute("insert into server(server,c_id,status) values (?,?,?)",(t),)
Pastoral answered 19/1, 2010 at 10:31 Comment(1)
Its fixed now. I have used this wrong method self.cursor.execute("insert into server(server,c_id,status) values (?,?,?)",(t[0],),(t[1],),(t[2],)) right method is self.cursor.execute("insert into server(server,c_id,status) values (?,?,?)",(t[0],t[1],t[2],))Pastoral
A
49
conn = sqlite3.connect('/path/to/your/sqlite_file.db')
c = conn.cursor()
for item in my_list:
  c.execute('insert into tablename values (?,?,?)', item)
Aindrea answered 19/1, 2010 at 10:36 Comment(7)
Thanks Dyno and Dominic - But it's not working - this is what i'm trying ----------- for record in list: print "--->",record cursor.execute("insert into process values (?,?,?)",record); ----------- getting errorPastoral
I have try, except part - try part will have for loop and execute stmt and except part has single message saying "insert failed" ...it just prints the message from except part and quits. How to debug this more effectively ?Pastoral
@lakshmipathi: If your exception handling is chewing up the error message then it isn't really handling it...Tersina
how to print error number .. atleast that would provide some hintPastoral
It wont' insert unless we use conn.commit()Vannoy
Hey guys, I so stuck and cannot for the life figure out what is going on. I cannot insert anything into my db from a list value. If i use my loop to insert say 'item' where the '?' is in values and dont have the item variable i will have inserted the right number of element into the db but with the text item. If I change it to the above to use the item in the list, then nothing is inserted. cur.execute("INSERT INTO TABLENAME(name) VALUES('item')") WORKS cur.execute("INSERT INTO TABLENAME(name) VALUES(?)",(item)) DOESNT WORK Any advice? I am stumpedEy
I think it isn't really recommended to leave out the column names. sql - Insert Query: Why is it a bad idea to not include column names? - Stack OverflowGiavani
L
98

there's a better way

# Larger example
rows = [('2006-03-28', 'BUY', 'IBM', 1000, 45.00),
        ('2006-04-05', 'BUY', 'MSOFT', 1000, 72.00),
        ('2006-04-06', 'SELL', 'IBM', 500, 53.00)]
c.executemany('insert into stocks values (?,?,?,?,?)', rows)
connection.commit()
Limpid answered 16/5, 2011 at 15:35 Comment(5)
I have a query about this because I have tried it but I get and error saying I have 6 columns and only provided information for 5. This is true but the 1st column is pk integer and you don't need to provide it with a value. Is there a way to over come this? –Gramophone
Figured it out. Just add the field headers after the db name as you would normally.Gramophone
Thx i used an approach based on this anwer in the wrapper github.com/WolfgangFahl/DgraphAndWeaviateTest/blob/master/…Ylem
@WolfgangFahl's wrapper now can be found here.Keon
@Keon thx for tixing my comment - in fact there is now a library pyLoDStorage that has the code and can be integrated via pypi github.com/WolfgangFahl/pyLoDStorage/blob/master/lodstorage/…Ylem
A
49
conn = sqlite3.connect('/path/to/your/sqlite_file.db')
c = conn.cursor()
for item in my_list:
  c.execute('insert into tablename values (?,?,?)', item)
Aindrea answered 19/1, 2010 at 10:36 Comment(7)
Thanks Dyno and Dominic - But it's not working - this is what i'm trying ----------- for record in list: print "--->",record cursor.execute("insert into process values (?,?,?)",record); ----------- getting errorPastoral
I have try, except part - try part will have for loop and execute stmt and except part has single message saying "insert failed" ...it just prints the message from except part and quits. How to debug this more effectively ?Pastoral
@lakshmipathi: If your exception handling is chewing up the error message then it isn't really handling it...Tersina
how to print error number .. atleast that would provide some hintPastoral
It wont' insert unless we use conn.commit()Vannoy
Hey guys, I so stuck and cannot for the life figure out what is going on. I cannot insert anything into my db from a list value. If i use my loop to insert say 'item' where the '?' is in values and dont have the item variable i will have inserted the right number of element into the db but with the text item. If I change it to the above to use the item in the list, then nothing is inserted. cur.execute("INSERT INTO TABLENAME(name) VALUES('item')") WORKS cur.execute("INSERT INTO TABLENAME(name) VALUES(?)",(item)) DOESNT WORK Any advice? I am stumpedEy
I think it isn't really recommended to leave out the column names. sql - Insert Query: Why is it a bad idea to not include column names? - Stack OverflowGiavani
I
15

Not a direct answer, but here is a function to insert a row with column-value pairs into sqlite table:

def sqlite_insert(conn, table, row):
    cols = ', '.join('"{}"'.format(col) for col in row.keys())
    vals = ', '.join(':{}'.format(col) for col in row.keys())
    sql = 'INSERT INTO "{0}" ({1}) VALUES ({2})'.format(table, cols, vals)
    conn.cursor().execute(sql, row)
    conn.commit()

Example of use:

sqlite_insert(conn, 'stocks', {
        'created_at': '2016-04-17',
        'type': 'BUY',
        'amount': 500,
        'price': 45.00})

Note, that table name and column names should be validated beforehand.

Ireneirenic answered 17/4, 2016 at 15:17 Comment(1)
What means ':' in vals before fieldnames? And how to validate the names?Unipod
K
8

Adapted from http://docs.python.org/library/sqlite3.html:

# Larger example
for t in [('2006-03-28', 'BUY', 'IBM', 1000, 45.00),
          ('2006-04-05', 'BUY', 'MSOFT', 1000, 72.00),
          ('2006-04-06', 'SELL', 'IBM', 500, 53.00),
         ]:
    c.execute('insert into stocks values (?,?,?,?,?)', t)
Kilocycle answered 19/1, 2010 at 10:34 Comment(0)
A
1

Adding to the provided answer, I think there are even better improvements.

1) Using Dictionary instead of list

Why?, better readability, maintainability and possibly speed (getting from a dictionary is way faster than access data via list indexing, in large list)

import sqlite3

connection: sqlite3.Connection = sqlite3.connect(":memory:")
connection.row_factory = sqlite3.Row    # This will make return a dictionary like object instead of tuples
cursor: sqlite3.Cursor = connection.cursor()

# create db
cursor.execute("CREATE TABLE user(name VARCHAR[100] UNIQUE, age INT, sex VARCHAR[1])")

data = (
    {"name": "laks", "age": 444, "sex": "M"},
    {"name": "kam", "age": 445, "sex": "M"},
    {"name": "kam2", "age": 445, "sex": "M"},
)
cursor.executemany("INSERT INTO user VALUES(:name, :age, :sex)", data)

cursor.execute("SELECT * FROM user WHERE age >= ?", (445, ))
users = cursor.fetchall()
for user in users:
    print(user["name"], user["age"], user["sex"])

Output

kam 445 M
kam2 445 M

2) Use a context manager to automatically commit - rollback

This is way better to handle transaction automatically and rollback.

import sqlite3

connection: sqlite3.Connection = sqlite3.connect(":memory:")
connection.row_factory = sqlite3.Row    # This will make return a dictionary like object instead of tuples
cursor: sqlite3.Cursor = connection.cursor()

# create db
# con.rollback() is called after the with block finishes with an exception,
# the exception is still raised and must be caught
try:
    with connection:
        cursor.execute("CREATE TABLE user(name VARCHAR[100] UNIQUE, age INT, sex VARCHAR[1])")
except sqlite3.IntegrityError as error:
    print(f"Error while creating the database, {error}")

data = (
    {"name": "laks", "age": 444, "sex": "M"},
    {"name": "kam", "age": 445, "sex": "M"},
    {"name": "kam2", "age": 445, "sex": "M"},
)

# con.rollback() is called after the with block finishes with an exception,
# the exception is still raised and must be caught
try:
    with connection:
        cursor.executemany("INSERT INTO user VALUES(:name, :age, :sex)", data)
except sqlite3.IntegrityError as error:
    print(f"Error while creating the database, {error}")

# Will add again same data, will throw an error now!
try:
    with connection:
        cursor.executemany("INSERT INTO user VALUES(:name, :age, :sex)", data)
except sqlite3.IntegrityError as error:
    print(f"Error while creating the database, {error}")


cursor.execute("SELECT * FROM user WHERE age >= ?", (445, ))
users = cursor.fetchall()
for user in users:
    print(user["name"], user["age"], user["sex"])

Output

Error while creating the database, UNIQUE constraint failed: user.name
kam 445 M
kam2 445 M

Documentatin

Abash answered 29/4, 2023 at 18:12 Comment(0)
X
0

This will work for a multiple row df having the dataframe as df with the same name of the columns in the df as the db.

tuples = list(df.itertuples(index=False, name=None))

columns_list = df.columns.tolist()
marks = ['?' for _ in columns_list]
columns_list = f'({(",".join(columns_list))})'
marks = f'({(",".join(marks))})'

table_name = 'whateveryouwant'

c.executemany(f'INSERT OR REPLACE INTO {table_name}{columns_list} VALUES {marks}', tuples)
conn.commit()
Xanthochroid answered 5/9, 2019 at 20:31 Comment(0)
S
0

rowPosition = 0 for one_row in cursor:

        self.tableWidget.insertRow(rowPosition)
        self.tableWidget.setItem(rowPosition,0,QtWidgets.QTableWidgetItem(str(one_row[0])))
        self.tableWidget.setItem(rowPosition,1,QtWidgets.QTableWidgetItem(str(one_row[1])))
        self.tableWidget.setItem(rowPosition,2,QtWidgets.QTableWidgetItem(str(one_row[2])))
        self.tableWidget.setItem(rowPosition,3,QtWidgets.QTableWidgetItem(str(one_row[3])))
        rowPosition+=1
Subdivide answered 22/6, 2023 at 8:59 Comment(1)
Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.Mortification
O
-5
#The Best way is to use `fStrings` (very easy and powerful in python3)   
#Format: f'your-string'   
#For Example:

mylist=['laks',444,'M']

cursor.execute(f'INSERT INTO mytable VALUES ("{mylist[0]}","{mylist[1]}","{mylist[2]}")')

#THATS ALL!! EASY!!
#You can use it with for loop!
Oakie answered 22/6, 2019 at 20:39 Comment(1)
Please note that this approach is vulnerable to SQL injection attacks.Thresher

© 2022 - 2024 — McMap. All rights reserved.