SQLAlchemy IN clause

Question

I'm trying to do this query in sqlalchemy

SELECT id, name FROM user WHERE id IN (123, 456)

I would like to bind the list [123, 456] at execution time.

Halfdan

Answer 1

How about

session.query(MyUserClass).filter(MyUserClass.id.in_((123,456))).all()

edit: Without the ORM, it would be

session.execute(
    select(
        [MyUserTable.c.id, MyUserTable.c.name], 
        MyUserTable.c.id.in_((123, 456))
    )
).fetchall()

select() takes two parameters, the first one is a list of fields to retrieve, the second one is the where condition. You can access all fields on a table object via the c (or columns) property.

Whodunit

Answer 2

Assuming you use the declarative style (i.e. ORM classes), it is pretty easy:

query = db_session.query(User.id, User.name).filter(User.id.in_([123,456]))
results = query.all()

db_session is your database session here, while User is the ORM class with __tablename__ equal to "users".

Seda

Use ~ (~User.id.in_([123,456])) or not_ from sqlalchemy.sql.expression (not_(User.id.in_([123,456]))). – Seda

Here is a link to the doc. – Rufena

Answer 3

An alternative way is using raw SQL mode with SQLAlchemy, I use SQLAlchemy 0.9.8, python 2.7, MySQL 5.X, and MySQL-Python as connector, in this case, a tuple is needed. My code listed below:

id_list = [1, 2, 3, 4, 5] # in most case we have an integer list or set
s = text('SELECT id, content FROM myTable WHERE id IN :id_list')
conn = engine.connect() # get a mysql connection
rs = conn.execute(s, id_list=tuple(id_list)).fetchall()

Hope everything works for you.

Supramolecular

If you use raw SQL for such simple queries you are better of using psycopg2 or other connector directly. – Girhiny

Update is not working – Cotyledon

Answer 4

Just wanted to share my solution using sqlalchemy and pandas in python 3. Perhaps, one would find it useful.

import sqlalchemy as sa
import pandas as pd
engine = sa.create_engine("postgresql://postgres:my_password@my_host:my_port/my_db")
values = [val1,val2,val3]   
query = sa.text(""" 
                SELECT *
                FROM my_table
                WHERE col1 IN :values; 
""")
query = query.bindparams(values=tuple(values))
df = pd.read_sql(query, engine)

Pyelitis

_mysql_connector.MySQLInterfaceError: Python type tuple cannot be converted – Tabber

My problem is that it's doing 'ARRAY[]' psycopg2.errors.UndefinedFunction: operator does not exist: bigint = integer[] LINE 4: WHERE ancestor_concept_id IN(ARRAY[44810261,4225055,... – Doughy

Answer 5

With the expression API, which based on the comments is what this question is asking for, you can use the in_ method of the relevant column.

To query

SELECT id, name FROM user WHERE id in (123,456)

use

myList = [123, 456]
select = sqlalchemy.sql.select([user_table.c.id, user_table.c.name], user_table.c.id.in_(myList))
result = conn.execute(select)
for row in result:
    process(row)

This assumes that user_table and conn have been defined appropriately.

Avril

Answer 6

Or maybe use .in_(list), similar to what @Carl has already suggested as

 stmt = select(
        id,
         name
      ).where(
        id.in_(idlist),
      )

Complete code assuming you have the data model in User class:

def fetch_name_ids(engine, idlist):
    # create an empty dataframe
    df = pd.DataFrame()
    try:
        # create session with engine
        session = Session(engine, future=True)
         stmt = select(
            User.id,
            User.name
          ).where(
            User.id.in_(idlist),
          )
        data = session.execute(stmt)

        df = pd.DataFrame(data.all())
        if len(df) > 0:
            df.columns = data.keys()
        else:
            columns = data.keys()
            df = pd.DataFrame(columns=columns)

    except SQLAlchemyError as e:
        error = str(e.__dict__['orig'])
        session.rollback()
        raise error
     else:

        session.commit()
     finally:

        engine.dispose()
        session.close()


      return df

Moke

Answer 7

Some people think it is unsafe because of SQL Injection:

But this is only working with Native Query:

This question posted a solution to the select query, unfortunately, it is not working for the update query. Using this solution would help even in the select conditions.

Update Query Solution:

id_list = [1, 2, 3, 4, 5] # in most cases we have an integer list or set
query = 'update myTable set content = 1 WHERE id IN {id_list}'.format(id_list=tuple(id_list))
conn.execute(query)

Note: Use a tuple instead of a list.

Cotyledon

Some people might not find this safe because of sql injection, but this is the only thing I could get to work. For sql injection protection, the within my function I did the equivalent of id_list = [int(x) for x in id_list] just in case a string was passed, so python will throw an error in that case – Doughy

Building the query as a string defeats the whole purpose of using SQLAlchemy in the first place. Instead use something like update(myTable).values(content=1).where(myTable.c.id.in_(id_list)) (using SQLAlchemy Core - remove the .c if myTable is an ORM model) – Anfractuosity

Answer 8

Just an addition to the answers above.

If you want to execute a SQL with an "IN" statement you could do this:

ids_list = [1,2,3]
query = "SELECT id, name FROM user WHERE id IN %s" 
args = [(ids_list,)] # Don't forget the "comma", to force the tuple
conn.execute(query, args)

Two points:

• There is no need for Parenthesis for the IN statement(like "... IN(%s) "), just put "...IN %s"
• Force the list of your ids to be one element of a tuple. Don't forget the " , " : (ids_list,)

EDIT Watch out that if the length of list is one or zero this will raise an error!

Odontoblast

I think you should do args = [(tuple(ids_list),)], please double check. – Uncharted

The OP is asking how to do this in sqlalchemy, not using raw dbapi call... – Athalee

This does not work with a single item list as it produces (item, ) and it's incorrect in SQL – Lollard