How do I convert a database row into a struct
Asked Answered
M

8

116

Let's say I have a struct:

type User struct {
    Name  string
    Id    int
    Score int
}

And a database table with the same schema. What's the easiest way to parse a database row into a struct? I've added an answer below but I'm not sure it's the best one.

Morrie answered 23/6, 2013 at 21:22 Comment(0)
M
45

Here's one way to do it - just assign all of the struct values manually in the Scan function.

func getUser(name string) (*User, error) {
    var u User
    // this calls sql.Open, etc.
    db := getConnection()
    // note the below syntax only works for postgres
    err := db.QueryRow("SELECT * FROM users WHERE name = $1", name).Scan(&u.Id, &u.Name, &u.Score)
    if err != nil {
        return &User{}, err
    } else {
        return &u, nil
    }
}
Morrie answered 23/6, 2013 at 21:22 Comment(6)
@eslammostafa at what point this code can have problems with NULL values?Coldiron
@Coldiron at score for example, i meant null values that are coming from the database.Kenay
@eslammostafa I understand, this is really a painColdiron
@Coldiron in the mean time i use sql.NullString and sql.NullInt64 ..etc to handle null values, it's a little extra work though.Kenay
@eslammostafa the problem there is if you use sql.NullString and convert the struct to JSON then the generated is not friendly, needing to use a VO or something like thatColdiron
there is a way to fix this, to write a JSONMarshal yourself for your struct, or just use a package called Null it helps with the sql nulls in json, yaml ..etcKenay
S
116

Go package tests often provide clues as to ways of doing things. For example, from database/sql/sql_test.go,

func TestQuery(t *testing.T) {
    /* . . . */
    rows, err := db.Query("SELECT|people|age,name|")
    if err != nil {
            t.Fatalf("Query: %v", err)
    }
    type row struct {
            age  int
            name string
    }
    got := []row{}
    for rows.Next() {
            var r row
            err = rows.Scan(&r.age, &r.name)
            if err != nil {
                    t.Fatalf("Scan: %v", err)
            }
            got = append(got, r)
    }
    /* . . . */
}

func TestQueryRow(t *testing.T) {
    /* . . . */
    var name string
    var age int
    var birthday time.Time
    err := db.QueryRow("SELECT|people|age,name|age=?", 3).Scan(&age)
    /* . . . */
}

Which, for your question, querying a row into a structure, would translate to something like:

var row struct {
    age  int
    name string
}
err = db.QueryRow("SELECT|people|age,name|age=?", 3).Scan(&row.age, &row.name)

I know that looks similar to your solution, but it's important to show how to find a solution.

Seve answered 23/6, 2013 at 22:51 Comment(5)
if the easy way is to manually bind columns to struct fields I'm wondering what's the hard wayArgyrol
Unfortunately, this is not very convenient especially in case of bigger structs - binding manually to struct properties is complete failure... Using jmoiron/sqlx or some other libraries is more efficient...Ayesha
i keep getting back nothing from rows.Scan(). all the vars are set to be empty.Imbecility
What the heck is the pipe syntax here? "SELECT|people" ? I don't see any reference to that in the godocs.Entanglement
Had the same WTF moment as Brian. Turns out its a fake driver they made purely for testing sql/database (golang.org/src/database/sql/fakedb_test.go) I was really hoping I could use it for new code!Preemie
A
76

I recommend github.com/jmoiron/sqlx.

From the README:

sqlx is a library which provides a set of extensions on go's standard database/sql library. The sqlx versions of sql.DB, sql.TX, sql.Stmt, et al. all leave the underlying interfaces untouched, so that their interfaces are a superset on the standard ones. This makes it relatively painless to integrate existing codebases using database/sql with sqlx.

Major additional concepts are:

  • Marshal rows into structs (with embedded struct support), maps, and slices
  • Named parameter support including prepared statements
  • Get and Select to go quickly from query to struct/slice

The README also includes a code snippet demonstrating scanning a row into a struct:

type Place struct {
    Country       string
    City          sql.NullString
    TelephoneCode int `db:"telcode"`
}
// Loop through rows using only one struct
place := Place{}
rows, err := db.Queryx("SELECT * FROM place")
for rows.Next() {
    err := rows.StructScan(&place)
    if err != nil {
        log.Fatalln(err)
    } 
    fmt.Printf("%#v\n", place)
}

Note that we didn't have to manually map each column to a field of the struct. sqlx has some default mappings for struct fields to database columns, as well as being able to specify database columns using tags (note the TelephoneCode field of the Place struct above). You can read more about that in the documentation.

Amling answered 12/10, 2016 at 23:44 Comment(1)
Old question, but this can help someone in the future: jmoiron.github.io/sqlx/#safetyPenguin
M
45

Here's one way to do it - just assign all of the struct values manually in the Scan function.

func getUser(name string) (*User, error) {
    var u User
    // this calls sql.Open, etc.
    db := getConnection()
    // note the below syntax only works for postgres
    err := db.QueryRow("SELECT * FROM users WHERE name = $1", name).Scan(&u.Id, &u.Name, &u.Score)
    if err != nil {
        return &User{}, err
    } else {
        return &u, nil
    }
}
Morrie answered 23/6, 2013 at 21:22 Comment(6)
@eslammostafa at what point this code can have problems with NULL values?Coldiron
@Coldiron at score for example, i meant null values that are coming from the database.Kenay
@eslammostafa I understand, this is really a painColdiron
@Coldiron in the mean time i use sql.NullString and sql.NullInt64 ..etc to handle null values, it's a little extra work though.Kenay
@eslammostafa the problem there is if you use sql.NullString and convert the struct to JSON then the generated is not friendly, needing to use a VO or something like thatColdiron
there is a way to fix this, to write a JSONMarshal yourself for your struct, or just use a package called Null it helps with the sql nulls in json, yaml ..etcKenay
S
7
rows, err := connection.Query("SELECT `id`, `username`, `email` FROM `users`")

if err != nil {
    panic(err.Error())
}

for rows.Next() {
    var user User

    if err := rows.Scan(&user.Id, &user.Username, &user.Email); err != nil {
        log.Println(err.Error())
    }

    users = append(users, user)
}

Full example

Sherrylsherurd answered 24/10, 2017 at 7:37 Comment(0)
C
7

Here is a library just for that: scany.

You can use it like that:

type User struct {
    Name  string
    Id    int
    Score int
}

// db is your *sql.DB instance
// ctx is your current context.Context instance

// Use sqlscan.Select to query multiple records.
var users []*User
sqlscan.Select(ctx, db, &users, `SELECT name, id, score FROM users`)

// Use sqlscan.Get to query exactly one record.
var user User
sqlscan.Get(ctx, db, &user, `SELECT name, id, score FROM users WHERE id=123`)

It's well documented and easy to work with.

Disclaimer: I am the author of this library.

Cingulum answered 17/7, 2020 at 17:9 Comment(0)
W
1

there's package just for that: sqlstruct

unfortunately, last time I checked it did not support embedded structs (which are trivial to implement yourself - i had a working prototype in a few hours).

just committed the changes I made to sqlstruct

Whaling answered 24/6, 2013 at 7:53 Comment(0)
R
0

There are already some good answers here, I would like to add an SQL library I wrote just for tackling this issue of how complex it is to do a simple query on the most used Golang libraries; its called KSQL (not the Kafka one, the K stands for KISS or Keep It Stupid Simple).

Using it you could scan a user into a struct like this:

import (
    "context"
    "fmt"

    "github.com/vingarcia/ksql"
    "github.com/vingarcia/ksql/adapters/kpgx"
)

type User struct {
    Id    int    `ksql:"id"`
    Name  string `ksql:"name"`
    Score int    `ksql:"score"`
}

// Just a simple struct containing the table name and
// the name of the ID column or columns:
var UsersTable = ksql.NewTable("users", "id")

func main() {
    ctx := context.Background()
    dsn := "host=localhost user=postgres password=postgres dbname=postgres port=5432 sslmode=disable TimeZone=UTC"
    db, err := kpgx.New(ctx, dsn, ksql.Config{})
    if err != nil {
        panic("failed to connect database")
    }

    // Lets insert a user so the query below has something to return:
    err = db.Insert(ctx, UsersTable, &User{
        Name:  "SomeUser",
        Score: 42,
    })
    if err != nil {
        panic(err)
    }

    var user User
    err = db.QueryOne(ctx, &user, "FROM users WHERE name = $1", "SomeUser")
    if err != nil {
        panic(err)
    }

    fmt.Printf("%+v\n", user) // {Id:1 Name:SomeUser Score:42}
}
Riverhead answered 14/7, 2023 at 20:32 Comment(0)
G
-1

use : go-models-mysql sqlbuilder

val, err = m.ScanRowType(row, (*UserTb)(nil))

or the full code

import (
    "database/sql"
    "fmt"

    lib "github.com/eehsiao/go-models-lib"
    mysql "github.com/eehsiao/go-models-mysql"
)

// MyUserDao : extend from mysql.Dao
type MyUserDao struct {
    *mysql.Dao
}

// UserTb : sql table struct that to store into mysql
type UserTb struct {
    Name       sql.NullString `TbField:"Name"`
    Id         int            `TbField:"Id"`
    Score      int            `TbField:"Score"`
}

// GetFirstUser : this is a data logical function, you can write more logical in there
// sample data logical function to get the first user
func (m *MyUserDao) GetFirstUser() (user *User, err error) {

    m.Select("Name", "Id", "Score").From("user").Limit(1)
    fmt.Println("GetFirstUser", m.BuildSelectSQL().BuildedSQL())
    var (
        val interface{}
        row *sql.Row
    )

    if row, err = m.GetRow(); err == nil {
        if val, err = m.ScanRowType(row, (*UserTb)(nil)); err == nil {
            u, _ := val.(*UserTb)

            user = &User{
                Name:       lib.Iif(u.Name.Valid, u.Nae.String, "").(string),
                Id:         u.Id,
                Score:      u.Score,
            }
        }
    }
    row, val = nil, nil

    return
}
Guileless answered 18/9, 2019 at 3:12 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.