Django JSONField inside ArrayField

Question

I have a problem inserting to a field using ArrayField with JSONField inside.

models.py

locations = ArrayField(JSONField(null = True,blank = True), blank=True, null = True)

Insert

location_arr = [{"locations" : "loc1","amount":Decimal(100.00)},{"locations" : "loc2","amount":Decimal(200.25)}]
instance.locations = location_arr
instance.save()

When I do this, I got

column "locations" is of type jsonb[] but expression is of type text[]

LINE 1: ...d" = 2517, "locations" = ARRAY['{"loc...

Hint: You will need to rewrite or cast the expression.

So I tried to dump it using:

import json
location_arr = [{"locations" : "loc1","amount":Decimal(100.00)},{"locations" : "loc2","amount":Decimal(200.25)}]
instance.locations = json.dumps(location_arr)
instance.save()

then I got this

LINE 1: ...d" = 2517, "locations" = '[{"loc":...

DETAIL: "[" must introduce explicitly-specified array dimensions.

I am using:

1. Django 1.9
2. Python 2.7
3. Postgres 9.4.10
4. psycopg2 2.6.2

Reticulation

This is known bug. – Tire

This was fixed in Django==2.2a1 – N

Answer 1

Arrays

First of all, let's take a close look at this important text from the Postgresql Arrays document.

Tip: Arrays are not sets; searching for specific array elements can be a sign of database misdesign. Consider using a separate table with a row for each item that would be an array element. This will be easier to search, and is likely to scale better for a large number of elements.

Most of the time, you should not be using arrays.

JSONB

JSONB is available in Django as the JSONField type. This field is more scalable and flexible than array fields and can be searched more efficiently. However if you find yourself searching inside JSONB fields all the time the above statement about Arrays is equally valid for JSONB.

Now what do you have in your system? A an array that holds JSONB field. This is a disaster waiting to happen. Please normalize your data.

Recap

so when to use ArrayField?

On the rare occasion when you don't need to search in that column and you don't need to use that column for a join.

Extract

Thanks. so when to use ArrayField? – Reticulation

On the rare occaision when you don't need to search in that column and you don't need to use that column for a join. – Extract

How about we use custom lookups like __contains for searching? docs.djangoproject.com/en/1.10/ref/contrib/postgres/fields/… – Reticulation

and that doesn't search the arrayfield? – Extract

no worries glad to have been of service. All the best with your project – Extract

This answer is quite out-of-date - now that Postgres supports GIN indexes on JSONB columns (with the crazy fast ? operator for array/key membership), storing tags in a column instead of a join table can help you avoid massive join fan-out during queries and is a totally reasonable solution. – Flam

On the contrary Ben, you are following a well known SQL anti pattern by putting comma seperated data in a column. – Extract

Answer 2

I have encountered the same scenario. Here is the way how I solved it

models.py

from django.contrib.postgres.fields.jsonb import JSONField as JSONBField
location = JSONBField(default=list,null=True,blank=True)

insert

model_object.location = [{"locations" : "loc1","amount":Decimal(100.00)},{"locations" : "loc2","amount":Decimal(200.25)}]

update

model_object.location.append({"locations" : "loc1","amount":Decimal(100.00)})
model_object.save()

This worked for me in Django - 2.0.2 Postgres - 9.5 psycopg2 - 2.7.4 python - 3.4.3

Flameproof

now how do you search for a particular location inside JSONBField, let's say with locations="loc1"? – Crossquestion

Answer 3

You can sidestep this issue by using the JSONField as the column field type with a list as the root element.

from django.contrib.postgres.fields import JSONField
class MyDBArray(models.Model):
    array_data = models.JSONField(default=list)

my_db_array = MyDBArray(array_data=[1, 2, 3])
my_db_array.save()

You would need to validate in the save method that the array_data field is actually list-like.

Motivation

Answer 4

This was fixed in the latest unreleased version of Django 2.2a1

pip install Django==2.2a1

PS I believe that it will work with versions >= 2.2a1

N

Answer 5

I guess the easiest way is to turn the field from an array of jsonfield into a jsonfield. Just add a key to the location_arr.

Related thread: ArrayField with JSONField as base_field in Django

Lehmann