Initializing a pydantic dataclass from json

Question

I'm in the process of converting existing dataclasses in my project to pydantic-dataclasses, I'm using these dataclasses to represent models I need to both encode-to and parse-from json.

Here's an example of my current approach that is not good enough for my use case, I have a class A that I want to both convert into a dict (to later be converted written as json) and to read from that dict. But the only way I can find to parse the json back into a model gives me back the underlying BaseModel and not the dataclass.

note that I'm using the asdict function to convert the dataclass to a dict as it's what the pydantic_encoder uses to convert the dataclass to json, and using the pydantic_encoder what the documentation recommends to convert a pydantic-dataclass to json: https://pydantic-docs.helpmanual.io/usage/dataclasses/

from dataclasses import asdict
from pydantic.dataclasses import dataclass
from pydantic import BaseModel

@dataclass
class A:
    x: str

a = A("string")
a_dict = asdict(a)
parsed_a = A.__pydantic_model__.parse_obj(a_dict)

print(f"type of a: {type(a)}")
print(f"type of parsed_a: {type(parsed_a)}")

print(f"a is instance of A: {isinstance(a, A)}")
print(f"parsed_a is instance of A: {isinstance(parsed_a, A)}")

print(f"a is instance of BaseModel: {isinstance(a, BaseModel)}")
print(f"parsed_a is instance of BaseModel: {isinstance(parsed_a, BaseModel)}")

output:

type of a: <class '__main__.A'>
type of parsed_a: <class '__main__.A'>
a is instance of A: True
parsed_a is instance of A: False
a is instance of BaseModel: False
parsed_a is instance of BaseModel: True

Is there maybe a way to initialize A from the parsed BaseModel?

Bookkeeping

There should be a from_json() classmethod for BaseModel – Slaty

Answer 1

To build on others' answers, since the right/best answer to this appears to constantly change:

Using Pydantic's TypeAdapter is currently the way to go. Updated @pceccon example:

from pydantic.dataclasses import dataclass
from pydantic import TypeAdapter
import dataclasses
import json

@dataclass
class User:
  id: int
  name: str

user = User(id=123, name="James")
user_json = json.dumps(dataclasses.asdict(user))
print(user_json)  # '{"id": 123, "name": "James"}'

user = TypeAdapter(User).validate_json(user_json)
print(user)  # User(id=123, name='James')
print(user.name)  # 'James'

if you want PyCharm (or whatever editor) to give you autocompletion/intellisense, use a type hint:

user: User = TypeAdapter(User).validate_json(user_json)

if you have a dictionary instead of json, you can use TypeAdapter.validate_python instead:

user: User = TypeAdapter(User).validate_python(user_dict)

This works recursively for your model/class definitions.

Granddaughter

Answer 2

I think I arrive a little bit late to the party, but I think this answer may come handy for future users having the same question.

To convert the dataclass to json you can use the combination that you are already using using (asdict plus json.dump).

from pydantic.dataclasses import dataclass

@dataclass
class User:
  id: int
  name: str

user = User(id=123, name="James")
d = asdict(user)  # {'id': 123, 'name': 'James'
user_json = json.dumps(d)
print(user_json)  # '{"id": 123, "name": "James"}'

# Or directly with pydantic_encoder
json.dumps(user, default=pydantic_encoder)

Then from the raw json you can use a BaseModel and the parse_raw method.

If you want to deserialize json into pydantic instances, I recommend you using the parse_raw method:

user = User.__pydantic_model__.parse_raw('{"id": 123, "name": "James"}')
print(user)
# id=123 name='James'

Otherwise, if you want to keep the dataclass:

json_raw = '{"id": 123, "name": "James"}'
user_dict = json.loads(json_raw)
user = User(**user_dict)

Motto

does the parsing work recursively? – Scroggins

Why not just User.parse_raw('{"id": 123, "name": "James"}') – Felicitation

Because pydantic dataclass do not have access to this method directly. You first have to pick the BaseModel using the __pydandic_model__ attribute – Motto

In Pydantic 2, it looks like we need User.__pydantic_validator__.validate_json(json_str) – Thigh

Answer 3

Pydantic recommends using parse_raw to deserialize JSON strings.

Class definition

from pydantic import BaseModel

class ResponseData(BaseModel):
    status_code: int
    text: str
    reason: str
    
    class Config:
        orm_mode = True

JSON to BaseModel conversion

x = ResponseData(status_code=200, text="", reason="")
json = x.json()
response = ResponseData.parse_raw(json)
assert x == response
print(response.dict())

Warrantable

To my knowledge, this is the current recommended method, as of February 2023: docs.pydantic.dev/usage/exporting_models/… – Hulky

This only works when your classes inherit from BaseModel not when using the @dataclass decorator. docs.pydantic.dev/latest/usage/dataclasses – Thigh

Update for January 2024: parse_raw() has been deprecated and you are now to use model_validate_json(). Reference: docs.pydantic.dev/latest/migration – Guberniya

Answer 4

Now it's possible to solve this issue in a simple manner, in my opinion. Adding to @Guillem example:

from pydantic.dataclasses import dataclass
from pydantic.tools import parse_obj_as
import dataclasses
import json

@dataclass
class User:
  id: int
  name: str

user = User(id=123, name="James")
user_json = json.dumps(dataclasses.asdict(user))
print(user_json)  # '{"id": 123, "name": "James"}'

user_dict = json.loads(user_json)
user = parse_obj_as(User, user_dict)
print(user)  # User(id=123, name='James')

It works for recursively as well.

Refusal

parse_obj_as is deprecated as of Pydantic v2 docs.pydantic.dev/latest/migration – Thigh

The penultimate line becomes user =TypeAdapter(User).validate_python(user_dict) if you made it this far. – Crus

Answer 5

I managed to address this issue by unpacking the properties of the parsed BaseModel and initializing the dataclass with them.

this solution works recursively for Pydantic dataclasses whose properties are either pydantic-dataclasses or primitives

please note that this solution doesn't work for for Unions and Generics (yet).

 def pydantic_dataclass_from_json_dict(json_dict: dict, pydantic_dataclass_type) -> Any:
    base_model = pydantic_dataclass_type.__pydantic_model__.parse_obj(json_dict)
    base_mode_fields = base_model.__fields__
    dataclass_fields = dataclasses.fields(pydantic_dataclass_type)

    values = []
    for base_model_field_name, base_model_field in base_mode_fields.items():
        value = getattr(base_model, base_model_field_name)
        dataclass_field = [field for field in dataclass_fields if field.name == base_model_field.name][0]
        if is_dataclass(dataclass_field):
            converted_value = pydantic_dataclass_from_json_dict(value, dataclass_field.type)
            values.append(converted_value)
        else:
            values.append(value)

    dataclass_object = pydantic_dataclass_type(*values)
    return dataclass_object

Bookkeeping

Answer 6

Well, I tried the above ways but it's error because pydantic_model is not existing BaseModel attribute.

So I checked functions in the class model. I found 'parse_raw()' in it and tried instead the above ways It's good working

Let use parse_raw() function

Warrantable

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. – Accordingly