pydantic exclude multiple fields from model

Question

In pydantic is there a cleaner way to exclude multiple fields from the model, something like:

class User(UserBase):
    class Config:        
        exclude = ['user_id', 'some_other_field']

I am aware that following works.

class User(UserBase):

    class Config:       
        fields = {'user_id': {'exclude':True}, 
                   'some_other_field': {'exclude':True}
                 }

but I was looking for something cleaner like django rest framework, where you could just specify a list of exclude or include per.

Reikoreilly

I came here looking for a clean solution for excluding fields from the OpenAPI description generated by Pydantic 2 but found there doesn't seem to be much native support. The docs say that you need to implement __get_pydantic_json_schema__ which isn't too hard but does require about 3 different imports from the bowels of Pydantic. https://docs.pydantic.dev/latest/usage/json_schema/#modifying-the-schema – Wilkie

I don't know how I missed it before but Pydantic 2 uses typing.ClassVar so that "Attributes annotated with typing.ClassVar are properly treated by Pydantic as class variables, and will not become fields on model instances". So just wrap the field type with ClassVar e.g. instead of foo: int = 1 use foo: ClassVar[int] = 1. So this excludes fields from the model, and the schema, but it also removes the field from instances, which may not be what you want. – Wilkie

Answer 1

Pydantic will exclude the class variables which begin with an underscore. so if it fits your use case, you can rename your attribues.

class User(UserBase):
    _user_id=str
    some_other_field=str
    ....

Weaponeer

one possible issue is that strict type validation doesn't work for class variables with an underscore. you can pass a non-string to _user_id and it can cause the code to break somewhere you don't expect – Ubald

I found,, using Pydantic 2, that Pydantic throws an error during model creation when underscores are used: "NameError: Fields must not use names with leading underscores; e.g., use 'correlation_id' instead of '_correlation_id'." – Wilkie

Per the newest FastAPI docs: “As of Pydantic v2.1.0, you will receive a NameError if trying to use the Field function with a private attribute. Because private attributes are not treated as fields, the Field() function cannot be applied.” @Wilkie Try using PrivateAttr() instead. docs.pydantic.dev/latest/concepts/models/… – Domeniga

Thanks @pythlang, but unfortunately I want some field behaviour, but I just don't want them appearing in the OpenAPI description. They're secret internal fields. I used a similar approach to that here except I created them as fields with a description string of "__exclude__".. I then interrupted the OpenAPI JSON dump to look for that description and delete those fields. A bit hacky and no better than the question here but it seems to be the prudent route for my use-case. – Wilkie

Answer 2

To exclude a field you can also use exclude in Field:

from pydantic import BaseModel, Field

class Mdl(BaseModel):
    val: str = Field(
        exclude=True,
        title="val"
    )

however, the advantage of adding excluded parameters in the Config class seems to be that you can get the list of excluded parameters with

print(Mdl.Config.exclude)

Blackcap

This doesn't work with version 2.6.1 – Frenchify

This answer was tested with pydantic < 2, so they might have changed something in version 2 – Blackcap

Answer 3

I wrote something like this for my json :

from pydantic import BaseModel


class CustomBase(BaseModel):
    def json(self, **kwargs):
        include = getattr(self.Config, "include", set())
        if len(include) == 0:
            include = None
        exclude = getattr(self.Config, "exclude", set())
        if len(exclude) == 0:
            exclude = None
        return super().json(include=include, exclude=exclude, **kwargs)

    

class User(CustomBase):
    name: str = ...
    family: str = ...

    class Config:
        exclude = {"family"}


u = User(**{"name": "milad", "family": "vayani"})

print(u.json())

you can overriding dict and other method like.

Skinflint

Answer 4

You can either do it in the field level or when serializing the model (see docs). Currently there is a stale issue for having include / exclude in the model_config.

from typing import Annotated
from pydantic import BaseModel, Field

class A(BaseModel):
    foo: str = Field('bar', exclude=True)
    bar: Annotated[int, Field(exclude=True)] = 'bar'
    baz: str = 'bar'

a = A()
assert 'baz' not in a.model_dump(exclude={'baz'})
assert 'baz' in a.model_dump()
assert 'foo' not in a.model_dump()

Inverness

Answer 5

A possible solution is creating a new class based in the baseclass using create_model:

from pydantic import BaseModel, create_model

def exclude_id(baseclass, to_exclude: list):
    # Here we just extract the fields and validators from the baseclass
    fields = baseclass.__fields__
    validators = {'__validators__': baseclass.__validators__}
    new_fields = {key: (item.type_, ... if item.required else None)
                  for key, item in fields.items() if key not in to_exclude}
    return create_model(f'{baseclass.__name__}Excluded', **new_fields, __validators__=validators)


class User(BaseModel):
    ID: str
    some_other: str

list_to_exclude = ['ID']
UserExcluded = exclude_id(User, list_to_exclude)

UserExcluded(some_other='hola')

Which will return:

> UserExcluded(some_other='hola')

Which is a copy of the baseclass but with no parameter 'ID'.

If you have the id in the validators you may want also to exclude those validators.

Latinalatinate

You need to add config__=baseclass.__config that include Config in new model. For example, it is very important when you use orm_mode=True etc – Universal

Answer 6

For pydantic2 you can use do something like:

class User(UserBase):
    user_id: Annotated[int, Field(exclude=True)] 
    some_other_field: str

This can be much cleaner

Lacrimator

Answer 7

At the time I'm posting this answer, the stable release of Pydantic is version 2.7.x

The PrivateAttr class in Pydantic 2.x provides a solution. You can mark one or more fields in your model class as private by prefixing each field name with an underscore and assigning that field to PrivateAttr. Here is an example:

class ExampleModelClassAlpha(BaseModel):
    name: str
    power_animal: Optional[str] = None
    _private: str = PrivateAttr(default=None)

Here is a simple Pytest that demonstrates how it works:

import logging
import pytest

from typing import Optional
from pydantic import BaseModel, PrivateAttr


logger = logging.getLogger(__name__)


class ExampleModelClassAlpha(BaseModel):
    name: str
    power_animal: Optional[str] = None
    _private: str = PrivateAttr(default=None)


# -------------------------------------------------------------------------------------------------
# noinspection PyArgumentList
@pytest.mark.unit
class TestSummaOpenapiModel:

    # ---------------------------------------------------------------------------------------------
    def test_x(self):
        emc_alpha: ExampleModelClassAlpha = ExampleModelClassAlpha(
            name='John Smith',
            _private='this is a private field that I want to exclude from the JSON'
        )

        the_json_string: str = emc_alpha.model_dump_json(indent=4, exclude_none=True)

        logger.info("\n%s", the_json_string)
        assert 'power_animal' not in the_json_string
        assert 'this is a private field' not in the_json_string

The test should pass, and the output should look like this:

{
    "name": "John Smith"
}

Dorie