How to validate more than one field of a Pydantic model?
Asked Answered
S

6

39

I want to validate three model Fields of a Pydantic model. To do this, I am importing root_validator from pydantic, however I am getting the error below:

from pydantic import BaseModel, ValidationError, root_validator
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'root_validator' from 'pydantic' (C:\Users\Lenovo\AppData\Local\Programs\Python\Python38-32\lib\site-packages\pydantic\__init__.py)

I tried this:

@validator
def validate_all(cls, v, values, **kwargs):
    ...

I am inheriting my pydantic model from some common fields parent model. Values showing only parent class fields, but not my child class fields. For example:

class Parent(BaseModel):
    name: str
    comments: str

class Customer(Parent):
    address: str
    phone: str
    
    @validator
    def validate_all(cls, v, values, **kwargs):
        #here values showing only (name and comment) but not address and phone.
        ...
Sequel answered 23/4, 2020 at 16:49 Comment(3)
if from pydantic import root_validator raises an ImportError, this is most probably because you do not have the right version of pydantic... Which version do you use ?Alloway
pydantic==0.32.2Sequel
Latest is 1.5.1 ... pypi.org/project/pydanticAlloway
A
43

To extend on the answer of Rahul R, this example shows in more detail how to use the pydantic validators.

This example contains all the necessary information to answer your question.

Note, that there is also the option to use a @root_validator, as mentioned by Kentgrav, see the example at the bottom of the post for more details.

import pydantic

class Parent(pydantic.BaseModel):
    name: str
    comments: str

class Customer(Parent):
    address: str
    phone: str

    # If you want to apply the Validator to the fields "name", "comments", "address", "phone"
    @pydantic.validator("name", "comments", "address", "phone")
    @classmethod
    def validate_all_fields_one_by_one(cls, field_value):
        # Do the validation instead of printing
        print(f"{cls}: Field value {field_value}")

        return field_value  # this is the value written to the class field

    # if you want to validate to content of "phone" using the other fields of the Parent and Child class
    @pydantic.validator("phone")
    @classmethod
    def validate_one_field_using_the_others(cls, field_value, values, field, config):
        parent_class_name = values["name"]
        parent_class_address = values["address"] # works because "address" is already validated once we validate "phone"
        # Do the validation instead of printing
        print(f"{field_value} is the {field.name} of {parent_class_name}")

        return field_value 

Customer(name="Peter", comments="Pydantic User", address="Home", phone="117")

Output

<class '__main__.Customer'>: Field value Peter
<class '__main__.Customer'>: Field value Pydantic User
<class '__main__.Customer'>: Field value Home
<class '__main__.Customer'>: Field value 117
117 is the phone number of Peter
Customer(name='Peter', comments='Pydantic User', address='Home', phone='117')

To answer your question in more detail:

Add the fields to validate to the @validator decorator directly above the validation function.

  • @validator("name") uses the field value of "name" (e.g. "Peter") as input to the validation function. All fields of the class and its parent classes can be added to the @validator decorator.
  • the validation function (validate_all_fields_one_by_one) then uses the field value as the second argument (field_value) for which to validate the input. The return value of the validation function is written to the class field. The signature of the validation function is def validate_something(cls, field_value) where the function and variable names can be chosen arbitrarily (but the first argument should be cls). According to Arjan (https://youtu.be/Vj-iU-8_xLs?t=329), also the @classmethod decorator should be added.

If the goal is to validate one field by using other (already validated) fields of the parent and child class, the full signature of the validation function is def validate_something(cls, field_value, values, field, config) (the argument names values,field and config must match) where the value of the fields can be accessed with the field name as key (e.g. values["comments"]).

Edit1: If you want to check only input values of a certain type, you could use the following structure:

@validator("*") # validates all fields
def validate_if_float(cls, value):
    if isinstance(value, float):
        # do validation here
    return value

Edit2: Easier way to validate all fields together using @root_validator:

import pydantic

class Parent(pydantic.BaseModel):
    name: str
    comments: str

class Customer(Parent):
    address: str
    phone: str

    @pydantic.root_validator()
    @classmethod
    def validate_all_fields_at_the_same_time(cls, field_values):
        # Do the validation instead of printing
        print(f"{cls}: Field values are: {field_values}")
        assert field_values["name"] != "invalid_name", f"Name `{field_values['name']}` not allowed."
        return field_values

Output:

Customer(name="valid_name", comments="", address="Street 7", phone="079")
<class '__main__.Customer'>: Field values are: {'name': 'valid_name', 'comments': '', 'address': 'Street 7', 'phone': '079'}
Customer(name='valid_name', comments='', address='Street 7', phone='079')
Customer(name="invalid_name", comments="", address="Street 7", phone="079")
ValidationError: 1 validation error for Customer
__root__
  Name `invalid_name` not allowed. (type=assertion_error)
Auriol answered 22/7, 2021 at 13:59 Comment(4)
How would you go about validating all fields of a certain type? I know there is a * input to the @validator() decorator that will use the validator for all fields. However, I'd like to use a specific validator to validate all floats for example.Wilton
Since Python is dynamically typed, I think you need the validator for all fields @validator("*") and make a type check inside with isinstance(value, float). If the type is float, do your validation, otherwise, return the input argument return value. However, this validation will activate for all inputs of type float, independent of the types in the class declaration.Auriol
Thank you this makes senseWilton
I don't think you need the @classmethod decorator because @validator already returns a classmethod; see this issue.Stanhope
I
12

Option 1 - Using the @validator decorator

As per the documentation, "a single validator can be applied to multiple fields by passing it multiple field names" (and "can also be called on all fields by passing the special value '*'"). Thus, you could add the fields you wish to validate to the validator decorator, and using field.name attribute you can check which one to validate each time the validator is called. If a field does not pass the validation, you could raise ValueError, "which will be caught and used to populate ValidationError" (see "Note" section here). If you need to validate a field based on other field(s), you have to check first if they have already been validated using values.get() method, as shown in this answer (Update 2). The below demonstrates an example, where fields such as name, country_code, and phone number (based on the provided country_code) are validated. The regex patterns provided are just examples for the purposes of this demo, and are based on this and this answer..

from pydantic import BaseModel, validator
import re

name_pattern = re.compile(r'[a-zA-Z\s]+$')
country_codes = {"uk", "us"}
UK_phone_pattern = re.compile(r'^(\+44\s?7\d{3}|\(?07\d{3}\)?)\s?\d{3}\s?\d{3}$')  # UK mobile phone number. Valid example: +44 7222 555 555
US_phone_pattern = re.compile(r'^(\([0-9]{3}\) |[0-9]{3}-)[0-9]{3}-[0-9]{4}$')  # US phone number. Valid example: (123) 123-1234
phone_patterns = {"uk": UK_phone_pattern, "us": US_phone_pattern}

class Parent(BaseModel):
    name: str
    comments: str
    
class Customer(Parent):
    address: str
    country_code: str
    phone: str

    @validator('name', 'country_code', 'phone')
    def validate_atts(cls, v, values, field):
        if field.name == "name":
            if not name_pattern.match(v): raise ValueError(f'{v} is not a valid name.')
        elif field.name == "country_code":
             if not v.lower() in country_codes: raise ValueError(f'{v} is not a valid country code.')
        elif field.name == "phone" and values.get('country_code'):
            c_code = values.get('country_code').lower()
            if not phone_patterns[c_code].match(v): raise ValueError(f'{v} is not a valid phone number.')
        return v

Update - Pydantic V2 Example

In Pydantic V2, @validator has been deprecated, and was replaced by @field_validator. If you want to access values from another field inside a @field_validator, this may be possible using ValidationInfo.data, which is a dict of field name to field value.

from pydantic import BaseModel, ValidationInfo, field_validator
import re

# ... the rest of the code is the same as above


class Customer(Parent):
    address: str
    country_code: str
    phone: str

    @field_validator('name', 'country_code', 'phone')
    @classmethod
    def validate_atts(cls, v: str, info: ValidationInfo):
        if info.field_name == 'name':
            if not name_pattern.match(v): raise ValueError(f'{v} is not a valid name.')
        elif info.field_name == 'country_code':
             if not v.lower() in country_codes: raise ValueError(f'{v} is not a valid country code.')
        elif info.field_name == 'phone' and info.data.get('country_code'):
            c_code = info.data.get('country_code').lower()
            if not phone_patterns[c_code].match(v): raise ValueError(f'{v} is not a valid phone number.')
        return v

Option 2 - Using the @root_validator decorator

Another approach would be using the @root_validator, which allows validation to be performed on the entire model's data.

from pydantic import BaseModel, root_validator
import re

name_pattern = re.compile(r'[a-zA-Z\s]+$')
country_codes = {"uk", "us"}
UK_phone_pattern = re.compile(r'^(\+44\s?7\d{3}|\(?07\d{3}\)?)\s?\d{3}\s?\d{3}$')  # UK mobile phone number. Valid example: +44 7222 555 555
US_phone_pattern = re.compile(r'^(\([0-9]{3}\) |[0-9]{3}-)[0-9]{3}-[0-9]{4}$')  # US phone number. Valid example: (123) 123-1234
phone_patterns = {"uk": UK_phone_pattern, "us": US_phone_pattern}

class Parent(BaseModel):
    name: str
    comments: str
    
class Customer(Parent):
    address: str
    country_code: str
    phone: str

    @root_validator()
    def validate_atts(cls, values):
        name = values.get('name')
        comments = values.get('comments')
        address = values.get('address')
        country_code = values.get('country_code')
        phone = values.get('phone')
        
        if name is not None and not name_pattern.match(name): 
            raise ValueError(f'{name} is not a valid name.')
        if country_code is not None and not country_code.lower() in country_codes: 
            raise ValueError(f'{country_code} is not a valid country code.')
        if phone is not None and country_code is not None:
            if not phone_patterns[country_code.lower()].match(phone): 
                raise ValueError(f'{phone} is not a valid phone number.')
                
        return values

Update - Pydantic V2 Example

In Pydantic V2, @root_validator has been deprecated, and was replaced by @model_validator. Model validators can be mode='before', mode='after' or mode='wrap'. In this case, mode='after' is suited best. As described in the documentation:

mode='after' validators are instance methods and always receive an instance of the model as the first argument. You should not use (cls, ModelType) as the signature, instead just use (self) and let type checkers infer the type of self for you. Since these are fully type safe they are often easier to implement than mode='before' validators. If any field fails to validate, mode='after' validators for that field will not be called.

Using mode='after'
from pydantic import BaseModel, model_validator
import re

# ... the rest of the code is the same as above


class Customer(Parent):
    address: str
    country_code: str
    phone: str

    @model_validator(mode='after')
    def validate_atts(self):
        name = self.name
        comments = self.comments
        address = self.address
        country_code = self.country_code
        phone = self.phone
        
        if name is not None and not name_pattern.match(name): 
            raise ValueError(f'{name} is not a valid name.')
        if country_code is not None and not country_code.lower() in country_codes: 
            raise ValueError(f'{country_code} is not a valid country code.')
        if phone is not None and country_code is not None:
            if not phone_patterns[country_code.lower()].match(phone): 
                raise ValueError(f'{phone} is not a valid phone number.')
                
        return self
Using mode='before'

In case you would rather using mode='before, you could this as follows. Note that in this case, you should, however, perform your own checks on whether the field values are in the expected format (e.g., str in the example below), before moving on with further processing/validation (e.g., converting values to lowercase, string values comparisons, etc.)—not included below.

from pydantic import BaseModel, model_validator
from typing import Any
import re

# ... the rest of the code is the same as above


class Customer(Parent):
    address: str
    country_code: str
    phone: str

    @model_validator(mode='before')
    @classmethod
    def validate_atts(cls, data: Any):
        if isinstance(data, dict):
            name = data.get('name')
            comments = data.get('comments')
            address = data.get('address')
            country_code = data.get('country_code')
            phone = data.get('phone')
            
            if name is not None and not name_pattern.match(name): 
                raise ValueError(f'{name} is not a valid name.')
            if country_code is not None and not country_code.lower() in country_codes: 
                raise ValueError(f'{country_code} is not a valid country code.')
            if phone is not None and country_code is not None:
                if not phone_patterns[country_code.lower()].match(phone): 
                    raise ValueError(f'{phone} is not a valid phone number.')
                
        return data

Test Examples for Options 1 & 2

from pydantic import ValidationError

# should throw "Value error, (123) 123-1234 is not a valid phone number."
try:
    Customer(name='john', comments='hi', address='some address', country_code='UK', phone='(123) 123-1234')
except ValidationError as e:
    print(e)

# should work without errors
print(Customer(name='john', comments='hi', address='some address', country_code='UK', phone='+44 7222 555 555'))
Ingenious answered 24/2, 2022 at 20:54 Comment(0)
C
11

You need to pass the fields as arguments of the decorator.

class Parent(BaseModel):
    name: str
    comments: str

class Customer(Parent):
    address: str
    phone: str

    @validator("name", "coments", "address", "phone")
    def validate_all(cls, v, values, **kwargs):
Christachristabel answered 24/6, 2020 at 4:24 Comment(1)
This would be a better answer if you explained how the code you provided answers the question.Runic
B
6

First off, if you are having an error importing root_validator, I would update pydantic.

pip install -U pydantic

A lot of the examples above show you how to use the same validator on multiple values one at a time. Or they add a lot of unnecessary complexity to accomplish what you want. You can simply use the following code to validate multiple fields at the same time in the same validator using the root_validator decorator.:

from pydantic import root_validator
from pydantic import BaseModel

class Parent(BaseModel):
    name: str = "Peter"
    comments: str = "Pydantic User"

class Customer(Parent):
    address: str = "Home"
    phone: str = "117"

    @root_validator
    def validate_all(cls, values):
         print(f"{values}")
         values["phone"] = "111-111-1111"
         values["address"] = "1111 Pydantic Lane"
         print(f"{values}")
         return values

Output:

{'name': 'Peter', 'comments': 'Pydantic User', 'address': 'Home', 'phone': '117'}

{'name': 'Peter', 'comments': 'Pydantic User', 'address': '1111 Pydantic Lane', 'phone': '111-111-1111'}
Berth answered 18/3, 2022 at 15:41 Comment(0)
M
2

Many of the answers here address how to add validators to a single pydantic model. I'll add how to share validators between models - and a few other advanced techniques.

Note: The following for Pydantic V2.

Option 1. Multiple fields in validate(...) decorator:

The previous answers to this Q provide the simplest and easiest way to validate multiple fields - that is, provide multiple field names to a single validator:


    ...

    @field_validator("field_1", "field_2", ...)
    def my_validator(...):
        ...

The existing answers are more than sufficient, I recommend reading them to understand further.

Option 2. Define a single validation function and reuse between parents and child:

This essentially allows you to import / reuse validators throughout your project.

from pydantic import field_validator, BaseModel


def must_be_title_case(v: str) -> str:
    """Validator to be used throughout"""
    if v != v.title():
        raise ValueError("must be title cased")
    return v



class Parent(BaseModel):
    name: str = "Peter"
    comments: str = "Pydantic User"
    
    validate_fields = field_validator("name", "comments")(must_be_title_case)


class Customer(Parent):
    address: str = "Home"
    phone: str = "117"
     
    validate_fields = field_validator("address", "phone")(must_be_title_case)

Alternatively, you could define the field validation on the child only, for all fields if you wish:

class Parent(BaseModel):
    name: str = "Peter"
    comments: str = "Pydantic User"


class Customer(Parent):
    address: str = "Home"
    phone: str = "117"
     
    validate_fields = field_validator("name", "comments", "address", "phone")(must_be_title_case)

Option 3. Define your validation as an Annotated Validator:

This allows you to define reusable validated "types" - a very high degree of flexibility:

from typing_extensions import Annotated

from pydantic import BaseModel, ValidationError, field_validator
from pydantic.functional_validators import AfterValidator


# Same function as before
def must_be_title_case(v: str) -> str:
    """Validator to be used throughout"""
    if v != v.title():
        raise ValueError("must be title cased")
    return v


# Define your annotated (validated) type:
MySpecialString = Annotated[str, AfterValidator(must_be_title_case)]


# Now use the custom type in your models
class Customer(Parent):
    address: MySpecialString = "Home"
    phone: MySpecialString = "117"

class Parent(BaseModel):
    name: MySpecialString = "Peter"
    comments: MySpecialString = "Pydantic User"

To explain a bit what's happening here in the annotated type:

  • The base type is a string
  • Pydantic will try to coerce an input value into a string. This is considered the "core validation" step
  • After pydantic's validation, we will run our validator function (declared by AfterValidator) - if this succeeds, the returned value will be set.
  • You can instead choose to declare BeforeValidator in the annotation, which will run our function before Pydantic tries coercing the values.

Option 4. Validate fields against each other:

The previous methods show how you can validate multiple fields individually. But what if you want to compare 2 values?

A common example is to compare 2 optional date values – if both are set, ensure one is larger than the other. I'll be demonstrating this below:

The best way to compare multiple fields is with a model_validator (aka root_validator in v1):

class MyModel(BaseModel):
    date_1: Optional[datetime] = None
    date_2: Optional[datetime] = None

    @model_validator(mode="after")
    def validate_dates(self):
        """Date 1 must always be larger than date 2, if they are both set"""
        if self.date_1 and self.date_2:
            if self.date_1 < self.date_2:
                raise ValueError("date_2 cannot be larger than date_1")
        return self

Notice mode="after" – this allows pydantic to perform its own validation first (coercing values into datetime objects + setting defaults).

You can technically do something similar with field validators, but other field values are not guaranteed to be set on the model at validation time – see extra notes on Pydantic's documentation.


I hope this gives you sufficient background for designing your solution.

Marashio answered 5/2 at 19:24 Comment(0)
C
-1

This example contains all the necessary information to answer your question.

    class User(BaseModel):
        name: Optional[str] = ""

        class Config:
            validate_assignment = True

        @validator("name")
            def set_name(cls, name):
            return name or "foo"
Cocoon answered 16/11, 2021 at 11:29 Comment(1)
As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center.Brout

© 2022 - 2024 — McMap. All rights reserved.