1

I try to parse and validate a CSV file using the following Pydantic (V2) model.

from typing import Optional
from pydantic import BaseModel, field_validator
    
class PydanticProduct(BaseModel):
    fid: Optional[float]
    water: Optional[float]

    class ConfigDict:
        from_attributes = True

    @field_validator('fid', 'water',  mode='before')
    def check_empty_fields(cls, value) -> float:
        if value == '':
            return None
        try:
            float_value = float(value)
        except ValueError:
            raise ValueError("Unable to parse string as a number. Please provide a valid number.")
        return float_value

If the CSV file contents look like this

fid,water
1.0,81.3
1.25,26.3
3.0,31.5

the fields will get converted from strings to float correctly.

On the other hand if the CSV file contents have empty strings

fid,water
1.0,
,26.3
3.0,31.5

then I will get the following error

Input should be a valid number, unable to parse string as a number [type=float_parsing, input_value='', input_type=str]

I tried to use the @field_validator with the mode="before" but the problem is that the validation is not executed.

In addition, I noticed that the validation is not executed even if there are no errors (aka. the case without empty strings)

1 Answer 1

3

From the field validator documentation (v2.9)

A few things to note on validators:

  • @field_validators are "class methods", so the first argument value they receive is the UserModel class, not an instance of UserModel. We recommend you use the @classmethod decorator on them below the @field_validator decorator to get proper type checking.

So you will need to add the missing @classmethod decorator, which needs to after the @field_validator decorator, as decorators get applied from the inside-out.

from typing import Optional
from pydantic import BaseModel, field_validator
    
class PydanticProduct(BaseModel):
    fid: Optional[float]
    water: Optional[float]

    class ConfigDict:
        from_attributes = True

    @field_validator('fid', 'water',  mode='before')
    @classmethod
    def check_empty_fields(cls, value) -> float:
        if value == '':
            return None
        try:
            float_value = float(value)
        except ValueError:
            raise ValueError("Unable to parse string as a number. Please provide a valid number.")
        return float_value

Update - In the v2.10 Pydantic docs, the wording was changed as the docs were rewritten to combine the Annotated and Field Validator methods in the documentation, but the behavior and usage of the @field_validator and @class_method decorators is the same. You will now find the following statement:

Four different types of validators can be used. They can all be defined using the annotated pattern or using the field_validator() decorator, applied on a class method:

Sign up to request clarification or add additional context in comments.

4 Comments

For me it was about the order of the annotations. I was using the @classmethod before field_validator.
Decorators get applied from the innermost layers first. Recall that they are just syntactic sugar for wrapping a function with another function, where the inner function gets passed to the outer function as a parameter, which then returns the a modified or replacement function.
I cannot find the sentence in the current documentation.
@Ziyuan In the current documentation you can find the following sentence before the examples of the four types of validators "Four different types of validators can be used. They can all be defined using the annotated pattern or using the field_validator() decorator, applied on a class method:". So there are now two possible methods - the annotated pattern, or apply the decorator to a class method. I will figure out the best way to edit my answer with the updated info.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.