82

I'm making an API with FastAPI and Pydantic.

I would like to have some PATCH endpoints, where 1 or N fields of a record could be edited at once. Moreover, I would like the client to only pass the necessary fields in the payload.

Example:

class Item(BaseModel):
    name: str
    description: str
    price: float
    tax: float


@app.post("/items", response_model=Item)
async def post_item(item: Item):
    ...

@app.patch("/items/{item_id}", response_model=Item)
async def update_item(item_id: str, item: Item):
    ...

In this example, for the POST request, I want every field to be required. However, in the PATCH endpoint, I don't mind if the payload only contains, for example, the description field. That's why I wish to have all fields as optional.

Naive approach:

class UpdateItem(BaseModel):
    name: Optional[str] = None
    description: Optional[str] = None
    price: Optional[float] = None
    tax: Optional[float]

But that would be terrible in terms of code repetition.

Any better option?

2
  • 3
    From my experience in multiple teams using pydantic, you should (really) consider having those models duplicated in your code, just like you presented as an example. I think you shouldn't try to do what you're trying to do. Having it automatic mightseem like a quick win, but there are so many drawbacks behind, beginning with a lower readability. What happens when you have some special cases, like, the id of your item is not optional anymore. What happens when you have X, Y and Z special other cases ? Such special cases are so easily solved by having multiple explicit schema. You'll see. :) Commented Jan 20, 2024 at 13:31
  • Future readers might find this answer helpful as well. Commented Feb 27, 2024 at 11:07

18 Answers 18

52

This method prevents data validation

Read this by @Anime Bk: https://stackoverflow.com/a/75011200

Solution with metaclasses

I've just come up with the following:


class AllOptional(pydantic.main.ModelMetaclass):
    def __new__(cls, name, bases, namespaces, **kwargs):
        annotations = namespaces.get('__annotations__', {})
        for base in bases:
            annotations.update(base.__annotations__)
        for field in annotations:
            if not field.startswith('__'):
                annotations[field] = Optional[annotations[field]]
        namespaces['__annotations__'] = annotations
        return super().__new__(cls, name, bases, namespaces, **kwargs)

Use it as:

class UpdatedItem(Item, metaclass=AllOptional):
    pass

So basically it replace all non optional fields with Optional

Any edits are welcome!

With your example:

from typing import Optional

from fastapi import FastAPI
from pydantic import BaseModel
import pydantic

app = FastAPI()

class Item(BaseModel):
    name: str
    description: str
    price: float
    tax: float


class AllOptional(pydantic.main.ModelMetaclass):
    def __new__(self, name, bases, namespaces, **kwargs):
        annotations = namespaces.get('__annotations__', {})
        for base in bases:
            annotations.update(base.__annotations__)
        for field in annotations:
            if not field.startswith('__'):
                annotations[field] = Optional[annotations[field]]
        namespaces['__annotations__'] = annotations
        return super().__new__(self, name, bases, namespaces, **kwargs)

class UpdatedItem(Item, metaclass=AllOptional):
    pass

# This continues to work correctly
@app.get("/items/{item_id}", response_model=Item)
async def get_item(item_id: int):
    return {
        'name': 'Uzbek Palov',
        'description': 'Palov is my traditional meal',
        'price': 15.0,
        'tax': 0.5,
    }

@app.patch("/items/{item_id}") # does using response_model=UpdatedItem makes mypy sad? idk, i did not check
async def update_item(item_id: str, item: UpdatedItem):
    return item
Sign up to request clarification or add additional context in comments.

6 Comments

Hey - the solution doesn't appear to work for nested models, as in, if I have a model as an attribute of another and apply the metaclass to both of these objects, parse_obj will through validation errors. Any thoughts?
@hiimBacon does Maxim's solution work for that case?
It is possible in Pydantic V2?
@Fyzzys I haven't used Pydantic for a long time, it seems it still uses the Optional[T] syntax. I believe this code should work for the new version, you could try it out and reply if this is the case.
Regarding the ModelMetaclass , I received an error on import and because it has moved. Use this statement now: from pydantic._internal._model_construction import ModelMetaclass
|
27

Good news and bad news:

Bad: it's a wontfix, even in pydantic v2: https://github.com/pydantic/pydantic/issues/3120

Good: @adriangb - one of the core devs of pydantic - made a solution, which I translated into a neat decorator. It works for nested models.

Here it goes:

from typing import Optional, Type, Any, Tuple
from copy import deepcopy

from pydantic import BaseModel, create_model
from pydantic.fields import FieldInfo


def partial_model(model: Type[BaseModel]):
    def make_field_optional(field: FieldInfo, default: Any = None) -> Tuple[Any, FieldInfo]:
        new = deepcopy(field)
        new.default = default
        new.annotation = Optional[field.annotation]  # type: ignore
        return new.annotation, new
    return create_model(
        f'Partial{model.__name__}',
        __base__=model,
        __module__=model.__module__,
        **{
            field_name: make_field_optional(field_info)
            for field_name, field_info in model.__fields__.items()
        }
    )

The original code is here.

Usage:

@partial_model
class Model(BaseModel):
    i: int
    f: float
    s: str


Model(i=1)

Pydantic 2

As per @martintrapp's input, this solution also goes well with pydantic 2. The only thing you'll have to update is

model.__fields__.items() needs to be changed to model.model_fields.items() to make it work

19 Comments

I got the following error with this decorator: File "pydantic/json.py", line 90, in pydantic.json.pydantic_encoder / TypeError: Object of type 'ModelField' is not JSON serializable
It breaks the pickling on the created model object.
Nice! Best solution for Pydantic v2 so far. model.__fields__.items() needs to be changed to model.model_fields.items() to make it work.
This is an outstanding solution for pydantic v2.
This should be the accepted solution, it works in Pydantic v2 including the auto-generated OpenAPI swagger docs.
|
18

Modified @Drdilyor's solution

Added checking for nesting of models.

from pydantic.main import ModelMetaclass, BaseModel
from typing import Any, Dict, Optional, Tuple

class _AllOptionalMeta(ModelMetaclass):
    def __new__(self, name: str, bases: Tuple[type], namespaces: Dict[str, Any], **kwargs):
        annotations: dict = namespaces.get('__annotations__', {})

        for base in bases:
            for base_ in base.__mro__:
                if base_ is BaseModel:
                    break

                annotations.update(base_.__annotations__)

        for field in annotations:
            if not field.startswith('__'):
                annotations[field] = Optional[annotations[field]]

        namespaces['__annotations__'] = annotations

        return super().__new__(mcs, name, bases, namespaces, **kwargs)

6 Comments

Is there a way to make this general so it works with any pydantic model, Rather than inheriting from PydanticModel?
Never mind figured it out
How did you figure it out ?
@Cwellan class MyPydanticModelAllOptional(MyPydanticModel, metaclass=AllOptional) See @Drdilyor 's answer
Thanks, Maxim. Is there a way to make this work for fields that are nested models?
|
15

The problem is once FastAPI sees item: Item in your route definition, it will try to initialize an Item type from the request body, and you can't declare your model's fields to be optional sometimes depending on some conditional, such as depending on which route it is used.

I have 3 solutions:

Solution #1: Separate Models

I would say that having separate models for the POST and PATCH payloads seems to be the more logical and readable approach. It might lead to duplicated code, yes, but I think clearly defining which route has an all-required or an all-optional model balances out the maintainability cost.

The FastAPI docs has a section for partially updating models with PUT or PATCH that uses Optional fields, and there's a note at the end that says something similar:

Notice that the input model is still validated.

So, if you want to receive partial updates that can omit all the attributes, you need to have a model with all the attributes marked as optional (with default values or None).

So...

class NewItem(BaseModel):
    name: str
    description: str
    price: float
    tax: float

class UpdateItem(BaseModel):
    name: Optional[str] = None
    description: Optional[str] = None
    price: Optional[float] = None
    tax: Optional[float] = None

@app.post('/items', response_model=NewItem)
async def post_item(item: NewItem):
    return item

@app.patch('/items/{item_id}',
           response_model=UpdateItem,
           response_model_exclude_none=True)
async def update_item(item_id: str, item: UpdateItem):
    return item

Solution #2: Declare as All-Required, but Manually Validate for PATCH

You can define your model to have all-required fields, then define your payload as a regular Body parameter on the PATCH route, and then initialize the actual Item object "manually" depending on what's available in the payload.

from fastapi import Body
from typing import Dict

class Item(BaseModel):
    name: str
    description: str
    price: float
    tax: float

@app.post('/items', response_model=Item)
async def post_item(item: Item):
    return item

@app.patch('/items/{item_id}', response_model=Item)
async def update_item(item_id: str, payload: Dict = Body(...)):
    item = Item(
        name=payload.get('name', ''),
        description=payload.get('description', ''),
        price=payload.get('price', 0.0),
        tax=payload.get('tax', 0.0),
    )
    return item

Here, the Item object is initialized with whatever is in the payload, or some default if there isn't one. You'll have to manually validate if none of the expected fields are passed, ex.:

from fastapi import HTTPException

@app.patch('/items/{item_id}', response_model=Item)
async def update_item(item_id: str, payload: Dict = Body(...)):
    # Get intersection of keys/fields
    # Must have at least 1 common
    if not (set(payload.keys()) & set(Item.__fields__)):
        raise HTTPException(status_code=400, detail='No common fields')
    ...
$ cat test2.json
{
    "asda": "1923"
}
$ curl -i -H'Content-Type: application/json' --data @test2.json --request PATCH localhost:8000/items/1
HTTP/1.1 400 Bad Request
content-type: application/json

{"detail":"No common fields"}

The behavior for the POST route is as expected: all the fields must be passed.

Solution #3: Declare as All-Optional But Manually Validate for POST

Pydantic's BaseModel's dict method has exclude_defaults and exclude_none options for:

  • exclude_defaults: whether fields which are equal to their default values (whether set or otherwise) should be excluded from the returned dictionary; default False

  • exclude_none: whether fields which are equal to None should be excluded from the returned dictionary; default False

This means, for both POST and PATCH routes, you can use the same Item model, but now with all Optional[T] = None fields. The same item: Item parameter can also be used.

class Item(BaseModel):
    name: Optional[str] = None
    description: Optional[str] = None
    price: Optional[float] = None
    tax: Optional[float] = None

On the POST route, if not all the fields were set, then exclude_defaults and exclude_none will return an incomplete dict, so you can raise an error. Else, you can use the item as your new Item.

@app.post('/items', response_model=Item)
async def post_item(item: Item):
    new_item_values = item.dict(exclude_defaults=True, exclude_none=True)

    # Check if exactly same set of keys/fields
    if set(new_item_values.keys()) != set(Item.__fields__):
        raise HTTPException(status_code=400, detail='Missing some fields..')

    # Use `item` or `new_item_values`
    return item
$ cat test_empty.json
{
}
$ curl -i -H'Content-Type: application/json' --data @test_empty.json --request POST localhost:8000/items
HTTP/1.1 400 Bad Request
content-type: application/json

{"detail":"Missing some fields.."}

$ cat test_incomplete.json 
{
    "name": "test-name",
    "tax": 0.44
}
$ curl -i -H'Content-Type: application/json' --data @test_incomplete.json --request POST localhost:8000/items
HTTP/1.1 400 Bad Request
content-type: application/json

{"detail":"Missing some fields.."}

$ cat test_ok.json
{
    "name": "test-name",
    "description": "test-description",
    "price": 123.456,
    "tax": 0.44
}
$ curl -i -H'Content-Type: application/json' --data @test_ok.json --request POST localhost:8000/items
HTTP/1.1 200 OK
content-type: application/json

{"name":"test-name","description":"test-description","price":123.456,"tax":0.44}

On the PATCH route, if at least 1 value is not default/None, then that will be your update data. Use the same validation from Solution 2 to fail if none of the expected fields were passed in.

@app.patch('/items/{item_id}', response_model=Item)
async def update_item(item_id: str, item: Item):
    update_item_values = item.dict(exclude_defaults=True, exclude_none=True)

    # Get intersection of keys/fields
    # Must have at least 1 common
    if not (set(update_item_values.keys()) & set(Item.__fields__)):
        raise HTTPException(status_code=400, detail='No common fields')

    update_item = Item(**update_item_values)

    return update_item
$ cat test2.json
{
    "asda": "1923"
}
$ curl -i -s -H'Content-Type: application/json' --data @test2.json --request PATCH localhost:8000/items/1
HTTP/1.1 400 Bad Request
content-type: application/json

{"detail":"No common fields"}

$ cat test2.json
{
    "description": "test-description"
}
$ curl -i -s -H'Content-Type: application/json' --data @test2.json --request PATCH localhost:8000/items/1
HTTP/1.1 200 OK
content-type: application/json

{"name":null,"description":"test-description","price":null,"tax":null}

2 Comments

Thanks ! Great explanations. So, it looks like solution 2 is better than 3 as the manual validation for PATCH has to be done in both, while POST validation only in 3. But I agree solution 1 is easier to read when you are not alone in a project ...
Ah Pydantic, where instead of one model, we now need at least 3.
10

Apparently this doesn't work with Pydantic 2

Using a decorator

Using separate models seems like a bad idea for large projects. Lots of effectively duplicated code making it much harder to maintain. The goal of this is reusability and flexibility

from typing import Optional, get_type_hints, Type

from pydantic import BaseModel


def make_optional(
    include: Optional[list[str]] = None,
    exclude: Optional[list[str]] = None,
):
    """Return a decorator to make model fields optional"""

    if exclude is None:
        exclude = []

    # Create the decorator
    def decorator(cls: Type[BaseModel]):
        type_hints = get_type_hints(cls)
        fields = cls.__fields__
        if include is None:
            fields = fields.items()
        else:
            # Create iterator for specified fields
            fields = ((name, fields[name]) for name in include if name in fields)
            # Fields in 'include' that are not in the model are simply ignored, as in BaseModel.dict
        for name, field in fields:
            if name in exclude:
                continue
            if not field.required:
                continue
            # Update pydantic ModelField to not required
            field.required = False
            # Update/append annotation
            cls.__annotations__[name] = Optional[type_hints[name]]
        return cls

    return decorator

Usage

In the context of fast-api models

class ModelBase(pydantic.BaseModel):
  a: int
  b: str


class ModelCreate(ModelBase):
  pass

# Make all fields optional
@make_optional()
class ModelUpdate(ModelBase):
  pass
  • By default, all fields are made optional.
  • include specifies which fields to make optional; all other fields remain unchanged.
  • exclude specifies which fields not to affect.
  • exclude takes precedence over include.
# Make only `a` optional
@make_optional(include=["a"])
class ModelUpdate(ModelBase):
  pass

# Make only `b` optional
@make_optional(exclude=["a"])
class ModelUpdate(ModelBase):
  pass

Note: pydantic appears to make copies of the fields when you inherit from a base class, which is why it's ok to change them in-place

7 Comments

This is a nice clean alternative and a creative idea to use a decorator. I wonder if you could help with 2 improvements: 1. clarify use of include and exclude parameters to the decorator, as it's not clear to me which "way round" the sense / meaning is of those. Perhaps illustrate with a usage example. 2. please fix up to apply to nested models (in my case I only have two sub-objects, so only one extra level)
@NeilG Thanks for the feedback. I've added clarification about the include & exclude fields. I need to spend a little more time to think about the nested models which I can't right now. It's not ideal, but you could try something like this: python class A(BaseModel): ... class B(BaseModel): a: A ... @make_optional() class OptB(B): a: make_optional()(A) ... This is not so bad for just one level but yeah, it would be worth adding support for nested models.
@NeilG sorry about the formatting, it will only let me edit every 5 mins and I didn't know you can't add code blocks to comments
This solution worked for me, however, you lose the config from the base model class. In my case, I needed to add this back in by adding it to the create_model call like this: create_model(f'{baseclass.__name__}Update', **_fields, __validators__=validators, __config__=baseclass.Config)
This doesn't work with pydantic2 for anyone who is wondering
|
9

🚨⛔️ Pay attention, @Drdilyor's solution prevent Fields validation.

It seems that @Drdilyor's solution cancels all the fields validation

Let's say you have :

from typing import Optional
import pydantic
from pydantic import BaseModel, Field

class AllOptional(pydantic.main.ModelMetaclass):
    def __new__(self, name, bases, namespaces, **kwargs):
        annotations = namespaces.get('__annotations__', {})
        for base in bases:
            annotations.update(base.__annotations__)
        for field in annotations:
            if not field.startswith('__'):
                annotations[field] = Optional[annotations[field]]
        namespaces['__annotations__'] = annotations
        return super().__new__(self, name, bases, namespaces, **kwargs)

class A(BaseModel):
    a:int = Field(gt=1)

class AO(A, metaclass=AllOptional):
    pass

AO(a=-1) # This will pass through the validation even that it's wrong ⛔️

A simple alternative

class AllOptional(pydantic.main.ModelMetaclass):
    def __new__(mcls, name, bases, namespaces, **kwargs):
        cls = super().__new__(mcls, name, bases, namespaces, **kwargs)
        for field in cls.__fields__.values():
            field.required=False
        return cls

2 Comments

Thanks for the warning, @Amine_Bk. I haven't tested but if this is correct then I will use Zuir's answer stackoverflow.com/a/72365032/134044 because I think it's taking the same approach you are (functionally iterating over already built instance) but doesn't require going down into __new__ and can just be applied as a dependency with no other distraction in the code.
Thanks for the catch! I will link this in my solution, you can edit this into my answer if you want
6

For my case creating a new class was the only solution that worked, but packed into a function it is quite convenient:

from pydantic import BaseModel, create_model
from typing import Optional
from functools import lru_cache

@lru_cache(maxsize=None) # avoids creating many classes with same name
def make_optional(baseclass: Type[BaseModel]) -> Type[BaseModel]:
    # Extracts the fields and validators from the baseclass and make fields optional
    fields = baseclass.__fields__
    validators = {'__validators__': baseclass.__validators__}
    optional_fields = {key: (Optional[item.type_], None)
                       for key, item in fields.items()}
    return create_model(f'{baseclass.__name__}Optional', **optional_fields,
                        __validators__=validators)

class Item(BaseModel):
    name: str
    description: str
    price: float
    tax: float

ItemOptional = make_optional(Item)

Comparing after and before:

> Item.__fields__

{'name': ModelField(name='name', type=str, required=True),
 'description': ModelField(name='description', type=str, required=True),
 'price': ModelField(name='price', type=float, required=True),
 'tax': ModelField(name='tax', type=float, required=True)}

> ItemOptional.__fields__

{'name': ModelField(name='name', type=Optional[str], required=False, default=None),
 'description': ModelField(name='description', type=Optional[str], required=False, default=None),
 'price': ModelField(name='price', type=Optional[float], required=False, default=None),
 'tax': ModelField(name='tax', type=Optional[float], required=False, default=None)}

It does work, and also it allows you to filter out some fields in the dict_comprehension if it is required.

Moreover in fastapi this approach allows you to do something like this:

@app.post("/items", response_model=Item)
async def post_item(item: Item = Depends()):
    ...

@app.patch("/items/{item_id}", response_model=Item)
async def update_item(item_id: str, item: make_optional(Item) = Depends()):
    ...

Which reduces a lot the boilerplate, using the same approach you can also make a function that makes optional the fields and also exclude a field in case your Item has an ID field,the id would will be repeated in your PATCH call. That can be solved like this:

def make_optional_no_id(baseclass):
    ... # same as make optional
    optional_fields = {key: (Optional[item.type_], None) 
                       for key, item in fields.items() if key != 'ID'} # take out here ID
    ... # you can also take out also validators of ID

@app.patch("/items/{item_id}", response_model=Item)
async def update_item(item: make_optional_no_id(Item) = Depends()):

11 Comments

I think I like this one even more than the decorator, because you can just stick it in a dependency and then there's nothing else to distract. This makes a lot of sense. I wonder if you could please update it to support nested models?
It's being many months since I did this code, but if you provide a snippet supporting that case I will update my answer. In any case if you use this approach several times with the same class I recommend to cache the result to prevent having many classes with the same name. I have edited my answer to prevent that.
Did you just add the lru_cache? I'm not sure if I saw it there before. I guess that deals with that problem very nicely. Thanks for your offer. If it's ok with you I might just create a new question and link it here. I'm not sure where to put the snippet otherwise.
yes lru_cache is the new edition, and sure, as you want.
Thanks again for your help and support @Ziur_Olpa. I have posted a new question here: stackoverflow.com/q/75167317/134044
|
5

I really liked how @mishnea used a decorator in his answer and ended up using this response (modified for more flexibility).

from copy import deepcopy
from typing import Any, Callable, Optional, TypeVar

from pydantic import BaseModel, create_model
from pydantic.fields import FieldInfo

T = TypeVar("T", bound="BaseModel")


def partial_model(
    include: Optional[list[str]] = None, exclude: Optional[list[str]] = None
) -> Callable[[type[T]], type[T]]:
    """Return a decorator to make model fields optional"""

    if exclude is None:
        exclude = []

    def decorator(model: type[T]) -> type[T]:
        def make_optional(
            field: FieldInfo, default: Any = None
        ) -> tuple[Any, FieldInfo]:
            new = deepcopy(field)
            new.default = default
            new.annotation = Optional[field.annotation or Any]
            return new.annotation, new

        fields = model.model_fields
        if include is None:
            fields = fields.items()
        else:
            fields = ((k, v) for k, v in fields.items() if k in include)

        return create_model(
            model.__name__,
            __base__=model,
            __module__=model.__module__,
            **{
                field_name: make_optional(field_info)
                for field_name, field_info in fields
                if exclude is None or field_name not in exclude
            },  # type: ignore
        )

    return decorator

This solution works for Pydantic v2

Comments

2

This simple trick works for me: build a new model class dynamically and modify fields to be optional as needed.

def make_partial_model(model: Type[BaseModel], optional_fields: Optional[list[str]] = None) -> Type[BaseModel]:
    class NewModel(model):
        ...

    for field in NewModel.__fields__.values():
        if not optional_fields or field in optional_fields:
            field.required = False

    NewModel.__name__ = f'Partial{model.__name__}'
    return NewModel

PartialRequest = cast(Type[RequestModel], make_partial_model(RequestModel))

Comments

2

Given the original accepted answer is deprecated in pydantic v2, ModelMetaclass is not public anymore, also it invalidates the pydantic field validation, the below solution is somehow more elegant, in my opinion, it also handles submodules recursively and it uses the suggested way of dynamcally creating models with pydantic.create_model.

The only issue with this implementation is that mypy does not handle dynamic models. Please let me know if there is a better option that also works with mypy.

Usage example based on the example from the quetion:

class Item(BaseModel):
    name: str
    description: str
    price: float
    tax: float


@app.post("/items", response_model=Item)
async def post_item(item: Partial[Item]):
    ...
Model = typing.TypeVar("Model", bound=BaseModel)
class Partial(typing.Generic[Model]):
"""Generate a new class with all attributes optionals.

Notes:
    This will wrap a class inheriting form BaseModel and will recursively
    convert all its attributes and its children's attributes to optionals.

Example:
    Partial[SomeModel]
"""

def __new__(
    cls,
    *args: object,  # noqa :ARG003
    **kwargs: object,  # noqa :ARG003
) -> "Partial[Model]":
    """Cannot instantiate.

    Raises:
        TypeError: Direct instantiation not allowed.
    """
    raise TypeError("Cannot instantiate abstract Partial class.")

def __init_subclass__(
    cls,
    *args: object,
    **kwargs: object,
) -> typing.NoReturn:
    """Cannot subclass.

    Raises:
       TypeError: Subclassing not allowed.
    """
    raise TypeError("Cannot subclass {}.Partial".format(cls.__module__))

def __class_getitem__(  # type: ignore[override]
    cls,
    wrapped_class: type[Model],
) -> type[Model]:
    """Convert model to a partial model with all fields being optionals."""

    def _make_field_optional(
        field: pydantic.fields.FieldInfo,
    ) -> tuple[object, pydantic.fields.FieldInfo]:
        tmp_field = copy.deepcopy(field)

        annotation = field.annotation
        # If the field is a BaseModel, then recursively convert it's
        # attributes to optionals.
        if type(annotation) is type(BaseModel):
            tmp_field.annotation = typing.Optional[Partial[annotation]]  # type: ignore[assignment, valid-type]
            tmp_field.default = {}
        else:
            tmp_field.annotation = typing.Optional[field.annotation]  # type: ignore[assignment]
            tmp_field.default = None
        return tmp_field.annotation, tmp_field

    return pydantic.create_model(  # type: ignore[no-any-return, call-overload]
        f"Partial{wrapped_class.__name__}",
        __base__=wrapped_class,
        __module__=wrapped_class.__module__,
        **{
            field_name: _make_field_optional(field_info)
            for field_name, field_info in wrapped_class.model_fields.items()
        },
    )

Comments

1

For Pydantic v2, such a class can be created with create_model(). In v2, required is no longer an attribute of FieldInfo object so you cannot do the trick of field_info.required = False.

from pydantic import BaseModel, create_model

class Item(BaseModel):
    name: str
    description: str
    price: float
    tax: float

UpdateItem = create_model(
    'UpdateItem',
    __base__=Item,
    **{k: (v.annotation, None) for k, v in Item.model_fields.items()}
)

Then,

In [410]: Item.model_fields
Out[410]: 
{'name': FieldInfo(annotation=str, required=True),
 'description': FieldInfo(annotation=str, required=True),
 'price': FieldInfo(annotation=float, required=True),
 'tax': FieldInfo(annotation=float, required=True)}

In [411]: UpdateItem.model_fields
Out[411]: 
{'name': FieldInfo(annotation=str, required=False),
 'description': FieldInfo(annotation=str, required=False),
 'price': FieldInfo(annotation=float, required=False),
 'tax': FieldInfo(annotation=float, required=False)}

In [412]: UpdateItem()
Out[412]: UpdateItem(name=None, description=None, price=None, tax=None)

3 Comments

This is the way for Pydantic V2. I would recommend adding the base model as a parameter to create_model() to ensure any model_config is also copied over.
In this problem, __base__ is not required and If UpdateItem is to be configured or validated differently from the base class Item, __base__ cannot be used together with __config__ or __validators__ arguments. So, I'm not sure if I would set __base__=Item.
This solution works but fails to validate. I only tested with a field with conint(ge=0). @winwin's solution seems to keep validation.
1

Modified @Drdilyor's answer

I've made a version that lets you define required arguments in the child class (like the ID of the id of the item you want to update for example) :

class AllOptional(ModelMetaclass):
def __new__(self, name, bases, namespaces, **kwargs):
    annotations = namespaces.get('__annotations__', {})
    for base in bases:
        optionals = {
            key: Optional[value] if not key.startswith('__') else value for key, value in base.__annotations__.items()
        }
        annotations.update(optionals)

    namespaces['__annotations__'] = annotations
    return super().__new__(self, name, bases, namespaces, **kwargs)

Comments

1

By the deprecation of ModelMetaclass as a public class, I propose that you do the same (as all the above) but through a function, instead of the OOP approach.

def convert_to_optional(schema):
   return {k: Optional[v] for k, v in schema.__annotations__.items()}

And apply it to the __annotations__ of the parent class, as in:

class UpdateItem(Item):
   __annotations__ = convert_to_optional(Item)

I hope it helps, also so people don't have to do private import (from pydantic._internal._model_construction import ModelMetaclass).

Comments

1

this will create a model class with all fields Optional annotated and None as default value.

from typing import Optional
from pydantic import create_model

class Item(BaseModel):
    name: str
    description: str
    price: float
    tax: float

UpdateItem = create_model(
    "UpdateItem", 
    **{k: (Optional[v], None) for k, v in Item.__annotations__.items()})

UpdateItem()
# UpdateItem(name=None, description=None, price=None, tax=None)

Comments

0

Setting fields to Optional is a bit bad though, as it changes your python types. If you ever have a field you would like to be able to set to None via a patch request, you will run into a problem.

For us, using default values works better. Let me explain.

You model would be something like this:

from typing import Optional
from uuid import UUID, uuid4

import pydantic

class PatchPoll(pydantic.BaseModel):
    id: UUID = pydantic.Field(default_factory=uuid4)
    subject: str = pydantic.Field(max_length=1024, default="")
    description: Optional[str] = pydantic.Field(max_length=1024 * 1024, default="")


class Poll(PatchPoll):
    id: UUID
    subject: str = pydantic.Field(max_length=1024)
    description: Optional[str] = pydantic.Field(max_length=1024 * 1024)

You can then use PatchPoll without as many attributes as you like. When it comes to actually applying the patch, make sure you're using __fields_set__ to only update fields which were specified by the client.

>>> PatchPoll()
PatchPoll(id=UUID('dcd80011-e81e-41fb-872b-4f82839a2a76'), subject='', description='')
>>> PatchPoll().__fields_set__
set()
>>> PatchPoll(subject="jskdlfjk").__fields_set__
{'subject'}

I know, it's a little bit of plumbing, but the added value of clean types in python makes it worth it IMHO.

As a bonus, you could even use a function to create the regular versions from the patch version:

def remove_defaults(baseclass: Type[T]) -> Type[T]:
    validators = {"__validators__": baseclass.__validators__}
    fields = baseclass.__fields__

    def remove_default(item: pydantic.fields.ModelField) -> pydantic.fields.FieldInfo:
        info = item.field_info
        if info.default == pydantic.fields.Undefined and not info.default_factory:
            raise RuntimeError("Field has no default")

        # Funny enough, if we don't keep the default for Optional types,
        # openapi-generator will not make it optional at all.
        if item.allow_none:
            return copy.copy(item.field_info)

        return pydantic.Field(
            alias=item.field_info.alias,
            title=item.field_info.title,
            description=item.field_info.description,
            exclude=item.field_info.exclude,
            include=item.field_info.include,
            const=item.field_info.const,
            gt=item.field_info.gt,
            ge=item.field_info.ge,
            lt=item.field_info.lt,
            le=item.field_info.le,
            multiple_of=item.field_info.multiple_of,
            allow_inf_nan=item.field_info.allow_inf_nan,
            max_digits=item.field_info.max_digits,
            decimal_places=item.field_info.decimal_places,
            min_items=item.field_info.min_items,
            max_items=item.field_info.max_items,
            unique_items=item.field_info.unique_items,
            min_length=item.field_info.min_length,
            max_length=item.field_info.max_length,
            allow_mutation=item.field_info.allow_mutation,
            regex=item.field_info.regex,
            discriminator=item.field_info.discriminator,
            repr=item.field_info.repr,
        )

    nondefault_fields = {
        key: (item.type_, remove_default(item)) for key, item in fields.items()
    }

    return pydantic.create_model(
        __model_name=f"{baseclass.__name__}Optional",
        __base__=baseclass,
        __validators__=validators,
        **nondefault_fields,
    )


class PatchPoll(pydantic.BaseModel):
    id: UUID = pydantic.Field(default_factory=uuid4)
    subject: str = pydantic.Field(max_length=1024, default="")
    description: Optional[str] = pydantic.Field(max_length=1024 * 1024, default="")


class Poll(remove_defaults(PatchPoll)):
    ...

Comments

0

Pydantic V2 variant:

import inspect
import typing
from pydantic._internal._model_construction import ModelMetaclass
from pydantic._internal._generics import PydanticGenericMetadata
from pydantic.fields import FieldInfo
from pydantic_core import PydanticUndefined


class MakeOptional(ModelMetaclass):
    def __new__(
        mcs,  # noqa: N804
        cls_name: str,
        bases: tuple[type[typing.Any], ...],
        namespace: dict[str, typing.Any],
        __pydantic_generic_metadata__: typing.Optional[PydanticGenericMetadata] = None,
        __pydantic_reset_parent_namespace__: bool = True,
        _create_model_module: typing.Optional[str] = None,
        **kwargs: typing.Any,
    ) -> type:
        if len(bases) > 1:
            raise NotImplementedError(
                "`MakeOptional` can't work with more then one base class"
            )
        if not bases or not issubclass(bases[0], pd.BaseModel):
            raise TypeError('Must be inherited from pydantic.BaseModel')

        annotations: dict[str, object] = {}

        for base in inspect.getmro(bases[0]):
            if not issubclass(base, pd.BaseModel):
                continue

            annotations.update(getattr(base, '__annotations__', {}))
            for field_name in getattr(base, '__annotations__', {}):
                if field_name.startswith('__'):
                    continue
                field_info = base.model_fields.get(field_name)
                if not field_info:
                    continue
                field_info.annotation = typing.Optional[field_info.annotation]  # type: ignore
                new_annotation = field_info.rebuild_annotation()
                new_field = FieldInfo.from_annotation(field_info.rebuild_annotation())
                if new_field.default is PydanticUndefined:
                    new_field.default = None
                annotations[field_name] = new_annotation
                setattr(base, field_name, new_field)
        namespace['__annotations__'] = annotations
        return super().__new__(
            mcs,
            cls_name,
            bases,
            namespace,
            __pydantic_generic_metadata__,
            __pydantic_reset_parent_namespace__,
            _create_model_module,
            **kwargs,
        )

Usage:

class ResponseModelOptional(
    ResponseModelStrict,
    metaclass=MakeOptional,
):
    pass

4 Comments

This looks promising (doesn't use create_model so maybe it could be picked), but it would be great to see this with functioning imports and a working example of usage. From what I can tell, some of these imports look to be using private (underscored) APIs?
@Brendan Added usage example. There is no private imports, __pydantic_generic_metadata__ and others it's pydantic arg names and I saved these names for no trigger pylint rule "too-many-args". So, this code passes mypy/pylint/ruff
Are you able to post a full example with the imports you used? I can't find any reference to e.g. ModelMetaclass or PydanticGenericMetadata in the pydantic docs.
I edited to add support for imports. This answer is quite nice, but unfortunately doesn't seem to work for my, admittedly very niche, use case (involving using schema from this to dynamically generate pyspark UDF return types).
0

Based on @slvmrc's and @ogtega's answers. Here's a fusion of the two which I have being using so far.

import copy
import typing
import pydantic
import functools
import weakref

Model = typing.TypeVar("Model", bound=pydantic.BaseModel)
_Depth: typing.TypeAlias = typing.Union[bool, int]
_Prefix: typing.TypeAlias = str

DEFAULT_PREFIX = "Partial"
TOP_LEVEL = 0

# Cache for created models
_model_cache = weakref.WeakValueDictionary()


@typing.overload
def partial(
    model_cls: typing.Optional[typing.Type[Model]] = None,  # noqa :ARG006
) -> typing.Type[Model]: ...


@typing.overload
def partial(
    *,
    include: typing.Optional[typing.List[str]] = None,
    depth: _Depth = TOP_LEVEL,
    prefix: typing.Optional[_Prefix] = None,
) -> typing.Callable[[typing.Type[Model]], typing.Type[Model]]: ...


@typing.overload
def partial(
    *,
    exclude: typing.Optional[typing.List[str]] = None,
    depth: _Depth = TOP_LEVEL,
    prefix: typing.Optional[_Prefix] = None,
) -> typing.Callable[[typing.Type[Model]], typing.Type[Model]]: ...


def _make_optional(
    field: pydantic.fields.FieldInfo,
    default: typing.Any,
    depth: _Depth,
    prefix: typing.Optional[_Prefix],
) -> tuple[object, pydantic.fields.FieldInfo]:
    """Helper function to make a field optional.

    :param field: The field to make optional
    :param default: Default value for the optional field
    :param depth: How deep to make nested models optional
    :param prefix: String to prepend to nested model names
    :returns: Tuple of (annotation, field_info)
    :raises ValueError: If depth is negative
    """
    tmp_field = copy.deepcopy(field)
    annotation = field.annotation or typing.Any

    if isinstance(depth, int) and depth < 0:
        raise ValueError("Depth cannot be negative")

    if (
        isinstance(annotation, type)
        and issubclass(annotation, pydantic.BaseModel)
        and depth
    ):
        model_key = (annotation, depth, prefix)
        if model_key not in _model_cache:
            _model_cache[model_key] = partial(
                depth=depth - 1 if isinstance(depth, int) else depth,
                prefix=prefix,
            )(annotation)
        annotation = _model_cache[model_key]

    tmp_field.annotation = typing.Optional[annotation]
    tmp_field.default = default
    return tmp_field.annotation, tmp_field


def partial(
    model_cls: typing.Optional[typing.Type[Model]] = None,  # noqa :ARG006
    *,
    include: typing.Optional[typing.List[str]] = None,
    exclude: typing.Optional[typing.List[str]] = None,
    depth: _Depth = TOP_LEVEL,
    prefix: typing.Optional[_Prefix] = None,
) -> typing.Callable[[typing.Type[Model]], typing.Type[Model]]:
    """
    Create a partial Pydantic model with optional fields.

    This decorator allows you to create a new model based on an existing one,
    where specified fields become optional. It's particularly useful for update
    operations where only some fields may be provided.

    :param model_cls: The Pydantic model to make partial
    :param include: List of field names to make optional. If None, all fields are included
    :param exclude: List of field names to keep required. If None, no fields are excluded
    :param depth: How deep to make nested models optional:
        - 0: Only top-level fields
        - n: n levels deep
        - True: All levels
    :param prefix: String to prepend to the new model's name
    :returns: A decorator function that creates a new model with optional fields
    :raises ValueError: If both include and exclude are provided
    :raises ValueError: If depth is negative

    Example:
        ```python
        @partial
        class UserUpdateSchema(UserSchema):
            pass

        # Make specific fields optional
        @partial(include=['name', 'email'])
        class UserPartialSchema(UserSchema):
            pass

        # Keep certain fields required
        @partial(exclude=['id'])
        class UserUpdateSchema(UserSchema):
            pass
        ```

    - Uses model caching to avoid recreating identical partial models
    """
    if include is not None and exclude is not None:
        raise ValueError("Cannot specify both include and exclude")

    if exclude is None:
        exclude = []

    @functools.lru_cache(maxsize=32)
    def create_partial_model(model_cls: typing.Type[Model]) -> typing.Type[Model]:
        """
        Create a new Pydantic model with optional fields.

        Cached model creation to avoid regenerating same models.
        """
        fields = model_cls.model_fields
        if include is None:
            fields = fields.items()
        else:
            fields = ((k, v) for k, v in fields.items() if k in include)

        return pydantic.create_model(
            f"{prefix or ''}{model_cls.__name__}",
            __base__=model_cls,
            __module__=model_cls.__module__,
            **{
                field_name: _make_optional(
                    field_info,
                    default=field_info.default
                    if field_info.default is not pydantic.fields.PydanticUndefined
                    else None,
                    depth=depth,
                    prefix=prefix,
                )
                for field_name, field_info in fields
                if exclude is None or field_name not in exclude
            },
        )

    if model_cls is None:
        return create_partial_model
    return create_partial_model(model_cls)


class _ModelConfig(typing.NamedTuple):
    """Configuration for partial model creation."""

    model: typing.Type[Model]
    depth: _Depth
    prefix: _Prefix


def _create_model_config(*args: typing.Any) -> _ModelConfig:
    """
    Factory function to create and validate model configuration.

    :raises TypeError: If arguments are invalid
    """
    if not args:
        raise TypeError("Model type argument is required")

    if len(args) > 3:
        raise TypeError(f"Expected at most 3 arguments, got {len(args)}")

    model, *rest = args
    if not (isinstance(model, type) and issubclass(model, pydantic.BaseModel)):
        raise TypeError(f"Expected BaseModel subclass, got {type(model)}")

    if not rest:
        return _ModelConfig(model, TOP_LEVEL, DEFAULT_PREFIX)

    depth = rest[0]
    if not isinstance(depth, (int, bool)):
        if not isinstance(depth, str):
            raise TypeError(
                f"Expected int, bool or str for depth/prefix, got {type(depth)}"
            )
        # Case where first arg is prefix
        return _ModelConfig(model, TOP_LEVEL, depth)

    prefix = rest[1] if len(rest) > 1 else DEFAULT_PREFIX
    if not isinstance(prefix, str):
        raise TypeError(f"Expected str for prefix, got {type(prefix)}")

    return _ModelConfig(model, depth, prefix)


class Partial(typing.Generic[Model]):
    """
    Type hint for creating partial Pydantic models.

    Supports three forms of instantiation:
    1. Partial[Model]  # Uses default depth and prefix
    2. Partial[Model, depth]  # Uses default prefix
    3. Partial[Model, depth, prefix]
    4. Partial[Model, prefix]  # Uses default depth

    :param Model: The Pydantic model to make partial
    :param depth: How deep to make fields optional (int, bool)
    :param prefix: Prefix for the generated model name (str)

    Example:
        ```python
        class User(BaseModel):
            name: str
            age: int

        # These are all valid:
        PartialUser = Partial[User]  # depth=0, prefix="Partial"
        UpdateUser = Partial[User, "Update"]  # depth=0, prefix="Update"
        DeepUpdateUser = Partial[User, True, "Update"]  # All nested fields optional
        ```
    """

    def __class_getitem__(  # type: ignore[override]
        cls,
        wrapped: typing.Union[typing.Type[Model], typing.Tuple[typing.Any, ...]],
    ) -> typing.Type[Model]:
        """Converts model to a partial model with optional fields."""
        args = wrapped if isinstance(wrapped, tuple) else (wrapped,)
        config = _create_model_config(*args)

        return partial(
            depth=config.depth,
            prefix=config.prefix,
        )(config.model)  # type: ignore[no-any-return, return-value]

    def __new__(
        cls,
        *args: object,  # noqa :ARG003
        **kwargs: object,  # noqa :ARG003
    ) -> "Partial[Model]":
        """Cannot instantiate.

        :raises TypeError: Direct instantiation not allowed.
        """
        raise TypeError("Cannot instantiate abstract Partial class.")

    def __init_subclass__(
        cls,
        *args: object,
        **kwargs: object,
    ) -> typing.NoReturn:
        """Cannot subclass.

        :raises TypeError: Subclassing not allowed.
        """
        raise TypeError("Cannot subclass {}.Partial".format(cls.__module__))

1 Comment

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.
-1

In FastAPI documentation, now we have:

response_model_exclude_unset=True

With this parameter, we won't show in the response any optional value that we defined as = None.

@router.get(
    "/mypath",
    response_model=ResourcesListResponse,
    response_model_exclude_unset=True,
)
async def get_my_func( ...

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.