Skip to content

feat(pydevlake): migrate Python plugin framework to Pydantic v2 / SQLModel 0.0.38#1

Draft
DoDiODev wants to merge 1 commit into
pr/python-311from
pr/pydantic-v2
Draft

feat(pydevlake): migrate Python plugin framework to Pydantic v2 / SQLModel 0.0.38#1
DoDiODev wants to merge 1 commit into
pr/python-311from
pr/pydantic-v2

Conversation

@DoDiODev

Copy link
Copy Markdown
Owner

Summary

Migrates the pydevlake Python plugin framework and the azuredevops remote plugin from Pydantic v1 to Pydantic v2 and SQLModel 0.0.38, plus related dependency bumps and runtime pins.

Stacked PR (Draft): This PR is based on pr/python-311 (Python 3.9 → 3.11, upstream PR apache#9005). It will be rebased onto main and marked Ready once the base is merged. Review only the delta against pr/python-311.

Scope

Framework (pydevlake)

  • Replace v1 APIs: .json()/.dict()model_dump_json()/model_dump(), @validator@field_validator, inner class Configmodel_config = ConfigDict(...), Field(..., const=True)Literal, __root__RootModel.
  • extractor.autoextract: construct via model_validate so SQLModel table=True models coerce raw values (strdatetime/Enum), which v2 no longer does implicitly on __init__.
  • model_info: read schema via model_json_schema, drop $defs, flatten Optional anyOf into a flat type expected by the Go backend.
  • Add explicit = None defaults to Optional fields (required under v2) across the domain layer, tool/domain models, DomainModel.id, and the azuredevops plugin models.
  • testing.assert_stream_convert: re-validate both operands to preserve v1 coercion semantics in equality checks.

Dependencies / tooling

  • pyproject.toml: sqlmodel ^0.0.38, pydantic ^2.13, inflect ^7.5, fire ^0.7, jsonpointer ^3.1, pytest ^9.1; regenerated poetry.locks.
  • uv.sh: force-reinstall local path deps so pydevlake source changes are always picked up (uv caches path deps by the static 0.1.0 dev version).
  • requirements.txt: pin setuptools/wheel/dbt-mysql/dbt-postgres.

Excluded

  • No sqlglot change (separate PR).
  • No fork-specific artifacts.

Testing

  • make unit-test-pythongreen (azuredevops 8 passed/1 skipped, pydevlake 10 passed).
  • make e2e-test-go-pluginsgreen (31 Go plugin e2e suites, no failures).
  • Running server loads the azuredevops remote plugin cleanly (plugin-info handshake + migrations via RPC, no errors) — verifies the v2 wire format / JSON schema against the Go backend.

Rollback

Revert this commit; the framework returns to the Pydantic v1 / SQLModel 0.0.8 state of the base branch.

…Model 0.0.38

Migrate the pydevlake framework and the azuredevops remote plugin from
Pydantic v1 to Pydantic v2 and SQLModel 0.0.38, and bump the related
dependencies (inflect, fire, jsonpointer, pytest) and pin the Python
runtime dependencies.

Framework changes:
- Replace v1 APIs: .json()/.dict() -> model_dump_json()/model_dump(),
  @validator -> @field_validator, inner class Config -> model_config
  via ConfigDict, Field(..., const=True) -> Literal, __root__ -> RootModel.
- extractor.autoextract: construct models via model_validate so that
  SQLModel table=True models coerce raw values (str -> datetime/Enum),
  which v2 no longer does implicitly on __init__.
- model_info: read the schema via model_json_schema, drop $defs and
  flatten Optional anyOf into a flat "type" expected by the Go backend.
- Give Optional fields explicit "= None" defaults (required under v2)
  across the domain layer, tool/domain models, DomainModel.id and the
  azuredevops plugin models.
- testing.assert_stream_convert: re-validate both operands so equality
  comparisons keep the v1 coercion semantics.

Tooling:
- uv.sh: force-reinstall local path dependencies so pydevlake source
  changes are always picked up (uv caches path deps by version, which
  stays 0.1.0 during development).
- requirements.txt: pin setuptools/wheel/dbt-mysql/dbt-postgres.

Verified with make unit-test-python and make e2e-test-go-plugins (both
green) and a running server that loads the azuredevops remote plugin.

Signed-off-by: DoDiODev <DoDiDev@proton.me>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant