Skip to content

[Bug][python] YAML workflow loader cannot resolve task_type "K8S" (Kubernetes.yaml example fails to parse) #176

Description

@initiallyqq

The shipped example examples/yaml_define/Kubernetes.yaml declares task_type: K8S but fails to load:

pydolphinscheduler.exceptions.PyDSTaskNoFoundException: cant not find task K8S

The YAML loader resolves a task type to its task class by class name only. In
src/pydolphinscheduler/core/yaml_workflow.py, get_task_cls() builds
{type_.capitalize(): type_ for type_ in tasks.__all__} and looks up
task_type.capitalize(). So it expects "Kubernetes" (the class name), but the
documented/constant task type is K8S:

  • "K8S".capitalize()"K8s", which is not in the class-name map.
  • The internal constant is TaskType.KUBERNETES = "K8S" (constants.py), i.e. the
    rest of the SDK and the backend use "K8S", but the YAML loader never consults
    TaskType — only class names. The loader and the rest of the SDK disagree on the
    K8S identifier.

There is a second bug in the same example: it uses camelCase field names
minCpuCores / minMemorySpace, but the Kubernetes task constructor expects
snake_case min_cpu_cores / min_memory_space. After getting past the loader you
then hit:

TypeError: __init__() missing 2 required positional arguments: 'min_cpu_cores' and 'min_memory_space'

Why CI never caught it

Kubernetes.yaml is the only one of the 22 examples omitted from the parametrize list in
tests/core/test_yaml_workflow.py::test_get_create_workflow (all others are covered).
So the broken example ships untested and CI stays green.

To Reproduce

from pydolphinscheduler.core.yaml_workflow import create_workflow
create_workflow("examples/yaml_define/Kubernetes.yaml")
# -> PyDSTaskNoFoundException: cant not find task K8S

Or just the resolver:

from pydolphinscheduler.core.yaml_workflow import get_task_cls
get_task_cls("K8S")   # -> PyDSTaskNoFoundException: cant not find task K8S
get_task_cls("Kubernetes")  # OK (only the class name works today)

Which version are you using?

Reproduced on:

  • PyPI release apache-dolphinscheduler 4.1.0 (latest release).
  • main branch HEADget_task_cls, Kubernetes.yaml, and the parametrize
    list are byte-for-byte the same as 4.1.0, confirming this is a current defect
    rather than a released-only regression.

Expected behavior

task_type: K8S (the documented constant value) should resolve to the Kubernetes
task class, and the shipped Kubernetes.yaml example should parse and be covered by CI.

Additional context

I'll open a PR that (1) makes the loader fall back to TaskType constant values,
(2) fixes the camelCase fields in the example, and (3) adds the example +
K8S/k8s/Kubernetes cases to the tests.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions