[Feature Request]: Show workflow and pipeline descriptions in Hop Server status pages and API responses

[DBiLeSi]

What would you like to happen?

Apache Hop already lets users maintain a short description and longer extended description for workflows and pipelines in the Hop GUI properties. Hop Server also exposes workflowStatus / pipelineStatus pages and XML/JSON responses for running executions, but the server status UI currently focuses on execution status, action/transform details, logs, and the canvas preview rather than making the artifact description easy to see.

As Hop projects grow more complex, it becomes increasingly important to understand not only what a workflow does, but also why it exists, what business process it supports, and what operational assumptions it has. Good action and pipeline names help explain the “what,” but they are not enough for the “why.” This is especially true in Hop Server, where users may monitor or trigger workflows remotely and may not be able to drill down easily into called pipelines and subworkflows from the server view.

Proposed Behavior

1. Show the workflow or pipeline description and optionally extended description in the Hop Server workflowStatus and pipelineStatus HTML pages.
2. Include the same metadata in workflowStatus?json=Y, workflowStatus?xml=Y, pipelineStatus?json=Y, and pipelineStatus?xml=Y.
3. For asynchronous workflows or pipelines, make the artifact description available as an API Endpoint.
4. Optionally show descriptions for called child pipelines/subworkflows when that metadata is available.

Use Case
Teams often trigger Hop workflows remotely from external schedulers such as Rundeck, Apache Airflow, Control-M, Jenkins, Argo Workflows, Prefect, or Dagster. In those setups, descriptions are often duplicated in the scheduler UI because Hop Server does not expose the workflow/pipeline description clearly enough.
If Hop Server returned the description and extended description, external schedulers could automatically sync that metadata from Hop. Teams would only need to maintain the description in one place: the Hop workflow or pipeline itself.

Why This Helps

• Makes Hop Server status pages easier to understand for operators.
• Reduces duplicate documentation in external schedulers.
• Helps teams maintain one source of truth for workflow purpose and operational context.
• Improves supportability when projects contain many workflows, pipelines, and subworkflows.
• Helps explain the “why” behind a workflow, not just the action sequence.

Suggested Acceptance Criteria

• workflowStatus HTML displays workflow description and has an expandable/collapsible area for extended description.
• pipelineStatus HTML does the same for pipelines.
• JSON/XML status responses include description and extendedDescription fields.
• Empty descriptions are handled gracefully and do not clutter the UI.
• Existing status behavior remains backward-compatible.

Issue Priority

Priority: 3

Issue Component

Component: API

[mattcasters]

I'm not against improving the Hop Server web UI. However, please do not use the server status page as it's a not persisting its state. Consider it to be capable of dissapearing at any given time because it does. Rather, persist execution information in a location like a folder, a database, OpenSearch, Neo4j, ...

[DBiLeSi]

Thank you for the quick reply. Maybe let me clarify this a little. I think I want two separate things.

The first is the description of the workflow, which is executed as an asynchronous workflow on the server. The scenario here is: I notice that an execution did not go through as planned and can quickly see in the Server UI which workflow it was, along with some information about the workflow that I included in the description. This would help me identify the workflow causing the problem more quickly and hopefully provide the context I need to fix it.

The second scenario is the following: I have a HOP Server running with many asynchronous workflows. These workflows are triggered by a separate scheduling process, such as Rundeck or Airflow. The challenge is that, in the external scheduler, I would like to have a description of what each workflow does.

I can maintain this information by hand, but then I have to maintain two descriptions: one in HOP for the developers and one in the scheduler for the people who sometimes trigger the workflows manually or want to know what the workflows do. For example, which files are ingested and from where, or in which database table the final result is saved.

My idea is that, if the asynchronous workflow could expose the description of the underlying HOP workflow, this description could be used in the scheduler. This would mean that developers only need to maintain one description or piece of documentation, which could then be synchronized periodically between the HOP Server and the scheduler.

This would give us a single source of truth for the workflow documentation, located directly in the workflows themselves. I understand that the execution is stateless, but I am not interested in the execution. I am more interested in information about the underlying workflow, such as its description, extended description, and path.

The latter scenario is the more interesting one for me.

[hansva]

Interesting use case, but for your second scenario wouldn't it be better to get that information back in your initial request?
Having to query the status endpoints to get this information would cause extra work on the scheduling side.

So you probalby also want to enrich the asyncRun response to have more information about the workflow you are starting?

[DBiLeSi]

Yes, that would be a good way to implement that. Alternativly a new endpoint which does not start an async run and only returns the info.

[mattcasters]

There is AsyncStatusServlet providing /hop/asyncStatus.
More can be added to plugins/misc/async