[srusia-delphix]

Problem

Jira: DLPX-99548

DLPX-87054 fixed a Teradata masking failure on Kettle/Hop 1.2.0 by adding a HOP_COMPATIBILITY_DB_IGNORE_TIMEZONE system property to ValueMetaBase. That flag no longer exists anywhere in this fork's delphix/hop-2.17 branch (nor in stock Apache Hop 2.10.0 through 2.17.0 — checked directly against the 2.10.0-rc1, 2.15.0-rc1, 2.16.0-rc1 and 2.17.0-rc1 tags), so the original DLPX-87054 failure mode is back on Hop 2.x:

Two different data types are being set for parameter N (675 & 773)

The Teradata JDBC driver assigns a different SQL type to a PreparedStatement timestamp parameter depending on which overload binds it:

• setTimestamp(idx, ts, Calendar) → type 773 (TIMESTAMP WITH TIME ZONE)
• setTimestamp(idx, ts) / setNull(idx, Types.TIMESTAMP) → type 675 (plain TIMESTAMP)

A batch insert that mixes null and non-null rows for the same nullable TIMESTAMP column binds both types to one parameter, and Teradata rejects the batch outright.

This surfaced during the dms-core-gate Hop 1.2.0 → 2.17 upgrade (DLPX-97999), via PR review on delphix/dms-core-gate#2337 (review comment). dms-core-gate still sets this flag today for Teradata connectors (RunnerUtil.java, gated on ExtendedPlatform::isTeradataConnector), so it silently becomes a no-op the moment the engine takes on this Hop version — until Teradata customers hit the batch-insert failure again.

Evaluation

setPreparedStatementValue()'s TYPE_DATE branch unconditionally uses the Calendar overload whenever getDateFormatTimeZone() != null. dateFormatTimeZone defaults to TimeZone.getDefault() in the ValueMetaBase constructor, so it is essentially never null — meaning every Teradata timestamp binds via the Calendar overload (773) regardless of whether the row's value is null or not, reproducing the exact DLPX-87054 symptom.

In Hop 1.2.0, the same method short-circuited the Calendar overload with if (ignoreTimezone || this.getDateFormatTimeZone() == null), backed by an ignoreTimezone field read once from HOP_COMPATIBILITY_DB_IGNORE_TIMEZONE in the constructor. Neither the field nor the constant survived the 1.2.0 → 2.x migration — this is an upstream Hop regression, not something a Delphix patch introduced or dropped.

Solution

Restore the flag exactly as 1.2.0 had it:

• Const.HOP_COMPATIBILITY_DB_IGNORE_TIMEZONE — the system-property name, with a @Variable doc matching the style of the neighboring HOP_EMPTY_STRING_DIFFERS_FROM_NULL / HOP_LENIENT_STRING_TO_NUMBER_CONVERSION flags.
• ValueMetaBase.ignoreTimezone — read once in the constructor, same as the two flags above.
• All three Calendar-overload call sites in setPreparedStatementValue() (setDate, and both setTimestamp branches) gated on ignoreTimezone || this.getDateFormatTimeZone() == null.

Default behavior (property unset) is unchanged — this only takes effect when a caller explicitly sets HOP_COMPATIBILITY_DB_IGNORE_TIMEZONE=Y, exactly as dms-core-gate already does for Teradata connectors.

Testing Done

check	result
ValueMetaBaseSetPreparedStmntValueTest	6 tests, 0 failures (4 existing + 2 added)
ValueMetaBaseTest	67 tests, 0 failures

Two tests added to ValueMetaBaseSetPreparedStmntValueTest, mirroring the existing testDateRegular / testTimestampRegular style in that file:

• testDateWithIgnoreTimezoneSkipsCalendarOverload — with the property set, asserts setDate(idx, date) is called and setDate(idx, date, Calendar) is not.
• testTimestampWithIgnoreTimezoneSkipsCalendarOverload — same shape for setTimestamp.

RestoreHopEnvironment now also clears HOP_COMPATIBILITY_DB_IGNORE_TIMEZONE between tests, matching how it already clears the two neighboring compatibility flags.

Still outstanding. This has not yet been validated against a live or emulated Teradata connection — the tests above confirm the PreparedStatement call shape, not an actual Teradata batch insert. Before dms-core-gate takes a dependency on this for Teradata masking: publish an updated 2.17.0-dlpx1 artifact to masking-maven (the one currently published predates this fix), bump dms-core-gate's hopVersion, and confirm with a real Teradata run.

Notes to Reviewers

This is a straight restoration of 1.2.0 behavior, not new design — the diff should read as "put back what upstream removed." The Jira ID in the code comment follows the convention of the other Delphix patches already in this fork (DLPX-46230 / DLPX-82789 / DLPX-51415 in ValueMetaBase).

[Copilot]

Copilot review overview

🟡 Changes recommended

Equality handling, JUnit 5 isolation, and converted-date branch coverage remain incomplete.

Review effort: Balanced
Findings: 2 · 1

Open (3)

• Include behavior flag in equals() and hashCode() · New
• Reset compatibility property in RestoreHopEnvironmentExtension · New
• Add coverage for converted-Date timestamp binding · New

What changed in this PR

Restores the compatibility flag needed to prevent Teradata timestamp batch-insert type conflicts.

Changes:

• Reintroduces HOP_COMPATIBILITY_DB_IGNORE_TIMEZONE.
• Uses non-calendar JDBC date/timestamp overloads when enabled.
• Adds focused tests and environment cleanup.

File	Description
Const.java	Defines and documents the compatibility property.
ValueMetaBase.java	Reads and applies the property during JDBC binding.
ValueMetaBaseSetPreparedStmntValueTest.java	Tests date and timestamp overload selection.
RestoreHopEnvironment.java	Clears the property in JUnit 4 tests.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[AnoopKumarDelphix]

LGTM