Skip to content

Fix: normalize datetime values to strings in unit tests#4696

Merged
georgesittas merged 1 commit intomainfrom
jo/normalize_datetime_values
Jun 10, 2025
Merged

Fix: normalize datetime values to strings in unit tests#4696
georgesittas merged 1 commit intomainfrom
jo/normalize_datetime_values

Conversation

@georgesittas
Copy link
Copy Markdown
Collaborator

Fixes the issue discussed in this Slack thread: https://tobiko-data.slack.com/archives/C044BRE5W4S/p1749046530513469

@georgesittas georgesittas force-pushed the jo/normalize_datetime_values branch from df850d4 to bd49e17 Compare June 9, 2025 17:11
Copy link
Copy Markdown
Collaborator

@VaggelisD VaggelisD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't really have a concrete comment, unless I'm downplaying the power of str(x), my hunch is this will cause false-negative comparisons again but I don't have a better solution at hand.

Comment thread sqlmesh/core/test/definition.py
Comment thread tests/core/test_test.py
query:
rows:
- id: id1
agg_timestamp_col: ["2024-01-02T15:00:00.000000"]
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would the same output comparison work properly if we compared an equivalent datetime value from a different string? E.g `2024-01-02T15:00:00.00" or one based on a different timezone that results in the same time?

Ideally we'd normalize all datetime types into a common representation at a common timezone (?)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Collaborator

@VaggelisD VaggelisD Jun 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like it's on the same timezone from a quick look, I was referring to a scenario such as :

inputs:
   ...
   time_col: "2020-01-01 05:00:00+05"

outputs:
   ...
   time_col: "2020-01-01 00:00:00+00"

this test should be correct, right? But if it's a string comparison (or at best, normalized to a certain precision with zeroes) then it'd fail

Copy link
Copy Markdown
Collaborator Author

@georgesittas georgesittas Jun 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that we encourage specifying time zones in the timestamp values (in the sense that it's unsupported).

The +zone syntax is only mentioned in relation to the execution_time variable in the docs, and I think that was intentional at the time.

@georgesittas georgesittas merged commit d96fac7 into main Jun 10, 2025
25 checks passed
@georgesittas georgesittas deleted the jo/normalize_datetime_values branch June 10, 2025 07:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants