Test the format of a German date with a short weekday and no year

[MidnightDesign]

I've stumbled upon an inconsistency between V8 and JavaScriptCore. Formatting a date with a short weekday and no year in German results in the following strings:

V8: Sa., 24. Dezember
JavaScriptCore: Sa. 24. Dezember (no comma after the weekday)

When the year is included (year: 'numeric'), JavaScriptCore does add a comma.

[Ms2ger]

Please add copyright and frontmatter: https://github.com/tc39/test262/blob/main/CONTRIBUTING.md#test-case-style

On what basis did you decide which implementation was correct?

[MidnightDesign]

On what basis did you decide which implementation was correct?

Because of the consistent behavior when including the year.

Edit: Maybe the version without a comma is the "correct"/spec-compliant version. Either way, the behavior between engines is inconsistent.

[MidnightDesign]

Can someone help me out with the esid? I don't think the local styles themselves are defined in the spec, are they?

[MidnightDesign]

I'm not sure if I'm reading it right, but if I do, I've found a source: In CLDR 42, in the file common/main/de.xml, there's the entry

<dateFormatItem id="MMMMEd">E, d. MMMM</dateFormatItem>

Edit: GitHub link

[ptomato]

Hi @MidnightDesign, thanks for contributing! We really appreciate it when people investigate browser bugs and are sufficiently motivated to make the web better that they make it all the way here to test262 😄

Unfortunately this kind of test raises questions about what's in scope for test262 that don't have a clear answer. As you correctly noted, the local styles themselves are not defined in the spec. The closest we get is the definition of [[LocaleData]], specifically the [[LocaleData]].[[<locale>]].[[formats]].[[<calendar>]].[[pattern]] field. The contents are "implementation-defined" which technically puts this out of scope for test262 because an engine could pretty much have any value they like and still be compliant. In practice, of course, that's not what happens: engines either have no locale data and no Intl object at all, or sensible locale data.

Additionally we don't want to couple tests too closely to the locale data, because sometimes the formats do get updated from the CLDR side. It's not good if engines update their locale data and tests start failing (which recently happened). That leads to untrustworthy compliance statistics.

On the other hand, I think everyone acknowledges it's beneficial for the web if browsers produce the same output in locale methods. So, pragmatically, we do have a few tests that are specifically coupled to the locale data (example).

I'm sorry I don't have a better answer for you. Probably there are divergent opinions among test262 maintainers and consumers as well. I understand a complicated response to a simple PR is probably demotivating, but I hope this helps explain the context and competing priorities that exist around this kind of test.

My personal opinion on a way forward would be the following, but others may disagree.

I tested your example using the eshost tool and found that on all three major browsers, the output has the comma. So I assume your results on JSC are on an old version, and I'd start by looking at the source code of JSC to see what bug they fixed that changed the result to include the comma. (e.g. Outdated CLDR data? Post-processing that other engines don't do?) Ideally that would surface a bug that we could test more generally in a test262 test that would be less tightly coupled to a specific version of the locale data. If that proves impossible, then depending on what the cause was, it might be worth having a tightly coupled test.

[MidnightDesign]

@ptomato Thank you for your very detailed response and trying to not discourage me. I will go back and try to make this more reproducible in the next days. But I do remember that I definitely used the latest Safari on the latest OSX.

Just to give you some background on why this is so important to me: I'm using Next.js which renders the page using Node on the server and then hydrates it on the client. If the HTML of the server and the client disagree, React throws an error. Well, in this case, Node.js renders the date differently from Safari, and this is spamming our Rollbar:

So I'm not just very picky about my commas in dates, but there's an actual problem I'm trying to solve. : )

[ptomato]

Cool, then I think my advice about JSC still applies, but maybe in reverse; we must both be testing with different versions, and one is newer than the other 😄

According to eshost, the version I tested with was WebKit/WebKit@4176964. I see a date of 2022-05-19 on that commit.

[MidnightDesign]

I just tested it in Safari 16.1 (18614.2.9.1.12) on MacOS Ventura (using Browserstack):

Maybe you provided slightly different options to the DateTimeFormat constructor.

I do get the problem with adding these kinds of tests here, but I also can't think of a better place.

[ptomato]

I've tested again today with JavaScriptCore built from the very latest commit:

>>> new Date(2022, 11, 24).toLocaleString('de', {weekday: 'short', day: 'numeric', month: 'long'})
"Sa., 24. Dezember"
>>> new Intl.DateTimeFormat('de', {weekday: 'short', day: 'numeric', month: 'long'}).format(new Date(2022, 11, 24))
"Sa., 24. Dezember"

I don't know which JSC commit Safari 16.1 was branched off of, but that might be a good place to start. Another possibility that occurred to me is that Safari the browser could be using different internationalization data than JSC does when built by itself?

[MidnightDesign]

Another possibility that occurred to me is that Safari the browser could be using different internationalization data than JSC does when built by itself?

I would pretty much assume that that's the case. Well, I guess I'll just have to add the year everywhere. Should I go ahead and close this PR?

[MidnightDesign]

For what it's worth, Safari also renders date time ranges in German differently from all other engines: It uses a different kind of dash/hyphen and different kinds of spaces around it.

[ptomato]

I'm still pretty sure that must be due to a CLDR update. (#3676 is the recent fallout in test262 due to CLDR 42 using different kinds of spaces than previously.)

[ptomato]

@MidnightDesign Thanks for waiting, I still owe you an answer on this.

I discussed it with some of our other maintainers and while this is indeed out of scope, there is precedent for specific bugs, and I think there's some benefit.

I also did a bit of digging and as far as I can tell, the E, d. MMMM format has not changed since 2014 in CLDR. JavaScriptCore seems to have the correct behaviour, with the comma, as of current head. So, whatever went wrong with the localization data of this buggy version of Safari, may well have been prevented by such a test.

I'd ask that we do structure the test so that it's robust against unrelated things changing in the CLDR data, for example the space separator issue that I mentioned earlier. So, for example, I think it'd be sufficient to ensure the string contains the substring Sa., rather than comparing the whole string.

[github-actions]

1 new or modified test were run on 7 engines.

View workflow run

Test	Engine262	GraalJS	JavaScriptCore	QuickJS	SpiderMonkey	V8	XS
intl402/DateTimeFormat/prototype/format/german-weekday-no-year.js	❌	✅	✅	❌	✅	✅	❌

[MidnightDesign]

@ptomato I just stumbled upon this old PR of mine. Sorry for not responding for over 3 years... : ) I just updated the assertion.

Edit: Feel free to close this if it doesn't belong in test262.