Entity Provider for java.nio.file.Path

[mkarg]

Closing #1274

This pull request provides the needed changes in the spec document and the TCK to support java.nio.file.Path entity providers.

[jim-krueger]

Looks good. I see no reason not to add this.

[jim-krueger]

Upon further review: I have some comments/questions.

[mkarg]

Will answer open question ASAP, unfortunately CORONA hit me, so it might take a few days. Sorry!

[jim-krueger]

Get well Markus, no rush. Also, it seems like now would be a good time to create a release plan for 4.1 (or 5.0 if needed) and add this to it.

@spericas / others, do you concur?

[spericas]

Get well Markus, no rush. Also, it seems like now would be a good time to create a release plan for 4.1 (or 5.0 if needed) and add this to it.

@spericas / others, do you concur?

Do we have any release drivers for 4.1?

[mkarg]

Get well Markus, no rush. Also, it seems like now would be a good time to create a release plan for 4.1 (or 5.0 if needed) and add this to it.
@spericas / others, do you concur?

Do we have any release drivers for 4.1?

As long as we all commit to really getting 5.0 thru the door in time, then I have no driver for 4.1. Looking at how things worked out in the past months, I doubt that this time it will work out better than the years before, so maybe we should start with 4.1 instead of 5.0...

[mkarg]

Kindly asking everybody to vote on this PR. Thanks! :-)

[jansupol]

lgtm

[spericas]

What is the real benefit in natively supporting java.nio.file.Path when you have java.nio.file.Path#toFile? Just trying to understand the rationale here.

[mkarg]

What is the real benefit in natively supporting java.nio.file.Path when you have java.nio.file.Path#toFile? Just trying to understand the rationale here.

Besides migrating JAX-RS's API to modern Java, it mostly is performance (not necessarily speed, but mostly power savings). In short, there is a huge difference how a file is handled by the JRE (all work done by JRE) internally to how a path is (if any possible, all work offloaded to OS). Falling back to files, when you actually have a path, is squandering valuable resources. In modern Java, nobody should ever go with file but always with path. If you are interested in more details, we should spin-off the thread.

[spericas]

What is the real benefit in natively supporting java.nio.file.Path when you have java.nio.file.Path#toFile? Just trying to understand the rationale here.

Besides migrating JAX-RS's API to modern Java, it mostly is performance (not necessarily speed, but mostly power savings). In short, there is a huge difference how a file is handled by the JRE (all work done by JRE) internally to how a path is (if any possible, all work offloaded to OS). Falling back to files, when you actually have a path, is squandering valuable resources. In modern Java, nobody should ever go with file but always with path. If you are interested in more details, we should spin-off the thread.

@jansupol How does Jersey handle java.nio.file.Path in the impl?

[spericas]

What is the real benefit in natively supporting java.nio.file.Path when you have java.nio.file.Path#toFile? Just trying to understand the rationale here.

Besides migrating JAX-RS's API to modern Java, it mostly is performance (not necessarily speed, but mostly power savings). In short, there is a huge difference how a file is handled by the JRE (all work done by JRE) internally to how a path is (if any possible, all work offloaded to OS). Falling back to files, when you actually have a path, is squandering valuable resources. In modern Java, nobody should ever go with file but always with path. If you are interested in more details, we should spin-off the thread.

@jansupol How does Jersey handle java.nio.file.Path in the impl?

From what I gather, the implementations for Path and File are basically identical:

• https://github.com/eclipse-ee4j/jersey/blob/3.1/core-common/src/main/java/org/glassfish/jersey/message/internal/PathProvider.java
• https://github.com/eclipse-ee4j/jersey/blob/3.1/core-common/src/main/java/org/glassfish/jersey/message/internal/FileProvider.java

Based on this, it's difficult to justify the advantages over the costs of needing to update all implementations, docs, etc. that this PR proposes.

[mkarg]

What is the real benefit in natively supporting java.nio.file.Path when you have java.nio.file.Path#toFile? Just trying to understand the rationale here.

Besides migrating JAX-RS's API to modern Java, it mostly is performance (not necessarily speed, but mostly power savings). In short, there is a huge difference how a file is handled by the JRE (all work done by JRE) internally to how a path is (if any possible, all work offloaded to OS). Falling back to files, when you actually have a path, is squandering valuable resources. In modern Java, nobody should ever go with file but always with path. If you are interested in more details, we should spin-off the thread.

@jansupol How does Jersey handle java.nio.file.Path in the impl?

From what I gather, the implementations for Path and File are basically identical:

* https://github.com/eclipse-ee4j/jersey/blob/3.1/core-common/src/main/java/org/glassfish/jersey/message/internal/PathProvider.java

* https://github.com/eclipse-ee4j/jersey/blob/3.1/core-common/src/main/java/org/glassfish/jersey/message/internal/FileProvider.java

Based on this, it's difficult to justify the advantages over the costs of needing to update all implementations, docs, etc. that this PR proposes.

No need to convince me. It was me who wrote "PathProvider". In fact, I would even go as far to say that support for "File" should become deprecated eventually. It simply is dead legacy.

As an OpenJDK NIO2 contributor I like to make everybody understand that "File" is a rather outdated concept, while "Path" is what every new software should go with. The difference is not what Jersey does, but what the JRE does. For those interested in the details, I would be more than happy to explain in a separate thread.

Edit: The core idea of this MR is to modernize JAX-RS, and using "Path" should be a no-brainer, actually. Sticking with File but not supporting Path is simply ridiculous and draws a picture of an old man's ancient not actively maintained API.

[spericas]

What is the real benefit in natively supporting java.nio.file.Path when you have java.nio.file.Path#toFile? Just trying to understand the rationale here.

Besides migrating JAX-RS's API to modern Java, it mostly is performance (not necessarily speed, but mostly power savings). In short, there is a huge difference how a file is handled by the JRE (all work done by JRE) internally to how a path is (if any possible, all work offloaded to OS). Falling back to files, when you actually have a path, is squandering valuable resources. In modern Java, nobody should ever go with file but always with path. If you are interested in more details, we should spin-off the thread.

@jansupol How does Jersey handle java.nio.file.Path in the impl?

From what I gather, the implementations for Path and File are basically identical:

* https://github.com/eclipse-ee4j/jersey/blob/3.1/core-common/src/main/java/org/glassfish/jersey/message/internal/PathProvider.java

* https://github.com/eclipse-ee4j/jersey/blob/3.1/core-common/src/main/java/org/glassfish/jersey/message/internal/FileProvider.java

Based on this, it's difficult to justify the advantages over the costs of needing to update all implementations, docs, etc. that this PR proposes.

No need to convince me. It was me who wrote "PathProvider". In fact, I would even go as far to say that support for "File" should become deprecated eventually. It simply is dead legacy.

As an OpenJDK NIO2 contributor I like to make everybody understand that "File" is a rather outdated concept, while "Path" is what every new software should go with. The difference is not what Jersey does, but what the JRE does. For those interested in the details, I would be more than happy to explain in a separate thread.

From an API perspective, I understand that Path may be desirable in some cases (yet, this is a bit of a weak argument IMO). From an implementation perspective, which seems to be your argument above, it seems that there can be zero difference if the implementation of File uses Path as the one in Jersey --again both providers seem to be doing exactly the same.

[mkarg]

From an API perspective, I understand that Path may be desirable in some cases (yet, this is a bit of a weak argument IMO). From an implementation perspective, which seems to be your argument above, it seems that there can be zero difference if the implementation of File uses Path as the one in Jersey --again both providers seem to be doing exactly the same.

Santiago, I think there is a misunderstanding, so let me clarify -- even if I still think that it would be better to start a new thread elsewhere, as this explanation is partly off-topic IMHO as it is about OpenJDK and Jersey, not about JAX-RS.

Maybe my reasoning was misleading. I do not see why you link this PR with my current implementation inside Jersey in particular (which is something which might change further over time, and which I actually changed in the NIO2-area several times just recently, so for this PR it plays absolutely no role how good or not-so-well done my solution in Jersey is currently or will be in future); the idea of this PR is to allow optimized implementations, which might or might not happen eventually or might or might not exist currently. Current Jersey itelf might or might not experience a benefit from Path-instead-of-File already, but that is simply irrelevant for opening the potential provided by native Path support. Other implementations might do better or never optimize at all; it is their choice - and even that is irrelevant. The point is, that the intended resource optimization happens not within any JAX-RS implementation, but it happens within the JRE!

Looking at OpenJDK's source code of File::toPath and Path::toFile we can see that these conversions imply (at least) the following drawbacks:

• Path::toFile only works with paths created by the default provider. All other providers will throw UnsupportedOperationException. This means, as of today, JAX-RS is incompatible with these providers! Or in other words, the missing native Path support in JAX-RS is (more or less) a showstopper for applications that work with these providers (yes, there is always the workaround of using raw streams instead of typed entities, but not using JAX-RS features is definitively not what we want people to do).
• Due to different parsing and toString algorithms, it might happen that the string after conversion actually differs from the original string. This might lead to weird bugs at runtime which are (more or less) hard to find and/or hard to work around. Path and File are even applying different rulesets what strings they can hold, so it could happen, that a string is not convertable at all (the reason is that a Path MAY be a C-implementation which is dependent of the provider and/or filesystem, while a File is more or less a simple FS/OS-independent Java-implemented string wrapper). This implies (besides other problems) that File::toPath might fail with InvalidPathException (another hard-to-workaround showstopper we won't people to experience at runtime)!
• File::toPath is synchronized, at least for the first invocation of each file instance, which implies a speed penalty. As each invocation of Path::toFile creates a new instance, effectively this means that conversion-and-reconversion always induces that speed penalty!
• File::toPath accesses a volatile variable as part of the aforementioned caching algorithm. As a result, all invocations of File::toPath imply passing-by the cache, or in other words, are pretty slow.
• Creating a Path or File instance implies (in most cases) one or many C calls via JNI, which implies a speed penalty, which sums up when performing conversion-and-reconversion.

Jersey: I already optimized Jersey (at least in part) for NIO2, i. e. for native use of Path, so it omits (in many cases, but not in all - something I like to fix in future) most of these drawbacks. As I hope is now clear, the optimization is not obvious to see in the Jersey source code, as Jersey only paves the way for not running into these JRE-internal obstacles by preventing unnecessary / multiple conversion. As a result, Jersey is able to do the same work with less resources (which might or might not lead to performance optimizations depending on the actual use case and environment). If you look carefully, you will notice that the two source codes you refer to are slightly different: One is using conversion (which is a bad thing, as we learned now), the other is not (or at least, not that often - until I eventually fixed that). Also see that the Path based provider bears potential for future optimization (which I would be happen to contribute), while typically no more optimizations will flow into the File based provider - just as the OpenJDK Team frequently optimizes the modern NIO2 API under the hood, while optimiztations to the 30yrs old IO API happen only sparingly).

Regarding your non-technical arguments, I like to say the following:

• Driver: The major driver for adopting this PR is allowing implementations to spare resources (whether or not sparing actually happens is not under my control, but at least implementors have a chance to optimize it, as there is no more force to convert-and-reconvert). Nevertheless, from the aspect of an OpenJDK contributor, also a major driver definitively is to trigger people to use Path everywhere and get rid of File ASAP. This includes triggering JAX-RS implementors to replace files by paths as a side effect of implementing this PR, but it also includes triggering JAX-RS users to do the same in their applications. NIO2 is key for future performance optimizations measured in both, speed and resource consumption. The earlier everybody switches from IO to NIO2, the better - particular in I/O-savvy applications like REST servers.
• Efforts: The work in Jersey is already done mostly (I did that in very few hours), and I am willing to contribute the missing bricks; Jan even already voted +1 for this PR. The work in other implementations should be few hours only, and I could contribute if needed; Jim already wrote "Looks good. I see no reason not to add this.". The work in JAX-RS itself is already done in this PR. So I actually cannot see what or whom you mean with "efforts".
• Users more and more experience Java, and in particular JAX-RS, as "ancient". If we want to stop that trend, we need to adopt modern features from current JREs. This implies, among other things, Path. I am really concerned you started this discussion as apparently readers could think the project lead is not planning to adapt JAX-RS to Java's future at all, leading to more people going away from JAX-RS. So I really beg you to please not stand in the way of the adoption of modern Java features. Thank you.

[spericas]

From an API perspective, I understand that Path may be desirable in some cases (yet, this is a bit of a weak argument IMO). From an implementation perspective, which seems to be your argument above, it seems that there can be zero difference if the implementation of File uses Path as the one in Jersey --again both providers seem to be doing exactly the same.

Santiago, I think there is a misunderstanding, so let me clarify -- even if I still think that it would be better to start a new thread elsewhere, as this explanation is partly off-topic IMHO as it is about OpenJDK and Jersey, not about JAX-RS.

The discussion is very much about JAX-RS, the side note about implementations was prompted by your performance claims.

Maybe my reasoning was misleading. I do not see why you link this PR with my current implementation inside Jersey in particular (which is something which might change further over time, and which I actually changed in the NIO2-area several times just recently, so for this PR it plays absolutely no role how good or not-so-well done my solution in Jersey is currently or will be in future); the idea of this PR is to allow optimized implementations, which might or might not happen eventually or might or might not exist currently. Current Jersey itelf might or might not experience a benefit from Path-instead-of-File already, but that is simply irrelevant for opening the potential provided by native Path support. Other implementations might do better or never optimize at all; it is their choice - and even that is irrelevant. The point is, that the intended resource optimization happens not within any JAX-RS implementation, but it happens within the JRE!

JAX-RS implementations drive the JRE, the JRE does not drive itself. I'm not opposed to this new feature, but as I stated above, I'm not convinced it is worth the effort. Hopefully others can comment and we can proceed with it.

[jansupol]

Given we support the File, I would not want to restrain the customers from using NIO API,

On the other hand, I see the real usage of Path much lower than the usage of the Status Codes defined in RFC 9110, which still did not make it to Jakarta REST Response.Status.

[mkarg]

Now that we all have exchanged our personal opinions, in the name of modern Java, I do beg all committers to vote +1. Thanks.

[mkarg]

As the PR is finally authored, and as Jersey is already implementing this feature, can anybody please tell me a good reason why we not simply merge this into the spec? I am willing to invest all needed efforts into all compliant implementations to fulfil this change if this helps.

[mkarg]

Kindly asking to resume this discussion. This PR would help to let JAX-RS look more modern, it is already supported by Jersey, and it would be just straight forward. Please don't stand in the way on modernization. Thanks.

[spericas]

As stated before, I'm +0 on this one

[mkarg]

I do not see why we should not adopt this MR as-is. Kindly asking for votes from @arjantijms @jamezp.

Side note: Jersey supports this since years.

[jamezp]

I'm still of the opinion, even more so now, that we don't use the xml_binding tag. It's not longer a requirement in the platform and XML binding is not a requirement for the Jakarta REST spec either.

Introducing TCK tests that may not run for a new requirement doesn't seem right to me.

[mkarg]

I see your point (I did not know that what I did "is not running"). What do you want me to change in this PR to get it accepted?

[jamezp]

I see your point (I did not know that what I did "is not running"). What do you want me to change in this PR to get it accepted?

With the exception of ee.jakarta.tck.ws.rs.spec.provider.overridestandard.JAXRSClientIT.readWritePathProviderTest(), I think all you need to do is remove the @Tag("xml_binding"). Unless I've missed something, I don't see where Jakarta XML Binding is required for these tests.

[mkarg]

With the exception of ee.jakarta.tck.ws.rs.spec.provider.overridestandard.JAXRSClientIT.readWritePathProviderTest(), I think all you need to do is remove the @Tag("xml_binding"). Unless I've missed something, I don't see where Jakarta XML Binding is required for these tests.

Thank you! If have remove @Tag.

Removed @Tag, requested re-review.

[mkarg]

@arjantijms Jersey is already supporting this since several releases (I implemented it long time ago). Kindly asking for your vote hereby, as it therefore does not imply any negative impact or effort for Jersey. Thanks. 🙂

[jamezp]

Minor comment on the assertion id's, but that might be to my lack of understanding how they are used.

[mkarg]

@jakartaee/ee4j-rest-committers Kindly asking more committers for votes.