DatalinkRecordMixin's record.getdataset() falls back to access_url

[stvoutsin]

Description

DatalinkRecordMixin.getdataset() catches any failure in the DataLink path and falls back to the base Record.getdataset():

pyvo/pyvo/dal/adhoc.py

Lines 435 to 447 in a7c6e6c

	@stream_decode_content
	def getdataset(self, timeout=None):
	try:
	url = next(self.getdatalink().bysemantics('#this')).access_url
	response = self._session.get(url, stream=True, timeout=timeout)
	try:
	response.raise_for_status()
	except requests.RequestException as ex:
	raise DALServiceError.from_except(ex, url)
	return response.raw
	except (DALServiceError, ValueError, StopIteration):
	# this should go to Record.getdataset()
	return super().getdataset(timeout=timeout)

The fallback fetches the record's access.reference column without checking the access_format, which may cause confusion for services that follow the Datalink-indirection model where access_url points at a DataLink links endpoint insstead of the data product directly.
So in this scenario, if the first request fails the fallback will fetch the links endpoint and return the VOTable as the dataset, and the user doesn't get any error messages they could otherwise use to discover what went wrong.

Also, note that this path reads .access_url directly instead of getdataurl() which will make error rows lead to a misleading error.

I think we need to fix this by:

• Skip the fallback when the record's access_format indicates a DataLink document
• Propagate the error_message from a DataLink error row instead of swallowing it?

If this seems reasonable and I'm not missing something I'm happy to put in a PR

[msdemlei]

On Tue, Jul 28, 2026 at 01:02:57PM -0700, stvoutsin wrote: stvoutsin created an issue (astropy/pyvo#776) I think we need to fix this by: - Skip the fallback when the record's access_format indicates a DataLink document - Propagate the error_message from a DataLink error row instead of swallowing it? If this seems reasonable and I'm not missing something I'm happy to put in a PR

I'd probably not veto it, but I'd at least briefly consider whether the behaviour of returning #this from getdataset() when it's a datalink document isn't the less surprising thing. I'd say the main counterargument is that if people return datalink instead of the full file is that the full file is large. At least that's what I do. Following this reasoning, blindly pulling Gigabytes or Terabytes sounds like a bad idea. In an interactive application, I think what should happen in the datalink-as-dataset is a dialogue popping up saying "here's your access options for this dataset". What's the closest thing we can do in pyVO?

[stvoutsin]

I think in general I agree with the notion that if on failure getdataset() would otherwise return a datalink document, try to resolve #this out of it instead. Perhaps it makes sense as a retry once to handle transient error cases and otherwise throw an error.

For the access options I'm not sure I have a great answer, but perhaps the exception itself is that, we include the error message if present and list any other non-error rows available and then the client can choose what to do.

[msdemlei]

On Mon, Aug 03, 2026 at 11:15:31AM -0700, stvoutsin wrote: stvoutsin left a comment (astropy/pyvo#776) I think in general I agree with the notion that if on failure `getdataset()` would otherwise return a datalink document, try to resolve `#this` out of it instead. Perhaps it makes sense as a

Ok, let's do that then and think about some mitigation for the problem of suddenly pulling many Gigabytes when the problem arises. Or should we have a parameter maxsize to getdataset such that: getdataset() throws an error if we can work out that the dataset we're retrieving is larger than some resonable default (0.5 GiB?). And if people are sure they want huge data, they'd call getdataset(maxsize=1e10) I think I'd like that feature: not really intrusive, but it will prevent accidents. We won't catch cases where there's no content-size through datalink or http, but that would be up to the data publishers to fix.

retry once to handle transient error cases and otherwise throw an error.

I'd avoid automatic retrys. Let's propagate transient errors.

For the access options I'm not sure I have a great answer, but perhaps the exception itself is that, we include the error message if present and list any other non-error rows available and then the client can choose what to do.

I think at least with the built-in huge-file protection, we'd be good enough.