Skip to content

Commit 9ecff0c

Browse files
bsipoczbsipocz
authored
Merge pull request #698 from lmichel/mango-next
2 parents 18186cc + 76a9ce9 commit 9ecff0c

32 files changed

Lines changed: 2957 additions & 947 deletions

‎CHANGES.rst‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,11 @@ Enhancements and Fixes
1212

1313
- Add retry option to AsyncTAPJob.fetch_result for transient failures [#696]
1414

15+
- Upgrade of the ``MivotViewer`` API (``xml_viewer`` module removed, support of
16+
multiple mapped objects per row, partial redesign of the public API) -
17+
Improve the gateway between annotations and ``SkyCoord`` objects -
18+
revamp the viewer documentation. [#698]
19+
1520

1621
Deprecations and Removals
1722
-------------------------

‎docs/mivot/example.rst‎

Lines changed: 149 additions & 120 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,10 @@
1+
.. _mivot-examples:
2+
13
************************************************************
24
MIVOT (``pyvo.mivot``): How to use annotated data - Examples
35
************************************************************
46

5-
Photometric properties readout
7+
Photometric Properties Readout
68
==============================
79

810
This example is based on VOTables provided by the ``XTapDB`` service.
@@ -22,32 +24,30 @@ to tell the server to annotate the queried data.
2224

2325
(*Please read the comment inside the code snippet carefully to fully understand the process*)
2426

25-
.. code-block:: python
26-
27-
import pytest
28-
from pyvo.utils import activate_features
29-
from pyvo.dal import TAPService
30-
from pyvo.mivot.utils.xml_utils import XmlUtils
31-
from pyvo.mivot.utils.dict_utils import DictUtils
32-
from pyvo.mivot.viewer.mivot_viewer import MivotViewer
33-
34-
# Enable MIVOT-specific features in the pyvo library
35-
activate_features("MIVOT")
36-
37-
service = TAPService('https://xcatdb.unistra.fr/xtapdb')
38-
result = service.run_sync(
39-
"""
40-
SELECT TOP 5 * FROM "public".mergedentry
41-
""",
42-
format="application/x-votable+xml;content=mivot"
43-
)
44-
45-
# The MIVOT viewer generates the model view of the data
46-
m_viewer = MivotViewer(result, resolve_ref=True)
47-
48-
# Print out the Mivot annotations read out of the VOtable
49-
# This statement is just for a pedagogic purpose (access to a private attribute)
50-
XmlUtils.pretty_print(m_viewer._mapping_block)
27+
.. doctest-skip::
28+
29+
>>> from pyvo.utils import activate_features
30+
>>> from pyvo.dal import TAPService
31+
>>> from pyvo.mivot.utils.xml_utils import XmlUtils
32+
>>> from pyvo.mivot.viewer.mivot_viewer import MivotViewer
33+
>>>
34+
>>> # Enable MIVOT-specific features in the pyvo library
35+
>>> activate_features("MIVOT")
36+
>>>
37+
>>> service = TAPService('https://xcatdb.unistra.fr/xtapdb')
38+
>>> result = service.run_sync(
39+
... """
40+
... SELECT TOP 5 * FROM "public".mergedentry
41+
... """,
42+
... format="application/x-votable+xml;content=mivot"
43+
... )
44+
>>>
45+
>>> # The MIVOT viewer generates the model view of the data
46+
>>> m_viewer = MivotViewer(result, resolve_ref=True)
47+
>>>
48+
>>> # Print out the Mivot annotations read out of the VOtable
49+
>>> # This statement is just for a pedagogic purpose (access to a private attribute)
50+
>>> XmlUtils.pretty_print(m_viewer._mapping_block)
5151

5252

5353
In this first step we just queried the service and we built the object that will process the Mivot annotations.
@@ -63,48 +63,45 @@ The Mivot block printing output is too long to be listed here. However, the scre
6363
``MangoObject`` instance which holds all the mapped properties.
6464

6565
At instantiation time, the viewer reads the first data row, which must exist,
66-
in order to construct a Python object that reflects the mapped model.
66+
in order to construct the Python objects that reflect the mapped models and
67+
to make the data available through them.
6768

68-
.. code-block:: python
69+
.. doctest-skip::
6970

70-
# Build a Python object matching the TEMPLATES content and
71-
# which leaves are set with the values of the first row
72-
mango_object = m_viewer.dm_instance
73-
74-
# Print out the content of the Python object
75-
# This statement is just for a pedagogic purpose
76-
DictUtils.print_pretty_json(mango_object.to_dict())
71+
>>> # Discover the Python objects matching the TEMPLATES content
72+
>>> for dm_instance in m_viewer.dm_instances;
73+
>>> print(dm_instance)
74+
<MivotInstance: dmtype="mango:MangoObject">
7775

7876
The annotations are consumed by this dynamic Python object which leaves are set with the data of the current row.
79-
You can explore the structure of this object by using the printed dictionary or standard object paths as shown below.
77+
You can explore the structure of this object by using standard object paths as shown below.
8078

8179
Now, we can iterate through the table data and retrieve an updated Mivot instance for each row.
8280

83-
.. code-block:: python
84-
85-
while m_viewer.next_row_view():
86-
if mango_object.dmtype == "mango:MangoObject":
87-
print(f"Read source {mango_object.identifier.value} {mango_object.dmtype}")
88-
for mango_property in mango_object.propertyDock:
89-
if mango_property.dmtype == "mango:Brightness":
90-
if mango_property.value.value:
91-
mag_value = mango_property.value.value
92-
mag_error = mango_property.error.sigma.value
93-
phot_cal = mango_property.photCal
94-
spectral_location = phot_cal.photometryFilter.spectralLocation
95-
mag_filter = phot_cal.identifier.value
96-
spectral_location = phot_cal.photometryFilter.spectralLocation
97-
mag_wl = spectral_location.value.value
98-
sunit = spectral_location.unitexpression.value
99-
100-
print(f" flux at {mag_wl} {sunit} (filter {mag_filter}) is {mag_value:.2e} +/- {mag_error:.2e}")
101-
102-
Read source 4XMM J054329.3-682106 mango:MangoObject
81+
.. doctest-skip::
82+
83+
>>> mango_object = m_viewer.dm_instances[0]
84+
>>> while m_viewer.next_row_view():
85+
>>> if mango_object.dmtype == "mango:MangoObject":
86+
>>> print(f"Read source {mango_object.identifier.value} {mango_object.dmtype}")
87+
>>> for mango_property in mango_object.propertyDock:
88+
>>> if mango_property.dmtype == "mango:Brightness":
89+
>>> if mango_property.value.value:
90+
>>> mag_value = mango_property.value.value
91+
>>> mag_error = mango_property.error.sigma.value
92+
>>> phot_cal = mango_property.photCal
93+
>>> spectral_location = phot_cal.photometryFilter.spectralLocation
94+
>>> mag_filter = phot_cal.identifier.value
95+
>>> spectral_location = phot_cal.photometryFilter.spectralLocation
96+
>>> mag_wl = spectral_location.value.value
97+
>>> sunit = spectral_location.unitexpression.value
98+
>>> print(f" flux at {mag_wl} {sunit} (filter {mag_filter}) is {mag_value:.2e} +/- {mag_error:.2e}")
99+
Read source 4XMM J054329.3-682106 mango:MangoObject
103100
flux at 0.35 keV (filter XMM/EPIC/EB1) is 8.35e-14 +/- 3.15e-14
104101
flux at 0.75 keV (filter XMM/EPIC/EB2) is 3.26e-15 +/- 5.45e-15
105102
flux at 6.1 keV (filter XMM/EPIC/EB8) is 8.68e-14 +/- 6.64e-14
106-
...
107-
...
103+
...
104+
...
108105

109106
The same code can easily be connected with matplotlib to plot SEDs as shown below (code not provided).
110107

@@ -113,11 +110,11 @@ The same code can easily be connected with matplotlib to plot SEDs as shown belo
113110
:width: 500
114111
:alt: XMM SED
115112

116-
It is to noted that the current table row keeps available through the Mivot viewer.
113+
It is to be noted that the current table row keeps available through the Mivot viewer.
117114

118-
.. code-block:: python
115+
.. code-block:: python
119116
120-
row = m_viewer.table_row
117+
row = m_viewer.table_row
121118
122119
123120
.. important::
@@ -129,7 +126,7 @@ It is to noted that the current table row keeps available through the Mivot view
129126

130127
The same client code can be reused in many places with many datasets, provided they are annotated.
131128

132-
EpochPosition property readout
129+
EpochPosition Property Readout
133130
==============================
134131

135132
This example is based on a VOtable resulting on a Vizier cone search.
@@ -138,58 +135,64 @@ which models a full source's astrometry at a given date.
138135

139136

140137
.. warning::
141-
At the time of writing, Vizier only mapped positions and proper motions (when available),
138+
At the time of writing (Q1 2025), Vizier only mapped positions and proper motions (when available),
142139
and the definitive epoch class had not been adopted.
143140
Therefore, this implementation may differ a little bit from the standard model.
144141

145142
Vizier does not wrap the source properties in a MANGO object,
146143
but rather lists them in the Mivot *TEMPLATES*.
147-
The annotation reader must support both designs.
144+
The annotation reader supports both designs.
148145

149146
In the first step below, we run a standard cone search query by using the standard PyVO API.
150-
151-
.. code-block:: python
152-
153-
import pytest
154-
import astropy.units as u
155-
from astropy.coordinates import SkyCoord
156-
from pyvo.dal.scs import SCSService
157-
158-
from pyvo.utils import activate_features
159-
from pyvo.mivot.viewer.mivot_viewer import MivotViewer
160-
from pyvo.mivot.features.sky_coord_builder import SkyCoordBuilder
161-
from pyvo.mivot.utils.dict_utils import DictUtils
162-
163-
# Enable MIVOT-specific features in the pyvo library
164-
activate_features("MIVOT")
165-
166-
scs_srv = SCSService("https://vizier.cds.unistra.fr/viz-bin/conesearch/V1.5/I/239/hip_main")
167-
168-
query_result = scs_srv.search(
169-
pos=SkyCoord(ra=52.26708 * u.degree, dec=59.94027 * u.degree, frame='icrs'),
170-
radius=0.5)
171-
172-
# The MIVOt viewer generates the model view of the data
173-
m_viewer = MivotViewer(query_result, resolve_ref=True)
174-
175147
Once the query is finished, we can get a reference to the object that will process the Mivot annotations.
176148

177-
.. code-block:: python
178-
179-
# Build a Python object matching the TEMPLATES content and
180-
# which leaves are set with the values of the first row
181-
mango_property = m_viewer.dm_instance
182-
183-
# Print out the content of the Python object
184-
# This statement is just for a pedagogic purpose
185-
DictUtils.print_pretty_json(mango_property.to_dict())
186-
187-
The annotations are consumed by this dynamic Python object which leaves are set with the data of the current row.
188-
You can explore the structure of this object by using standard object paths or by browsing the dictionary shown below.
189-
190-
.. code-block:: json
191-
192-
{
149+
.. doctest-skip::
150+
151+
>>> import astropy.units as u
152+
>>> from astropy.coordinates import SkyCoord
153+
>>> from pyvo.dal.scs import SCSService
154+
>>> from pyvo.utils import activate_features
155+
>>> from pyvo.mivot.viewer.mivot_viewer import MivotViewer
156+
>>> from pyvo.mivot.features.sky_coord_builder import SkyCoordBuilder
157+
>>>
158+
>>> # Enable MIVOT-specific features in the pyvo library
159+
>>> activate_features("MIVOT")
160+
>>>
161+
>>> scs_srv = SCSService("https://vizier.cds.unistra.fr/viz-bin/conesearch/V1.5/I/239/hip_main")
162+
>>>
163+
>>> query_result = scs_srv.search(
164+
... pos=SkyCoord(ra=52.26708 * u.degree, dec=59.94027 * u.degree, frame='icrs'),
165+
... radius=0.5)
166+
>>>
167+
>>> # The MIVOT viewer generates the model view of the data
168+
>>> m_viewer = MivotViewer(query_result, resolve_ref=True)
169+
170+
We can now discover which data model classes the data is mapped to.
171+
172+
.. doctest-skip::
173+
174+
>>> # Get a set of Python objects matching the TEMPLATES content and
175+
>>> # which leaves are set with the values of the first row
176+
>>> for dm_instance in m_viewer.dm_instances;
177+
>>> print(dm_instance)
178+
<MivotInstance: dmtype="mango:EpochPosition">
179+
180+
The first instance can be accessed by the ``m_viewer.dm_instance`` getter.
181+
This is a simple shorcut aiming at simplifying the code.
182+
183+
.. doctest-skip::
184+
185+
>>> dm_instance = m_viewer.dm_instance
186+
>>> print(dm_instance.dmtype)
187+
mango:EpochPosition
188+
189+
We can also provide a complete instance representation that includes all fields in the entire hierarchy.
190+
191+
.. doctest-skip::
192+
193+
>>> # Print out the json serialization of the Python object
194+
>>> print(repr(dm_instance))
195+
{
193196
"dmtype": "mango:EpochPosition",
194197
"longitude": {
195198
"dmtype": "ivoa:RealQuantity",
@@ -231,22 +234,22 @@ You can explore the structure of this object by using standard object paths or b
231234
"spaceRefFrame": {
232235
"dmtype": "ivoa:string",
233236
"value": "ICRS"
234-
}
235-
}
236-
}
237-
}
238-
237+
}
238+
}
239+
}
240+
}
239241

240-
The reader can transform ``EpochPosition`` instances into ``SkyCoord`` instances.
241-
These can then be used for further scientific processing.
242+
The reader can transform ``EpochPosition`` instances into ``SkyCoord`` instances.
243+
These can then be used for further scientific processing.
242244

243-
.. code-block:: python
245+
.. doctest-skip::
244246

245-
while m_viewer.next_row_view():
246-
if mango_property.dmtype == "mango:EpochPosition":
247-
scb = SkyCoordBuilder(mango_property.to_dict())
248-
# do whatever process with the SkyCoord object
249-
print(scb.build_sky_coord())
247+
>>> while m_viewer.next_row_view():
248+
>>> mango_property = m_viewer.dm_instance
249+
>>> if mango_property.dmtype == "mango:EpochPosition":
250+
>>> scb = SkyCoordBuilder(mango_property)
251+
>>> # do whatever process with the SkyCoord object
252+
>>> print(scb.build_sky_coord())
250253

251254
.. important::
252255
Similar to the previous example, this code can be used with any VOTable with data mapped to MANGO.
@@ -255,5 +258,31 @@ You can explore the structure of this object by using standard object paths or b
255258
It avoids the need for users to build SkyCoord objects by hand from VOTable fields,
256259
which is never an easy task.
257260

261+
Homework
262+
========
263+
264+
Simbad has released (Q3 2025) an annotated version of its Cone Search.
265+
It's a good case to exercise this API.
266+
267+
268+
.. code-block:: python
269+
270+
SERVER = "https://simbad.cds.unistra.fr/cone?"
271+
VERB = 2
272+
RA = 269.452076* u.degree
273+
DEC = 4.6933649* u.degree
274+
SR = 0.1* u.degree
275+
MAXREC = 100
276+
RESPONSEFORMAT = "mivot"
277+
278+
scs_srv = SCSService(SERVER)
279+
280+
query_result = scs_srv.search(
281+
pos=SkyCoord(ra=RA, dec=DEC, frame='icrs'),
282+
radius=SR,
283+
verbosity=VERB,
284+
RESPONSEFORMAT=RESPONSEFORMAT,
285+
MAXREC=MAXREC)
286+
258287
259-
The next section provides some tips to use the API documented in the :ref:`annoter page <mivot-annoter>`.
288+
*The next section provides some tips to use the API documented in the* :ref:`annoter page <mivot-annoter>`.

0 commit comments

Comments
 (0)