ci: always use PR target branch as benchmark reference

[sbrunato]

Always use Pull Request target branch as reference in benchmark github action instead of develop

[github-actions]

Benchmark comparison

Compared baseline: origin/develop
Compared candidate: f4cc4b7

------------------------------- benchmark: 9 tests, 2 sources -------------------------------
Name (time in us)                                 baseline Median  candidate Median   ΔMedian
---------------------------------------------------------------------------------------------
test_benchmark_eoproduct_assets_population                 8.1980            8.3610     +2.0%
test_benchmark_eoproduct_instantiation                    15.9160           14.5525     -8.6%
test_benchmark_stac_reader_fetch_recursive             2,507.2280        2,546.2930     +1.6%
test_benchmark_cli_version                            12,766.8730       12,723.6650     -0.3%
test_benchmark_cli_without_args_subprocess            98,928.4185       98,663.7600     -0.3%
test_benchmark_core_search_with_provider              98,097.5745       97,976.9760     -0.1%
test_benchmark_core_search_with_count                127,175.9155      125,188.7880     -1.6%
test_benchmark_cli_list_collections                  190,845.6340      190,753.6080     -0.0%
test_benchmark_eodag_instantiation_subprocess        702,844.3250      695,219.9210     -1.1%
---------------------------------------------------------------------------------------------

Legend:
  Cyan: reference source for comparison. Green: improvement, Red: regression.
  Δ: percentage change from reference source.

[github-actions]

Test Results

    4 files  ±0      4 suites  ±0   4m 17s ⏱️ ±0s
  890 tests ±0    890 ✅ ±0  0 💤 ±0  0 ❌ ±0 
3 610 runs  ±0  3 608 ✅ ±0  2 💤 ±0  0 ❌ ±0 

Results for commit 9a2113d. ± Comparison against base commit 942a833.

♻️ This comment has been updated with latest results.

[github-actions]

Code Coverage (Ubuntu)

Details

Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                     19       2  89.47%   60, 69
cli.py                                         259      11  95.75%   99-110, 376, 658
config.py                                      336      27  91.96%   71-73, 76, 79, 82, 86, 90, 94-96, 590-592, 746-748, 772-776, 795, 803, 833-838, 840
crunch.py                                        2       0  100.00%
api/__init__.py                                  0       0  100.00%
api/collection.py                              151       8  94.70%   177, 213, 216, 323, 362, 365, 383, 386
api/core.py                                    777      58  92.54%   262, 548, 596, 639, 679, 699, 740-745, 770, 854-873, 887, 893, 1046, 1051, 1150, 1189-1190, 1286-1287, 1313, 1344-1345, 1371, 1384, 1445-1446, 1477-1478, 1559-1564, 1569-1572, 1684, 1909, 2066, 2070, 2179, 2267-2268
api/provider.py                                389      42  89.20%   128-132, 199, 203-204, 341-362, 430, 546, 550-551, 556-559, 570, 646-656, 841-842, 890, 897, 911, 934-937, 971-978, 982-983
api/search_result.py                           138      12  91.30%   111, 123, 133, 154, 208, 266, 278, 406, 461-464
api/product/__init__.py                         26       4  84.62%   51-54
api/product/_assets.py                          78       5  93.59%   141, 231-232, 244-248
api/product/_product.py                        351      26  92.59%   226, 382, 408-411, 448, 467-468, 577, 606, 613, 653, 703, 709, 844, 887-890, 899-902, 956, 1022, 1034, 1101
api/product/metadata_mapping.py                887      57  93.57%   136-138, 259-264, 288, 347-348, 428, 448-449, 454, 517-518, 555, 576-579, 602, 614-615, 656, 679, 704-707, 709-714, 793, 799, 815, 823, 1054, 1207, 1229, 1238-1242, 1259-1264, 1397, 1420, 1429, 1451, 1456, 1523, 1620, 1641, 1667, 1681, 1779, 1874, 1970, 2036, 2080
api/product/drivers/__init__.py                 11       0  100.00%
api/product/drivers/base.py                     32       0  100.00%
api/product/drivers/generic.py                  11       0  100.00%
api/product/drivers/sentinel1.py                33       0  100.00%
api/product/drivers/sentinel2.py                33       0  100.00%
plugins/__init__.py                              0       0  100.00%
plugins/base.py                                 38       3  92.11%   50, 84-85
plugins/manager.py                             192      17  91.15%   103-108, 138, 220, 242, 260-261, 273, 312-313, 413-416, 428-429
plugins/apis/__init__.py                         0       0  100.00%
plugins/apis/base.py                             8       0  100.00%
plugins/apis/ecmwf.py                           95      10  89.47%   163-165, 213-214, 240-242, 273-274
plugins/apis/usgs.py                           175      25  85.71%   158, 246, 279, 322-324, 329, 357-358, 363, 393-400, 411-416, 438-444
plugins/authentication/__init__.py               6       1  83.33%   32
plugins/authentication/aws_auth.py             124      35  71.77%   52-54, 69-70, 142-149, 177-203, 226, 258-262, 279, 303, 319-320
plugins/authentication/base.py                  26       4  84.62%   58, 71, 94, 108
plugins/authentication/eoiam.py                 98       1  98.98%   170
plugins/authentication/generic.py               16       3  81.25%   50, 55, 65
plugins/authentication/header.py                19       0  100.00%
plugins/authentication/keycloak.py              46       4  91.30%   154, 177-182
plugins/authentication/openid_connect.py       243      21  91.36%   101-102, 110-128, 175, 183, 197, 217, 363-366, 392, 433, 623-626
plugins/authentication/qsauth.py                34       1  97.06%   91
plugins/authentication/sas_auth.py              58       3  94.83%   68, 89, 135
plugins/authentication/token.py                141       9  93.62%   192, 229, 309-310, 358-362
plugins/authentication/token_exchange.py        36      14  61.11%   75, 93-121
plugins/crunch/__init__.py                       6       0  100.00%
plugins/crunch/base.py                          12       0  100.00%
plugins/crunch/filter_date.py                   47       0  100.00%
plugins/crunch/filter_latest_intersect.py       52       6  88.46%   87-88, 95-96, 98-102
plugins/crunch/filter_latest_tpl_name.py        35       0  100.00%
plugins/crunch/filter_overlap.py                66      10  84.85%   118-121, 137-163
plugins/crunch/filter_property.py               30       2  93.33%   81-84
plugins/download/__init__.py                     4       0  100.00%
plugins/download/aws.py                        403      77  80.89%   271, 305, 354-357, 387-388, 396-400, 480-483, 523-525, 529, 560-561, 567-571, 602, 667-675, 739-833, 845-850, 888, 914, 959-961, 1013
plugins/download/base.py                       297      24  91.92%   148, 178, 394-395, 439, 473, 550-554, 584, 619-620, 645-654, 713, 734, 756, 764
plugins/download/http.py                       581      74  87.26%   242, 284-287, 349-352, 355, 362-367, 399-400, 417, 432, 493, 528, 542, 556, 569-573, 589-594, 605, 624, 660-663, 684, 694, 701, 757, 865, 897, 929-938, 974, 1019-1024, 1033, 1048-1050, 1054, 1057, 1072-1073, 1083, 1158, 1210, 1253-1254, 1266, 1276, 1332-1333, 1363, 1390, 1410, 1490-1491
plugins/search/__init__.py                      25       0  100.00%
plugins/search/base.py                         220      20  90.91%   119, 123, 147-153, 162, 228-231, 264, 314, 346, 367, 500, 585-588, 597
plugins/search/build_search_result.py          555      82  85.23%   138-139, 175, 179, 199, 258, 427-441, 520-521, 570, 582-584, 608, 610, 675, 683-687, 707, 717, 743, 792, 817, 879-884, 945, 972, 975, 1030-1051, 1092, 1119-1120, 1129-1138, 1228, 1243, 1246, 1251, 1270-1279, 1330, 1333, 1336, 1338, 1445-1446, 1490, 1499-1501, 1558, 1606-1616
plugins/search/cop_ghsl.py                     410      84  79.51%   73-74, 109-110, 174, 176, 218, 249-252, 298, 314, 360, 364-369, 384, 406, 427, 461, 477, 512, 527, 541-542, 556-558, 565, 569-571, 574-577, 595-655, 666-681
plugins/search/cop_marine.py                   297      65  78.11%   56, 64-66, 82, 104, 106, 109, 175-176, 194, 218, 224, 228, 250-251, 259, 294-297, 303, 313, 326, 334, 340-344, 350-353, 359-412, 428-431, 481, 493, 537-538
plugins/search/creodias_s3.py                   29       1  96.55%   59
plugins/search/csw.py                          112      87  22.32%   99-100, 104-105, 113-170, 176-189, 197-229, 246-287
plugins/search/eumetsat_ds.py                   24       0  100.00%
plugins/search/geodes.py                        66       0  100.00%
plugins/search/oar.py                           82      13  84.15%   50, 57-60, 172-180, 186-188, 195
plugins/search/qssearch.py                     889     108  87.85%   381, 425-426, 543-544, 567-568, 580-584, 816-822, 880, 946, 980, 987, 1058, 1079, 1082-1083, 1101, 1110-1111, 1129, 1200, 1209, 1214-1231, 1240, 1255, 1264, 1277, 1300, 1339, 1412, 1435, 1450, 1523-1524, 1526-1528, 1530-1531, 1618, 1625, 1733-1737, 1803, 1806, 1810-1811, 1830-1833, 1845, 1867-1884, 1892-1898, 1933-1935, 1956-1962, 1969, 2007, 2023, 2046, 2051-2052, 2067, 2073, 2087, 2193, 2197, 2208, 2232, 2245, 2253-2263, 2289
plugins/search/stac_list_assets.py              25      10  60.00%   44-51, 75-85
plugins/search/static_stac_search.py            84      17  79.76%   99-127, 166-169, 182
resources/__init__.py                            0       0  100.00%
resources/shp/__init__.py                        0       0  100.00%
types/__init__.py                              180      43  76.11%   58, 71-75, 86-98, 126-128, 135-140, 155, 165, 247, 280, 290-306, 311, 313, 335, 340, 348, 358
types/bbox.py                                   39      19  51.28%   46-61, 72-74, 85-87, 99-101, 113-115, 123
types/download_args.py                          10       0  100.00%
types/queryables.py                            113       0  100.00%
types/search_args.py                            71      18  74.65%   67-71, 78-95, 110
types/stac_extensions.py                       162       4  97.53%   455-461, 640
types/stac_metadata.py                         159      22  86.16%   98, 120, 153, 186-188, 202-216, 225-232, 259-262
utils/__init__.py                              658      62  90.58%   57, 186-241, 254-256, 274, 313-318, 329-330, 351, 431-435, 510-514, 594, 634-635, 664, 1046-1049, 1100, 1119-1120, 1152, 1170-1171, 1335, 1337, 1345, 1349, 1355, 1357, 1359, 1365, 1367, 1467, 1599, 1664, 1839
utils/cache.py                                  22       0  100.00%
utils/dates.py                                 191       1  99.48%   182
utils/deserialize.py                            74      10  86.49%   53, 55, 103, 108-109, 148, 162, 184-185, 191
utils/env.py                                     3       0  100.00%
utils/exceptions.py                             58       0  100.00%
utils/free_text_search.py                       65       2  96.92%   83, 91
utils/import_system.py                          28      19  32.14%   64-78, 89-99
utils/logging.py                                42       3  92.86%   42-43, 61
utils/notebook.py                               44      23  47.73%   25-29, 36-41, 58-62, 72-78, 83-87
utils/repr.py                                   38       0  100.00%
utils/requests.py                               55      29  47.27%   51-52, 64, 85-96, 107-124, 128
utils/s3.py                                    266      11  95.86%   204-205, 246, 262, 284, 506, 553-554, 622, 697, 723
utils/stac_reader.py                           113      44  61.06%   63-85, 95-97, 101, 138, 154-159, 206-216, 226-256
utils/streamresponse.py                         91       4  95.60%   106, 166, 175, 187
utils/yaml.py                                   42       7  83.33%   49, 65, 70-74
TOTAL                                        11763    1404  88.06%

Diff against develop

Filename      Stmts    Miss  Cover
----------  -------  ------  --------
TOTAL             0       0  +100.00%

Results for commit: 9a2113d

Minimum allowed coverage is 70%

♻️ This comment has been updated with latest results

[github-actions]

Code Coverage (Windows)

Details

Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                     19       2  89.47%   60, 69
cli.py                                         259      11  95.75%   99-110, 376, 658
config.py                                      336      27  91.96%   71-73, 76, 79, 82, 86, 90, 94-96, 590-592, 746-748, 772-776, 795, 803, 833-838, 840
crunch.py                                        2       0  100.00%
api/__init__.py                                  0       0  100.00%
api/collection.py                              151       8  94.70%   177, 213, 216, 323, 362, 365, 383, 386
api/core.py                                    777      58  92.54%   262, 548, 596, 639, 679, 699, 740-745, 770, 854-873, 887, 893, 1046, 1051, 1150, 1189-1190, 1286-1287, 1313, 1344-1345, 1371, 1384, 1445-1446, 1477-1478, 1559-1564, 1569-1572, 1684, 1909, 2066, 2070, 2179, 2267-2268
api/provider.py                                389      42  89.20%   128-132, 199, 203-204, 341-362, 430, 546, 550-551, 556-559, 570, 646-656, 841-842, 890, 897, 911, 934-937, 971-978, 982-983
api/search_result.py                           138      12  91.30%   111, 123, 133, 154, 208, 266, 278, 406, 461-464
api/product/__init__.py                         26       4  84.62%   51-54
api/product/_assets.py                          78       5  93.59%   141, 231-232, 244-248
api/product/_product.py                        351      26  92.59%   226, 382, 408-411, 448, 467-468, 577, 606, 613, 653, 703, 709, 844, 887-890, 899-902, 956, 1022, 1034, 1101
api/product/metadata_mapping.py                887      57  93.57%   136-138, 259-264, 288, 347-348, 428, 448-449, 454, 517-518, 555, 576-579, 602, 614-615, 656, 679, 704-707, 709-714, 793, 799, 815, 823, 1054, 1207, 1229, 1238-1242, 1259-1264, 1397, 1420, 1429, 1451, 1456, 1523, 1620, 1641, 1667, 1681, 1779, 1874, 1970, 2036, 2080
api/product/drivers/__init__.py                 11       0  100.00%
api/product/drivers/base.py                     32       0  100.00%
api/product/drivers/generic.py                  11       0  100.00%
api/product/drivers/sentinel1.py                33       0  100.00%
api/product/drivers/sentinel2.py                33       0  100.00%
plugins/__init__.py                              0       0  100.00%
plugins/base.py                                 38       3  92.11%   50, 84-85
plugins/manager.py                             192      17  91.15%   103-108, 138, 220, 242, 260-261, 273, 312-313, 413-416, 428-429
plugins/apis/__init__.py                         0       0  100.00%
plugins/apis/base.py                             8       0  100.00%
plugins/apis/ecmwf.py                           95      10  89.47%   163-165, 213-214, 240-242, 273-274
plugins/apis/usgs.py                           175      25  85.71%   158, 246, 279, 322-324, 329, 357-358, 363, 393-400, 411-416, 438-444
plugins/authentication/__init__.py               6       1  83.33%   32
plugins/authentication/aws_auth.py             124      35  71.77%   52-54, 69-70, 142-149, 177-203, 226, 258-262, 279, 303, 319-320
plugins/authentication/base.py                  26       4  84.62%   58, 71, 94, 108
plugins/authentication/eoiam.py                 98       1  98.98%   170
plugins/authentication/generic.py               16       3  81.25%   50, 55, 65
plugins/authentication/header.py                19       0  100.00%
plugins/authentication/keycloak.py              46       4  91.30%   154, 177-182
plugins/authentication/openid_connect.py       243      21  91.36%   101-102, 110-128, 175, 183, 197, 217, 363-366, 392, 433, 623-626
plugins/authentication/qsauth.py                34       1  97.06%   91
plugins/authentication/sas_auth.py              58       3  94.83%   68, 89, 135
plugins/authentication/token.py                141       9  93.62%   192, 229, 309-310, 358-362
plugins/authentication/token_exchange.py        36      14  61.11%   75, 93-121
plugins/crunch/__init__.py                       6       0  100.00%
plugins/crunch/base.py                          12       0  100.00%
plugins/crunch/filter_date.py                   47       0  100.00%
plugins/crunch/filter_latest_intersect.py       52       6  88.46%   87-88, 95-96, 98-102
plugins/crunch/filter_latest_tpl_name.py        35       0  100.00%
plugins/crunch/filter_overlap.py                66      10  84.85%   118-121, 137-163
plugins/crunch/filter_property.py               30       2  93.33%   81-84
plugins/download/__init__.py                     4       0  100.00%
plugins/download/aws.py                        403      77  80.89%   271, 305, 354-357, 387-388, 396-400, 480-483, 523-525, 529, 560-561, 567-571, 602, 667-675, 739-833, 845-850, 888, 914, 959-961, 1013
plugins/download/base.py                       297      26  91.25%   148, 178, 253-255, 394-395, 439, 473, 550-554, 584, 619-620, 645-654, 713, 734, 756, 764
plugins/download/http.py                       581      74  87.26%   242, 284-287, 349-352, 355, 362-367, 399-400, 417, 432, 493, 528, 542, 556, 569-573, 589-594, 605, 624, 660-663, 684, 694, 701, 757, 865, 897, 929-938, 974, 1019-1024, 1033, 1048-1050, 1054, 1057, 1072-1073, 1083, 1158, 1210, 1253-1254, 1266, 1276, 1332-1333, 1363, 1390, 1410, 1490-1491
plugins/search/__init__.py                      25       0  100.00%
plugins/search/base.py                         220      20  90.91%   119, 123, 147-153, 162, 228-231, 264, 314, 346, 367, 500, 585-588, 597
plugins/search/build_search_result.py          555      82  85.23%   138-139, 175, 179, 199, 258, 427-441, 520-521, 570, 582-584, 608, 610, 675, 683-687, 707, 717, 743, 792, 817, 879-884, 945, 972, 975, 1030-1051, 1092, 1119-1120, 1129-1138, 1228, 1243, 1246, 1251, 1270-1279, 1330, 1333, 1336, 1338, 1445-1446, 1490, 1499-1501, 1558, 1606-1616
plugins/search/cop_ghsl.py                     410      84  79.51%   73-74, 109-110, 174, 176, 218, 249-252, 298, 314, 360, 364-369, 384, 406, 427, 461, 477, 512, 527, 541-542, 556-558, 565, 569-571, 574-577, 595-655, 666-681
plugins/search/cop_marine.py                   297      65  78.11%   56, 64-66, 82, 104, 106, 109, 175-176, 194, 218, 224, 228, 250-251, 259, 294-297, 303, 313, 326, 334, 340-344, 350-353, 359-412, 428-431, 481, 493, 537-538
plugins/search/creodias_s3.py                   29       1  96.55%   59
plugins/search/csw.py                          112      87  22.32%   99-100, 104-105, 113-170, 176-189, 197-229, 246-287
plugins/search/eumetsat_ds.py                   24       0  100.00%
plugins/search/geodes.py                        66       0  100.00%
plugins/search/oar.py                           82      13  84.15%   50, 57-60, 172-180, 186-188, 195
plugins/search/qssearch.py                     889     108  87.85%   381, 425-426, 543-544, 567-568, 580-584, 816-822, 880, 946, 980, 987, 1058, 1079, 1082-1083, 1101, 1110-1111, 1129, 1200, 1209, 1214-1231, 1240, 1255, 1264, 1277, 1300, 1339, 1412, 1435, 1450, 1523-1524, 1526-1528, 1530-1531, 1618, 1625, 1733-1737, 1803, 1806, 1810-1811, 1830-1833, 1845, 1867-1884, 1892-1898, 1933-1935, 1956-1962, 1969, 2007, 2023, 2046, 2051-2052, 2067, 2073, 2087, 2193, 2197, 2208, 2232, 2245, 2253-2263, 2289
plugins/search/stac_list_assets.py              25      10  60.00%   44-51, 75-85
plugins/search/static_stac_search.py            84      17  79.76%   99-127, 166-169, 182
resources/__init__.py                            0       0  100.00%
resources/shp/__init__.py                        0       0  100.00%
types/__init__.py                              180      43  76.11%   58, 71-75, 86-98, 126-128, 135-140, 155, 165, 247, 280, 290-306, 311, 313, 335, 340, 348, 358
types/bbox.py                                   39      19  51.28%   46-61, 72-74, 85-87, 99-101, 113-115, 123
types/download_args.py                          10       0  100.00%
types/queryables.py                            113       0  100.00%
types/search_args.py                            71      18  74.65%   67-71, 78-95, 110
types/stac_extensions.py                       162       4  97.53%   455-461, 640
types/stac_metadata.py                         159      22  86.16%   98, 120, 153, 186-188, 202-216, 225-232, 259-262
utils/__init__.py                              658      62  90.58%   57, 186-241, 254-256, 274, 313-318, 329-330, 351, 431-435, 510-514, 594, 634-635, 664, 1046-1049, 1100, 1119-1120, 1152, 1170-1171, 1335, 1337, 1345, 1349, 1355, 1357, 1359, 1365, 1367, 1467, 1599, 1664, 1839
utils/cache.py                                  22       0  100.00%
utils/dates.py                                 191       1  99.48%   182
utils/deserialize.py                            74      10  86.49%   53, 55, 103, 108-109, 148, 162, 184-185, 191
utils/env.py                                     3       0  100.00%
utils/exceptions.py                             58       0  100.00%
utils/free_text_search.py                       65       2  96.92%   83, 91
utils/import_system.py                          28      19  32.14%   64-78, 89-99
utils/logging.py                                42       3  92.86%   42-43, 61
utils/notebook.py                               44      23  47.73%   25-29, 36-41, 58-62, 72-78, 83-87
utils/repr.py                                   38       0  100.00%
utils/requests.py                               55      29  47.27%   51-52, 64, 85-96, 107-124, 128
utils/s3.py                                    266      11  95.86%   204-205, 246, 262, 284, 506, 553-554, 622, 697, 723
utils/stac_reader.py                           113      44  61.06%   63-85, 95-97, 101, 138, 154-159, 206-216, 226-256
utils/streamresponse.py                         91       4  95.60%   106, 166, 175, 187
utils/yaml.py                                   42       7  83.33%   49, 65, 70-74
TOTAL                                        11763    1406  88.05%

Diff against develop

Filename      Stmts    Miss  Cover
----------  -------  ------  --------
TOTAL             0       0  +100.00%

Results for commit: 9a2113d

Minimum allowed coverage is 70%

♻️ This comment has been updated with latest results