fix(core): avoid s3 fsspec storage options for http assets

[sbrunato]

Using EOProduct.get_storage_options(), avoids s3 fsspec storage options for http assets

[github-actions]

Benchmark comparison

Compared baseline: origin/develop
Compared candidate: 8cfc331

------------------------------- benchmark: 9 tests, 2 sources -------------------------------
Name (time in us)                                 baseline Median  candidate Median   ΔMedian
---------------------------------------------------------------------------------------------
test_benchmark_eoproduct_assets_population                12.6540           13.2050     +4.4%
test_benchmark_eoproduct_instantiation                    29.9110           30.0070     +0.3%
test_benchmark_stac_reader_fetch_recursive             4,173.8780        4,062.3710     -2.7%
test_benchmark_cli_version                            15,991.8910       16,341.3625     +2.2%
test_benchmark_cli_without_args_subprocess           139,440.6475      134,980.9415     -3.2%
test_benchmark_core_search_with_provider             167,174.8110      164,463.6170     -1.6%
test_benchmark_core_search_with_count                221,342.4750      215,190.6160     -2.8%
test_benchmark_cli_list_collections                  277,336.1190      279,293.9910     +0.7%
test_benchmark_eodag_instantiation_subprocess      1,063,104.1190    1,063,361.7770     +0.0%
---------------------------------------------------------------------------------------------

Legend:
  Cyan: reference source for comparison. Green: improvement, Red: regression.
  Δ: percentage change from reference source.

[github-actions]

Test Results

    4 files  ±0      4 suites  ±0   4m 3s ⏱️ -11s
  972 tests +1    972 ✅ +1  0 💤 ±0  0 ❌ ±0 
3 966 runs  +4  3 964 ✅ +4  2 💤 ±0  0 ❌ ±0 

Results for commit 97f96ed. ± Comparison against base commit 086ad68.

[github-actions]

Code Coverage (Ubuntu)

Details

Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                     19       2  89.47%   60, 69
cli.py                                         259      11  95.75%   99-110, 376, 658
config.py                                      336      28  91.67%   71-73, 76, 79, 82, 86, 90, 94-96, 590-592, 643, 750-752, 776-780, 799, 807, 837-842, 844
crunch.py                                        2       0  100.00%
api/__init__.py                                  0       0  100.00%
api/collection.py                              151       7  95.36%   177, 213, 323, 362, 365, 383, 386
api/core.py                                    777      22  97.17%   262, 548, 596, 639, 679, 699, 770, 867-871, 893, 1046, 1051, 1313, 1384, 1684, 1909, 2066, 2070, 2179
api/provider.py                                389      42  89.20%   128-132, 199, 203-204, 341-362, 430, 546, 550-551, 556-559, 570, 646-656, 841-842, 890, 897, 911, 934-937, 971-978, 982-983
api/search_result.py                           138      12  91.30%   111, 123, 133, 154, 208, 266, 278, 406, 461-464
api/product/__init__.py                         26       4  84.62%   51-54
api/product/_assets.py                          78       5  93.59%   141, 231-232, 244-248
api/product/_product.py                        351      26  92.59%   226, 382, 408-411, 448, 467-468, 577, 606, 613, 653, 703, 709, 846, 889-892, 901-904, 958, 1024, 1036, 1103
api/product/metadata_mapping.py                887      57  93.57%   136-138, 259-264, 288, 347-348, 428, 448-449, 454, 517-518, 555, 576-579, 602, 614-615, 656, 679, 704-707, 709-714, 793, 799, 815, 823, 1054, 1207, 1229, 1238-1242, 1259-1264, 1397, 1420, 1429, 1451, 1456, 1523, 1620, 1641, 1667, 1681, 1779, 1874, 1970, 2036, 2080
api/product/drivers/__init__.py                 11       0  100.00%
api/product/drivers/base.py                     32       0  100.00%
api/product/drivers/generic.py                  11       0  100.00%
api/product/drivers/sentinel1.py                33       0  100.00%
api/product/drivers/sentinel2.py                33       0  100.00%
plugins/__init__.py                              0       0  100.00%
plugins/base.py                                 38       3  92.11%   50, 84-85
plugins/manager.py                             192      17  91.15%   103-108, 138, 220, 242, 260-261, 273, 312-313, 416-419, 431-432
plugins/apis/__init__.py                         0       0  100.00%
plugins/apis/base.py                             8       0  100.00%
plugins/apis/ecmwf.py                           95      10  89.47%   163-165, 213-214, 240-242, 273-274
plugins/apis/usgs.py                           175      25  85.71%   158, 246, 279, 322-324, 329, 357-358, 363, 393-400, 411-416, 438-444
plugins/authentication/__init__.py               6       1  83.33%   32
plugins/authentication/aws_auth.py             124      32  74.19%   69-70, 142-149, 177-203, 226, 258-262, 279, 303, 319-320
plugins/authentication/base.py                  26       4  84.62%   61, 74, 97, 111
plugins/authentication/eoiam.py                 98       1  98.98%   170
plugins/authentication/generic.py               16       3  81.25%   50, 55, 65
plugins/authentication/header.py                19       0  100.00%
plugins/authentication/keycloak.py              46       4  91.30%   154, 177-182
plugins/authentication/openid_connect.py       243      21  91.36%   101-102, 110-128, 175, 183, 197, 217, 363-366, 392, 433, 623-626
plugins/authentication/qsauth.py                34       1  97.06%   91
plugins/authentication/sas_auth.py              58       3  94.83%   68, 89, 135
plugins/authentication/token.py                141       9  93.62%   192, 229, 309-310, 358-362
plugins/authentication/token_exchange.py        36       3  91.67%   75, 114-115
plugins/crunch/__init__.py                       6       0  100.00%
plugins/crunch/base.py                          12       0  100.00%
plugins/crunch/filter_date.py                   47       0  100.00%
plugins/crunch/filter_latest_intersect.py       52       6  88.46%   87-88, 95-96, 98-102
plugins/crunch/filter_latest_tpl_name.py        35       0  100.00%
plugins/crunch/filter_overlap.py                66      10  84.85%   118-121, 137-163
plugins/crunch/filter_property.py               30       2  93.33%   81-84
plugins/download/__init__.py                     4       0  100.00%
plugins/download/aws.py                        404      42  89.60%   271, 305, 354-357, 387-388, 396-400, 480-483, 523-525, 529, 560-561, 567-571, 602, 667-675, 763, 780, 795, 827-828, 891, 917, 962-964, 1016
plugins/download/base.py                       297      24  91.92%   148, 178, 394-395, 439, 473, 550-554, 584, 619-620, 645-654, 713, 734, 756, 764
plugins/download/http.py                       581      74  87.26%   242, 284-287, 349-352, 355, 362-367, 399-400, 417, 432, 493, 528, 542, 556, 569-573, 589-594, 605, 624, 660-663, 684, 694, 701, 757, 865, 897, 929-938, 974, 1019-1024, 1033, 1048-1050, 1054, 1057, 1072-1073, 1083, 1158, 1210, 1253-1254, 1266, 1276, 1332-1333, 1363, 1390, 1410, 1490-1491
plugins/search/__init__.py                      25       0  100.00%
plugins/search/base.py                         220      20  90.91%   119, 123, 147-153, 162, 228-231, 264, 314, 346, 367, 500, 585-588, 597
plugins/search/build_search_result.py          555      41  92.61%   258, 430-438, 520-521, 570, 582-584, 608, 610, 675, 683-687, 707, 717, 743, 792, 817, 879-884, 945, 972, 975, 1092, 1228, 1243, 1246, 1271, 1445-1446, 1490, 1499-1501, 1558
plugins/search/cop_ghsl.py                     410      27  93.41%   73-74, 109-110, 174, 176, 218, 298, 314, 360, 384, 406, 427, 461, 477, 512, 541-542, 556-558, 569-571, 613, 650, 654
plugins/search/cop_marine.py                   297      34  88.55%   56, 64-66, 82, 104, 106, 109, 175-176, 194, 218, 224, 228, 250-251, 259, 303, 313, 326, 334, 340-344, 361, 393-396, 411, 481, 493, 537-538
plugins/search/creodias_s3.py                   29       1  96.55%   59
plugins/search/csw.py                          118      12  89.83%   121, 165, 203, 214-225, 230
plugins/search/eumetsat_ds.py                   24       0  100.00%
plugins/search/geodes.py                        66       0  100.00%
plugins/search/oar.py                           82      13  84.15%   50, 57-60, 172-180, 186-188, 195
plugins/search/qssearch.py                     895      84  90.61%   381, 543-544, 567-568, 580-584, 880, 946, 980, 1058, 1079, 1082-1083, 1101, 1110-1111, 1129, 1200, 1209, 1221-1231, 1240, 1255, 1264, 1277, 1300, 1339, 1412, 1450, 1523-1524, 1526-1528, 1530-1531, 1803, 1806, 1810-1811, 1830-1833, 1845, 1867-1884, 1892-1898, 1956-1962, 1969, 2007, 2053, 2058-2059, 2089, 2209, 2213, 2224, 2248, 2261, 2269-2279, 2305
plugins/search/stac_list_assets.py              25       0  100.00%
plugins/search/static_stac_search.py            84      17  79.76%   99-127, 166-169, 182
resources/__init__.py                            0       0  100.00%
resources/shp/__init__.py                        0       0  100.00%
types/__init__.py                              180      43  76.11%   58, 71-75, 86-98, 126-128, 135-140, 155, 165, 247, 280, 290-306, 311, 313, 335, 340, 348, 358
types/bbox.py                                   39       0  100.00%
types/download_args.py                          10       0  100.00%
types/queryables.py                            113       0  100.00%
types/search_args.py                            71      18  74.65%   67-71, 78-95, 110
types/stac_extensions.py                       162       4  97.53%   455-461, 640
types/stac_metadata.py                         159      22  86.16%   98, 120, 153, 186-188, 202-216, 225-232, 259-262
utils/__init__.py                              658      19  97.11%   57, 216, 223, 239, 274, 351, 606, 646-647, 676, 1060-1063, 1133-1134, 1166, 1373, 1481, 1613
utils/cache.py                                  22       0  100.00%
utils/dates.py                                 191       1  99.48%   182
utils/deserialize.py                            74      10  86.49%   53, 55, 103, 108-109, 148, 162, 184-185, 191
utils/env.py                                     3       0  100.00%
utils/exceptions.py                             58       0  100.00%
utils/free_text_search.py                       65       2  96.92%   83, 91
utils/import_system.py                          28       6  78.57%   65-73
utils/logging.py                                42       3  92.86%   42-43, 61
utils/notebook.py                               44      13  70.45%   27, 41, 58-62, 72-78, 86-87
utils/repr.py                                   38       0  100.00%
utils/requests.py                               55       6  89.09%   94, 110, 118-120, 128
utils/s3.py                                    266      11  95.86%   204-205, 246, 262, 284, 506, 553-554, 622, 697, 723
utils/stac_reader.py                           113      42  62.83%   63-85, 101, 138, 154-159, 206-216, 226-256
utils/streamresponse.py                         91       4  95.60%   106, 166, 175, 187
utils/yaml.py                                   42       7  83.33%   49, 65, 70-74
TOTAL                                        11776     971  91.75%

Diff against develop

Filename      Stmts    Miss  Cover
----------  -------  ------  --------
TOTAL             0       0  +100.00%

Results for commit: 97f96ed

Minimum allowed coverage is 70%

♻️ This comment has been updated with latest results

[github-actions]

Code Coverage (Windows)

Details

Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                     19       2  89.47%   60, 69
cli.py                                         259      11  95.75%   99-110, 376, 658
config.py                                      336      28  91.67%   71-73, 76, 79, 82, 86, 90, 94-96, 590-592, 643, 750-752, 776-780, 799, 807, 837-842, 844
crunch.py                                        2       0  100.00%
api/__init__.py                                  0       0  100.00%
api/collection.py                              151       7  95.36%   177, 213, 323, 362, 365, 383, 386
api/core.py                                    777      22  97.17%   262, 548, 596, 639, 679, 699, 770, 867-871, 893, 1046, 1051, 1313, 1384, 1684, 1909, 2066, 2070, 2179
api/provider.py                                389      42  89.20%   128-132, 199, 203-204, 341-362, 430, 546, 550-551, 556-559, 570, 646-656, 841-842, 890, 897, 911, 934-937, 971-978, 982-983
api/search_result.py                           138      12  91.30%   111, 123, 133, 154, 208, 266, 278, 406, 461-464
api/product/__init__.py                         26       4  84.62%   51-54
api/product/_assets.py                          78       5  93.59%   141, 231-232, 244-248
api/product/_product.py                        351      26  92.59%   226, 382, 408-411, 448, 467-468, 577, 606, 613, 653, 703, 709, 846, 889-892, 901-904, 958, 1024, 1036, 1103
api/product/metadata_mapping.py                887      57  93.57%   136-138, 259-264, 288, 347-348, 428, 448-449, 454, 517-518, 555, 576-579, 602, 614-615, 656, 679, 704-707, 709-714, 793, 799, 815, 823, 1054, 1207, 1229, 1238-1242, 1259-1264, 1397, 1420, 1429, 1451, 1456, 1523, 1620, 1641, 1667, 1681, 1779, 1874, 1970, 2036, 2080
api/product/drivers/__init__.py                 11       0  100.00%
api/product/drivers/base.py                     32       0  100.00%
api/product/drivers/generic.py                  11       0  100.00%
api/product/drivers/sentinel1.py                33       0  100.00%
api/product/drivers/sentinel2.py                33       0  100.00%
plugins/__init__.py                              0       0  100.00%
plugins/base.py                                 38       3  92.11%   50, 84-85
plugins/manager.py                             192      17  91.15%   103-108, 138, 220, 242, 260-261, 273, 312-313, 416-419, 431-432
plugins/apis/__init__.py                         0       0  100.00%
plugins/apis/base.py                             8       0  100.00%
plugins/apis/ecmwf.py                           95      10  89.47%   163-165, 213-214, 240-242, 273-274
plugins/apis/usgs.py                           175      25  85.71%   158, 246, 279, 322-324, 329, 357-358, 363, 393-400, 411-416, 438-444
plugins/authentication/__init__.py               6       1  83.33%   32
plugins/authentication/aws_auth.py             124      32  74.19%   69-70, 142-149, 177-203, 226, 258-262, 279, 303, 319-320
plugins/authentication/base.py                  26       4  84.62%   61, 74, 97, 111
plugins/authentication/eoiam.py                 98       1  98.98%   170
plugins/authentication/generic.py               16       3  81.25%   50, 55, 65
plugins/authentication/header.py                19       0  100.00%
plugins/authentication/keycloak.py              46       4  91.30%   154, 177-182
plugins/authentication/openid_connect.py       243      21  91.36%   101-102, 110-128, 175, 183, 197, 217, 363-366, 392, 433, 623-626
plugins/authentication/qsauth.py                34       1  97.06%   91
plugins/authentication/sas_auth.py              58       3  94.83%   68, 89, 135
plugins/authentication/token.py                141       9  93.62%   192, 229, 309-310, 358-362
plugins/authentication/token_exchange.py        36       3  91.67%   75, 114-115
plugins/crunch/__init__.py                       6       0  100.00%
plugins/crunch/base.py                          12       0  100.00%
plugins/crunch/filter_date.py                   47       0  100.00%
plugins/crunch/filter_latest_intersect.py       52       6  88.46%   87-88, 95-96, 98-102
plugins/crunch/filter_latest_tpl_name.py        35       0  100.00%
plugins/crunch/filter_overlap.py                66      10  84.85%   118-121, 137-163
plugins/crunch/filter_property.py               30       2  93.33%   81-84
plugins/download/__init__.py                     4       0  100.00%
plugins/download/aws.py                        404      42  89.60%   271, 305, 354-357, 387-388, 396-400, 480-483, 523-525, 529, 560-561, 567-571, 602, 667-675, 763, 780, 795, 827-828, 891, 917, 962-964, 1016
plugins/download/base.py                       297      26  91.25%   148, 178, 253-255, 394-395, 439, 473, 550-554, 584, 619-620, 645-654, 713, 734, 756, 764
plugins/download/http.py                       581      74  87.26%   242, 284-287, 349-352, 355, 362-367, 399-400, 417, 432, 493, 528, 542, 556, 569-573, 589-594, 605, 624, 660-663, 684, 694, 701, 757, 865, 897, 929-938, 974, 1019-1024, 1033, 1048-1050, 1054, 1057, 1072-1073, 1083, 1158, 1210, 1253-1254, 1266, 1276, 1332-1333, 1363, 1390, 1410, 1490-1491
plugins/search/__init__.py                      25       0  100.00%
plugins/search/base.py                         220      20  90.91%   119, 123, 147-153, 162, 228-231, 264, 314, 346, 367, 500, 585-588, 597
plugins/search/build_search_result.py          555      41  92.61%   258, 430-438, 520-521, 570, 582-584, 608, 610, 675, 683-687, 707, 717, 743, 792, 817, 879-884, 945, 972, 975, 1092, 1228, 1243, 1246, 1271, 1445-1446, 1490, 1499-1501, 1558
plugins/search/cop_ghsl.py                     410      27  93.41%   73-74, 109-110, 174, 176, 218, 298, 314, 360, 384, 406, 427, 461, 477, 512, 541-542, 556-558, 569-571, 613, 650, 654
plugins/search/cop_marine.py                   297      34  88.55%   56, 64-66, 82, 104, 106, 109, 175-176, 194, 218, 224, 228, 250-251, 259, 303, 313, 326, 334, 340-344, 361, 393-396, 411, 481, 493, 537-538
plugins/search/creodias_s3.py                   29       1  96.55%   59
plugins/search/csw.py                          118      12  89.83%   121, 165, 203, 214-225, 230
plugins/search/eumetsat_ds.py                   24       0  100.00%
plugins/search/geodes.py                        66       0  100.00%
plugins/search/oar.py                           82      13  84.15%   50, 57-60, 172-180, 186-188, 195
plugins/search/qssearch.py                     895      84  90.61%   381, 543-544, 567-568, 580-584, 880, 946, 980, 1058, 1079, 1082-1083, 1101, 1110-1111, 1129, 1200, 1209, 1221-1231, 1240, 1255, 1264, 1277, 1300, 1339, 1412, 1450, 1523-1524, 1526-1528, 1530-1531, 1803, 1806, 1810-1811, 1830-1833, 1845, 1867-1884, 1892-1898, 1956-1962, 1969, 2007, 2053, 2058-2059, 2089, 2209, 2213, 2224, 2248, 2261, 2269-2279, 2305
plugins/search/stac_list_assets.py              25       0  100.00%
plugins/search/static_stac_search.py            84      17  79.76%   99-127, 166-169, 182
resources/__init__.py                            0       0  100.00%
resources/shp/__init__.py                        0       0  100.00%
types/__init__.py                              180      43  76.11%   58, 71-75, 86-98, 126-128, 135-140, 155, 165, 247, 280, 290-306, 311, 313, 335, 340, 348, 358
types/bbox.py                                   39       0  100.00%
types/download_args.py                          10       0  100.00%
types/queryables.py                            113       0  100.00%
types/search_args.py                            71      18  74.65%   67-71, 78-95, 110
types/stac_extensions.py                       162       4  97.53%   455-461, 640
types/stac_metadata.py                         159      22  86.16%   98, 120, 153, 186-188, 202-216, 225-232, 259-262
utils/__init__.py                              658      19  97.11%   57, 216, 223, 239, 274, 351, 606, 646-647, 676, 1060-1063, 1133-1134, 1166, 1373, 1481, 1613
utils/cache.py                                  22       0  100.00%
utils/dates.py                                 191       1  99.48%   182
utils/deserialize.py                            74      10  86.49%   53, 55, 103, 108-109, 148, 162, 184-185, 191
utils/env.py                                     3       0  100.00%
utils/exceptions.py                             58       0  100.00%
utils/free_text_search.py                       65       2  96.92%   83, 91
utils/import_system.py                          28       6  78.57%   65-73
utils/logging.py                                42       3  92.86%   42-43, 61
utils/notebook.py                               44      13  70.45%   27, 41, 58-62, 72-78, 86-87
utils/repr.py                                   38       0  100.00%
utils/requests.py                               55       6  89.09%   94, 110, 118-120, 128
utils/s3.py                                    266      11  95.86%   204-205, 246, 262, 284, 506, 553-554, 622, 697, 723
utils/stac_reader.py                           113      42  62.83%   63-85, 101, 138, 154-159, 206-216, 226-256
utils/streamresponse.py                         91       4  95.60%   106, 166, 175, 187
utils/yaml.py                                   42       7  83.33%   49, 65, 70-74
TOTAL                                        11776     973  91.74%

Diff against develop

Filename      Stmts    Miss  Cover
----------  -------  ------  --------
TOTAL             0       0  +100.00%

Results for commit: 97f96ed

Minimum allowed coverage is 70%

♻️ This comment has been updated with latest results