Skip to content

Introducing parallel variant of add_all_ta_features. - #310

Open
such3r wants to merge 1 commit into
bukosabino:masterfrom
such3r:master
Open

such3r wants to merge 1 commit into
bukosabino:masterfrom
such3r:master

Conversation

@such3r

@such3r such3r commented Dec 1, 2022

Copy link
Copy Markdown

Suggest simple process-based all-set indicators calculation parallelizing. Gives ~2x speedup compared to the initial implementation.

+ Add add_all_ta_features_parallel parallelizing all indicators calculation.
* Fix __init__.py and README.md.

@Harsh-Maheshwari Harsh-Maheshwari left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@such3r
Getting this error when running

line 609, in _job
    instance = globals()[obj](**{k: target[v] if k in ohlcv_keys else v for k, v in kwargs.items()})
TypeError: __init__() got an unexpected keyword argument 'close'

The above exception was the direct cause of the following exception:

TypeError                                 Traceback (most recent call last)
Cell In[4], line 1
----> 1 _ = add_all_ta_features_parallel(df,open="open", high="high", low="low", close="close", volume="volume", fillna=False)

File ~/utils/parallel_ta.py:1068, in add_all_ta_features_parallel(df, open, high, low, close, volume, fillna, colprefix, vectorized, n_jobs)
    964     init += [
    965         # Money Flow Indicator
    966         (
   (...)
   1063         ),
   1064     ]
   1066 columns_order = df.columns.tolist() + [column for *_, jobs in init for column, _ in jobs]
-> 1068 results = Parallel(n_jobs=n_jobs)(delayed(_job)(df, *args) for args in init)
   1070 for columns in results:
   1071     for column, values in columns.items():

@such3r

such3r commented Apr 18, 2024 •

Copy link
Copy Markdown
Author

@Harsh-Maheshwari, could you, please, provide the input data you used?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants