Skip to content
This repository was archived by the owner on Feb 2, 2024. It is now read-only.

Commit c8d370c

Browse files
PokhodenkoSAPokhodenkoSA
authored
Fix README (#786)
1 parent 4b7cfcd commit c8d370c

1 file changed

Lines changed: 84 additions & 65 deletions

File tree

‎README.rst‎

Lines changed: 84 additions & 65 deletions
Original file line numberDiff line numberDiff line change
@@ -10,44 +10,49 @@ Intel® Scalable Dataframe Compiler
1010
:target: https://dev.azure.com/IntelPython/HPAT/_build/latest?definitionId=2&branchName=master
1111
:alt: Azure Pipelines
1212

13+
.. _Numba*: https://numba.pydata.org/
14+
.. _Pandas*: https://pandas.pydata.org/
15+
.. _Sphinx*: https://www.sphinx-doc.org/
16+
1317
Numba* Extension For Pandas* Operations Compilation
1418
###################################################
1519

16-
Intel® Scalable Dataframe Compiler (Intel® SDC), which is an extension of `Numba* <https://numba.pydata.org/>`_
17-
that enables compilation of `Pandas* <https://pandas.pydata.org/>`_ operations. It automatically vectorizes and parallelizes
20+
Intel® Scalable Dataframe Compiler (Intel® SDC) is an extension of `Numba*`_
21+
that enables compilation of `Pandas*`_ operations. It automatically vectorizes and parallelizes
1822
the code by leveraging modern hardware instructions and by utilizing all available cores.
1923

20-
Intel SDC's documentation can be found `here <https://intelpython.github.io/sdc-doc/>`_.
24+
Intel® SDC documentation can be found `here <https://intelpython.github.io/sdc-doc/>`_.
2125

22-
Intel SDC uses special Numba build based on ``0.48.0`` tag for build and run.
26+
Intel® SDC uses special Numba build based on ``0.48.0`` tag for build and run.
2327
Required Numba version can be installed from ``intel/label/beta`` channel from the Anaconda Cloud.
24-
Note: For maximum performance and stability please use numba from ``intel/label/beta`` channel.
28+
29+
.. note::
30+
For maximum performance and stability, please use numba from ``intel/label/beta`` channel.
2531

2632
Installing Binary Packages (conda and wheel)
27-
----------------------------------
28-
Intel SDC is available on the Anaconda Cloud intel/label/beta channel.
29-
Distribution includes Intel SDC for Python 3.6 and Python 3.7 for Windows and Linux platforms.
33+
--------------------------------------------
3034

31-
Intel SDC conda package can be installed using the steps below:
32-
::
35+
Intel® SDC is available on the Anaconda Cloud ``intel/label/beta`` channel.
36+
Distribution includes Intel® SDC for Python 3.6 and Python 3.7 for Windows and Linux platforms.
3337

34-
> conda create -n sdc_env python=<3.7 or 3.6>
35-
> conda activate sdc_env
38+
Intel® SDC conda package can be installed using the steps below::
39+
40+
> conda create -n sdc-env python=<3.7 or 3.6>
41+
> conda activate sdc-env
3642
> conda install sdc -c intel/label/beta -c intel -c defaults -c conda-forge --override-channels
3743

38-
Intel SDC wheel package can be installed using the steps below:
39-
::
44+
Intel® SDC wheel package can be installed using the steps below::
4045

41-
> conda create -n sdc_env python=<3.7 or 3.6> pip
42-
> conda activate sdc_env
46+
> conda create -n sdc-env python=<3.7 or 3.6> pip
47+
> conda activate sdc-env
4348
> pip install --index-url https://pypi.anaconda.org/intel/label/beta/simple --extra-index-url https://pypi.anaconda.org/intel/simple --extra-index-url https://pypi.org/simple sdc
4449

4550

4651
Building Intel® SDC from Source on Linux
4752
----------------------------------------
4853

4954
We use `Anaconda <https://www.anaconda.com/download/>`_ distribution of
50-
Python for setting up Intel SDC build environment.
55+
Python for setting up Intel® SDC build environment.
5156

5257
If you do not have conda, we recommend using Miniconda3::
5358

@@ -56,21 +61,23 @@ If you do not have conda, we recommend using Miniconda3::
5661
./miniconda.sh -b
5762
export PATH=$HOME/miniconda3/bin:$PATH
5863

59-
Intel SDC uses special Numba build based on ``0.48.0`` tag for build and run.
64+
Intel® SDC uses special Numba build based on ``0.48.0`` tag for build and run.
6065
Required Numba version can be installed from ``intel/label/beta`` channel from the Anaconda Cloud.
61-
Note: For maximum performance and stability please use numba from ``intel/label/beta`` channel.
6266

63-
It is possible to build Intel SDC via conda-build or setuptools. Follow one of the
64-
cases below to install Intel SDC and its dependencies on Linux.
67+
.. note::
68+
For maximum performance and stability, please use numba from ``intel/label/beta`` channel.
69+
70+
It is possible to build Intel® SDC via conda-build or setuptools. Follow one of the
71+
cases below to install Intel® SDC and its dependencies on Linux.
6572

6673
Building on Linux with conda-build
6774
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
6875
::
6976

7077
PYVER=<3.6 or 3.7>
7178
NUMPYVER=<1.16 or 1.17>
72-
conda create -n CBLD python=$PYVER conda-build
73-
source activate CBLD
79+
conda create -n conda-build-env python=$PYVER conda-build
80+
source activate conda-build-env
7481
git clone https://github.com/IntelPython/sdc.git
7582
cd sdc
7683
conda build --python $PYVER --numpy $NUMPYVER --output-folder=<output_folder> -c intel/label/beta -c defaults -c intel -c conda-forge --override-channels buildscripts/sdc-conda-recipe
@@ -81,8 +88,8 @@ Building on Linux with setuptools
8188

8289
PYVER=<3.6 or 3.7>
8390
NUMPYVER=<1.16 or 1.17>
84-
conda create -n SDC -q -y -c intel/label/beta -c defaults -c intel -c conda-forge python=$PYVER numpy=$NUMPYVER numba=0.48.0 pandas=0.25.3 scipy pyarrow=0.15.1 gcc_linux-64 gxx_linux-64
85-
source activate SDC
91+
conda create -n sdc-env -q -y -c intel/label/beta -c defaults -c intel -c conda-forge python=$PYVER numpy=$NUMPYVER numba=0.48.0 pandas=0.25.3 scipy pyarrow=0.15.1 gcc_linux-64 gxx_linux-64
92+
source activate sdc-env
8693
git clone https://github.com/IntelPython/sdc.git
8794
cd sdc
8895
python setup.py install
@@ -98,17 +105,17 @@ Building Intel® SDC on Windows requires Build Tools for Visual Studio 2019 (wit
98105
* Install `Miniconda for Windows <https://repo.continuum.io/miniconda/Miniconda3-latest-Windows-x86_64.exe>`_.
99106
* Start 'Anaconda prompt'
100107

101-
It is possible to build Intel SDC via conda-build or setuptools. Follow one of the
102-
cases below to install Intel SDC and its dependencies on Windows.
108+
It is possible to build Intel® SDC via conda-build or setuptools. Follow one of the
109+
cases below to install Intel® SDC and its dependencies on Windows.
103110

104111
Building on Windows with conda-build
105112
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
106113
::
107114

108115
set PYVER=<3.6 or 3.7>
109116
set NUMPYVER=<1.16 or 1.17>
110-
conda create -n CBLD -q -y python=%PYVER% conda-build conda-verify vc vs2015_runtime vs2015_win-64
111-
conda activate CBLD
117+
conda create -n conda-build-env -q -y python=%PYVER% conda-build conda-verify vc vs2015_runtime vs2015_win-64
118+
conda activate conda-build-env
112119
git clone https://github.com/IntelPython/sdc.git
113120
cd sdc
114121
conda build --python %PYVER% --numpy %NUMPYVER% --output-folder=<output_folder> -c intel/label/beta -c defaults -c intel -c conda-forge --override-channels buildscripts\sdc-conda-recipe
@@ -119,8 +126,8 @@ Building on Windows with setuptools
119126

120127
set PYVER=<3.6 or 3.7>
121128
set NUMPYVER=<1.16 or 1.17>
122-
conda create -n SDC -c intel/label/beta -c defaults -c intel -c conda-forge python=%PYVER% numpy=%NUMPYVER% numba=0.48.0 pandas=0.25.3 scipy pyarrow=0.15.1
123-
conda activate SDC
129+
conda create -n sdc-env -c intel/label/beta -c defaults -c intel -c conda-forge python=%PYVER% numpy=%NUMPYVER% numba=0.48.0 pandas=0.25.3 scipy pyarrow=0.15.1
130+
conda activate sdc-env
124131
set INCLUDE=%INCLUDE%;%CONDA_PREFIX%\Library\include
125132
set LIB=%LIB%;%CONDA_PREFIX%\Library\lib
126133
git clone https://github.com/IntelPython/sdc.git
@@ -138,71 +145,83 @@ Troubleshooting Windows Build
138145
* For setting up Visual Studio, one might need go to registry at
139146
``HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\VisualStudio\SxS\VS7``,
140147
and add a string value named ``14.0`` whose data is ``C:\Program Files (x86)\Microsoft Visual Studio 14.0\``.
141-
* Sometimes if the conda version or visual studio version being used are not latest then building Intel SDC can throw some vague error about a keyword used in a file. So make sure you are using the latest versions.
142-
148+
* Sometimes if the conda version or visual studio version being used are not latest then
149+
building Intel® SDC can throw some vague error about a keyword used in a file.
150+
So make sure you are using the latest versions.
143151

144152
Building documentation
145153
----------------------
146-
Building Intel SDC User's Guide documentation requires pre-installed Intel SDC package along with compatible Pandas* version as well as Sphinx* 2.2.1 or later.
147154

148-
Intel SDC documentation includes Intel SDC examples output which is pasted to functions description in the API Reference.
155+
Building Intel® SDC User's Guide documentation requires pre-installed Intel® SDC package
156+
along with compatible `Pandas*`_ version as well as `Sphinx*`_ 2.2.1 or later.
157+
158+
Intel® SDC documentation includes Intel® SDC examples output which is pasted to functions description in the API Reference.
149159
In order to get the correct examples result it is required to install ``scipy`` package before documentation build:
150160
::
151161

152162
conda install scipy -c intel --override-channels
153163

154-
Use ``pip`` to install Sphinx* and extensions:
164+
Use ``pip`` to install `Sphinx*`_ and extensions:
155165
::
156166

157167
pip install sphinx sphinxcontrib-programoutput
158168

159-
Currently the build precedure is based on ``make`` located at ``./sdc/docs/`` folder. While it is not generally required we recommended that you clean up the system from previous documentaiton build by running
169+
Currently the build precedure is based on ``make`` located at ``./sdc/docs/`` folder.
170+
While it is not generally required we recommended that you clean up the system from previous documentaiton build by running:
160171
::
161172

162173
make clean
163174

164-
To build HTML documentation you will need to run
175+
To build HTML documentation you will need to run:
165176
::
166177

167178
make html
168179

169-
The built documentation will be located in the ``.sdc/docs/build/html`` directory. To preview the documentation open ``index.html``
170-
file.
180+
The built documentation will be located in the ``./sdc/docs/build/html`` directory.
181+
To preview the documentation open ``index.html`` file.
182+
183+
`Sphinx*`_ Generation Internals
184+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
171185

172-
Sphinx* Generation Internals
173-
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
174-
The documentation generation is controlled by ``conf.py`` script automatically invoked by Sphinx.
186+
The documentation generation is controlled by ``conf.py`` script automatically invoked by `Sphinx*`_.
175187
See `Sphinx documentation <http://www.sphinx-doc.org/en/master/usage/configuration.html>`_ for details.
176188

177-
The API Reference for Intel SDC User's Guide is auto-generated by inspecting ``pandas`` and ``sdc`` modules. That's why these modules must be pre-installed for documentation generation using Sphinx*. However, there is a possibility to skip API Reference auto-generation by setting environment variable ``SDC_DOC_NO_API_REF_STR=1``.
189+
The API Reference for Intel® SDC User's Guide is auto-generated by inspecting ``pandas`` and ``sdc`` modules.
190+
That's why these modules must be pre-installed for documentation generation using `Sphinx*`_.
191+
However, there is a possibility to skip API Reference auto-generation by setting environment variable ``SDC_DOC_NO_API_REF_STR=1``.
178192

179-
If the environment variable ``SDC_DOC_NO_API_REF_STR`` is unset then Sphinx's ``conf.py`` invokes ``generate_api_reference()`` function located in ``./sdc/docs/source/buildscripts/apiref_generator`` module. This function parses ``pandas`` and ``sdc`` docstrings for each API, combines those into single docstring and writes it into RST file with respective Pandas* API name. The auto-generated RST files are
193+
If the environment variable ``SDC_DOC_NO_API_REF_STR`` is unset then Sphinx's ``conf.py``
194+
invokes ``generate_api_reference()`` function located in ``./sdc/docs/source/buildscripts/apiref_generator`` module.
195+
This function parses ``pandas`` and ``sdc`` docstrings for each API, combines those into single docstring and
196+
writes it into RST file with respective `Pandas*`_ API name. The auto-generated RST files are
180197
located at ``./sdc/docs/source/_api_ref`` directory.
181198

182-
.. note:
183-
Sphinx will automatically clean the ``_api_ref`` directory on the next invocation of the documenation build.
199+
.. note::
200+
`Sphinx*`_ will automatically clean the ``_api_ref`` directory on the next invocation of the documenation build.
184201

185-
Intel SDC docstring decoration rules
186-
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
187-
Since SDC API Reference is auto-generated from respective Pandas* and Intel SDC docstrings there are certain rules that must be
202+
Intel® SDC docstring decoration rules
203+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
204+
205+
Since Intel® SDC API Reference is auto-generated from respective `Pandas*`_ and Intel® SDC docstrings there are certain rules that must be
188206
followed to accurately generate the API description.
189207

190-
1. Every SDC API must have the docstring.
191-
If developer does not provide the docstring then Sphinx will not be able to match Pandas docstring with respective SDC one. In this situation Sphinx assumes that SDC does not support such API and will include respective note in the API Reference that
208+
1. Every Intel® SDC API must have the docstring.
209+
If developer does not provide the docstring then `Sphinx*`_ will not be able to match `Pandas*`_ docstring with respective SDC one.
210+
In this situation `Sphinx*`_ assumes that SDC does not support such API and will include respective note in the API Reference that
192211
**This API is currently unsupported**.
193212

194213
2. Follow 'one function - one docstring' rule.
195214
You cannot have one docstring for multiple APIs, even if those are very similar. Auto-generator assumes every SDC API is covered by
196-
respective docstring. If Sphinx does not find the docstring for particular API then it assumes that SDC does not support such API
215+
respective docstring. If `Sphinx*`_ does not find the docstring for particular API then it assumes that SDC does not support such API
197216
and will include respective note in the API Reference that **This API is currently unsupported**.
198217

199-
3. Description (introductory section, the very first few paragraphs without a title) is taken from Pandas*.
200-
Intel SDC developers should not include API description in SDC docstring.
218+
3. Description (introductory section, the very first few paragraphs without a title) is taken from `Pandas*`_.
219+
Intel® SDC developers should not include API description in SDC docstring.
201220
But developers are encouraged to follow Pandas API description naming conventions
202221
so that the combined docstring appears consistent.
203222

204-
4. Parameters, Returns, and Raises sections' description is taken from Pandas* docstring.
205-
SDC developers should not include such descriptions in their SDC docstrings.
223+
4. Parameters, Returns, and Raises sections' description is taken from `Pandas*`_ docstring.
224+
Intel® SDC developers should not include such descriptions in their SDC docstrings.
206225
Rather developers are encouraged to follow Pandas naming conventions
207226
so that the combined docstring appears consistent.
208227

@@ -214,17 +233,17 @@ SDC developers should not include such descriptions in their SDC docstrings.
214233
********************************************
215234
Pandas API: <full pandas name, e.g. pandas.Series.nlargest>
216235

217-
<Intel SDC specific sections>
236+
<Intel® SDC specific sections>
218237

219238
Intel Scalable Dataframe Compiler Developer Guide
220239
*************************************************
221240
<Developer's Guide specific sections>
222241
"""
223242

224-
The first two lines must be the User Guide header. This is an indication to Sphinx that this section is intended for public API
243+
The first two lines must be the User Guide header. This is an indication to `Sphinx*`_ that this section is intended for public API
225244
and it will be combined with repsective Pandas API docstring.
226245

227-
Line 3 must specify what Pandas API this Intel SDC docstring does correspond to. It must start with ``Pandas API:`` followed by
246+
Line 3 must specify what Pandas API this Intel® SDC docstring does correspond to. It must start with ``Pandas API:`` followed by
228247
full Pandas API name that corresponds to this SDC docstring. Remember to include full name, for example, ``nlargest`` is not
229248
sufficient for auto-generator to perform the match. The full name must be ``pandas.Series.nlargest``.
230249

@@ -233,15 +252,15 @@ Developer's Guide and must not be included into User's Guide.
233252

234253
6. Examples, See Also, References sections are **NOT** taken from Pandas docstring. SDC developers are expected to complete these sections in SDC doctrings.
235254
This is so because respective Pandas sections are sometimes too Pandas specific and are not relevant to SDC. SDC developers have to
236-
rewrite those sections in Intel SDC style. Do not forget about User Guide header and Pandas API name prior to adding SDC specific
255+
rewrite those sections in Intel® SDC style. Do not forget about User Guide header and Pandas API name prior to adding SDC specific
237256
sections.
238257

239258
7. Examples section is mandatory for every SDC API. 'One API - at least one example' rule is applied.
240259
Examples are essential part of user experience and must accompany every API docstring.
241260

242261
8. Embed examples into Examples section from ``./sdc/examples``.
243262
Rather than writing example in the docstring (which is error-prone) embed relevant example scripts into the docstring. For example,
244-
here is an example how to embed example for ``pandas.Series.get()`` function into respective Intel SDC docstring:
263+
here is an example how to embed example for ``pandas.Series.get()`` function into respective Intel® SDC docstring:
245264

246265
::
247266

@@ -262,7 +281,7 @@ Developer's Guide and must not be included into User's Guide.
262281

263282
In the above snapshot the script ``series_getitem.py`` is embedded into the docstring. ``:lines: 27-`` allows to skip lengthy
264283
copyright header of the file. ``:caption:`` provides meaningful description of the example. It is a good tone to have the caption
265-
for every example. ``:name:`` is the Sphinx name that allows referencing example from other parts of the documentation. It is a good
284+
for every example. ``:name:`` is the `Sphinx*`_ name that allows referencing example from other parts of the documentation. It is a good
266285
tone to include this field. Please follow the naming convention ``ex_<example file name>`` for consistency.
267286

268287
Accompany every example with the expected output using ``.. code-block:: console`` decorator.
@@ -290,7 +309,7 @@ Developer's Guide and must not be included into User's Guide.
290309
Warning
291310
-------
292311

293-
Pay attention to indentation and required blank lines. Sphinx is very sensitive to that.
312+
Pay attention to indentation and required blank lines. `Sphinx*`_ is very sensitive to that.
294313

295314
11. If SDC API does not support all variations of respective Pandas API then Limitations section is mandatory.
296315
While there is not specific guideline how Limitations section must be written, a good style is to follow Pandas Parameters section
@@ -314,7 +333,7 @@ Running unit tests
314333
References
315334
##########
316335

317-
Intel SDC follows ideas and initial code base of High-Performance Analytics Toolkit (HPAT). These academic papers describe ideas and methods behind HPAT:
336+
Intel® SDC follows ideas and initial code base of High-Performance Analytics Toolkit (HPAT). These academic papers describe ideas and methods behind HPAT:
318337

319338
- `HPAT paper at ICS'17 <http://dl.acm.org/citation.cfm?id=3079099>`_
320339
- `HPAT at HotOS'17 <http://dl.acm.org/citation.cfm?id=3103004>`_

0 commit comments

Comments
 (0)