Skip to content

Commit f362a7e

Browse files
committed
Reformatted .rst files for 80 characters
1 parent 9c53fec commit f362a7e

3 files changed

Lines changed: 74 additions & 23 deletions

File tree

docs/source/core.rst

Lines changed: 32 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,9 @@ Map, emit, and sink
1515
map
1616
sink
1717

18-
You can create a basic pipeline by instantiating the ``Streamz`` object and then using methods like ``map``, ``accumulate``, and ``sink``.
18+
You can create a basic pipeline by instantiating the ``Streamz``
19+
object and then using methods like ``map``, ``accumulate``, and
20+
``sink``.
1921

2022
.. code-block:: python
2123
@@ -27,7 +29,10 @@ You can create a basic pipeline by instantiating the ``Streamz`` object and then
2729
source = Stream()
2830
source.map(increment).sink(print)
2931
30-
The ``map`` and ``sink`` methods both take a function and apply that function to every element in the stream. The ``map`` method returns a new stream with the modified elements while ``sink`` is typically used at the end of a stream for final actions.
32+
The ``map`` and ``sink`` methods both take a function and apply that
33+
function to every element in the stream. The ``map`` method returns a
34+
new stream with the modified elements while ``sink`` is typically used
35+
at the end of a stream for final actions.
3136

3237
To push data through our pipeline we call ``emit``
3338

@@ -383,14 +388,33 @@ want to read further about :doc:`collections <collections>`
383388
Metadata
384389
--------
385390

386-
Metadata can be emitted into the pipeline to accompany the data as a list of dictionaries. Most functions will pass the metadata to the downstream function without making any changes. However, functions that make the pipeline asynchronous require logic that dictates how and when the metadata will be passed downstream. Synchronous functions and asynchronous functions that have a 1:1 ratio of the number of values on the input to the number of values on the output will emit the metadata collection without any modification. However, functions that have multiple input streams or emit collections of data will emit the metadata associated with the emitted data as a collection.
391+
Metadata can be emitted into the pipeline to accompany the data as a
392+
list of dictionaries. Most functions will pass the metadata to the
393+
downstream function without making any changes. However, functions
394+
that make the pipeline asynchronous require logic that dictates how
395+
and when the metadata will be passed downstream. Synchronous functions
396+
and asynchronous functions that have a 1:1 ratio of the number of
397+
values on the input to the number of values on the output will emit
398+
the metadata collection without any modification. However, functions
399+
that have multiple input streams or emit collections of data will emit
400+
the metadata associated with the emitted data as a collection.
387401

388402

389403
Reference Counting and Checkpointing
390404
------------------------------------
391405

392-
Checkpointing is achieved in Streamz through the use of reference counting. With this method, a checkpoint can be saved when and only when data has progressed through all of the the pipeline without any issues. This prevents data loss and guarantees at-least-once semantics.
393-
394-
Any node that caches or holds data after it returns increments the reference counter associated with the given data by one. When a node is no longer holding the data, it will release it by decrementing the counter by one. When the counter changes to zero, a callback associated with the data is triggered.
395-
396-
References are passed in the metadata as a value of the `ref` keyword. Each metadata object contains only one reference counter object.
406+
Checkpointing is achieved in Streamz through the use of reference
407+
counting. With this method, a checkpoint can be saved when and only
408+
when data has progressed through all of the the pipeline without any
409+
issues. This prevents data loss and guarantees at-least-once
410+
semantics.
411+
412+
Any node that caches or holds data after it returns increments the
413+
reference counter associated with the given data by one. When a node
414+
is no longer holding the data, it will release it by decrementing the
415+
counter by one. When the counter changes to zero, a callback
416+
associated with the data is triggered.
417+
418+
References are passed in the metadata as a value of the `ref`
419+
keyword. Each metadata object contains only one reference counter
420+
object.

docs/source/dask.rst

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -123,17 +123,20 @@ Gotchas
123123
+++++++
124124

125125

126-
An important gotcha with ``DaskStream`` is that it is a subclass of ``Stream``, and so can be used as an input
127-
to any function expecting a ``Stream``. If there is no intervening ``.gather()``, then the downstream node will
128-
receive Dask futures instead of the data they represent::
126+
An important gotcha with ``DaskStream`` is that it is a subclass of
127+
``Stream``, and so can be used as an input to any function expecting a
128+
``Stream``. If there is no intervening ``.gather()``, then the
129+
downstream node will receive Dask futures instead of the data they
130+
represent::
129131

130132
source = Stream()
131133
source2 = Stream()
132134
a = source.scatter().map(inc)
133135
b = source2.combine_latest(a)
134136

135-
In this case, the combine operation will get real values from ``source2``, and Dask futures.
136-
Downstream nodes would be free to operate on the futures, but more likely, the line should be::
137+
In this case, the combine operation will get real values from
138+
``source2``, and Dask futures. Downstream nodes would be free to
139+
operate on the futures, but more likely, the line should be::
137140

138141
b = source2.combine_latest(a.gather())
139142

docs/source/plotting.rst

Lines changed: 34 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,29 @@
11
Visualizing streamz
22
===================
33

4-
A variety of tools are available to help you understand, debug, visualize your streaming objects:
4+
A variety of tools are available to help you understand, debug,
5+
visualize your streaming objects:
56

6-
- Most Streamz objects automatically display themselves in Jupyter notebooks, periodically updating their visual representation as text or tables by registering events with the Tornado IOLoop used by Jupyter
7-
- The network graph underlying a stream can be visualized using `dot` to render a PNG using `Stream.visualize(filename)`
8-
- Streaming data can be visualized using the optional separate packages hvPlot, HoloViews, and Panel (see below)
7+
- Most Streamz objects automatically display themselves in Jupyter
8+
notebooks, periodically updating their visual representation as text
9+
or tables by registering events with the Tornado IOLoop used by Jupyter
10+
- The network graph underlying a stream can be visualized using `dot` to
11+
render a PNG using `Stream.visualize(filename)`
12+
- Streaming data can be visualized using the optional separate packages
13+
hvPlot, HoloViews, and Panel (see below)
914

1015

1116
hvplot.streamz
1217
--------------
1318

14-
hvPlot is a separate plotting library providing Bokeh-based plots for Pandas dataframes and a variety of other object types, including streamz DataFrame and Series objects.
19+
hvPlot is a separate plotting library providing Bokeh-based plots for
20+
Pandas dataframes and a variety of other object types, including
21+
streamz DataFrame and Series objects.
1522

16-
See `hvplot.holoviz.org <https://hvplot.holoviz.org>`_ for instructions on how to install hvplot. Once it is installed, you can use the Pandas .plot() API to get a dynamically updating plot in Jupyter or in Bokeh/Panel Server:
23+
See `hvplot.holoviz.org <https://hvplot.holoviz.org>`_ for
24+
instructions on how to install hvplot. Once it is installed, you can
25+
use the Pandas .plot() API to get a dynamically updating plot in
26+
Jupyter or in Bokeh/Panel Server:
1727

1828
.. code-block:: python
1929
@@ -23,15 +33,29 @@ See `hvplot.holoviz.org <https://hvplot.holoviz.org>`_ for instructions on how t
2333
df = Random()
2434
df.hvplot(backlog=100)
2535
26-
See the `streaming section <https://hvplot.holoviz.org/user_guide/Streaming.html>`_ of the hvPlot user guide for more details, and the
27-
`dataframes.ipynb` example that comes with streamz for a simple runnable example.
36+
See the `streaming section
37+
<https://hvplot.holoviz.org/user_guide/Streaming.html>`_ of the hvPlot
38+
user guide for more details, and the `dataframes.ipynb` example that
39+
comes with streamz for a simple runnable example.
2840

2941

3042
HoloViews
3143
---------
32-
hvPlot is built on HoloViews, and you can also use HoloViews directly if you want more control over events and how they are processed. See the `HoloViews user guide <http://holoviews.org/user_guide/Streaming_Data.html>`_ for more details.
44+
45+
hvPlot is built on HoloViews, and you can also use HoloViews directly
46+
if you want more control over events and how they are processed. See
47+
the `HoloViews user guide
48+
<http://holoviews.org/user_guide/Streaming_Data.html>`_ for more
49+
details.
3350

3451

3552
Panel
3653
-----
37-
Panel is a general purpose dashboard and app framework, supporting a wide variety of displayable objects as "Panes". Panel provides a `streamz Pane <https://panel.holoviz.org/reference/panes/Streamz.html>`_ for rendering arbitrary streamz objects, and streamz DataFrames are handled by the Panel `DataFrame Pane <https://panel.holoviz.org/reference/panes/DataFrame.html>`_.
54+
55+
Panel is a general purpose dashboard and app framework, supporting a
56+
wide variety of displayable objects as "Panes". Panel provides a
57+
`streamz Pane
58+
<https://panel.holoviz.org/reference/panes/Streamz.html>`_ for
59+
rendering arbitrary streamz objects, and streamz DataFrames are
60+
handled by the Panel `DataFrame Pane
61+
<https://panel.holoviz.org/reference/panes/DataFrame.html>`_.

0 commit comments

Comments
 (0)