Skip to content
This repository was archived by the owner on Mar 24, 2021. It is now read-only.
This repository was archived by the owner on Mar 24, 2021. It is now read-only.

Suggest to document Partitioner Behavior in more details in Producer #1009

Description

@Shadowsong27

PyKafka version: 2.8.0

I m not sure I m doing this correctly, but based on what I have read in docs and source code,
to produce messages into a specific partition while having many consumers (so only one consumer
can consume this set of messages), you not only need to provide a partition_key when calling
produce, but also a HashingPartitionersince the default is RandomPartitioner.

I found this solution based on the above code and this docs string, took me a while,

:param partition_key: The key to use when deciding which partition to send this
            message to. This key is passed to the `partitioner`, which may or may not
            use it in deciding the partition. The default `RandomPartitioner` does not
            use this key, but the optional `HashingPartitioner` does.

If I am solving this problem using the intended approach, I would suggest to add the usage of
HashingPartitioner into the docs string of produce too, since when I first started I assumed that
providing partition_key is sufficient to achieve my use case.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions