fix(layer): cast polars decimal columns to float#1096
Conversation
polars' to_pandas() returns a Decimal column as a pandas object column of decimal.Decimal values, which plotnine treats as discrete so a continuous scale rejects it. Cast those object columns to float right after the conversion so they behave like any other numeric column. Closes has2k1#1033 Signed-off-by: Arpit Jain <arpitjain099@gmail.com>
has2k1
left a comment
has2k1
left a comment
There was a problem hiding this comment.
The root cause is pandas frames with Decimal columns, so we should fix that.
| ) | ||
|
|
||
|
|
||
| def _convert_decimal_columns(data: pd.DataFrame) -> pd.DataFrame: |
There was a problem hiding this comment.
Rename to _decimal_columns_to_float and put function at the bottom of the module.
Move the fix to the root cause: any pandas frame with a decimal.Decimal object column now gets that column cast to float once, right where the layer data is finalized in _make_layer_data. This drops the per-call-site casts on the polars to_pandas() paths and covers plain pandas frames too. The helper _decimal_columns_to_float uses a notna() mask instead of dropna() to peek at the first non-null value, since it runs for every dataframe and most object columns are strings. Closes has2k1#1033 Signed-off-by: Arpit Jain <arpitjain099@gmail.com>
|
Thanks, that makes sense. I moved the fix to the root: |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1096 +/- ##
==========================================
- Coverage 87.46% 87.39% -0.07%
==========================================
Files 208 210 +2
Lines 14703 14920 +217
Branches 1834 1889 +55
==========================================
+ Hits 12860 13040 +180
- Misses 1274 1302 +28
- Partials 569 578 +9 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
@arpitjain099, thank you. |
2 participants
A polars Decimal column comes through to_pandas() as a pandas object column of decimal.Decimal values, so plotnine reads it as discrete and a continuous scale rejects it with "Discrete value supplied to continuous scale" (#1033). This casts those Decimal object columns to float right after the conversion, following the cheap-hack approach you mentioned in the issue, so such a column now behaves like any other numeric one instead of needing a manual cast to Float. I added a small regression test that builds a plot from a Decimal column on a continuous scale. Thanks!