chore: fallback for spark.sql.mapKeyDedupPolicy == LAST_WIN

[comphead]

Which issue does this PR close?

Narrows the spark.sql.mapKeyDedupPolicy divergence for map_from_arrays and map_from_entries reported in #4680

Rationale for this change

Spark's ArrayBasedMapBuilder (used by both expressions) honors two dedup modes:

• EXCEPTION (default): raise [DUPLICATED_MAP_KEY] on duplicate keys.
• LAST_WIN: keep the last occurrence.

Comet is currently pinned to datafusion-spark 54.0.0, whose map_from_arrays / map_from_entries implement neither mode — duplicates simply pass through. That means:

• Under EXCEPTION (the default) with duplicate keys, Comet silently keeps duplicates instead of erroring — a data divergence tracked separately.
• Under LAST_WIN, Comet's output shape disagrees with Spark's per-key dedup — a config-driven divergence we can detect at plan time.

Upstream apache/datafusion#21720 wires both modes into datafusion-spark, but that fix has not yet landed in a release Comet pins to.

What changes are included in this PR?

• Add MapKeyDedupPolicySupport in serde/maps.scala — one helper checks spark.sql.mapKeyDedupPolicy == LAST_WIN.
• CometMapFromArrays and CometMapFromEntries return Incompatible when LAST_WIN is active, so planning falls back to Spark for that config only. EXCEPTION (default) continues to run natively.
• Sibling expressions (CreateMap, MapConcat, TransformKeys, ...) already delegate to Spark's doGenCode via CometCodegenDispatch and inherit dedup semantics for free — no gate needed.

Are these changes tested?

Yes — new SQL file tests exercise the fallback with spark.sql.mapKeyDedupPolicy=LAST_WIN for both expressions, on literal and column inputs:

• spark/src/test/resources/sql-tests/expressions/map/map_from_arrays_dedup_policy.sql
• spark/src/test/resources/sql-tests/expressions/map/map_from_entries_dedup_policy.sql

The map_from_entries test also disables the codegen dispatcher so the incompat branch surfaces as a genuine Spark fallback rather than in-pipeline codegen.

Follow-up

When Comet bumps to a datafusion-spark release containing #21720, this gate can be replaced by propagating spark.sql.mapKeyDedupPolicy → datafusion.spark.map_key_dedup_policy and removing the
Incompatible branch. str_to_map gets the same upgrade for free.

[andygrove]

Checking for default vs non-default value seems brittle. What if the default changes in a later Spark version? Shouldn't we check explicitly for EXCEPTION vs LAST_WIN?

[andygrove]

Thanks @comphead