Skip to content

Split multi-event lineup cells so each booking is its own row (fixes #11) - #12

Open
mslovich wants to merge 1 commit into
jfjelstul:masterfrom
mslovich:fix/multi-event-lineup-cells
Open

mslovich wants to merge 1 commit into
jfjelstul:masterfrom
mslovich:fix/multi-event-lineup-cells

Conversation

@mslovich

Copy link
Copy Markdown

Fixes #11.

The bug

The bookings table is documented as having one row per booking, but when a player has more than one event in the same Wikipedia lineup cell — e.g. a yellow card followed by the second yellow or red card that results in a sending-off — the scraped events string concatenates them with no comma between them ("yellow card 40' red card 42'"). The bookings and substitutions builders only split events on commas via separate_rows(events, sep = ","), so these collapse into a single row: the first event's minute is kept, the second event's minute is discarded, and the flags from both events are merged onto one row.

This is exactly the inconsistency reported in #11. Ray Wilkins' dismissal vs Morocco (1986) was stored as a single row flagged both yellow_card = 1 and red_card = 1 at 40', rather than a yellow at 40' and the sending-off at 42'.

Two one-off hand-patches already existed in the builder (Khalil Ghanim's booking, one substitution), confirming the bug but not generalizing it.

The fix

Before the comma-based separate_rows(), insert a comma after an event's minute whenever it is immediately followed by another event keyword, so each card / substitution becomes its own row. Applied in both the bookings and substitutions builders; it also subsumes the two existing hand-patches.

Effect on the data

Ray Wilkins (England vs Morocco, 1986) — #11:

minute yellow red 2nd yellow sending_off
before 40' 1 1 0 1
after 40' 1 0 0 0
after 42' 0 1 0 1

Totals:

rows yellow red 2nd yellow sending_off
before 3178 3092 102 70 172
after 3263 3162 102 70 172
  • bookings: 3178 → 3263 (+85 first-cards recovered)
  • red_card, second_yellow_card, sending_off totals unchanged — no cards or dismissals fabricated; the recovered rows are the previously-dropped first cards
  • sending-off minutes now point at the dismissal card, not the earlier first yellow
  • all 70 second-yellow dismissals now have their preceding first-yellow row (was 1 of 70)

Why substitutions is fixed in code but not regenerated

The same fix is applied to the substitutions builder because it shares the identical root cause, but no substitutions data files are changed in this PR. The current scraped data contains no remaining multi-event substitution cells (the only such case was already covered by an existing hand-patch), so running the patched builder produces output byte-identical to the committed substitutions data (verified: 10,222 rows in, 10,222 rows out, no diff). The committed substitutions files therefore already match the fixed code — there is nothing to rebuild. The change is included as a defensive, consistent fix so the bug cannot reappear in substitutions if the source data is ever re-scraped or extended.

Regenerated artifacts

data/bookings.RData, data-csv/bookings.csv, the bookings table in data-sqlite/worldcup.db, and data-json/worldcup.json. Only the bookings table changed; every other table (including substitutions) is byte/row-identical.

When a player has more than one event in the same Wikipedia lineup cell
(e.g. a first yellow at 36' and a second yellow at 88'), the scraped
`events` string concatenates them with no comma between them. The
bookings and substitutions builders only split on commas via
separate_rows(), so these collapsed into a single row: the first event's
minute was kept, the second event's minute was discarded, and the flags
from both events were merged onto one row (e.g. yellow_card=1 AND
second_yellow_card=1 on the 36' row, with the actual sending-off minute
of 88' lost). Two one-off hand-patches already existed for individual
instances of this, confirming the bug but not generalizing it.

Insert a comma after an event's minute whenever it is immediately
followed by another event keyword, so each card / substitution becomes
its own row.

Effect on the data:
- bookings: 3178 -> 3263 rows (+85 first-cards recovered)
- yellow_card total: 3092 -> 3162; red_card, second_yellow_card and
  sending_off totals unchanged (no fabricated cards or dismissals)
- sending-off minutes now point at the dismissal card, not the earlier
  first yellow
- substitutions: no change in the current data (no remaining multi-event
  sub cells), but the same fix is applied defensively

Regenerated data/bookings.RData, data-csv/bookings.csv, the bookings
table in data-sqlite/worldcup.db, and data-json/worldcup.json. Only the
bookings table changed; all other tables are byte/row-identical.
mslovich added a commit to mslovich/worldcup-malloy that referenced this pull request Jun 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Historical Inconsistencies

1 participant