Skip to content

Process large bulk event uploads in background#3611

Merged
bernardhanna merged 2 commits into
masterfrom
feature/bulk-upload-large-files
Jul 1, 2026
Merged

Process large bulk event uploads in background#3611
bernardhanna merged 2 commits into
masterfrom
feature/bulk-upload-large-files

Conversation

@bernardhanna

Copy link
Copy Markdown
Collaborator

Summary

  • Moves bulk event validation and import into background queue jobs so uploads with thousands of rows no longer hit gateway timeouts.
  • Adds a processing page with status polling, then redirects to preview or report when done.
  • For uploads over 500 rows, preview shows problem rows only; report shows created count without listing every event link.
  • Raises upload limit from 10 MB to 50 MB for large Excel/CSV files.

Test plan

  • Confirm queue workers are running on prod (--timeout=3600 recommended)
  • Upload a ~3000–6000 row CSV at /admin/bulk-upload — should reach processing page quickly, then preview
  • Run import — processing page → report with created count
  • Verify file over 10 MB but under 50 MB is accepted
  • php artisan test tests/Unit/BulkEventUploadCacheTest.php

Deploy notes

  • Requires queue workers processing ValidateBulkEventUploadJob and ProcessBulkEventImportJob
  • If multiple app servers: BULK_UPLOAD_TEMP_DISK=s3
  • Server upload_max_filesize / post_max_size must allow 50 MB (Forge PHP settings)

Made with Cursor

bernardhanna and others added 2 commits July 1, 2026 11:46
Validation and import no longer run inside the HTTP request, avoiding
gateway timeouts on files with thousands of rows. Adds a processing
page with status polling, caches preview/report data, skips per-row
info logging, and shows failures-only preview for uploads over 500 rows.

Co-authored-by: Cursor <cursoragent@cursor.com>
Allows large Excel/CSV files (~6000 rows) to pass validation before
background processing.

Co-authored-by: Cursor <cursoragent@cursor.com>
@bernardhanna bernardhanna merged commit 451e235 into master Jul 1, 2026
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant