Skip to content

Add support for larger reads via MaxPages config option#199

Merged
geertj merged 2 commits into
jacobsa:masterfrom
abhishek10004:abhishek/larger_reads_only
Jun 30, 2026
Merged

Add support for larger reads via MaxPages config option#199
geertj merged 2 commits into
jacobsa:masterfrom
abhishek10004:abhishek/larger_reads_only

Conversation

@abhishek10004

@abhishek10004 abhishek10004 commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

Add MaxPages and MaxWrite configuration to support larger reads/writes and smaller buffers

Description

This PR adds support for configuring MaxPages and MaxWrite in MountConfig (Linux only). This enables:

  1. Larger Reads: Supporting read sizes larger than 1 MiB by increasing MaxPages.
  2. Larger Writes: Supporting write sizes larger than 1 MiB by increasing MaxWrite (and MaxPages accordingly).
  3. Smaller Memory Footprint: Reducing memory usage by configuring a smaller MaxWrite, which dynamically reduces the size of the pre-allocated FUSE input message buffers.

We also introduce sanitization and validation logic to ensure MaxPages and MaxWrite are compatible (i.e., MaxPages must be large enough to hold MaxWrite).

Key Changes

1. Configuration

  • mount_config.go: Added MaxPages uint16 and MaxWrite uint32 to MountConfig with detailed documentation on their default behaviors and interactions.

2. Connection & Buffer Management

  • connection.go:
    • Added inMessageBufSize to Connection to track the dynamic buffer size needed for incoming messages (PageSize + MaxWrite).
    • Added sanitizeMaxPagesAndWrite() to handle default values when one or both of MaxPages and MaxWrite are unset, and to validate that MaxPages * PageSize >= MaxWrite.
    • Updated Connection.Init() to negotiate the configured MaxPages and MaxWrite with the kernel.
  • internal/buffer/in_message.go: Updated NewInMessage() to accept a dynamic buffer size instead of using a hardcoded constant.
  • freelists.go: Updated getInMessage() to pass the connection's dynamic inMessageBufSize when allocating new message buffers.

3. API & Documentation Updates

  • fuseops/ops.go: Updated documentation for ReadFileOp.Dst and ReadFileOp.Data:
    • Clarified that Dst will be nil when MaxPages is configured for reads larger than the pre-allocated buffer.
    • Clarified that the file system must use Data (vectored read) when Dst is nil.

4. Tests

  • connection_test.go: Added TestSanitizeMaxPagesAndWrite covering various scenarios:
    • Neither option configured (defaults applied).
    • Only MaxWrite configured (dynamic MaxPages calculation).
    • Only MaxPages configured (dynamic MaxWrite calculation).
    • Both configured (valid and invalid combinations).

Testing Done

  • Added and ran TestSanitizeMaxPagesAndWrite in connection_test.go.

Comment thread mount_config.go
@abhishek10004 abhishek10004 force-pushed the abhishek/larger_reads_only branch 2 times, most recently from 7ca4480 to 04b7821 Compare June 30, 2026 14:10
@abhishek10004 abhishek10004 force-pushed the abhishek/larger_reads_only branch from 04b7821 to 3173d12 Compare June 30, 2026 17:33
Comment thread fuseops/ops.go
@geertj geertj merged commit a124548 into jacobsa:master Jun 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants