Skip to content

Mowenhao13/FluteGo

BranchesTags

Repository files navigation

FluteGo - File Delivery over Unidirectional Transport in Go implementation

Unicast File Transfer Solution for Small-Scale Scalable Deployments

Acknowledgments

  • Protocol inspiration: ypo/flute - FLUTE implementation in Rust

Usage example

alt text

alt text

RFC

This library implements the following RFCs

RFC Title Link
RFC 6726 FLUTE - File Delivery over Unidirectional Transport https://www.rfc-editor.org/rfc/rfc6726.html
RFC 5052 Forward Error Correction (FEC) Building Block https://www.rfc-editor.org/rfc/rfc5052
RFC 5510 Reed-Solomon Forward Error Correction (FEC) Schemes https://www.rfc-editor.org/rfc/rfc5510.html

Structure

sequenceDiagram
    participant MS as Meta Sender
    participant MR as Meta Receiver
    participant FS as File Sender
    participant FR as File Receiver
    
    Note over MS,MR: Step 1: Metadata Transfer
    MS->>MR: Send File Metadata<br/>(OTI, File Size, File Name, Type)
    
    Note over MR,FR: Step 2: Receiver Preparation
    MR->>FR: Start File Receiver<br/>Open Corresponding Port
    
    Note over MS,FS: Step 3: Data Transmission
    MS->>FS: Start File Sender
    FS->>FR: Asynchronously Send File Data
    
    Note over FR: Step 4: Resource Cleanup
    FR->>FR: Data Receiving Completed<br/>Close Port
Loading

开始使用前

静态 ARP 配置说明

快速开始

  1. Start receiver first

  2. Then start sender

RaptorQ 恢复能力说明

  • RaptorQ 需要 ≥K 个不同符号才能解码一个 chunk，K = ceil(chunkSize / symbolSize)。
  • send-redundancy-ratio 控制每个 chunk 的额外符号数：totalSymbols = ceil(K × ratio)。
  • 当 percentage = 100/(ratio) × 100% 时，发送端恰好发送 K 个基符号（无冗余），此时只要丢 1 个包就可能导致某个 chunk 解码失败。
  • 推荐 ratio = 1/(packetLossRate+0.01) 作为安全值：例如丢包 5% 用 ratio=1.20，丢包 10% 用 ratio=1.50。
  • 实际测试中因为丢包的随机分布，有时即使理论符号数足够也可能解码失败（关键符号集中丢失）。
  • 大文件（1GB+）比小文件更容易恢复成功，因为丢包分散在更多 chunk 上，每个 chunk 丢包概率更低。

编译命令

一键编译所有平台

# 发送端
GOOS=darwin GOARCH=amd64 go build -o release/flute_sender_darwin_amd64 ./cmd/flute_sender/
GOOS=darwin GOARCH=arm64 go build -o release/flute_sender_darwin_arm64 ./cmd/flute_sender/
GOOS=windows GOARCH=amd64 go build -o release/flute_sender_windows_amd64.exe ./cmd/flute_sender/

# 接收端
GOOS=darwin GOARCH=amd64 go build -o release/flute_receiver_darwin_amd64 ./cmd/flute_receiver/
GOOS=darwin GOARCH=arm64 go build -o release/flute_receiver_darwin_arm64 ./cmd/flute_receiver/
GOOS=windows GOARCH=amd64 go build -o release/flute_receiver_windows_amd64.exe ./cmd/flute_receiver/

编译说明

平台 GOOS GOARCH 输出文件名
macOS Intel darwin amd64 flute_sender_darwin_amd64 / flute_receiver_darwin_amd64
macOS Apple Silicon (M1/M2/M3/M4) darwin arm64 flute_sender_darwin_arm64 / flute_receiver_darwin_arm64
Windows 64-bit windows amd64 flute_sender_windows_amd64.exe / flute_receiver_windows_amd64.exe

编译当前平台

# 直接编译（默认当前平台）
go build -o flute_sender ./cmd/flute_sender/
go build -o flute_receiver ./cmd/flute_receiver/

编译产物统一输出到 release/ 目录，方便分发。

性能测试

两种测试场景：

  • Mac→Win：Apple M4/16GB/macOS 26.2（发送端）→ AMD Ryzen/32GB/Win 11（接收端），不限速
  • Win→Mac：AMD Ryzen 9 7940HX@5.2GHz/32GB/Win 11（发送端）→ Apple M4/16GB/macOS 26.2（接收端），500 Mbps 限速（另有无限速对照）

所有测试均在正常网络环境下进行，无人工丢包。FEC 配置：RaptorQ（1.25×）/ NoCode，chunk=32KB，symbol=1400B。取前 3 次成功传输平均值。

单文件传输

Mac→Win（Mac 发送 → Win 接收，不限速）

FEC 文件大小 耗时 有效速率
NoCode 1 GB 27.5 s 313 Mbps
NoCode 500 MB 13.1 s 328 Mbps
NoCode 100 MB 2.5 s 331 Mbps
RaptorQ 1 GB 38.2 s 227 Mbps
RaptorQ 500 MB 16.1 s 267 Mbps
RaptorQ 100 MB 3.2 s 262 Mbps

Win→Mac（Win 发送 → Mac 接收，不限速）

FEC 文件大小 耗时 有效速率
NoCode 1 GB 11.9 s 725 Mbps
NoCode 500 MB 5.9 s 731 Mbps
NoCode 100 MB 1.1 s 754 Mbps
RaptorQ 1 GB 13.2 s 653 Mbps
RaptorQ 500 MB 6.6 s 648 Mbps
RaptorQ 100 MB 1.2 s 672 Mbps

Win→Mac（Win 发送 → Mac 接收，500 Mbps 限速）

FEC 文件大小 耗时 有效速率
NoCode 1 GB 16.8 s 512 Mbps
NoCode 500 MB 8.1 s 528 Mbps
NoCode 100 MB 1.2 s 706 Mbps
RaptorQ 1 GB 21.6 s 397 Mbps
RaptorQ 500 MB 10.6 s 406 Mbps
RaptorQ 100 MB 1.7 s 504 Mbps
  • Win→Mac 不限速时发送端有效速率是 Mac→Win 的 2.0–2.8×，差距来自 Win 发送端 CPU（32核 vs 10核）及 NIC 驱动发包效率。
  • Win→Mac 500 Mbps 限速下 NoCode 接近跑满限速（512–528 Mbps），RaptorQ 因 FEC 冗余开销（28.9%）有效速率约 400 Mbps。
  • Mac→Win 不限速时 Mac 发送端 10 核性能有限，NoCode 约 313–331 Mbps、RaptorQ 约 227–267 Mbps。
  • RaptorQ 开销 28.9%（1.25× 冗余 + 8 字节头部），NoCode 仅 0.6% 头部开销。
  • Win 发送端内存稳定（14–19 MB），无 FEC 解码状态。
  • NoCode GC 高于 RaptorQ（sync.Pool 写缓冲无状态缓存），限速场景因耗时拉长 GC 最高。
  • 所有场景内存稳定，无泄漏。 -->

接收端收包统计

场景 FEC 预期包数 实际收包 比率
Mac→Win RaptorQ 786,432（1 GB） 983,040 125.00%
Mac→Win NoCode 精确相等 精确相等 100.00%
Win→Mac RaptorQ（不限速） 786,432（1 GB） 982,981–983,040 125.00%
Win→Mac RaptorQ（500M限速） 786,432（1 GB） 983,040 125.00%
Win→Mac NoCode 精确相等 精确相等 100.00%
  • RaptorQ 收包率精确匹配 基符号数 × 冗余比，不限速和限速均稳定 125.00%。
  • NoCode 零丢包，100% 精确匹配。

RaptorQ Recovery Formula

Parameters

Symbol Meaning How to compute
S symbol size (bytes) maxPacketSize - 8
C chunk size (bytes) OTI MaximumChunkSize (default 32768)
F file size (bytes) os.Stat()
R redundancy ratio --send-redundancy-ratio
D drop probability 1.0 - percentage/100

Derived values

baseSymbols   B = ceil(C / S)
totalSymbols  T = ceil(B * R)
chunkCount    N = ceil(F / C)
sendRate      p = 1 - D

Recovery condition

A transfer succeeds when every chunk receives enough symbols to decode. With random independent packet loss at rate D, the probability a single chunk fails is:

P(fail per chunk) = P( Binomial(T, p) < B )

For N chunks to all succeed with high confidence:

P(fail per chunk) * N  <  0.5    (expected failures < 1)

Verified boundaries (Windows localhost, RaptorQ, rate-limit=0)

100 MB file (N = 3,200 chunks):

R T p min Loss max Overhead Validated
1.30 32 0.95 5% +30% OK
1.50 36 0.90 10% +50% OK
1.60 39 0.85 15% +60% OK
2.00 48 0.75 25% +100% OK
2.50 60 0.60 40% +150% OK
3.00 72 0.60 40% +200% OK

1 GB file (N = 32,768 chunks):

R T p min Loss max Overhead Validated
1.30 32 0.95 5% +30% OK
1.50 36 0.90 10% +50% OK
1.75 42 0.85 15% +75% OK
2.00 48 0.80 20% +100% OK
2.50 60 0.70 30% +150% OK
3.00 72 0.60 40% +200% OK

Key insight

Larger files have more chunks → higher chance of an extreme outlier. For a 1 GB file (32,768 chunks) you need ~1 extra symbol per chunk of safety margin compared to a 100 MB file.

Quick reference

# 5% loss  -> ratio >= 1.3
# 10% loss -> ratio >= 1.5
# 15% loss -> ratio >= 1.75
# 20% loss -> ratio >= 2.0
# 30% loss -> ratio >= 2.5
# 40% loss -> ratio >= 3.0

About

File Delivery over Unidirectional Transport in Go implementation

Topics

golang

Resources

Readme
Activity

Stars

2 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases 4

v1.0.2 Latest
Jun 17, 2026
+ 3 releases

Packages

 
 
 

Contributors

Languages