fix(sync): 修复云同步多设备冲突下的静默覆盖与状态污染问题#1439
Open
cyfung1031 wants to merge 1 commit into
Open
Conversation
Closed
Member
|
我总感觉要被你改炸了。。。。。 filesystem这个包应该只专注于文件读写,冲突之类不要由这个包去处理,违背单一职责了 |
Collaborator
Author
每个 Provider 的冲突处理不一样呀 也是 fs.write(...) fs.delete(...) 有冲突的话要报错 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
概要
本 PR 修复云同步在多设备并发写入、删除、拉取/推送失败时可能出现的静默覆盖、状态污染和错误推进本地同步状态问题。
核心目标是:云同步写入脚本、元数据和
scriptcat-sync.json时,不再无条件基于过期远端状态覆盖;而是根据各 provider 能力使用version/digest/rev/ ETag 等信息建立写入前置检查或条件写入。若远端已被其他设备修改,当前设备会识别为冲突或失败,并停止继续更新scriptcat-sync.json与本地file_digest,避免把错误状态写回本地或云端。本 PR 同时补齐 provider 条件写入/删除能力、删除幂等性、请求错误类型化、Google Drive 重名保护、tombstone 删除收敛、同步失败通知、选中脚本导出失败处理、Service Worker alarm 错误处理以及相关测试。
主要改动
1. 扩展 filesystem 通用接口
FileInfo新增version?: stringFileCreateOptions新增:expectedDigestexpectedVersioncreateOnlyFileDeleteOptions:expectedDigestexpectedVersionFileSystem.delete()支持传入FileDeleteOptionsFileSystemError新增unsupportedfileConflictError()unsupportedConditionalWriteError()conflict/unsupported标识写入冲突和 provider 不支持的场景LimiterFileSystem.delete()会透传FileDeleteOptions,确保 limiter 包装后仍保留条件删除语义2. 条件 header 生成逻辑复用
buildConditionalHeaders(opts?: FileCreateOptions)buildExpectedHeaders(opts?: FileDeleteOptions)If-None-Match: *:用于createOnlyIf-Match:用于expectedVersion/expectedDigestIf-None-Match,继续使用 webdav client 的overwrite: false处理 create-only 语义3. Provider 写入/删除前置条件与冲突处理
S3
create()透传完整FileCreateOptionsdelete()支持FileDeleteOptionslist()暴露:digest: 去除引号后的 ETagversion: provider 原始 ETagPUT写入支持:If-None-Match: *用于createOnlyIf-Match用于expectedVersion/expectedDigestDELETE支持If-Match409/412统一转换为fileConflictError("s3", ...)delete()继续保持不存在时幂等成功list()不再为每个对象额外发送HEAD读取 metadata createtime,避免目录列表产生额外请求;创建时间使用对象LastModifiedWebDAV
create()透传FileCreateOptionsdelete()支持FileDeleteOptionslist()将 ETag 暴露为digest/versionIf-Match条件更新createOnly创建保护overwrite: false实现DELETE支持If-Match409/412统一转换为fileConflictError("webdav", ...)delete()对 404 保持幂等成功Dropbox
create()透传FileCreateOptionsdelete()支持FileDeleteOptionslist()将 Dropboxrev暴露为versionoverwritemode,不再先做 metadataexists()preflightexpectedVersion时使用 DropboxupdatemodecreateOnly时使用addmodeexpectedDigest但没有expectedVersion时通过unsupportedConditionalWriteError()明确报unsupported_conditional_writeincorrect_offset以及已类型化的FileSystemError(conflict: true)会统一转换 / 保持为 Dropbox 冲突错误rev/content_hash的 best-effort preflightGoogle Drive
create()透传FileCreateOptionsdelete()支持FileDeleteOptionslist()请求version字段,并暴露 opaque version token:fileId:versiondelete()遇到 typed not-found 时保持幂等成功,并清理 stale path cacheversion/md5Checksum的 best-effort preflightfindFileInDirectory()改为:findFilesInDirectory()fileConflictError("googledrive", ...)expectedVersion解析出fileId和versionversion412 versionMismatchgenerateIdsIf-None-Match: *createOnly会在创建后再次检查同名文件OneDrive
create()透传FileCreateOptionsdelete()支持FileDeleteOptionslist()将 eTag 暴露为:digestversiondelete()对 typed not-found 保持幂等成功DELETE支持If-MatchIf-None-Match: *用于createOnlyIf-Match用于expectedVersion/expectedDigestfailreplaceBaidu
expectedVersion明确标记为不支持,并通过unsupportedConditionalWriteError()抛出unsupported_conditional_writeexpectedDigest通过写入前list()做 best-effort preflightcreateOnly:rtype=0,要求百度服务端拒绝覆盖fileConflictError("baidu", ...)expectedDigest通过 preflight 后仍使用默认覆盖语义rtype=3delete()支持基于expectedDigest的 best-effort preflightdelete()对文件不存在 errno 保持幂等成功4. 云同步写入正确性改进
getWriteOptions(modifiedDate, remoteFile)createOnlyversion:使用expectedVersionversion但有digest:使用expectedDigestgetDeleteOptions(remoteFile)versionversion时回退到digestpushScript()现在接收远端脚本 / meta 文件信息,并分别带写入前置条件:${uuid}.user.js${uuid}.meta.jsoncreateOnlyversion/digest作为写入前置条件scriptcat-sync.json写入也改为使用远端version/digest前置条件list()远端状态,再带前置条件写入list()远端状态,再带前置条件删除 / 写 tombstone5. 同步失败时避免污染本地状态
syncOnceInternal()会检查 push / pull / status sync 的 rejected taskscriptcat-sync.jsonfile_digestscriptcat-sync.json写入失败时会被捕获:pullScript()失败后不再静默吞掉异常,而是继续抛出,让上层停止状态推进6. digest cache 与 tombstone digest 处理
FileDigestMapupdateFileDigest()会先读取云端列表fs.list()结果中,会再重试一次 liststorage.setfile_digest或scriptcat-sync.json的成功状态7. tombstone 删除收敛与 orphan 状态处理
pullScript()现在会先读取 meta.user.js.user.js和 tombstone.meta.json时,会优先处理 tombstone,避免残留脚本长期无法收敛.user.js、没有.meta.json时跳过本轮处理,避免另一台设备半上传时被误删expectedDigest8. 选中脚本备份导出失败处理
9. 同步失败通知与文案
notifySyncFailed(hasConflict: boolean, rejectedCount: number)notification.script_sync_failednotification.script_sync_failed_descnotification.script_sync_conflict_descde-DEen-USja-JPru-RUvi-VNzh-CNzh-TW10. Service Worker alarm 错误处理
cloudSyncalarm 调用链增加.catch()测试覆盖
本 PR 补充了各 provider、filesystem utils、limiter、备份导出和同步流程的单元测试,覆盖:
version暴露buildConditionalHeaders()行为:createOnly优先级高于 expected tokenexpectedVersion优先于expectedDigestexpectedDigest时生成If-MatchbuildExpectedHeaders()行为FileSystemError(conflict: true)unsupportedLimiterFileSystem.delete()透传FileDeleteOptionsversion保留原始 ETag,digest保留去引号 ETagoverwrite: falsertype=0expectedDigest成功时仍使用rtype=3pushScript()对新建文件使用createOnlypushScript()对已有文件传递远端expectedVersion/expectedDigestscriptcat-sync.json使用 create-only 或 expectedVersion / expectedDigest 条件写入scriptcat-sync.json写入失败时通知并跳过 digest 更新.user.js删除收敛.user.jswithout meta 的跳过逻辑cloudSyncalarm 异常捕获解决的问题
scriptcat-sync.json.user.js长期无法收敛