From 35178024bfc43a6d20bfbb883236e3d5cde42fa1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E9=93=8D=E7=9B=90=E7=94=9C=E4=B8=8D=E7=94=9C?= <51872789+long45343@users.noreply.github.com> Date: Tue, 18 Aug 2026 23:03:36 +0800 Subject: [PATCH 01/32] Create LICENSE --- LICENSE | 674 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 674 insertions(+) create mode 100644 LICENSE diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..f288702 --- /dev/null +++ b/LICENSE @@ -0,0 +1,674 @@ + GNU GENERAL PUBLIC LICENSE + Version 3, 29 June 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The GNU General Public License is a free, copyleft license for +software and other kinds of works. + + The licenses for most software and other practical works are designed +to take away your freedom to share and change the works. By contrast, +the GNU General Public License is intended to guarantee your freedom to +share and change all versions of a program--to make sure it remains free +software for all its users. We, the Free Software Foundation, use the +GNU General Public License for most of our software; it applies also to +any other work released this way by its authors. You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +them if you wish), that you receive source code or can get it if you +want it, that you can change the software or use pieces of it in new +free programs, and that you know you can do these things. + + To protect your rights, we need to prevent others from denying you +these rights or asking you to surrender the rights. Therefore, you have +certain responsibilities if you distribute copies of the software, or if +you modify it: responsibilities to respect the freedom of others. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must pass on to the recipients the same +freedoms that you received. You must make sure that they, too, receive +or can get the source code. And you must show them these terms so they +know their rights. + + Developers that use the GNU GPL protect your rights with two steps: +(1) assert copyright on the software, and (2) offer you this License +giving you legal permission to copy, distribute and/or modify it. + + For the developers' and authors' protection, the GPL clearly explains +that there is no warranty for this free software. For both users' and +authors' sake, the GPL requires that modified versions be marked as +changed, so that their problems will not be attributed erroneously to +authors of previous versions. + + Some devices are designed to deny users access to install or run +modified versions of the software inside them, although the manufacturer +can do so. This is fundamentally incompatible with the aim of +protecting users' freedom to change the software. The systematic +pattern of such abuse occurs in the area of products for individuals to +use, which is precisely where it is most unacceptable. Therefore, we +have designed this version of the GPL to prohibit the practice for those +products. If such problems arise substantially in other domains, we +stand ready to extend this provision to those domains in future versions +of the GPL, as needed to protect the freedom of users. + + Finally, every program is threatened constantly by software patents. +States should not allow patents to restrict development and use of +software on general-purpose computers, but in those that do, we wish to +avoid the special danger that patents applied to a free program could +make it effectively proprietary. To prevent this, the GPL assures that +patents cannot be used to render the program non-free. + + The precise terms and conditions for copying, distribution and +modification follow. + + TERMS AND CONDITIONS + + 0. Definitions. + + "This License" refers to version 3 of the GNU General Public License. + + "Copyright" also means copyright-like laws that apply to other kinds of +works, such as semiconductor masks. + + "The Program" refers to any copyrightable work licensed under this +License. Each licensee is addressed as "you". "Licensees" and +"recipients" may be individuals or organizations. + + To "modify" a work means to copy from or adapt all or part of the work +in a fashion requiring copyright permission, other than the making of an +exact copy. The resulting work is called a "modified version" of the +earlier work or a work "based on" the earlier work. + + A "covered work" means either the unmodified Program or a work based +on the Program. + + To "propagate" a work means to do anything with it that, without +permission, would make you directly or secondarily liable for +infringement under applicable copyright law, except executing it on a +computer or modifying a private copy. Propagation includes copying, +distribution (with or without modification), making available to the +public, and in some countries other activities as well. + + To "convey" a work means any kind of propagation that enables other +parties to make or receive copies. Mere interaction with a user through +a computer network, with no transfer of a copy, is not conveying. + + An interactive user interface displays "Appropriate Legal Notices" +to the extent that it includes a convenient and prominently visible +feature that (1) displays an appropriate copyright notice, and (2) +tells the user that there is no warranty for the work (except to the +extent that warranties are provided), that licensees may convey the +work under this License, and how to view a copy of this License. If +the interface presents a list of user commands or options, such as a +menu, a prominent item in the list meets this criterion. + + 1. Source Code. + + The "source code" for a work means the preferred form of the work +for making modifications to it. "Object code" means any non-source +form of a work. + + A "Standard Interface" means an interface that either is an official +standard defined by a recognized standards body, or, in the case of +interfaces specified for a particular programming language, one that +is widely used among developers working in that language. + + The "System Libraries" of an executable work include anything, other +than the work as a whole, that (a) is included in the normal form of +packaging a Major Component, but which is not part of that Major +Component, and (b) serves only to enable use of the work with that +Major Component, or to implement a Standard Interface for which an +implementation is available to the public in source code form. A +"Major Component", in this context, means a major essential component +(kernel, window system, and so on) of the specific operating system +(if any) on which the executable work runs, or a compiler used to +produce the work, or an object code interpreter used to run it. + + The "Corresponding Source" for a work in object code form means all +the source code needed to generate, install, and (for an executable +work) run the object code and to modify the work, including scripts to +control those activities. However, it does not include the work's +System Libraries, or general-purpose tools or generally available free +programs which are used unmodified in performing those activities but +which are not part of the work. For example, Corresponding Source +includes interface definition files associated with source files for +the work, and the source code for shared libraries and dynamically +linked subprograms that the work is specifically designed to require, +such as by intimate data communication or control flow between those +subprograms and other parts of the work. + + The Corresponding Source need not include anything that users +can regenerate automatically from other parts of the Corresponding +Source. + + The Corresponding Source for a work in source code form is that +same work. + + 2. Basic Permissions. + + All rights granted under this License are granted for the term of +copyright on the Program, and are irrevocable provided the stated +conditions are met. This License explicitly affirms your unlimited +permission to run the unmodified Program. The output from running a +covered work is covered by this License only if the output, given its +content, constitutes a covered work. This License acknowledges your +rights of fair use or other equivalent, as provided by copyright law. + + You may make, run and propagate covered works that you do not +convey, without conditions so long as your license otherwise remains +in force. You may convey covered works to others for the sole purpose +of having them make modifications exclusively for you, or provide you +with facilities for running those works, provided that you comply with +the terms of this License in conveying all material for which you do +not control copyright. Those thus making or running the covered works +for you must do so exclusively on your behalf, under your direction +and control, on terms that prohibit them from making any copies of +your copyrighted material outside their relationship with you. + + Conveying under any other circumstances is permitted solely under +the conditions stated below. Sublicensing is not allowed; section 10 +makes it unnecessary. + + 3. Protecting Users' Legal Rights From Anti-Circumvention Law. + + No covered work shall be deemed part of an effective technological +measure under any applicable law fulfilling obligations under article +11 of the WIPO copyright treaty adopted on 20 December 1996, or +similar laws prohibiting or restricting circumvention of such +measures. + + When you convey a covered work, you waive any legal power to forbid +circumvention of technological measures to the extent such circumvention +is effected by exercising rights under this License with respect to +the covered work, and you disclaim any intention to limit operation or +modification of the work as a means of enforcing, against the work's +users, your or third parties' legal rights to forbid circumvention of +technological measures. + + 4. Conveying Verbatim Copies. + + You may convey verbatim copies of the Program's source code as you +receive it, in any medium, provided that you conspicuously and +appropriately publish on each copy an appropriate copyright notice; +keep intact all notices stating that this License and any +non-permissive terms added in accord with section 7 apply to the code; +keep intact all notices of the absence of any warranty; and give all +recipients a copy of this License along with the Program. + + You may charge any price or no price for each copy that you convey, +and you may offer support or warranty protection for a fee. + + 5. Conveying Modified Source Versions. + + You may convey a work based on the Program, or the modifications to +produce it from the Program, in the form of source code under the +terms of section 4, provided that you also meet all of these conditions: + + a) The work must carry prominent notices stating that you modified + it, and giving a relevant date. + + b) The work must carry prominent notices stating that it is + released under this License and any conditions added under section + 7. This requirement modifies the requirement in section 4 to + "keep intact all notices". + + c) You must license the entire work, as a whole, under this + License to anyone who comes into possession of a copy. This + License will therefore apply, along with any applicable section 7 + additional terms, to the whole of the work, and all its parts, + regardless of how they are packaged. This License gives no + permission to license the work in any other way, but it does not + invalidate such permission if you have separately received it. + + d) If the work has interactive user interfaces, each must display + Appropriate Legal Notices; however, if the Program has interactive + interfaces that do not display Appropriate Legal Notices, your + work need not make them do so. + + A compilation of a covered work with other separate and independent +works, which are not by their nature extensions of the covered work, +and which are not combined with it such as to form a larger program, +in or on a volume of a storage or distribution medium, is called an +"aggregate" if the compilation and its resulting copyright are not +used to limit the access or legal rights of the compilation's users +beyond what the individual works permit. Inclusion of a covered work +in an aggregate does not cause this License to apply to the other +parts of the aggregate. + + 6. Conveying Non-Source Forms. + + You may convey a covered work in object code form under the terms +of sections 4 and 5, provided that you also convey the +machine-readable Corresponding Source under the terms of this License, +in one of these ways: + + a) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by the + Corresponding Source fixed on a durable physical medium + customarily used for software interchange. + + b) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by a + written offer, valid for at least three years and valid for as + long as you offer spare parts or customer support for that product + model, to give anyone who possesses the object code either (1) a + copy of the Corresponding Source for all the software in the + product that is covered by this License, on a durable physical + medium customarily used for software interchange, for a price no + more than your reasonable cost of physically performing this + conveying of source, or (2) access to copy the + Corresponding Source from a network server at no charge. + + c) Convey individual copies of the object code with a copy of the + written offer to provide the Corresponding Source. This + alternative is allowed only occasionally and noncommercially, and + only if you received the object code with such an offer, in accord + with subsection 6b. + + d) Convey the object code by offering access from a designated + place (gratis or for a charge), and offer equivalent access to the + Corresponding Source in the same way through the same place at no + further charge. You need not require recipients to copy the + Corresponding Source along with the object code. If the place to + copy the object code is a network server, the Corresponding Source + may be on a different server (operated by you or a third party) + that supports equivalent copying facilities, provided you maintain + clear directions next to the object code saying where to find the + Corresponding Source. Regardless of what server hosts the + Corresponding Source, you remain obligated to ensure that it is + available for as long as needed to satisfy these requirements. + + e) Convey the object code using peer-to-peer transmission, provided + you inform other peers where the object code and Corresponding + Source of the work are being offered to the general public at no + charge under subsection 6d. + + A separable portion of the object code, whose source code is excluded +from the Corresponding Source as a System Library, need not be +included in conveying the object code work. + + A "User Product" is either (1) a "consumer product", which means any +tangible personal property which is normally used for personal, family, +or household purposes, or (2) anything designed or sold for incorporation +into a dwelling. In determining whether a product is a consumer product, +doubtful cases shall be resolved in favor of coverage. For a particular +product received by a particular user, "normally used" refers to a +typical or common use of that class of product, regardless of the status +of the particular user or of the way in which the particular user +actually uses, or expects or is expected to use, the product. A product +is a consumer product regardless of whether the product has substantial +commercial, industrial or non-consumer uses, unless such uses represent +the only significant mode of use of the product. + + "Installation Information" for a User Product means any methods, +procedures, authorization keys, or other information required to install +and execute modified versions of a covered work in that User Product from +a modified version of its Corresponding Source. The information must +suffice to ensure that the continued functioning of the modified object +code is in no case prevented or interfered with solely because +modification has been made. + + If you convey an object code work under this section in, or with, or +specifically for use in, a User Product, and the conveying occurs as +part of a transaction in which the right of possession and use of the +User Product is transferred to the recipient in perpetuity or for a +fixed term (regardless of how the transaction is characterized), the +Corresponding Source conveyed under this section must be accompanied +by the Installation Information. But this requirement does not apply +if neither you nor any third party retains the ability to install +modified object code on the User Product (for example, the work has +been installed in ROM). + + The requirement to provide Installation Information does not include a +requirement to continue to provide support service, warranty, or updates +for a work that has been modified or installed by the recipient, or for +the User Product in which it has been modified or installed. Access to a +network may be denied when the modification itself materially and +adversely affects the operation of the network or violates the rules and +protocols for communication across the network. + + Corresponding Source conveyed, and Installation Information provided, +in accord with this section must be in a format that is publicly +documented (and with an implementation available to the public in +source code form), and must require no special password or key for +unpacking, reading or copying. + + 7. Additional Terms. + + "Additional permissions" are terms that supplement the terms of this +License by making exceptions from one or more of its conditions. +Additional permissions that are applicable to the entire Program shall +be treated as though they were included in this License, to the extent +that they are valid under applicable law. If additional permissions +apply only to part of the Program, that part may be used separately +under those permissions, but the entire Program remains governed by +this License without regard to the additional permissions. + + When you convey a copy of a covered work, you may at your option +remove any additional permissions from that copy, or from any part of +it. (Additional permissions may be written to require their own +removal in certain cases when you modify the work.) You may place +additional permissions on material, added by you to a covered work, +for which you have or can give appropriate copyright permission. + + Notwithstanding any other provision of this License, for material you +add to a covered work, you may (if authorized by the copyright holders of +that material) supplement the terms of this License with terms: + + a) Disclaiming warranty or limiting liability differently from the + terms of sections 15 and 16 of this License; or + + b) Requiring preservation of specified reasonable legal notices or + author attributions in that material or in the Appropriate Legal + Notices displayed by works containing it; or + + c) Prohibiting misrepresentation of the origin of that material, or + requiring that modified versions of such material be marked in + reasonable ways as different from the original version; or + + d) Limiting the use for publicity purposes of names of licensors or + authors of the material; or + + e) Declining to grant rights under trademark law for use of some + trade names, trademarks, or service marks; or + + f) Requiring indemnification of licensors and authors of that + material by anyone who conveys the material (or modified versions of + it) with contractual assumptions of liability to the recipient, for + any liability that these contractual assumptions directly impose on + those licensors and authors. + + All other non-permissive additional terms are considered "further +restrictions" within the meaning of section 10. If the Program as you +received it, or any part of it, contains a notice stating that it is +governed by this License along with a term that is a further +restriction, you may remove that term. If a license document contains +a further restriction but permits relicensing or conveying under this +License, you may add to a covered work material governed by the terms +of that license document, provided that the further restriction does +not survive such relicensing or conveying. + + If you add terms to a covered work in accord with this section, you +must place, in the relevant source files, a statement of the +additional terms that apply to those files, or a notice indicating +where to find the applicable terms. + + Additional terms, permissive or non-permissive, may be stated in the +form of a separately written license, or stated as exceptions; +the above requirements apply either way. + + 8. Termination. + + You may not propagate or modify a covered work except as expressly +provided under this License. Any attempt otherwise to propagate or +modify it is void, and will automatically terminate your rights under +this License (including any patent licenses granted under the third +paragraph of section 11). + + However, if you cease all violation of this License, then your +license from a particular copyright holder is reinstated (a) +provisionally, unless and until the copyright holder explicitly and +finally terminates your license, and (b) permanently, if the copyright +holder fails to notify you of the violation by some reasonable means +prior to 60 days after the cessation. + + Moreover, your license from a particular copyright holder is +reinstated permanently if the copyright holder notifies you of the +violation by some reasonable means, this is the first time you have +received notice of violation of this License (for any work) from that +copyright holder, and you cure the violation prior to 30 days after +your receipt of the notice. + + Termination of your rights under this section does not terminate the +licenses of parties who have received copies or rights from you under +this License. If your rights have been terminated and not permanently +reinstated, you do not qualify to receive new licenses for the same +material under section 10. + + 9. Acceptance Not Required for Having Copies. + + You are not required to accept this License in order to receive or +run a copy of the Program. Ancillary propagation of a covered work +occurring solely as a consequence of using peer-to-peer transmission +to receive a copy likewise does not require acceptance. However, +nothing other than this License grants you permission to propagate or +modify any covered work. These actions infringe copyright if you do +not accept this License. Therefore, by modifying or propagating a +covered work, you indicate your acceptance of this License to do so. + + 10. Automatic Licensing of Downstream Recipients. + + Each time you convey a covered work, the recipient automatically +receives a license from the original licensors, to run, modify and +propagate that work, subject to this License. You are not responsible +for enforcing compliance by third parties with this License. + + An "entity transaction" is a transaction transferring control of an +organization, or substantially all assets of one, or subdividing an +organization, or merging organizations. If propagation of a covered +work results from an entity transaction, each party to that +transaction who receives a copy of the work also receives whatever +licenses to the work the party's predecessor in interest had or could +give under the previous paragraph, plus a right to possession of the +Corresponding Source of the work from the predecessor in interest, if +the predecessor has it or can get it with reasonable efforts. + + You may not impose any further restrictions on the exercise of the +rights granted or affirmed under this License. For example, you may +not impose a license fee, royalty, or other charge for exercise of +rights granted under this License, and you may not initiate litigation +(including a cross-claim or counterclaim in a lawsuit) alleging that +any patent claim is infringed by making, using, selling, offering for +sale, or importing the Program or any portion of it. + + 11. Patents. + + A "contributor" is a copyright holder who authorizes use under this +License of the Program or a work on which the Program is based. The +work thus licensed is called the contributor's "contributor version". + + A contributor's "essential patent claims" are all patent claims +owned or controlled by the contributor, whether already acquired or +hereafter acquired, that would be infringed by some manner, permitted +by this License, of making, using, or selling its contributor version, +but do not include claims that would be infringed only as a +consequence of further modification of the contributor version. For +purposes of this definition, "control" includes the right to grant +patent sublicenses in a manner consistent with the requirements of +this License. + + Each contributor grants you a non-exclusive, worldwide, royalty-free +patent license under the contributor's essential patent claims, to +make, use, sell, offer for sale, import and otherwise run, modify and +propagate the contents of its contributor version. + + In the following three paragraphs, a "patent license" is any express +agreement or commitment, however denominated, not to enforce a patent +(such as an express permission to practice a patent or covenant not to +sue for patent infringement). To "grant" such a patent license to a +party means to make such an agreement or commitment not to enforce a +patent against the party. + + If you convey a covered work, knowingly relying on a patent license, +and the Corresponding Source of the work is not available for anyone +to copy, free of charge and under the terms of this License, through a +publicly available network server or other readily accessible means, +then you must either (1) cause the Corresponding Source to be so +available, or (2) arrange to deprive yourself of the benefit of the +patent license for this particular work, or (3) arrange, in a manner +consistent with the requirements of this License, to extend the patent +license to downstream recipients. "Knowingly relying" means you have +actual knowledge that, but for the patent license, your conveying the +covered work in a country, or your recipient's use of the covered work +in a country, would infringe one or more identifiable patents in that +country that you have reason to believe are valid. + + If, pursuant to or in connection with a single transaction or +arrangement, you convey, or propagate by procuring conveyance of, a +covered work, and grant a patent license to some of the parties +receiving the covered work authorizing them to use, propagate, modify +or convey a specific copy of the covered work, then the patent license +you grant is automatically extended to all recipients of the covered +work and works based on it. + + A patent license is "discriminatory" if it does not include within +the scope of its coverage, prohibits the exercise of, or is +conditioned on the non-exercise of one or more of the rights that are +specifically granted under this License. You may not convey a covered +work if you are a party to an arrangement with a third party that is +in the business of distributing software, under which you make payment +to the third party based on the extent of your activity of conveying +the work, and under which the third party grants, to any of the +parties who would receive the covered work from you, a discriminatory +patent license (a) in connection with copies of the covered work +conveyed by you (or copies made from those copies), or (b) primarily +for and in connection with specific products or compilations that +contain the covered work, unless you entered into that arrangement, +or that patent license was granted, prior to 28 March 2007. + + Nothing in this License shall be construed as excluding or limiting +any implied license or other defenses to infringement that may +otherwise be available to you under applicable patent law. + + 12. No Surrender of Others' Freedom. + + If conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot convey a +covered work so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you may +not convey it at all. For example, if you agree to terms that obligate you +to collect a royalty for further conveying from those to whom you convey +the Program, the only way you could satisfy both those terms and this +License would be to refrain entirely from conveying the Program. + + 13. Use with the GNU Affero General Public License. + + Notwithstanding any other provision of this License, you have +permission to link or combine any covered work with a work licensed +under version 3 of the GNU Affero General Public License into a single +combined work, and to convey the resulting work. The terms of this +License will continue to apply to the part which is the covered work, +but the special requirements of the GNU Affero General Public License, +section 13, concerning interaction through a network will apply to the +combination as such. + + 14. Revised Versions of this License. + + The Free Software Foundation may publish revised and/or new versions of +the GNU General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + + Each version is given a distinguishing version number. If the +Program specifies that a certain numbered version of the GNU General +Public License "or any later version" applies to it, you have the +option of following the terms and conditions either of that numbered +version or of any later version published by the Free Software +Foundation. If the Program does not specify a version number of the +GNU General Public License, you may choose any version ever published +by the Free Software Foundation. + + If the Program specifies that a proxy can decide which future +versions of the GNU General Public License can be used, that proxy's +public statement of acceptance of a version permanently authorizes you +to choose that version for the Program. + + Later license versions may give you additional or different +permissions. However, no additional obligations are imposed on any +author or copyright holder as a result of your choosing to follow a +later version. + + 15. Disclaimer of Warranty. + + THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY +APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT +HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY +OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, +THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM +IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF +ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + + 16. Limitation of Liability. + + IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS +THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY +GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE +USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF +DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD +PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), +EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF +SUCH DAMAGES. + + 17. Interpretation of Sections 15 and 16. + + If the disclaimer of warranty and limitation of liability provided +above cannot be given local legal effect according to their terms, +reviewing courts shall apply local law that most closely approximates +an absolute waiver of all civil liability in connection with the +Program, unless a warranty or assumption of liability accompanies a +copy of the Program in return for a fee. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +state the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . + +Also add information on how to contact you by electronic and paper mail. + + If the program does terminal interaction, make it output a short +notice like this when it starts in an interactive mode: + + Copyright (C) + This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, your program's commands +might be different; for a GUI interface, you would use an "about box". + + You should also get your employer (if you work as a programmer) or school, +if any, to sign a "copyright disclaimer" for the program, if necessary. +For more information on this, and how to apply and follow the GNU GPL, see +. + + The GNU General Public License does not permit incorporating your program +into proprietary programs. If your program is a subroutine library, you +may consider it more useful to permit linking proprietary applications with +the library. If this is what you want to do, use the GNU Lesser General +Public License instead of this License. But first, please read +. From 5a7fe747b26c0237c5c13be06efab806e0376b3e Mon Sep 17 00:00:00 2001 From: long45343 <1725334094@qq.com> Date: Tue, 18 Aug 2026 23:20:40 +0800 Subject: [PATCH 02/32] Add cross-platform release CI --- .github/workflows/ci.yml | 39 +++++++ .github/workflows/release.yml | 107 +++++++++++++++++++ .gitignore | 1 + README.md | 22 ++++ TextCascade.Server/TextCascade.Server.csproj | 9 ++ 5 files changed, 178 insertions(+) create mode 100644 .github/workflows/ci.yml create mode 100644 .github/workflows/release.yml diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 0000000..4d44bde --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,39 @@ +name: CI + +on: + push: + branches: [main] + pull_request: + branches: [main] + workflow_dispatch: + +jobs: + build-test: + name: Build and test + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Setup .NET + uses: actions/setup-dotnet@v4 + with: + dotnet-version: 10.0.x + dotnet-quality: ga + + - name: Restore + run: dotnet restore TextCascade.Server.slnx + + - name: Build + run: dotnet build TextCascade.Server.slnx --configuration Release --no-restore + + - name: Test + run: dotnet test TextCascade.Server.slnx --configuration Release --no-build --logger trx --results-directory ./TestResults + + - name: Upload test results + if: always() + uses: actions/upload-artifact@v4 + with: + name: test-results + path: ./TestResults + if-no-files-found: warn diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml new file mode 100644 index 0000000..33a35cd --- /dev/null +++ b/.github/workflows/release.yml @@ -0,0 +1,107 @@ +name: Release + +on: + push: + tags: + - 'v*.*.*' + workflow_dispatch: + +permissions: + contents: write + +jobs: + release: + name: Build and publish release + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Setup .NET + uses: actions/setup-dotnet@v4 + with: + dotnet-version: 10.0.x + dotnet-quality: ga + + - name: Restore + run: dotnet restore TextCascade.Server.slnx + + - name: Build + run: dotnet build TextCascade.Server.slnx --configuration Release --no-restore + + - name: Test + run: dotnet test TextCascade.Server.slnx --configuration Release --no-build + + - name: Resolve version + id: version + run: echo "version=${GITHUB_REF_NAME#v}" >> "$GITHUB_OUTPUT" + + - name: Publish Windows framework-dependent single file + run: > + dotnet publish TextCascade.Server/TextCascade.Server.csproj + --configuration Release --runtime win-x64 --self-contained false + -p:PublishSingleFile=true -p:DebugType=embedded + -p:IncludeNativeLibrariesForSelfExtract=true + -p:PublishDir=$GITHUB_WORKSPACE/artifacts/win-x64/ + + - name: Publish Linux framework-dependent single file + run: > + dotnet publish TextCascade.Server/TextCascade.Server.csproj + --configuration Release --runtime linux-x64 --self-contained false + -p:PublishSingleFile=true -p:DebugType=embedded + -p:IncludeNativeLibrariesForSelfExtract=true + -p:PublishDir=$GITHUB_WORKSPACE/artifacts/linux-x64/ + + - name: Stage release files + run: | + mkdir -p release/textcascade-server-windows-x64 release/textcascade-server-linux-x64 + cp artifacts/win-x64/TextCascade.Server.exe release/textcascade-server-windows-x64/ + cp artifacts/linux-x64/TextCascade.Server release/textcascade-server-linux-x64/ + cp deploy/textcascade.toml release/textcascade-server-windows-x64/ + cp deploy/textcascade.toml release/textcascade-server-linux-x64/ + cp deploy/textcascade-server.service release/textcascade-server-linux-x64/ + chmod +x release/textcascade-server-linux-x64/TextCascade.Server + file release/textcascade-server-linux-x64/TextCascade.Server + test -f release/textcascade-server-windows-x64/TextCascade.Server.exe + test -f release/textcascade-server-linux-x64/TextCascade.Server + + - name: Package Windows archive + run: | + cd release + zip -r ../TextCascade.Server-${{ steps.version.outputs.version }}-windows-x64.zip textcascade-server-windows-x64 + + - name: Package Linux archive + run: | + cd release + tar -czf ../TextCascade.Server-${{ steps.version.outputs.version }}-linux-x64.tar.gz textcascade-server-linux-x64 + + - name: Generate checksums + run: | + cd "$GITHUB_WORKSPACE" + sha256sum \ + TextCascade.Server-${{ steps.version.outputs.version }}-windows-x64.zip \ + TextCascade.Server-${{ steps.version.outputs.version }}-linux-x64.tar.gz \ + > checksums-sha256.txt + + - name: Upload build artifacts + uses: actions/upload-artifact@v4 + with: + name: textcascade-server-${{ steps.version.outputs.version }} + path: | + TextCascade.Server-${{ steps.version.outputs.version }}-windows-x64.zip + TextCascade.Server-${{ steps.version.outputs.version }}-linux-x64.tar.gz + checksums-sha256.txt + if-no-files-found: error + + - name: Create GitHub Release + if: startsWith(github.ref, 'refs/tags/') + uses: softprops/action-gh-release@v2 + with: + tag_name: ${{ github.ref_name }} + name: TextCascade Server ${{ github.ref_name }} + draft: false + prerelease: ${{ contains(github.ref_name, '-') }} + files: | + TextCascade.Server-${{ steps.version.outputs.version }}-windows-x64.zip + TextCascade.Server-${{ steps.version.outputs.version }}-linux-x64.tar.gz + checksums-sha256.txt diff --git a/.gitignore b/.gitignore index 0ba57b3..b145798 100644 --- a/.gitignore +++ b/.gitignore @@ -58,3 +58,4 @@ Desktop.ini *.log logs/ tmp/ +artifacts/ diff --git a/README.md b/README.md index 5a67d15..7029e05 100644 --- a/README.md +++ b/README.md @@ -144,6 +144,17 @@ dotnet test 测试覆盖协议解析、配置、登录限流、token 服务、用户文件、clip 与核心逻辑。 + +### 下载与发布 + +GitHub Release 提供两种 Framework-dependent 单文件包,目标机需预装 .NET 10 Runtime: + +- `TextCascade.Server--windows-x64.zip` +- `TextCascade.Server--linux-x64.tar.gz` + +包内附带主程序、配置模板;Linux 包另附 systemd unit。每次 Release 同时提供 SHA-256 校验文件。 + +推送 `v*.*.*` 标签(如 `v0.2.0`)会自动执行测试、构建双平台单文件包、生成校验和并发布 GitHub Release。`main` 分支和 Pull Request 会自动执行 restore/build/test CI。 ### 生产部署(systemd) 参考 `deploy/textcascade-server.service`: @@ -279,6 +290,17 @@ dotnet test Covers protocol parsing, config, login limiting, token service, users file, and clip/core logic. + +### Downloads and Releases + +GitHub Releases provides two framework-dependent single-file archives. The .NET 10 Runtime must be installed on the target machine: + +- `TextCascade.Server--windows-x64.zip` +- `TextCascade.Server--linux-x64.tar.gz` + +Each archive contains the executable and config template; the Linux archive also includes the systemd unit. Every Release includes a SHA-256 checksum file. + +Pushing a `v*.*.*` tag (for example `v0.2.0`) runs tests, builds both single-file archives, generates checksums, and publishes a GitHub Release. Pushes to `main` and pull requests run restore/build/test CI automatically. ### Production (systemd) See `deploy/textcascade-server.service`: diff --git a/TextCascade.Server/TextCascade.Server.csproj b/TextCascade.Server/TextCascade.Server.csproj index 4e35742..a1b9775 100644 --- a/TextCascade.Server/TextCascade.Server.csproj +++ b/TextCascade.Server/TextCascade.Server.csproj @@ -6,6 +6,15 @@ enable 0.2.0 TextCascade.Server + true + + + + false + true + true + embedded + $(RuntimeIdentifier) From 3a28a44c33484a722f7e63d5d6dcfdb9237c7c07 Mon Sep 17 00:00:00 2001 From: long45343 <1725334094@qq.com> Date: Tue, 18 Aug 2026 23:48:07 +0800 Subject: [PATCH 03/32] Fix server protocol bugs --- .../ConnectionStateTests.cs | 67 +++++++++++++++++++ .../ProtocolSerializationTests.cs | 47 +++++++++++++ TextCascade.Server/Cli.cs | 35 +++++++--- TextCascade.Server/Protocol.cs | 44 ++++++++---- TextCascade.Server/SyncServer.cs | 44 +++++++++++- 5 files changed, 215 insertions(+), 22 deletions(-) create mode 100644 TextCascade.Server.Tests/ConnectionStateTests.cs create mode 100644 TextCascade.Server.Tests/ProtocolSerializationTests.cs diff --git a/TextCascade.Server.Tests/ConnectionStateTests.cs b/TextCascade.Server.Tests/ConnectionStateTests.cs new file mode 100644 index 0000000..7b3c6bd --- /dev/null +++ b/TextCascade.Server.Tests/ConnectionStateTests.cs @@ -0,0 +1,67 @@ +using TextCascade.Server; + +namespace TextCascade.Server.Tests; + +public class ConnectionStateTests +{ + [Fact] + public void UnsolicitedPongIsRejectedUntilPingIsAwaited() + { + var state = new ConnectionStateBag(TextCascade.Server.Config.CreateDefaultConfig()); + + Assert.False(state.TryTakePongAwaiting()); + } + + [Fact] + public void ExpectedPongIsAcceptedOnceAndThenRejectedAgain() + { + var state = new ConnectionStateBag(TextCascade.Server.Config.CreateDefaultConfig()); + + state.MarkPingAwaitingPong(); + Assert.True(state.TryTakePongAwaiting()); + Assert.False(state.TryTakePongAwaiting()); + } +} + +public class CliPasswordInputTests +{ + [Fact] + public void DetectsPasswordStdinFlag() + { + Assert.True(Cli.HasPasswordStdin(new[] { "add", "--username", "alice", "--password-stdin" })); + Assert.False(Cli.HasPasswordStdin(new[] { "add", "--username", "alice" })); + } + + [Fact] + public void PasswordStdinReadsOneLineWithoutConsoleKeyInput() + { + var original = Console.In; + try + { + Console.SetIn(new StringReader("secret-password\n")); + var args = new[] { "add", "--username", "alice", "--password-stdin" }; + Assert.Equal("secret-password", Cli.ReadPassword("Password: ", args)); + } + finally + { + Console.SetIn(original); + } + } + + [Fact] + public void PasswordStdinRejectsEmptyInput() + { + var original = Console.In; + try + { + Console.SetIn(new StringReader(string.Empty)); + var args = new[] { "hash", "--password-stdin" }; + Assert.Throws(() => Cli.ReadPassword("Password: ", args)); + } + finally + { + Console.SetIn(original); + } + } +} + diff --git a/TextCascade.Server.Tests/ProtocolSerializationTests.cs b/TextCascade.Server.Tests/ProtocolSerializationTests.cs new file mode 100644 index 0000000..f7e3220 --- /dev/null +++ b/TextCascade.Server.Tests/ProtocolSerializationTests.cs @@ -0,0 +1,47 @@ +using System.Text; +using System.Text.Json; +using TextCascade.Server; + +namespace TextCascade.Server.Tests; + +public class ProtocolSerializationTests +{ + [Fact] + public void WelcomeLatestTimeUsesUtcSecondFormat() + { + var timestamp = DateTimeOffset.FromUnixTimeMilliseconds(1760000000123).ToUniversalTime(); + var latest = new LatestText("payload", 7, "hash", true, "client", "name", timestamp); + + var json = Encoding.UTF8.GetString(Protocol.SerializeWelcome(latest)); + + using var document = JsonDocument.Parse(json); + var actual = document.RootElement.GetProperty("latest").GetProperty("updatedAtUtc").GetString(); + Assert.Equal("2025-10-09T08:53:20Z", actual); + } + + [Fact] + public void ClipTimeUsesUtcSecondFormat() + { + var timestamp = DateTimeOffset.FromUnixTimeMilliseconds(1760000000999).ToUniversalTime(); + var latest = new LatestText("payload", 7, "hash", true, "client", "name", timestamp); + + var json = Encoding.UTF8.GetString(Protocol.SerializeClip("clip-1", latest)); + + using var document = JsonDocument.Parse(json); + var actual = document.RootElement.GetProperty("updatedAtUtc").GetString(); + Assert.Equal("2025-10-09T08:53:20Z", actual); + } + + [Fact] + public void ClipAckTimeUsesUtcSecondFormat() + { + var timestamp = DateTimeOffset.FromUnixTimeMilliseconds(1760000000500).ToUniversalTime(); + var latest = new LatestText("payload", 7, "hash", true, "client", "name", timestamp); + + var json = Encoding.UTF8.GetString(Protocol.SerializeClipAck("clip-1", latest)); + + using var document = JsonDocument.Parse(json); + var actual = document.RootElement.GetProperty("updatedAtUtc").GetString(); + Assert.Equal("2025-10-09T08:53:20Z", actual); + } +} diff --git a/TextCascade.Server/Cli.cs b/TextCascade.Server/Cli.cs index 2e15c3f..2e45a54 100644 --- a/TextCascade.Server/Cli.cs +++ b/TextCascade.Server/Cli.cs @@ -16,9 +16,7 @@ public static int RunCli(string[] args, IPasswordHasher? hasher = null) { if (args.Length == 0 || args[0] != "user") { - Console.Error.WriteLine("Usage: TextCascade.Server user [options]"); - Console.Error.WriteLine("Commands: add, passwd, disable, enable, delete, revoke-tokens, list, hash"); - return Error; + return PrintUsage(); } hasher ??= new Argon2PasswordHasher(); @@ -48,6 +46,7 @@ private static int PrintUsage() { Console.Error.WriteLine("Usage: TextCascade.Server user [options]"); Console.Error.WriteLine("Commands: add, passwd, disable, enable, delete, revoke-tokens, list, hash"); + Console.Error.WriteLine("Password commands accept --password-stdin (reads one line from stdin)."); return Error; } @@ -59,8 +58,8 @@ private static int CommandAddUser(string[] args, IPasswordHasher hasher) return Error; } - var password = ReadPassword("Password: "); - var confirm = ReadPassword("Confirm: "); + var password = ReadPassword("Password: ", args); + var confirm = HasPasswordStdin(args) ? password : ReadPassword("Confirm: ", args); if (!string.Equals(password, confirm, StringComparison.Ordinal)) { Console.Error.WriteLine("Passwords do not match."); @@ -109,7 +108,7 @@ private static int CommandPasswd(string[] args, IPasswordHasher hasher) return Error; } - var password = ReadPassword("New password: "); + var password = ReadPassword("New password: ", args); var hash = hasher.Hash(password, CreateArgon2Config(config)); users.Users[index] = users.Users[index] with { PasswordHash = hash }; UsersFile.SaveUsers(usersPath, users); @@ -212,7 +211,7 @@ private static int CommandListUsers(string[] args) private static int CommandHashPassword(string[] args, IPasswordHasher hasher) { - var password = ReadPassword("Password: "); + var password = ReadPassword("Password: ", args); var config = Config.CreateDefaultConfig(); var hash = hasher.Hash(password, CreateArgon2Config(config)); Console.WriteLine(hash); @@ -259,8 +258,28 @@ private static bool TryGetOption(string[] args, string name, out string value) return false; } - private static string ReadPassword(string prompt) + internal static bool HasPasswordStdin(string[] args) => HasFlag(args, "password-stdin"); + + private static bool HasFlag(string[] args, string name) + { + var flag = $"--{name}"; + return args.Any(arg => string.Equals(arg, flag, StringComparison.Ordinal)); + } + + internal static string ReadPassword(string prompt, string[] args) { + if (HasPasswordStdin(args)) + { + var line = Console.In.ReadLine(); + if (string.IsNullOrEmpty(line)) + { + Console.Error.WriteLine("--password-stdin requires one non-empty line."); + throw new ArgumentException("--password-stdin requires one non-empty line."); + } + + return line; + } + Console.Write(prompt); var builder = new StringBuilder(); while (true) diff --git a/TextCascade.Server/Protocol.cs b/TextCascade.Server/Protocol.cs index bb03e17..a46415d 100644 --- a/TextCascade.Server/Protocol.cs +++ b/TextCascade.Server/Protocol.cs @@ -110,7 +110,20 @@ public static LatestText From(ClipSnapshot snapshot, ulong version, string clien [JsonSerializable(typeof(PingMessage))] [JsonSerializable(typeof(ByeMessage))] [JsonSerializable(typeof(ProtocolErrorMessage))] -internal sealed partial class ServerJsonContext : JsonSerializerContext; +[JsonSourceGenerationOptions(DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull)] +internal sealed partial class ServerJsonContext : JsonSerializerContext +{ + public static ServerJsonContext Configured { get; } + + public static JsonSerializerOptions SerializationOptions { get; } = new(JsonSerializerDefaults.Web) + { + DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull, + WriteIndented = false, + Converters = { new UtcSecondDateTimeConverter() }, + }; + + static ServerJsonContext() => Configured = new ServerJsonContext(SerializationOptions); +} public sealed record WelcomeMessage( [property: JsonPropertyName("type")] string Type, @@ -148,6 +161,17 @@ public sealed record ProtocolErrorMessage( [property: JsonPropertyName("message")] string Message, [property: JsonPropertyName("referenceId")] string? ReferenceId); +internal sealed class UtcSecondDateTimeConverter : JsonConverter +{ + public override DateTimeOffset Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) + => reader.TokenType == JsonTokenType.String && DateTimeOffset.TryParse(reader.GetString(), CultureInfo.InvariantCulture, DateTimeStyles.None, out var value) + ? value.ToUniversalTime() + : throw new JsonException("Invalid date/time value."); + + public override void Write(Utf8JsonWriter writer, DateTimeOffset value, JsonSerializerOptions options) + => writer.WriteStringValue(value.ToUniversalTime().ToString("yyyy-MM-dd'T'HH:mm:ss'Z'", CultureInfo.InvariantCulture)); +} + public static class Protocol { public const int ProtocolVersion = 1; @@ -158,19 +182,13 @@ public static class Protocol public const int MaxHashBytes = 4096; - private static readonly JsonSerializerOptions WriteOptions = new(JsonSerializerDefaults.Web) - { - DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull, - WriteIndented = false, - }; - public static byte[] SerializeMessage(T message, JsonTypeInfo typeInfo) => JsonSerializer.SerializeToUtf8Bytes(message, typeInfo); public static byte[] SerializeWelcome(LatestText? latest) { var message = new WelcomeMessage("welcome", ProtocolVersion, latest); - return SerializeMessage(message, ServerJsonContext.Default.WelcomeMessage); + return SerializeMessage(message, ServerJsonContext.Configured.WelcomeMessage); } public static byte[] SerializeClip(string id, LatestText latest) @@ -185,25 +203,25 @@ public static byte[] SerializeClip(string id, LatestText latest) latest.FromClientId, latest.FromClientName, latest.UpdatedAtUtc); - return SerializeMessage(message, ServerJsonContext.Default.ClipMessage); + return SerializeMessage(message, ServerJsonContext.Configured.ClipMessage); } public static byte[] SerializeClipAck(string id, LatestText latest) { var message = new ClipAckMessage("clip_ack", id, latest.Version, latest.UpdatedAtUtc); - return SerializeMessage(message, ServerJsonContext.Default.ClipAckMessage); + return SerializeMessage(message, ServerJsonContext.Configured.ClipAckMessage); } public static byte[] SerializePing(DateTimeOffset nowUtc) => - SerializeMessage(new PingMessage("ping", nowUtc), ServerJsonContext.Default.PingMessage); + SerializeMessage(new PingMessage("ping", nowUtc), ServerJsonContext.Configured.PingMessage); public static byte[] SerializeBye(string reason = "server_shutdown") => - SerializeMessage(new ByeMessage("bye", reason), ServerJsonContext.Default.ByeMessage); + SerializeMessage(new ByeMessage("bye", reason), ServerJsonContext.Configured.ByeMessage); public static byte[] SerializeProtocolError(ProtocolError error) => SerializeMessage( new ProtocolErrorMessage("error", error.CodeName, error.Message, error.ReferenceId), - ServerJsonContext.Default.ProtocolErrorMessage); + ServerJsonContext.Configured.ProtocolErrorMessage); public static byte[] SerializeLoginResponse(AuthToken token, bool needsRehash = false) { diff --git a/TextCascade.Server/SyncServer.cs b/TextCascade.Server/SyncServer.cs index 486d7bb..2630dd1 100644 --- a/TextCascade.Server/SyncServer.cs +++ b/TextCascade.Server/SyncServer.cs @@ -37,6 +37,7 @@ public sealed class ConnectionStateBag private DateTimeOffset lastPingAt; private bool closed; private bool helloTimeoutStarted; + private bool pongAwaited; public Channel SendQueue { get; } public CancellationTokenSource Cts { get; } public bool HelloReceived { get; internal set; } @@ -54,6 +55,21 @@ public DateTimeOffset LastPingAt internal set { lock (gate) { lastPingAt = value; } } } + public void MarkPingAwaitingPong() + { + lock (gate) { pongAwaited = true; } + } + + public bool TryTakePongAwaiting() + { + lock (gate) + { + if (!pongAwaited) return false; + pongAwaited = false; + return true; + } + } + public bool IsClosed { get { lock (gate) { return closed; } } @@ -366,6 +382,7 @@ public void EnqueuePing(DateTimeOffset nowUtc) } connection.State.LastPingAt = nowUtc; + connection.State.MarkPingAwaitingPong(); if (!connection.State.TryEnqueueSend(bytes) && connection.State.MarkClosed()) { connection.State.Cts.Cancel(); @@ -687,6 +704,10 @@ public static async Task RunAsync(ConnectionContext provisional, TokenPayload pa var received = await ReceiveFrameAsync(provisional, config.Limits.MaxFrameBytes, provisional.State.Cts.Token); if (received.MessageType == WebSocketMessageType.Close) { + await provisional.Socket.CloseOutputAsync( + WebSocketCloseStatus.NormalClosure, + "client_closed", + CancellationToken.None); SyncServer.Instance.CancelConnection(provisional, "closed"); return; } @@ -851,7 +872,18 @@ private static async Task ReadLoopAsync(ConnectionContext connection, RuntimeCon break; } - if (received.MessageType == WebSocketMessageType.Close) break; + if (received.MessageType == WebSocketMessageType.Close) + { + try + { + await connection.Socket.CloseOutputAsync( + WebSocketCloseStatus.NormalClosure, + "client_closed", + CancellationToken.None); + } + catch (WebSocketException) { } + break; + } if (!Protocol.CheckFrameSize(received.Payload.Length, config)) { @@ -897,6 +929,16 @@ private static async Task ReadLoopAsync(ConnectionContext connection, RuntimeCon } break; case MessageKind.Pong: + if (!connection.State.TryTakePongAwaiting()) + { + var unsolicitedPong = Protocol.SerializeProtocolError(new ProtocolError( + ProtocolErrorCode.InvalidMessage, + "Pong received without an outstanding ping.", + null)); + await SendSafeAsync(connection, unsolicitedPong); + continue; + } + if (connection.Hub is null || !connection.Hub.TryWriteJob(new PongJob(connection, (ClientPong)parse.Message!))) { SyncServer.Instance.CancelConnection(connection, "user_loop_unavailable"); From 701828c1694f3633a02f4d4e8b795094cdede5ce Mon Sep 17 00:00:00 2001 From: long45343 <1725334094@qq.com> Date: Tue, 18 Aug 2026 23:48:36 +0800 Subject: [PATCH 04/32] Bump version to 0.2.1 --- TextCascade.Server/TextCascade.Server.csproj | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/TextCascade.Server/TextCascade.Server.csproj b/TextCascade.Server/TextCascade.Server.csproj index a1b9775..db51706 100644 --- a/TextCascade.Server/TextCascade.Server.csproj +++ b/TextCascade.Server/TextCascade.Server.csproj @@ -4,7 +4,7 @@ net10.0 enable enable - 0.2.0 + 0.2.1 TextCascade.Server true From 248a63538d36a9b3003032f503b18e3458a66b64 Mon Sep 17 00:00:00 2001 From: long45343 <1725334094@qq.com> Date: Sat, 22 Aug 2026 00:57:48 +0800 Subject: [PATCH 05/32] Release v0.2.5 --- README.md | 14 +- TextCascade.Server.Tests/ClipAndCoreTests.cs | 12 + .../ProtocolParseTests.cs | 6 +- .../ProtocolSerializationTests.cs | 2 +- .../RuntimeStateAndProtocolTests.cs | 140 +++++++ TextCascade.Server/AuthService.cs | 14 +- TextCascade.Server/Cli.cs | 91 +++-- TextCascade.Server/Core.cs | 23 ++ TextCascade.Server/Protocol.cs | 13 +- TextCascade.Server/RuntimeConfig.cs | 34 +- TextCascade.Server/RuntimeStateStore.cs | 129 +++++++ TextCascade.Server/ServerHost.cs | 47 ++- TextCascade.Server/SyncServer.cs | 347 ++++++++++++------ TextCascade.Server/TextCascade.Server.csproj | 2 +- deploy/textcascade-server.service | 8 +- deploy/textcascade.toml | 1 + 16 files changed, 685 insertions(+), 198 deletions(-) create mode 100644 TextCascade.Server.Tests/RuntimeStateAndProtocolTests.cs create mode 100644 TextCascade.Server/RuntimeStateStore.cs diff --git a/README.md b/README.md index 7029e05..d90d8f4 100644 --- a/README.md +++ b/README.md @@ -64,7 +64,7 @@ TextCascade-Server/ 3. **添加用户** ``` - dotnet TextCascade.Server.dll user add --username alice + dotnet TextCascade.Server.dll user add --config /etc/textcascade/textcascade.toml --username alice ``` CLI 子命令:`add`、`passwd`、`disable`、`enable`、`delete`、`revoke-tokens`、`list`、`hash`。 @@ -115,6 +115,7 @@ users_file = "users.json" 关键规则: - `token_secret_env` 指向环境变量名,secret 不写入 TOML;长度 < 32 字节则启动失败。 +- CLI 配置回退顺序为 `--config`、`TEXTCASCADE_CONFIG`、当前目录 `textcascade.toml`;`TEXTCASCADE_USERS_FILE` 与 `TEXTCASCADE_STATE_FILE` 仍可覆盖 TOML。 - TLS 始终启用;证书仅支持无密码格式(PEM bundle 或无密码 PFX),带密码 PFX 不支持。 - `max_frame_bytes` 必须大于 `max_text_bytes`(差额留给协议头)。 - 所有容量与时间配置必须 > 0,心跳超时必须大于心跳间隔。 @@ -158,8 +159,9 @@ GitHub Release 提供两种 Framework-dependent 单文件包,目标机需预装 ### 生产部署(systemd) 参考 `deploy/textcascade-server.service`: +- 以专用系统用户 `textcascade` 运行;先执行 `useradd --system --home /opt/textcascade-server --shell /usr/sbin/nologin textcascade`。 - 以 systemd 托管,`Restart=on-failure`,开启 `ProtectSystem`/`ProtectHome`/`PrivateTmp` 等加固项。 -- 配置与 users.json 放 `/etc/textcascade/`,程序放 `/opt/textcascade-server/`。 +- 配置与 users.json 放 `/etc/textcascade/`,运行状态放 `/var/lib/textcascade/`,程序放 `/opt/textcascade-server/`;目录属主设为 `textcascade:textcascade`。 - token secret 由 `/etc/textcascade/textcascade.env` 注入。 ### 许可 @@ -210,7 +212,7 @@ Built on ASP.NET Core Minimal API with native Kestrel WebSockets, TLS-terminated 3. **Add a user** ``` - dotnet TextCascade.Server.dll user add --username alice + dotnet TextCascade.Server.dll user add --config /etc/textcascade/textcascade.toml --username alice ``` CLI subcommands: `add`, `passwd`, `disable`, `enable`, `delete`, `revoke-tokens`, `list`, `hash`. @@ -257,10 +259,13 @@ clip_tokens_per_second = 2 [files] users_file = "users.json" +state_file = "textcascade.state.json" +state_file = "textcascade.state.json" ``` Key rules: - `token_secret_env` names an env var; the secret is never written to TOML and must be >= 32 bytes. +- CLI config fallback is `--config`, then `TEXTCASCADE_CONFIG`, then `textcascade.toml`; `TEXTCASCADE_USERS_FILE` and `TEXTCASCADE_STATE_FILE` still override TOML. - TLS is always on; only password-less certs are supported (PEM bundle or password-less PFX). - `max_frame_bytes` must exceed `max_text_bytes` (the difference covers the JSON header). - All capacity/time values must be > 0; heartbeat timeout must exceed the interval. @@ -304,8 +309,9 @@ Pushing a `v*.*.*` tag (for example `v0.2.0`) runs tests, builds both single-fil ### Production (systemd) See `deploy/textcascade-server.service`: +- Run as the dedicated `textcascade` system user; create it with `useradd --system --home /opt/textcascade-server --shell /usr/sbin/nologin textcascade`. - Managed by systemd with `Restart=on-failure` and hardening flags (`ProtectSystem`, `ProtectHome`, `PrivateTmp`). -- Config and `users.json` under `/etc/textcascade/`; binaries under `/opt/textcascade-server/`. +- Config and `users.json` under `/etc/textcascade/`, runtime state under `/var/lib/textcascade/`, and binaries under `/opt/textcascade-server/`; set directory ownership to `textcascade:textcascade`. - Token secret injected via `/etc/textcascade/textcascade.env`. ### License diff --git a/TextCascade.Server.Tests/ClipAndCoreTests.cs b/TextCascade.Server.Tests/ClipAndCoreTests.cs index 50c3501..6184282 100644 --- a/TextCascade.Server.Tests/ClipAndCoreTests.cs +++ b/TextCascade.Server.Tests/ClipAndCoreTests.cs @@ -47,6 +47,18 @@ public void SeenIdRingRetainsOriginalResultForDuplicateAck() Assert.Equal(original, result); } + [Fact] + public void SeenIdRingTreatsSameIdWithChangedContentAsNewClip() + { + var ring = new SeenIdRing(4); + var original = new LatestText("first", 1, "hash-1", false, "client", "name", DateTimeOffset.UtcNow); + ring.RememberId("same-id", original); + + Assert.False(ring.IsUnchangedDuplicate("same-id", "second", "hash-2", false, out _)); + Assert.True(ring.IsUnchangedDuplicate("same-id", "first", "hash-1", false, out var unchanged)); + Assert.Equal(original, unchanged); + } + [Fact] public void TokenBucketRefillsOverTime() { diff --git a/TextCascade.Server.Tests/ProtocolParseTests.cs b/TextCascade.Server.Tests/ProtocolParseTests.cs index b7021d0..d4f3b72 100644 --- a/TextCascade.Server.Tests/ProtocolParseTests.cs +++ b/TextCascade.Server.Tests/ProtocolParseTests.cs @@ -5,6 +5,8 @@ namespace TextCascade.Server.Tests; public class ProtocolParseTests { + private const string ValidHash = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; + private static RuntimeConfig NewConfig() => TextCascade.Server.Config.CreateDefaultConfig(); private static ParseResult Parse(string json) => @@ -13,7 +15,7 @@ private static ParseResult Parse(string json) => [Fact] public void ParsesHello() { - var result = Parse("{\"type\":\"hello\",\"clientId\":\"a\",\"clientName\":\"n\",\"lastServerVersion\":5,\"snapshot\":{\"payload\":\"p\",\"encrypted\":true,\"hash\":\"h\",\"localModifiedAtUtc\":\"2026-08-18T08:00:00Z\"}}"); + var result = Parse($"{{\"type\":\"hello\",\"clientId\":\"a\",\"clientName\":\"n\",\"lastServerVersion\":5,\"snapshot\":{{\"payload\":\"p\",\"encrypted\":true,\"hash\":\"{ValidHash}\",\"localModifiedAtUtc\":\"2026-08-18T08:00:00Z\"}}}}"); Assert.True(result.IsSuccess); Assert.Equal(MessageKind.Hello, result.Kind); @@ -26,7 +28,7 @@ public void ParsesHello() [Fact] public void ParsesClip() { - var result = Parse("{\"type\":\"clip\",\"id\":\"id1\",\"payload\":\"p\",\"encrypted\":true,\"hash\":\"h\"}"); + var result = Parse($"{{\"type\":\"clip\",\"id\":\"id1\",\"payload\":\"p\",\"encrypted\":true,\"hash\":\"{ValidHash}\"}}"); Assert.True(result.IsSuccess); Assert.Equal(MessageKind.Clip, result.Kind); diff --git a/TextCascade.Server.Tests/ProtocolSerializationTests.cs b/TextCascade.Server.Tests/ProtocolSerializationTests.cs index f7e3220..7823101 100644 --- a/TextCascade.Server.Tests/ProtocolSerializationTests.cs +++ b/TextCascade.Server.Tests/ProtocolSerializationTests.cs @@ -12,7 +12,7 @@ public void WelcomeLatestTimeUsesUtcSecondFormat() var timestamp = DateTimeOffset.FromUnixTimeMilliseconds(1760000000123).ToUniversalTime(); var latest = new LatestText("payload", 7, "hash", true, "client", "name", timestamp); - var json = Encoding.UTF8.GetString(Protocol.SerializeWelcome(latest)); + var json = Encoding.UTF8.GetString(Protocol.SerializeWelcome(latest, TextCascade.Server.Config.CreateDefaultConfig().Limits)); using var document = JsonDocument.Parse(json); var actual = document.RootElement.GetProperty("latest").GetProperty("updatedAtUtc").GetString(); diff --git a/TextCascade.Server.Tests/RuntimeStateAndProtocolTests.cs b/TextCascade.Server.Tests/RuntimeStateAndProtocolTests.cs new file mode 100644 index 0000000..b89fa76 --- /dev/null +++ b/TextCascade.Server.Tests/RuntimeStateAndProtocolTests.cs @@ -0,0 +1,140 @@ +using System.Text; +using Microsoft.Extensions.Logging.Abstractions; +using TextCascade.Server; + +namespace TextCascade.Server.Tests; + +public class RuntimeStateAndProtocolTests +{ + private static readonly DateTimeOffset TestStartTime = DateTimeOffset.FromUnixTimeSeconds(1760000000); + + [Fact] + public void StateStorePersistsHighestVersionAtomically() + { + var path = Path.Combine(Path.GetTempPath(), $"textcascade-state-{Guid.NewGuid():N}.json"); + try + { + var first = new RuntimeStateStore(path); + first.SaveVersion("alice", 7UL); + + var second = new RuntimeStateStore(path); + second.SaveVersion("alice", 5UL); + + Assert.Equal(7UL, second.GetVersion("alice")); + Assert.Equal(7UL, new RuntimeStateStore(path).GetVersion("alice")); + } + finally + { + if (File.Exists(path)) File.Delete(path); + } + } + + [Fact] + public void StateStoreRejectsInvalidFile() + { + var path = Path.Combine(Path.GetTempPath(), $"textcascade-state-{Guid.NewGuid():N}.json"); + try + { + File.WriteAllText(path, """{"entries":[{"username":"alice","version":0}]}""", Encoding.UTF8); + Assert.Throws(() => new RuntimeStateStore(path)); + } + finally + { + if (File.Exists(path)) File.Delete(path); + } + } + + [Fact] + public void ClipValidationAcceptsClientLocalHash() + { + var config = TextCascade.Server.Config.CreateDefaultConfig(); + Assert.True(Protocol.ValidateClipMessage(new ClientClip("id", "payload", true, new string('a', 64)), config)); + Assert.True(Protocol.ValidateClipMessage(new ClientClip("id", "payload", true, "client-local-hash"), config)); + Assert.False(Protocol.ValidateClipMessage(new ClientClip("id", "payload", true, ""), config)); + Assert.False(Protocol.ValidateClipMessage(new ClientClip("id", "payload", true, new string('a', 4097)), config)); + } + + [Fact] + public void ConfigParsesStateFilePath() + { + var path = Path.Combine(Path.GetTempPath(), $"textcascade-config-{Guid.NewGuid():N}.toml"); + try + { + File.WriteAllText(path, "[files]\nstate_file = \"/tmp/state.json\"\n"); + var config = TextCascade.Server.Config.LoadTomlConfig(path); + Assert.Equal("/tmp/state.json", config.Files.StateFile); + } + finally + { + if (File.Exists(path)) File.Delete(path); + } + } + + [Fact] + public void RecoveryWindowRestoresSnapshotAtPersistedVersion() + { + var path = Path.Combine(Path.GetTempPath(), $"textcascade-state-{Guid.NewGuid():N}.json"); + try + { + new RuntimeStateStore(path).SaveVersion("alice", 7UL); + var config = TextCascade.Server.Config.CreateDefaultConfig(); + var server = new SyncServer( + config, + new UsersFile(), + new RuntimeStateStore(path), + new Argon2PasswordHasher(), + new SystemClock(), + NullLogger.Instance); + var hub = new UserHub("alice", config, TestStartTime, server, 7UL); + var modified = DateTimeOffset.FromUnixTimeSeconds(1759999990); + hub.AcceptSnapshot(new ClientHello( + "client-a", + "Client A", + 7UL, + new ClipSnapshot("restored", false, "client-local-hash", modified))); + + hub.CloseRecoveryWindow(TestStartTime.AddSeconds(3)); + + Assert.NotNull(hub.Latest); + Assert.Equal(7UL, hub.Version); + Assert.Equal("restored", hub.Latest!.Payload); + } + finally + { + if (File.Exists(path)) File.Delete(path); + } + } + + [Fact] + public void RecoveryWindowIgnoresStaleSnapshot() + { + var path = Path.Combine(Path.GetTempPath(), $"textcascade-state-{Guid.NewGuid():N}.json"); + try + { + new RuntimeStateStore(path).SaveVersion("alice", 7UL); + var config = TextCascade.Server.Config.CreateDefaultConfig(); + var server = new SyncServer( + config, + new UsersFile(), + new RuntimeStateStore(path), + new Argon2PasswordHasher(), + new SystemClock(), + NullLogger.Instance); + var hub = new UserHub("alice", config, TestStartTime, server, 7UL); + hub.AcceptSnapshot(new ClientHello( + "client-a", + "Client A", + 6UL, + new ClipSnapshot("stale", false, "client-local-hash", TestStartTime))); + + hub.CloseRecoveryWindow(TestStartTime.AddSeconds(3)); + + Assert.Null(hub.Latest); + Assert.Equal(7UL, hub.Version); + } + finally + { + if (File.Exists(path)) File.Delete(path); + } + } +} diff --git a/TextCascade.Server/AuthService.cs b/TextCascade.Server/AuthService.cs index 434006b..d1688b8 100644 --- a/TextCascade.Server/AuthService.cs +++ b/TextCascade.Server/AuthService.cs @@ -7,10 +7,10 @@ namespace TextCascade.Server; public sealed class AuthService { - public static async Task HandleLoginAsync(HttpContext context, RuntimeConfig config, ILogger? logger = null) + public static async Task HandleLoginAsync(HttpContext context, RuntimeConfig config, SyncServer syncServer, ILogger? logger = null) { - var limiter = SyncServer.Instance.LoginLimiter; - var clock = SyncServer.Instance.Clock; + var limiter = syncServer.LoginLimiter; + var clock = syncServer.Clock; var ip = context.Connection.RemoteIpAddress?.ToString() ?? "unknown"; var now = clock.UtcNow; @@ -32,9 +32,9 @@ public static async Task HandleLoginAsync(HttpContext context, RuntimeConfig con return; } - var userLookup = SyncServer.Instance.UserLookup; + var userLookup = syncServer.UserLookup; var found = userLookup.TryGetValue(request.Username, out var user); - var passwordOk = found && user is not null && SyncServer.Instance.Hasher.Verify(request.Password, user.PasswordHash); + var passwordOk = found && user is not null && syncServer.Hasher.Verify(request.Password, user.PasswordHash); if (!passwordOk) { logger?.LogSecurityEvent("login", ("username", request.Username), ("ip", ip), ("success", false), ("reason", "invalid_credentials")); @@ -55,7 +55,7 @@ public static async Task HandleLoginAsync(HttpContext context, RuntimeConfig con // Spec §4.1: on parameter drift, emit a structured rehash warning rather than rewriting users.json. bool needsRehash = false; - if (SyncServer.Instance.Hasher.NeedsRehash(authenticatedUser.PasswordHash, Cli.CreateArgon2Config(config))) + if (syncServer.Hasher.NeedsRehash(authenticatedUser.PasswordHash, Cli.CreateArgon2Config(config))) { needsRehash = true; logger?.LogWarning("Argon2 password hash needs rehash for user {Username}; users.json was not rewritten.", authenticatedUser.Username); @@ -63,7 +63,7 @@ public static async Task HandleLoginAsync(HttpContext context, RuntimeConfig con var tokenService = new TokenService(config.TokenSecret!); var token = tokenService.CreateToken(authenticatedUser, now, TimeSpan.FromDays(config.Auth.TokenTtlDays)); - var bytes = Protocol.SerializeLoginResponse(token, needsRehash); + var bytes = Protocol.SerializeLoginResponse(token, config, needsRehash); context.Response.StatusCode = 200; context.Response.ContentType = "application/json"; await context.Response.BodyWriter.WriteAsync(bytes); diff --git a/TextCascade.Server/Cli.cs b/TextCascade.Server/Cli.cs index 2e45a54..6c52614 100644 --- a/TextCascade.Server/Cli.cs +++ b/TextCascade.Server/Cli.cs @@ -3,6 +3,7 @@ using System.Runtime.Versioning; using System.Security.Cryptography; using System.Text; +using System.Text.Json; using Isopoh.Cryptography.Argon2; namespace TextCascade.Server; @@ -28,16 +29,38 @@ public static int RunCli(string[] args, IPasswordHasher? hasher = null) } var rest = args.Skip(1).ToArray(); + if (!TryExtractConfigOption(ref rest, out var configPath)) + { + Console.Error.WriteLine("--config requires a path."); + return Error; + } + + configPath ??= Environment.GetEnvironmentVariable("TEXTCASCADE_CONFIG") is { Length: > 0 } environmentConfig + ? environmentConfig + : "textcascade.toml"; + RuntimeConfig config; + try + { + config = Config.CreateDefaultConfig(); + config = Config.LoadTomlConfig(configPath, config); + config = Config.ApplyEnvironmentOverrides(config); + } + catch (Exception exception) when (exception is InvalidOperationException or JsonException or DecoderFallbackException or IOException) + { + Console.Error.WriteLine($"Configuration error: {exception.Message}"); + return Error; + } + return rest switch { - { Length: > 0 } when rest[0] == "add" => CommandAddUser(rest, hasher), - { Length: > 0 } when rest[0] == "passwd" => CommandPasswd(rest, hasher), - { Length: > 0 } when rest[0] == "disable" => CommandSetDisabled(rest, disabled: true), - { Length: > 0 } when rest[0] == "enable" => CommandSetDisabled(rest, disabled: false), - { Length: > 0 } when rest[0] == "delete" => CommandDeleteUser(rest), - { Length: > 0 } when rest[0] == "revoke-tokens" => CommandRevokeTokens(rest), - { Length: > 0 } when rest[0] == "list" => CommandListUsers(rest), - { Length: > 0 } when rest[0] == "hash" => CommandHashPassword(rest, hasher), + { Length: > 0 } when rest[0] == "add" => CommandAddUser(rest, hasher, config), + { Length: > 0 } when rest[0] == "passwd" => CommandPasswd(rest, hasher, config), + { Length: > 0 } when rest[0] == "disable" => CommandSetDisabled(rest, disabled: true, config), + { Length: > 0 } when rest[0] == "enable" => CommandSetDisabled(rest, disabled: false, config), + { Length: > 0 } when rest[0] == "delete" => CommandDeleteUser(rest, config), + { Length: > 0 } when rest[0] == "revoke-tokens" => CommandRevokeTokens(rest, config), + { Length: > 0 } when rest[0] == "list" => CommandListUsers(rest, config), + { Length: > 0 } when rest[0] == "hash" => CommandHashPassword(rest, hasher, config), _ => PrintUsage(), }; } @@ -46,11 +69,40 @@ private static int PrintUsage() { Console.Error.WriteLine("Usage: TextCascade.Server user [options]"); Console.Error.WriteLine("Commands: add, passwd, disable, enable, delete, revoke-tokens, list, hash"); - Console.Error.WriteLine("Password commands accept --password-stdin (reads one line from stdin)."); + Console.Error.WriteLine("All commands accept --config ; fallback order is --config, TEXTCASCADE_CONFIG, then textcascade.toml."); return Error; } - private static int CommandAddUser(string[] args, IPasswordHasher hasher) + private static bool TryExtractConfigOption(ref string[] args, out string? configPath) + { + configPath = null; + var remaining = new List(); + for (var index = 0; index < args.Length; index++) + { + if (string.Equals(args[index], "--config", StringComparison.Ordinal)) + { + if (index + 1 >= args.Length) + { + return false; + } + + configPath = args[++index]; + } + else if (args[index].StartsWith("--config=", StringComparison.Ordinal)) + { + configPath = args[index]["--config=".Length..]; + } + else + { + remaining.Add(args[index]); + } + } + + args = remaining.ToArray(); + return true; + } + + private static int CommandAddUser(string[] args, IPasswordHasher hasher, RuntimeConfig config) { if (!TryGetOption(args, "username", out var username)) { @@ -66,7 +118,6 @@ private static int CommandAddUser(string[] args, IPasswordHasher hasher) return Error; } - var config = Config.CreateDefaultConfig(); var usersPath = config.Files.UsersFile; var users = LoadForWrite(usersPath); if (users.Users.Any(user => string.Equals(user.Username, username, StringComparison.Ordinal))) @@ -90,7 +141,7 @@ private static int CommandAddUser(string[] args, IPasswordHasher hasher) return Ok; } - private static int CommandPasswd(string[] args, IPasswordHasher hasher) + private static int CommandPasswd(string[] args, IPasswordHasher hasher, RuntimeConfig config) { if (!TryGetOption(args, "username", out var username)) { @@ -98,7 +149,6 @@ private static int CommandPasswd(string[] args, IPasswordHasher hasher) return Error; } - var config = Config.CreateDefaultConfig(); var usersPath = config.Files.UsersFile; var users = LoadForWrite(usersPath); var index = users.Users.FindIndex(user => string.Equals(user.Username, username, StringComparison.Ordinal)); @@ -116,7 +166,7 @@ private static int CommandPasswd(string[] args, IPasswordHasher hasher) return Ok; } - private static int CommandSetDisabled(string[] args, bool disabled) + private static int CommandSetDisabled(string[] args, bool disabled, RuntimeConfig config) { if (!TryGetOption(args, "username", out var username)) { @@ -124,7 +174,6 @@ private static int CommandSetDisabled(string[] args, bool disabled) return Error; } - var config = Config.CreateDefaultConfig(); var usersPath = config.Files.UsersFile; var users = LoadForWrite(usersPath); var index = users.Users.FindIndex(user => string.Equals(user.Username, username, StringComparison.Ordinal)); @@ -140,7 +189,7 @@ private static int CommandSetDisabled(string[] args, bool disabled) return Ok; } - private static int CommandDeleteUser(string[] args) + private static int CommandDeleteUser(string[] args, RuntimeConfig config) { if (!TryGetOption(args, "username", out var username)) { @@ -148,7 +197,6 @@ private static int CommandDeleteUser(string[] args) return Error; } - var config = Config.CreateDefaultConfig(); var usersPath = config.Files.UsersFile; var users = LoadForWrite(usersPath); var index = users.Users.FindIndex(user => string.Equals(user.Username, username, StringComparison.Ordinal)); @@ -164,7 +212,7 @@ private static int CommandDeleteUser(string[] args) return Ok; } - private static int CommandRevokeTokens(string[] args) + private static int CommandRevokeTokens(string[] args, RuntimeConfig config) { if (!TryGetOption(args, "username", out var username)) { @@ -172,7 +220,6 @@ private static int CommandRevokeTokens(string[] args) return Error; } - var config = Config.CreateDefaultConfig(); var usersPath = config.Files.UsersFile; var users = LoadForWrite(usersPath); var index = users.Users.FindIndex(user => string.Equals(user.Username, username, StringComparison.Ordinal)); @@ -196,9 +243,8 @@ private static int CommandRevokeTokens(string[] args) return Ok; } - private static int CommandListUsers(string[] args) + private static int CommandListUsers(string[] args, RuntimeConfig config) { - var config = Config.CreateDefaultConfig(); var users = File.Exists(config.Files.UsersFile) ? UsersFile.LoadUsers(config.Files.UsersFile) : new UsersFile(); Console.WriteLine($"nextTokenVersion: {users.NextTokenVersion}"); Console.WriteLine("username,disabled,tokenVersion"); @@ -209,10 +255,9 @@ private static int CommandListUsers(string[] args) return Ok; } - private static int CommandHashPassword(string[] args, IPasswordHasher hasher) + private static int CommandHashPassword(string[] args, IPasswordHasher hasher, RuntimeConfig config) { var password = ReadPassword("Password: ", args); - var config = Config.CreateDefaultConfig(); var hash = hasher.Hash(password, CreateArgon2Config(config)); Console.WriteLine(hash); return Ok; diff --git a/TextCascade.Server/Core.cs b/TextCascade.Server/Core.cs index 057f3a3..ff61597 100644 --- a/TextCascade.Server/Core.cs +++ b/TextCascade.Server/Core.cs @@ -178,6 +178,29 @@ public void RememberId(string id, LatestText? result) } } + public bool IsUnchangedDuplicate(string id, string payload, string hash, bool encrypted, out LatestText? latest) + { + lock (gate) + { + for (var index = 0; index < ids.Length; index++) + { + if (!string.Equals(ids[index], id, StringComparison.Ordinal)) + { + continue; + } + + latest = results[index]; + return latest is not null + && string.Equals(latest.Payload, payload, StringComparison.Ordinal) + && string.Equals(latest.Hash, hash, StringComparison.Ordinal) + && latest.Encrypted == encrypted; + } + + latest = null; + return false; + } + } + private void RememberInternal(string id, LatestText? result) { ids[next] = id; diff --git a/TextCascade.Server/Protocol.cs b/TextCascade.Server/Protocol.cs index a46415d..83360b4 100644 --- a/TextCascade.Server/Protocol.cs +++ b/TextCascade.Server/Protocol.cs @@ -185,7 +185,7 @@ public static class Protocol public static byte[] SerializeMessage(T message, JsonTypeInfo typeInfo) => JsonSerializer.SerializeToUtf8Bytes(message, typeInfo); - public static byte[] SerializeWelcome(LatestText? latest) + public static byte[] SerializeWelcome(LatestText? latest, LimitsConfig _) { var message = new WelcomeMessage("welcome", ProtocolVersion, latest); return SerializeMessage(message, ServerJsonContext.Configured.WelcomeMessage); @@ -223,7 +223,7 @@ public static byte[] SerializeProtocolError(ProtocolError error) => new ProtocolErrorMessage("error", error.CodeName, error.Message, error.ReferenceId), ServerJsonContext.Configured.ProtocolErrorMessage); - public static byte[] SerializeLoginResponse(AuthToken token, bool needsRehash = false) + public static byte[] SerializeLoginResponse(AuthToken token, RuntimeConfig config, bool needsRehash = false) { using var stream = new MemoryStream(); using var writer = new Utf8JsonWriter(stream); @@ -231,10 +231,10 @@ public static byte[] SerializeLoginResponse(AuthToken token, bool needsRehash = writer.WriteString("token", token.CompactToken); writer.WriteString("expiresAtUtc", DateTimeOffset.FromUnixTimeSeconds(token.Payload.ExpiresAtUnix).ToUniversalTime().ToString("O")); writer.WriteNumber("protocolVersion", ProtocolVersion); - writer.WriteNumber("maxTextBytes", RuntimeConfigAccessor.Current?.Limits.MaxTextBytes ?? 524288); - writer.WriteNumber("helloTimeoutSeconds", RuntimeConfigAccessor.Current?.Limits.HelloTimeoutSeconds ?? 5); - writer.WriteNumber("heartbeatIntervalSeconds", RuntimeConfigAccessor.Current?.Limits.HeartbeatIntervalSeconds ?? 30); - writer.WriteNumber("heartbeatTimeoutSeconds", RuntimeConfigAccessor.Current?.Limits.HeartbeatTimeoutSeconds ?? 60); + writer.WriteNumber("maxTextBytes", config.Limits.MaxTextBytes); + writer.WriteNumber("helloTimeoutSeconds", config.Limits.HelloTimeoutSeconds); + writer.WriteNumber("heartbeatIntervalSeconds", config.Limits.HeartbeatIntervalSeconds); + writer.WriteNumber("heartbeatTimeoutSeconds", config.Limits.HeartbeatTimeoutSeconds); if (needsRehash) { writer.WriteBoolean("needsRehash", true); @@ -422,7 +422,6 @@ private static bool ValidateClipSnapshot(ClipSnapshot snapshot, RuntimeConfig co { return snapshot.Payload.Length > 0 && Encoding.UTF8.GetByteCount(snapshot.Payload) <= config.Limits.MaxTextBytes - && snapshot.Hash.Length > 0 && Encoding.UTF8.GetByteCount(snapshot.Hash) <= MaxHashBytes && snapshot.LocalModifiedAtUtc.Offset == TimeSpan.Zero; } diff --git a/TextCascade.Server/RuntimeConfig.cs b/TextCascade.Server/RuntimeConfig.cs index a182d28..9071a24 100644 --- a/TextCascade.Server/RuntimeConfig.cs +++ b/TextCascade.Server/RuntimeConfig.cs @@ -36,7 +36,7 @@ public sealed record RateLimitConfig( int ClipBurst, int ClipTokensPerSecond); -public sealed record FilesConfig(string UsersFile); +public sealed record FilesConfig(string UsersFile, string StateFile); public sealed record RuntimeConfig( ServerConfig Server, @@ -46,17 +46,6 @@ public sealed record RuntimeConfig( FilesConfig Files, byte[]? TokenSecret = null); -public static class RuntimeConfigAccessor -{ - private static RuntimeConfig? current; - - public static RuntimeConfig? Current - { - get => current; - set => current = value; - } -} - public static class Config { public static RuntimeConfig CreateDefaultConfig() => new( @@ -64,7 +53,7 @@ public static class Config new AuthConfig(30, "TEXTCASCADE_TOKEN_SECRET", 19456, 2, 1), new LimitsConfig(524288, 589824, 16, 64, 5, 30, 60, 3, 4194304, 16), new RateLimitConfig(10, 5, 10000, 10, 2), - new FilesConfig("users.json")); + new FilesConfig("users.json", "textcascade.state.json")); public static RuntimeConfig LoadTomlConfig(string path, RuntimeConfig? defaults = null) { @@ -158,8 +147,13 @@ private static RuntimeConfig ApplyTomlModel(RuntimeConfig config, TomlTable mode if (TryGetTable(model, "files", out var files)) { - WarnUnknownKeys(files, "files", new[] { "users_file" }); - config = config with { Files = new FilesConfig(GetString(files, "users_file", config.Files.UsersFile, "files.users_file")) }; + WarnUnknownKeys(files, "files", new[] { "users_file", "state_file" }); + config = config with + { + Files = new FilesConfig( + GetString(files, "users_file", config.Files.UsersFile, "files.users_file"), + GetString(files, "state_file", config.Files.StateFile, "files.state_file")), + }; } return config; @@ -239,6 +233,11 @@ public static RuntimeConfig ApplyEnvironmentOverrides(RuntimeConfig config) config = config with { Files = config.Files with { UsersFile = usersFile } }; } + if (Environment.GetEnvironmentVariable("TEXTCASCADE_STATE_FILE") is { Length: > 0 } stateFile) + { + config = config with { Files = config.Files with { StateFile = stateFile } }; + } + if (Environment.GetEnvironmentVariable(config.Auth.TokenSecretEnv) is { Length: > 0 } secretText) { var secret = Encoding.UTF8.GetBytes(secretText); @@ -315,5 +314,10 @@ public static void ValidateConfig(RuntimeConfig config) { throw new InvalidOperationException("files.users_file must not be empty."); } + + if (string.IsNullOrWhiteSpace(config.Files.StateFile)) + { + throw new InvalidOperationException("files.state_file must not be empty."); + } } } diff --git a/TextCascade.Server/RuntimeStateStore.cs b/TextCascade.Server/RuntimeStateStore.cs new file mode 100644 index 0000000..2288ba1 --- /dev/null +++ b/TextCascade.Server/RuntimeStateStore.cs @@ -0,0 +1,129 @@ +using System.Text; +using System.Text.Json; +using System.Text.Json.Serialization; + +namespace TextCascade.Server; + +public sealed record RuntimeStateEntry(string Username, ulong Version); + +internal sealed record RuntimeStateFile(IReadOnlyList Entries); + +public sealed class RuntimeStateStore +{ + private readonly object gate = new(); + private readonly string path; + private readonly Dictionary versions; + + public RuntimeStateStore(string path) + { + this.path = path; + versions = Load(path); + } + + public ulong GetVersion(string username) + { + lock (gate) + { + return versions.TryGetValue(username, out var version) ? version : 0UL; + } + } + + public void SaveVersion(string username, ulong version) + { + lock (gate) + { + if (versions.TryGetValue(username, out var current) && version <= current) + { + return; + } + + versions[username] = version; + WriteAtomic(path, versions.Select(pair => new RuntimeStateEntry(pair.Key, pair.Value)) + .OrderBy(pair => pair.Username, StringComparer.Ordinal) + .ToList()); + } + } + + private static Dictionary Load(string path) + { + if (!File.Exists(path)) + { + return new Dictionary(StringComparer.Ordinal); + } + + try + { + var options = new JsonSerializerOptions + { + PropertyNamingPolicy = JsonNamingPolicy.CamelCase, + ReadCommentHandling = JsonCommentHandling.Disallow, + }; + using var stream = File.OpenRead(path); + var state = JsonSerializer.Deserialize(stream, options); + if (state is null) + { + throw new JsonException("State file is empty."); + } + + var result = new Dictionary(StringComparer.Ordinal); + foreach (var entry in state.Entries) + { + if (string.IsNullOrWhiteSpace(entry.Username) || entry.Version == 0 || !result.TryAdd(entry.Username, entry.Version)) + { + throw new InvalidOperationException("State file contains duplicate, empty, or zero versions."); + } + } + + return result; + } + catch (Exception exception) when (exception is JsonException or InvalidOperationException) + { + throw new InvalidOperationException($"Invalid runtime state file '{path}': {exception.Message}", exception); + } + } + + private static void WriteAtomic(string path, IReadOnlyList entries) + { + var options = new JsonSerializerOptions + { + PropertyNamingPolicy = JsonNamingPolicy.CamelCase, + DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull, + WriteIndented = true, + }; + var json = JsonSerializer.Serialize(new RuntimeStateFile(entries), options); + var temporary = path + "." + Guid.NewGuid().ToString("N") + ".tmp"; + try + { + using (var stream = new FileStream(temporary, FileMode.CreateNew, FileAccess.Write, FileShare.None)) + using (var writer = new StreamWriter(stream, new UTF8Encoding(encoderShouldEmitUTF8Identifier: false))) + { + writer.Write(json); + writer.Flush(); + stream.Flush(flushToDisk: true); + } + + if (File.Exists(path)) + { + try + { + File.Replace(temporary, path, destinationBackupFileName: null, ignoreMetadataErrors: true); + } + catch (PlatformNotSupportedException) + { + File.Move(temporary, path, overwrite: true); + } + } + else + { + File.Move(temporary, path); + } + } + finally + { + if (File.Exists(temporary)) + { + File.Delete(temporary); + } + } + } +} diff --git a/TextCascade.Server/ServerHost.cs b/TextCascade.Server/ServerHost.cs index 4b9db50..de1e5c5 100644 --- a/TextCascade.Server/ServerHost.cs +++ b/TextCascade.Server/ServerHost.cs @@ -27,6 +27,8 @@ public static int RunServer(string[] args) } RuntimeConfig config; + UsersFile users; + RuntimeStateStore stateStore; try { var configPath = args.Length > 0 && args[0] == "--config" ? args[1] : "textcascade.toml"; @@ -34,10 +36,10 @@ public static int RunServer(string[] args) config = Config.LoadTomlConfig(configPath, config); config = Config.ApplyEnvironmentOverrides(config); Config.ValidateConfig(config); - var users = UsersFile.LoadUsers(config.Files.UsersFile); - SyncServer.Instance.Initialize(config, users); + users = UsersFile.LoadUsers(config.Files.UsersFile); + stateStore = new RuntimeStateStore(config.Files.StateFile); } - catch (Exception exception) when (exception is InvalidOperationException or JsonException or DecoderFallbackException) + catch (Exception exception) when (exception is InvalidOperationException or JsonException or DecoderFallbackException or IOException) { Console.Error.WriteLine($"Configuration error: {exception.Message}"); return Error; @@ -56,7 +58,6 @@ public static int RunServer(string[] args) using (certificate) { - RuntimeConfigAccessor.Current = config; var builder = WebApplication.CreateBuilder(args); builder.WebHost.UseKestrel(ConfigureKestrel(config, certificate)); builder.Logging.ClearProviders(); @@ -65,14 +66,27 @@ public static int RunServer(string[] args) options.SingleLine = true; options.TimestampFormat = "yyyy-MM-ddTHH:mm:ssZ "; }); + builder.Services.AddSingleton(); + builder.Services.AddSingleton(); + builder.Services.AddSingleton(stateStore); + builder.Services.AddSingleton(serviceProvider => new SyncServer( + config, + users, + serviceProvider.GetRequiredService(), + serviceProvider.GetRequiredService(), + serviceProvider.GetRequiredService(), + serviceProvider.GetRequiredService>())); builder.Services.AddHostedService(); var app = builder.Build(); - SyncServer.Instance.Logger = app.Logger; app.UseWebSockets(); app.MapGet("/health", () => Results.Json(new { status = "ok" })); - app.MapPost("/api/v1/login", async context => await AuthService.HandleLoginAsync(context, config, app.Logger)); - app.MapGet("/api/v1/sync", async context => await SyncEndpoint.HandleAsync(context, config)); + app.MapPost("/api/v1/login", async context => await AuthService.HandleLoginAsync( + context, + config, + context.RequestServices.GetRequiredService(), + app.Logger)); + app.MapGet("/api/v1/sync", async context => await SyncEndpoint.HandleAsync(context, config, context.RequestServices.GetRequiredService())); app.MapMethods("/health", new[] { "HEAD" }, () => Results.Json(new { status = "ok" })); app.Run(); @@ -236,25 +250,32 @@ public sealed class HeartbeatScannerService : IHostedService, IDisposable { private Timer? timer; + private readonly SyncServer syncServer; + + public HeartbeatScannerService(SyncServer syncServer) + { + this.syncServer = syncServer; + } + public Task StartAsync(CancellationToken cancellationToken) { timer = new Timer(Scan, null, TimeSpan.FromSeconds(1), TimeSpan.FromSeconds(1)); return Task.CompletedTask; } - private static void Scan(object? state) + private void Scan(object? state) { var now = DateTimeOffset.UtcNow; - SyncServer.Instance.ScanHeartbeats(now); + syncServer.ScanHeartbeats(now); - var recoveryEnd = SyncServer.Instance.ProcessStartTime.AddSeconds( - SyncServer.Instance.Config.Limits.SnapshotWindowSeconds); + var recoveryEnd = syncServer.ProcessStartTime.AddSeconds( + syncServer.Config.Limits.SnapshotWindowSeconds); if (now < recoveryEnd) { return; } - foreach (var pair in SyncServer.Instance.Registry.All) + foreach (var pair in syncServer.Registry.All) { pair.Value.CloseRecoveryWindow(now); } @@ -263,7 +284,7 @@ private static void Scan(object? state) public async Task StopAsync(CancellationToken cancellationToken) { timer?.Change(Timeout.Infinite, 0); - await SyncServer.Instance.ShutdownAsync(TimeSpan.FromSeconds(2), DateTimeOffset.UtcNow); + await syncServer.ShutdownAsync(TimeSpan.FromSeconds(2), DateTimeOffset.UtcNow); } public void Dispose() diff --git a/TextCascade.Server/SyncServer.cs b/TextCascade.Server/SyncServer.cs index 2630dd1..1cb006c 100644 --- a/TextCascade.Server/SyncServer.cs +++ b/TextCascade.Server/SyncServer.cs @@ -123,6 +123,7 @@ public sealed class UserHub public TokenBucket ClipBucket { get; } public SeenIdRing SeenIds { get; } public DateTimeOffset ProcessStartTime { get; } + public DateTimeOffset LastActivityAt => new(new DateTime(Interlocked.Read(ref lastActivityTicks), DateTimeKind.Utc)); private readonly object connectionsGate = new(); private readonly List connections = new(); @@ -135,15 +136,22 @@ public sealed class UserHub private readonly List recoveryQueue = new(); private bool recoveryWindowClosed; - public UserHub(string username, RuntimeConfig config, DateTimeOffset processStart) + private readonly SyncServer server; + private readonly RuntimeStateStore runtimeStateStore; + private long lastActivityTicks; + + public UserHub(string username, RuntimeConfig config, DateTimeOffset processStart, SyncServer server, ulong initialVersion) { Username = username; this.config = config; + this.server = server; + this.runtimeStateStore = server.RuntimeStateStore; ProcessStartTime = processStart; UserChannel = Channel.CreateUnbounded(new UnboundedChannelOptions { SingleReader = true, SingleWriter = false }); ClipBucket = new TokenBucket(config.RateLimit.ClipBurst, config.RateLimit.ClipTokensPerSecond, processStart); SeenIds = new SeenIdRing(config.Limits.SeenIdCapacity); - Version = 0; + Version = initialVersion; + lastActivityTicks = processStart.UtcTicks; } public IReadOnlyList Connections @@ -156,13 +164,18 @@ public bool IsEmpty get { lock (connectionsGate) { return connections.Count == 0; } } } + internal object ScanGate => connectionsGate; + internal List ConnectionList => connections; + internal RuntimeConfig Config => config; + public void AddConnection(ConnectionContext connection) { lock (connectionsGate) { connections.Add(connection); } + MarkActivity(DateTimeOffset.UtcNow); var nowUtc = DateTimeOffset.UtcNow; if (recoveryWindowClosed) { - BroadcastToConnection(connection, Protocol.SerializeWelcome(Latest)); + BroadcastToConnection(connection, Protocol.SerializeWelcome(Latest, config.Limits)); return; } @@ -175,18 +188,32 @@ public void AddConnection(ConnectionContext connection) public bool RemoveConnection(ConnectionContext connection) { - lock (connectionsGate) { return connections.Remove(connection); } + bool removed; + lock (connectionsGate) { removed = connections.Remove(connection); } + if (removed) + { + MarkActivity(DateTimeOffset.UtcNow); + } + + return removed; } - public void StartIfIdle(Func processor) + public void StartIfIdle() { if (userLoop is null || userLoop.IsCompleted) { userLoop = Task.Run(async () => { - try { await processor(this); } + try { await RunUserLoopAsync(); } catch (OperationCanceledException) { } - catch (Exception) { } + catch (Exception exception) + { + server.Logger.LogError( + exception, + "User loop failed; rebuilding hub. username={Username}", + Username); + server.RebuildHub(this); + } }); } } @@ -223,7 +250,7 @@ private void ProcessJob(UserJob job, DateTimeOffset nowUtc) } break; case DisconnectJob disconnectJob: - SyncServer.Instance.CancelConnection(disconnectJob.Connection, disconnectJob.Reason); + server.CancelConnection(disconnectJob.Connection, disconnectJob.Reason); break; } } @@ -271,8 +298,21 @@ public void CloseRecoveryWindow(DateTimeOffset nowUtc) winner = CoreLogic.SelectSnapshotWinner(snapshotCandidates); if (winner is not null) { - Version = winner.Version; - Latest = new LatestText(winner.Snapshot.Payload, winner.Version, winner.Snapshot.Hash, winner.Snapshot.Encrypted, winner.ClientId, winner.ClientName, winner.Snapshot.LocalModifiedAtUtc); + var canRestoreLatest = winner.Version > Version + || (winner.Version == Version && Latest is null); + if (!canRestoreLatest) + { + winner = null; + } + else + { + if (winner.Version > Version) + { + runtimeStateStore.SaveVersion(Username, winner.Version); + } + Version = winner.Version; + Latest = new LatestText(winner.Snapshot.Payload, winner.Version, winner.Snapshot.Hash, winner.Snapshot.Encrypted, winner.ClientId, winner.ClientName, winner.Snapshot.LocalModifiedAtUtc); + } } clips = recoveryQueue.ToList(); recoveryQueue.Clear(); @@ -287,12 +327,14 @@ public void CloseRecoveryWindow(DateTimeOffset nowUtc) BroadcastWelcome(nowUtc); // Spec §6.2: empty hubs that survived until the recovery window closes are now removed. - SyncServer.Instance.Registry.RemoveIfEmpty(this, allowDuringRecovery: true); + server.Registry.RemoveIfEmpty(this, allowDuringRecovery: true); + + MarkActivity(nowUtc); } private void BroadcastWelcome(DateTimeOffset nowUtc) { - var bytes = Protocol.SerializeWelcome(Latest); + var bytes = Protocol.SerializeWelcome(Latest, config.Limits); foreach (var connection in Connections) { if (!connection.State.TryEnqueueSend(bytes) && connection.State.MarkClosed()) @@ -315,9 +357,16 @@ public void EnsureRecoveryWindowClosed(DateTimeOffset nowUtc) } } + private void MarkActivity(DateTimeOffset nowUtc) + { + Interlocked.Exchange(ref lastActivityTicks, nowUtc.UtcTicks); + } + + internal void MarkActivityForScan(DateTimeOffset nowUtc) => MarkActivity(nowUtc); + public void ApplyClip(ClientClip clip, ConnectionContext sender, DateTimeOffset nowUtc) { - if (SeenIds.TryGetResult(clip.Id, out var duplicateLatest)) + if (SeenIds.IsUnchangedDuplicate(clip.Id, clip.Payload, clip.Hash, clip.Encrypted, out var duplicateLatest)) { var ackBytes = Protocol.SerializeClipAck(clip.Id, duplicateLatest ?? Latest ?? new LatestText(string.Empty, Version, string.Empty, false, sender.ClientId, sender.ClientName, nowUtc)); if (!sender.State.TryEnqueueSend(ackBytes) && sender.State.MarkClosed()) @@ -327,9 +376,19 @@ public void ApplyClip(ClientClip clip, ConnectionContext sender, DateTimeOffset return; } + if (SeenIds.TryGetResult(clip.Id, out _)) + { + server.Logger.LogWarning( + "Replacing reused clip id. username={Username} clipId={ClipId} clientId={ClientId} previousVersion={PreviousVersion}", + Username, + clip.Id, + sender.ClientId, + Version); + } + if (!ClipBucket.TryAcquire(nowUtc)) { - SyncServer.Instance.Logger?.LogSecurityEvent("reject", + server.Logger.LogSecurityEvent("reject", ("username", Username), ("code", "rate_limited"), ("bytes", Encoding.UTF8.GetByteCount(clip.Payload))); @@ -342,27 +401,39 @@ public void ApplyClip(ClientClip clip, ConnectionContext sender, DateTimeOffset } var next = CoreLogic.NextVersion(Version); + runtimeStateStore.SaveVersion(Username, next); Version = next; var latest = new LatestText(clip.Payload, next, clip.Hash, clip.Encrypted, sender.ClientId, sender.ClientName, nowUtc); Latest = latest; SeenIds.RememberId(clip.Id, latest); - SyncServer.Instance.Logger?.LogSecurityEvent("clip", + server.Logger.LogSecurityEvent("clip", ("username", Username), ("version", latest.Version), + ("clipId", clip.Id), ("bytes", Encoding.UTF8.GetByteCount(clip.Payload)), ("fromClientId", sender.ClientId), ("encrypted", clip.Encrypted)); var broadcastBytes = Protocol.SerializeClip(clip.Id, latest); + var deliveries = new List(); foreach (var connection in Connections) { if (ReferenceEquals(connection, sender)) continue; - if (!connection.State.TryEnqueueSend(broadcastBytes) && connection.State.MarkClosed()) + var queued = connection.State.TryEnqueueSend(broadcastBytes); + deliveries.Add($"{connection.ClientId}:{(queued ? "queued" : "full")}"); + if (!queued && connection.State.MarkClosed()) { connection.State.Cts.Cancel(); } } + server.Logger.LogInformation( + "Clip broadcast. username={Username} version={Version} clipId={ClipId} recipients=[{Recipients}]", + Username, + next, + clip.Id, + string.Join(",", deliveries)); + var ackBytesFinal = Protocol.SerializeClipAck(clip.Id, latest); if (!sender.State.TryEnqueueSend(ackBytesFinal) && sender.State.MarkClosed()) { @@ -370,26 +441,6 @@ public void ApplyClip(ClientClip clip, ConnectionContext sender, DateTimeOffset } } - public void EnqueuePing(DateTimeOffset nowUtc) - { - var bytes = Protocol.SerializePing(nowUtc); - foreach (var connection in Connections) - { - if (!connection.State.HelloReceived - || nowUtc - connection.State.LastPingAt < TimeSpan.FromSeconds(config.Limits.HeartbeatIntervalSeconds)) - { - continue; - } - - connection.State.LastPingAt = nowUtc; - connection.State.MarkPingAwaitingPong(); - if (!connection.State.TryEnqueueSend(bytes) && connection.State.MarkClosed()) - { - connection.State.Cts.Cancel(); - } - } - } - private static void BroadcastToConnection(ConnectionContext connection, byte[] payload) { if (!connection.State.TryEnqueueSend(payload) && connection.State.MarkClosed()) @@ -448,6 +499,11 @@ public void RemoveIfEmpty(UserHub hub, bool allowDuringRecovery) if (!allowDuringRecovery && hub.IsRecoveryWindowOpen(DateTimeOffset.UtcNow)) return; hubs.TryRemove(hub.Username, out _); } + + public bool Remove(UserHub hub) + { + return hubs.TryRemove(new KeyValuePair(hub.Username, hub)); + } } public interface IClock @@ -462,55 +518,109 @@ public sealed class SystemClock : IClock public sealed class SyncServer { - public static SyncServer Instance { get; } = new(); - - private UserRegistry registry = new(); + private readonly UserRegistry registry = new(); private readonly List pendingHellos = new(); private readonly object pendingGate = new(); + private readonly IPasswordHasher hasher; + private readonly IClock clock; + private readonly RuntimeStateStore runtimeStateStore; + private readonly IReadOnlyDictionary userLookup; public UserRegistry Registry => registry; - public IPasswordHasher Hasher { get; set; } = new Argon2PasswordHasher(); + public IPasswordHasher Hasher => hasher; public SlidingWindowLoginLimiter LoginLimiter { get; } = new(); - public IClock Clock { get; set; } = new SystemClock(); - public ILogger? Logger { get; set; } - public IReadOnlyDictionary UserLookup { get; set; } = new Dictionary(StringComparer.Ordinal); - public DateTimeOffset ProcessStartTime { get; set; } = DateTimeOffset.UtcNow; - public RuntimeConfig Config { get; private set; } = TextCascade.Server.Config.CreateDefaultConfig(); - - public void Initialize(RuntimeConfig config, UsersFile users) + public IClock Clock => clock; + public ILogger Logger { get; } + public IReadOnlyDictionary UserLookup => userLookup; + public DateTimeOffset ProcessStartTime { get; } + public RuntimeConfig Config { get; } + public RuntimeStateStore RuntimeStateStore => runtimeStateStore; + + public SyncServer( + RuntimeConfig config, + UsersFile users, + RuntimeStateStore runtimeStateStore, + IPasswordHasher hasher, + IClock clock, + ILogger logger) { Config = config; - registry = new UserRegistry(); - UserLookup = UsersFile.BuildUserLookup(users); - ProcessStartTime = DateTimeOffset.UtcNow; + userLookup = UsersFile.BuildUserLookup(users); + this.runtimeStateStore = runtimeStateStore; + this.hasher = hasher; + this.clock = clock; + Logger = logger; + ProcessStartTime = clock.UtcNow; } public UserHub GetOrCreateHub(string username, RuntimeConfig runtimeConfig) { - var hub = registry.GetOrAdd(username, name => new UserHub(name, runtimeConfig, ProcessStartTime)); - hub.StartIfIdle(hub => hub.RunUserLoopAsync()); + var initialVersion = runtimeStateStore.GetVersion(username); + var hub = registry.GetOrAdd(username, name => new UserHub(name, runtimeConfig, ProcessStartTime, this, initialVersion)); + hub.StartIfIdle(); return hub; } public void ScanHeartbeats(DateTimeOffset nowUtc) { - var timeout = RuntimeConfigAccessor.Current?.Limits.HeartbeatTimeoutSeconds ?? 60; + var timeout = Config.Limits.HeartbeatTimeoutSeconds; foreach (var pair in registry.All) { - pair.Value.EnqueuePing(nowUtc); - foreach (var connection in pair.Value.Connections) + var hub = pair.Value; + List? timedOut = null; + lock (hub.ScanGate) { - if (!connection.State.HelloReceived && connection.State.HelloDeadline is { } deadline && nowUtc >= deadline) + var pingInterval = TimeSpan.FromSeconds(hub.Config.Limits.HeartbeatIntervalSeconds); + var pingBytes = Protocol.SerializePing(nowUtc); + for (var index = hub.ConnectionList.Count - 1; index >= 0; index--) { - EnqueueHelloTimeout(connection); - continue; + var connection = hub.ConnectionList[index]; + if (!connection.State.HelloReceived && connection.State.HelloDeadline is { } deadline && nowUtc >= deadline) + { + timedOut ??= new List(); + timedOut.Add(connection); + continue; + } + + if (connection.State.HelloReceived && nowUtc - connection.State.LastPingAt >= pingInterval) + { + connection.State.LastPingAt = nowUtc; + connection.State.MarkPingAwaitingPong(); + if (!connection.State.TryEnqueueSend(pingBytes) && connection.State.MarkClosed()) + { + connection.State.Cts.Cancel(); + } + } + + if (nowUtc - connection.State.LastSeen >= TimeSpan.FromSeconds(timeout)) + { + timedOut ??= new List(); + timedOut.Add(connection); + } + + hub.MarkActivityForScan(nowUtc); } - var elapsed = nowUtc - connection.State.LastSeen; - if (elapsed.TotalSeconds >= timeout) + } + + if (timedOut is not null) + { + foreach (var connection in timedOut) { - CancelConnection(connection, "heartbeat_timeout"); + if (!connection.State.HelloReceived) + { + EnqueueHelloTimeout(connection); + } + else + { + CancelConnection(connection, "heartbeat_timeout"); + } } } + + if (hub.IsEmpty && nowUtc - hub.LastActivityAt >= TimeSpan.FromMinutes(10)) + { + registry.RemoveIfEmpty(hub, allowDuringRecovery: false); + } } List expired = new(); @@ -531,6 +641,21 @@ public void ScanHeartbeats(DateTimeOffset nowUtc) } } + public void RebuildHub(UserHub hub) + { + if (!registry.Remove(hub)) + { + return; + } + + foreach (var connection in hub.Connections) + { + CancelConnection(connection, "user_loop_failed"); + } + + Registry.RemoveIfEmpty(hub, allowDuringRecovery: true); + } + public void RegisterPendingHello(ConnectionContext connection) { lock (pendingGate) { pendingHellos.Add(connection); } @@ -552,7 +677,7 @@ private void EnqueueHelloTimeout(ConnectionContext connection) _ = CloseAfterHelloTimeoutAsync(connection); } - private static async Task CloseAfterHelloTimeoutAsync(ConnectionContext connection) + private async Task CloseAfterHelloTimeoutAsync(ConnectionContext connection) { try { @@ -566,11 +691,11 @@ private static async Task CloseAfterHelloTimeoutAsync(ConnectionContext connecti } catch (Exception) { - Instance.EnqueueImmediateClose(connection, "server_busy"); + EnqueueImmediateClose(connection, "server_busy"); } finally { - Instance.CancelConnection(connection, "hello_timeout"); + CancelConnection(connection, "hello_timeout"); } } @@ -582,7 +707,7 @@ public void CancelConnection(ConnectionContext connection, string reason) connection.Hub?.RemoveConnection(connection); if (connection.Hub is not null) { - Logger?.LogSecurityEvent("disconnect", + Logger.LogSecurityEvent("disconnect", ("username", connection.Username), ("clientId", connection.ClientId), ("connectionId", connection.ConnectionId), @@ -646,7 +771,7 @@ private static async Task CloseConnectionAsync(ConnectionContext connection, Web public static class SyncEndpoint { - public static async Task HandleAsync(HttpContext context, RuntimeConfig config) + public static async Task HandleAsync(HttpContext context, RuntimeConfig config, SyncServer server) { var tokenHeader = context.Request.Headers.Authorization.ToString(); if (!tokenHeader.StartsWith("Bearer ", StringComparison.Ordinal)) @@ -658,7 +783,7 @@ public static async Task HandleAsync(HttpContext context, RuntimeConfig config) var compactToken = tokenHeader["Bearer ".Length..]; var now = DateTimeOffset.UtcNow; var tokenService = new TokenService(config.TokenSecret!); - if (!tokenService.TryVerifyToken(compactToken, now, SyncServer.Instance.UserLookup, out var payload)) + if (!tokenService.TryVerifyToken(compactToken, now, server.UserLookup, out var payload)) { context.Response.StatusCode = 401; return; @@ -680,7 +805,7 @@ public static async Task HandleAsync(HttpContext context, RuntimeConfig config) using var socket = await context.WebSockets.AcceptWebSocketAsync(subProtocol); var connectionId = Guid.NewGuid().ToString("N"); var provisional = new ConnectionContext(connectionId, payload.Subject, "pending", "pending", socket, null!, config); - await ConnectionHandler.RunAsync(provisional, payload, config); + await ConnectionHandler.RunAsync(provisional, payload, config, server); } internal static string? SelectSubProtocol(IList requested) @@ -695,9 +820,9 @@ public static async Task HandleAsync(HttpContext context, RuntimeConfig config) public static class ConnectionHandler { - public static async Task RunAsync(ConnectionContext provisional, TokenPayload payload, RuntimeConfig config) + public static async Task RunAsync(ConnectionContext provisional, TokenPayload payload, RuntimeConfig config, SyncServer server) { - SyncServer.Instance.RegisterPendingHello(provisional); + server.RegisterPendingHello(provisional); ClientHello hello; try { @@ -708,7 +833,7 @@ await provisional.Socket.CloseOutputAsync( WebSocketCloseStatus.NormalClosure, "client_closed", CancellationToken.None); - SyncServer.Instance.CancelConnection(provisional, "closed"); + server.CancelConnection(provisional, "closed"); return; } @@ -723,7 +848,8 @@ await SendAndClosePreHelloAsync( provisional, error, WebSocketCloseStatus.PolicyViolation, - "invalid_hello"); + "invalid_hello", + server); return; } @@ -732,23 +858,23 @@ await SendAndClosePreHelloAsync( catch (FrameTooLargeException) { var error = Protocol.SerializeProtocolError(new ProtocolError(ProtocolErrorCode.FrameTooLarge, "frame_too_large", null)); - await SendAndClosePreHelloAsync(provisional, error, WebSocketCloseStatus.MessageTooBig, "frame_too_large"); + await SendAndClosePreHelloAsync(provisional, error, WebSocketCloseStatus.MessageTooBig, "frame_too_large", server); return; } catch (OperationCanceledException) { // Hello timeout is owned by the unified heartbeat scanner; here the socket was // cancelled for another reason (e.g. shutdown). Fall through to unified cleanup. - SyncServer.Instance.CancelConnection(provisional, "cancelled"); + server.CancelConnection(provisional, "cancelled"); return; } catch (WebSocketException) { - SyncServer.Instance.CancelConnection(provisional, "socket_error"); + server.CancelConnection(provisional, "socket_error"); return; } - var hub = SyncServer.Instance.GetOrCreateHub(payload.Subject, config); + var hub = server.GetOrCreateHub(payload.Subject, config); var connection = new ConnectionContext( provisional.ConnectionId, payload.Subject, @@ -758,23 +884,23 @@ await SendAndClosePreHelloAsync( hub, config); hub.AddConnection(connection); - SyncServer.Instance.Logger?.LogSecurityEvent("connect", + server.Logger.LogSecurityEvent("connect", ("username", connection.Username), ("clientId", connection.ClientId), ("connectionId", connection.ConnectionId)); connection.State.HelloReceived = true; connection.State.LastSeen = DateTimeOffset.UtcNow; - SyncServer.Instance.UnregisterPendingHello(provisional); + server.UnregisterPendingHello(provisional); if (!hub.TryWriteJob(new HelloJob(connection, hello))) { - SyncServer.Instance.CancelConnection(connection, "user_loop_unavailable"); + server.CancelConnection(connection, "user_loop_unavailable"); return; } var sendTask = ConnectionSendLoopAsync(connection); - var readTask = ReadLoopAsync(connection, config); + var readTask = ReadLoopAsync(connection, config, server); await Task.WhenAll(sendTask, readTask); - SyncServer.Instance.CancelConnection(connection, "disconnected"); + server.CancelConnection(connection, "disconnected"); } private static async Task ReceiveFrameAsync( @@ -804,7 +930,8 @@ private static async Task SendAndClosePreHelloAsync( ConnectionContext connection, byte[] error, WebSocketCloseStatus status, - string reason) + string reason, + SyncServer server) { try { @@ -816,39 +943,15 @@ private static async Task SendAndClosePreHelloAsync( } catch (Exception) { - SyncServer.Instance.EnqueueImmediateClose(connection, "server_busy"); - } - finally - { - SyncServer.Instance.CancelConnection(connection, reason); - } - } - - private static async Task CloseAfterProtocolErrorAsync( - ConnectionContext connection, - WebSocketCloseStatus status, - string reason) - { - // Give the send loop a short opportunity to flush the queued error frame. - await Task.Delay(100); - try - { - if (connection.Socket.State == WebSocketState.Open) - { - await connection.Socket.CloseAsync(status, reason, CancellationToken.None); - } - } - catch (Exception) - { - // Cancellation below is still the unified cleanup path. + server.EnqueueImmediateClose(connection, "server_busy"); } finally { - SyncServer.Instance.CancelConnection(connection, reason); + server.CancelConnection(connection, reason); } } - private static async Task ReadLoopAsync(ConnectionContext connection, RuntimeConfig config) + private static async Task ReadLoopAsync(ConnectionContext connection, RuntimeConfig config, SyncServer server) { try { @@ -862,13 +965,13 @@ private static async Task ReadLoopAsync(ConnectionContext connection, RuntimeCon catch (FrameTooLargeException) { var oversized = Protocol.SerializeProtocolError(new ProtocolError(ProtocolErrorCode.FrameTooLarge, "frame_too_large", null)); - await SendSafeAsync(connection, oversized); + await SendSafeAsync(connection, oversized, server); await Task.Delay(100, connection.State.Cts.Token); if (connection.Socket.State == WebSocketState.Open) { await connection.Socket.CloseAsync(WebSocketCloseStatus.MessageTooBig, "frame_too_large", CancellationToken.None); } - SyncServer.Instance.CancelConnection(connection, "frame_too_large"); + server.CancelConnection(connection, "frame_too_large"); break; } @@ -888,21 +991,21 @@ await connection.Socket.CloseOutputAsync( if (!Protocol.CheckFrameSize(received.Payload.Length, config)) { var error = Protocol.SerializeProtocolError(new ProtocolError(ProtocolErrorCode.FrameTooLarge, "frame_too_large", null)); - await SendSafeAsync(connection, error); + await SendSafeAsync(connection, error, server); await connection.Socket.CloseAsync(WebSocketCloseStatus.MessageTooBig, "frame_too_large", CancellationToken.None); - SyncServer.Instance.CancelConnection(connection, "frame_too_large"); + server.CancelConnection(connection, "frame_too_large"); break; } var parse = Protocol.ParseClientMessage(received.Payload, config); if (!parse.IsSuccess) { - SyncServer.Instance.Logger?.LogSecurityEvent("reject", + server.Logger.LogSecurityEvent("reject", ("username", connection.Username), ("code", parse.Error?.CodeName ?? "invalid_message"), ("bytes", received.Payload.Length)); var error = Protocol.SerializeProtocolError(parse.Error!); - await SendSafeAsync(connection, error); + await SendSafeAsync(connection, error, server); continue; } @@ -913,19 +1016,19 @@ await connection.Socket.CloseOutputAsync( var hub = connection.Hub; if (hub is null) { - SyncServer.Instance.CancelConnection(connection, "user_loop_unavailable"); + server.CancelConnection(connection, "user_loop_unavailable"); break; } var decision = hub.ClassifyClip(clip, connection); if (decision == RecoveryDecision.QueueFull) { - SyncServer.Instance.CancelConnection(connection, "recovery_queue_full"); + server.CancelConnection(connection, "recovery_queue_full"); } else if (decision == RecoveryDecision.ProcessNow && !hub.TryWriteJob(new ClipJob(connection, clip))) { - SyncServer.Instance.CancelConnection(connection, "user_loop_unavailable"); + server.CancelConnection(connection, "user_loop_unavailable"); } break; case MessageKind.Pong: @@ -935,13 +1038,13 @@ await connection.Socket.CloseOutputAsync( ProtocolErrorCode.InvalidMessage, "Pong received without an outstanding ping.", null)); - await SendSafeAsync(connection, unsolicitedPong); + await SendSafeAsync(connection, unsolicitedPong, server); continue; } if (connection.Hub is null || !connection.Hub.TryWriteJob(new PongJob(connection, (ClientPong)parse.Message!))) { - SyncServer.Instance.CancelConnection(connection, "user_loop_unavailable"); + server.CancelConnection(connection, "user_loop_unavailable"); } break; } @@ -964,11 +1067,11 @@ private static async Task ConnectionSendLoopAsync(ConnectionContext connection) catch (WebSocketException) { } } - private static async Task SendSafeAsync(ConnectionContext connection, byte[] payload) + private static async Task SendSafeAsync(ConnectionContext connection, byte[] payload, SyncServer server) { if (!connection.State.TryEnqueueSend(payload)) { - SyncServer.Instance.EnqueueImmediateClose(connection, "server_busy"); + server.EnqueueImmediateClose(connection, "server_busy"); return; } } diff --git a/TextCascade.Server/TextCascade.Server.csproj b/TextCascade.Server/TextCascade.Server.csproj index db51706..61d9f16 100644 --- a/TextCascade.Server/TextCascade.Server.csproj +++ b/TextCascade.Server/TextCascade.Server.csproj @@ -4,7 +4,7 @@ net10.0 enable enable - 0.2.1 + 0.2.5 TextCascade.Server true diff --git a/deploy/textcascade-server.service b/deploy/textcascade-server.service index b479b03..fac8210 100644 --- a/deploy/textcascade-server.service +++ b/deploy/textcascade-server.service @@ -5,11 +5,13 @@ After=network-online.target [Service] Type=simple -User=root -Group=root +User=textcascade +Group=textcascade WorkingDirectory=/opt/textcascade-server EnvironmentFile=/etc/textcascade/textcascade.env -ExecStart=/usr/bin/dotnet /opt/textcascade-server/TextCascade.Server.dll serve --config /etc/textcascade/textcascade.toml +ExecStart=/opt/textcascade-server/TextCascade.Server serve --config /etc/textcascade/textcascade.toml +StateDirectory=textcascade +ConfigurationDirectory=textcascade Restart=on-failure RestartSec=5s UMask=0077 diff --git a/deploy/textcascade.toml b/deploy/textcascade.toml index 843b93c..f9e6fa4 100644 --- a/deploy/textcascade.toml +++ b/deploy/textcascade.toml @@ -5,3 +5,4 @@ certificate_path = "/etc/cert/test.pem" [files] users_file = "/etc/textcascade/users.json" +state_file = "/var/lib/textcascade/textcascade.state.json" From eeefa35a195d2e4c6b86e5c79abd94933ddaf690 Mon Sep 17 00:00:00 2001 From: long45343 <1725334094@qq.com> Date: Sat, 22 Aug 2026 14:32:26 +0800 Subject: [PATCH 06/32] add spec of server --- docs/server-spec.md | 857 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 857 insertions(+) create mode 100644 docs/server-spec.md diff --git a/docs/server-spec.md b/docs/server-spec.md new file mode 100644 index 0000000..76ed45e --- /dev/null +++ b/docs/server-spec.md @@ -0,0 +1,857 @@ +# TextCascade 轻量文本同步服务端规格 + +状态：函数级设计已定稿，已按审查决策台账修订 +日期：2026-08-18 +协议目标：不兼容原 ClipCascade，只做轻量、可靠、高性能的文本最新值同步 + +## 1. 目标与非目标 + +### 1.1 目标 + +- 仅同步文本最新值：每用户只保存一份当前文本，不保存历史。 +- 无数据库：账号使用 `users.json`，文本与版本只在内存中。 +- 服务端重启可恢复：客户端用无状态 token 重连，并在恢复窗口内上报 snapshot。 +- 低空闲资源占用：无数据库轮询、无磁盘写入、无 WebUI、无 metrics endpoint。 +- 明确边界：协议错误显式返回，慢连接被隔离或断开，绝不拖垮整个服务。 +- 三端协议实现：服务端与桌面端使用 C#，Android 端使用 Kotlin；三端手写模型，由服务端契约测试约束。 + +### 1.2 非目标 + +- 不兼容原 ClipCascade 的 Spring、CSRF、JSESSIONID、STOMP 协议。 +- 不支持图片、文件、剪贴板历史、离线消息队列、逐设备 ACK 状态。 +- 不支持多实例分布式部署、数据库后端、管理后台或 WebUI。 +- 不提供 Prometheus metrics；内部只保留轻量计数器。 +- 不做 Socket.IO；使用原生 WebSocket。 + +## 2. 已定架构 + +### 2.1 运行时与进程 + +- 技术栈：ASP.NET Core Minimal API + Kestrel 原生 WebSocket。 +- 目标框架：`net10.0`；产品版本采用 SemVer，从 `0.1.0` 开始。 +- 进程模型：单进程；生产环境由 systemd 或 Windows Service 托管并负责崩溃自动重启。 +- TLS：Kestrel 直接终止 TLS；不提供生产/开发模式开关，所有部署都禁止明文 HTTP 登录。 +- 部署产物：框架依赖单文件；目标机必须预装对应 .NET Runtime。 + +### 2.2 核心对象 + +- `ConnectionContext`：不可变稳定属性，包括连接 ID、用户名、clientId、socket、认证信息。 +- `ConnectionStateBag`：可变运行时状态，包括 lastSeen、关闭标记、发送 Channel；修改必须收敛到少数明确函数。 +- `UserHub`：每个在线用户一个 hub，持有最新值、版本号、幂等队列、令牌桶与用户 Channel。 +- `UserRegistry`：`ConcurrentDictionary`，不同用户天然并发。 +- `LatestText`：不可变 record，包含 payload、version、来源、更新时间；更新即替换引用。 + +### 2.3 并发模型 + +1. 每个连接一个独立 `ReadLoopAsync`。 +2. 读循环只负责收帧、解析、验证，然后把用户级 job 投递到 UserHub Channel。 +3. 每个 UserHub 一个 `UserLoopAsync` 单消费者，串行处理该用户的 clip、连接、断开与恢复 job。 +4. 广播时只序列化一次 UTF-8 字节，并把同一份字节投递到每个连接的有界发送 Channel。 +5. 每个连接一个 `ConnectionSendLoopAsync`，慢连接只积压自己的队列。 +6. 发送队列满立即取消该连接，不等待 drain，不补发应用层 error 或 WebSocket close frame。 + +## 3. 配置与用户 + +### 3.1 配置函数 + +- `CreateDefaultConfig()`：内置安全默认值。 +- `LoadTomlConfig(path)`：读取可选 TOML 配置并覆盖默认值。 +- `ApplyEnvironmentOverrides(config)`：环境变量覆盖敏感项与非默认部署值。 +- `ValidateConfig(config)`：启动时强校验；非法值 fail-fast。 +- `BuildWebHost(config)`：创建 Minimal API 应用并绑定 Kestrel。 + +默认配置文件示例： + +```toml +[server] +bind = "0.0.0.0" +port = 8443 +certificate_path = "certs/server.pfx" + +[auth] +token_ttl_days = 30 +token_secret_env = "TEXTCASCADE_TOKEN_SECRET" +argon2_memory_kib = 19456 +argon2_iterations = 2 +argon2_parallelism = 1 + +[limits] +max_text_bytes = 524288 +max_frame_bytes = 589824 +send_queue_capacity = 16 +seen_id_capacity = 64 +hello_timeout_seconds = 5 +heartbeat_interval_seconds = 30 +heartbeat_timeout_seconds = 60 +snapshot_window_seconds = 3 +snapshot_total_bytes = 4194304 +recovery_clip_queue_capacity = 16 + +[rate_limit] +login_ip_per_minute = 10 +login_user_per_minute = 5 +max_keys = 10000 +clip_burst = 10 +clip_tokens_per_second = 2 + +[files] +users_file = "users.json" +``` + +规则： + +- 服务端不提供 production/environment 模式开关，安全校验不因部署环境降低。 +- `token_secret_env` 指向环境变量名；token secret 不写入 TOML。 +- token secret 必须由环境变量提供，长度至少 32 字节；缺失或过短时启动失败。 +- TLS 始终启用；`certificate_path` 必须指向服务端可用证书。 +- 证书仅支持无密码格式：`.pem` / `.crt` 必须是包含叶证书与未加密私钥的 PEM bundle，`.pfx` 必须可无密码加载；带密码证书不支持，遇到需要密码的 PFX 时启动失败。 +- TOML 使用宽松解析：必须以 UTF-8 读取；未知键忽略并输出 warning；重复键采用后值并输出 warning；结构或类型非法仍 fail-fast。 +- `max_frame_bytes` 必须大于 `max_text_bytes`，差额留给 JSON 协议头。 +- 所有容量与时间配置必须大于 0，心跳超时必须大于心跳间隔。 + +### 3.2 用户存储 + +文件：`users.json` + +```json +{ + "nextTokenVersion": 3, + "users": [ + { + "username": "alice", + "passwordHash": "$argon2id$...", + "tokenVersion": 1, + "disabled": false + } + ] +} +``` + +函数： + +- `LoadUsers(path)`：启动时全量读取。 +- `ValidateUsers(users)`：校验 `nextTokenVersion` 必填且大于所有用户 `tokenVersion`；校验用户名唯一、哈希格式、正数 `long` tokenVersion 与 disabled 字段。 +- `BuildUserLookup(users)`：构造只读用户查找表。 + +说明： + +- 不热加载用户文件，避免在线连接认证状态与文件状态竞态。 +- `tokenVersion` 不是软件版本，而是账号 token 作废计数器。 +- `tokenVersion` 与 `nextTokenVersion` 使用有符号 64 位整数（`long`），只允许正数，创建与递增时溢出即 fail-fast。 +- `nextTokenVersion` 是全局水位；新增用户取当前水位作为 `tokenVersion`，随后水位加一。 +- `revoke-tokens` 将目标用户 `tokenVersion` 更新为当前水位，随后水位加一，保证未来新建任何账号都不会复用已撤销版本。 +- CLI 写入用户文件前必须先通过 `ValidateUsers(users)`；水位递增溢出或校验失败时放弃替换并保留原文件。 +- CLI 使用 PID 锁文件实现单实例：同一时刻只允许一个 TextCascade CLI 进程运行；检测到仍存活的其他 CLI 进程时，新实例直接失败退出。 +- PID 锁文件覆盖 CLI 生命周期；实现必须识别并回收陈旧 PID、进程已退出但锁文件残留的情况，并在 Windows 与 Linux 上行为一致。 +- 支持直接删除用户条目，不保留墓碑；之后重建同名用户时从全局水位取新 `tokenVersion`，因此不会落入旧 token 的版本空档。 +- 删除或修改用户文件后需重启服务生效；重启后已删除用户不存在，其 token 因用户查找失败而失效。 + +### 3.3 用户 CLI + +入口在同一服务端可执行文件中，不提供 WebUI。 + +```bash +TextCascade.Server user add --username alice +TextCascade.Server user passwd --username alice +TextCascade.Server user disable --username alice +TextCascade.Server user enable --username alice +TextCascade.Server user delete --username alice +TextCascade.Server user revoke-tokens --username alice +TextCascade.Server user list +TextCascade.Server user hash +``` + +函数： + +- `RunCli(args)`：识别 `user` 子命令。 +- `CommandAddUser()`：生成 Argon2id 哈希，从全局水位分配 `tokenVersion`，并原子重写用户文件。 +- `CommandDeleteUser()`：直接删除用户条目并原子重写文件，不写入墓碑。 +- `CommandHashPassword()`：只输出密码哈希。 +- `CommandListUsers()`：只输出用户名、禁用状态与 tokenVersion，不输出哈希。 + +CLI 写入 `users.json` 时先持有 PID 单实例锁，再使用临时文件加原子替换；服务端运行中修改文件不会热生效，需重启。服务端不写入用户文件。 + +## 4. HTTP API + +### 4.1 登录 + +```http +POST /api/v1/login +Content-Type: application/json +``` + +请求： + +```json +{ + "username": "alice", + "password": "raw-password" +} +``` + +函数： + +- `MapLoginEndpoint()`：薄 Endpoint，只处理 HTTP 请求与响应。 +- `AuthService.LoginAsync()`：执行认证、tokenVersion 校验、token 签发。 +- `ParseLoginRequest()`：限制请求体 16KB、JSON 深度 3。 +- `AuthenticateUser()`：Argon2id 常数时间校验。 +- `CreateLoginFailure()`：统一返回 `invalid_credentials`。 +- `CreateRateLimitResult()`：统一返回 `429`。 + +成功： + +```json +{ + "token": "", + "expiresAtUtc": "2026-09-17T00:00:00Z", + "protocolVersion": 1, + "maxTextBytes": 524288, + "helloTimeoutSeconds": 5, + "heartbeatIntervalSeconds": 30, + "heartbeatTimeoutSeconds": 60 +} +``` + +失败： + +```http +401 Unauthorized +``` + +```json +{ + "error": "invalid_credentials", + "message": "Invalid username or password." +} +``` + +规则： + +- 客户端通过 TLS 发送原始密码；客户端不做 Argon2id。 +- 用户不存在与密码错误返回相同错误，避免枚举用户。 +- Argon2id 参数变化时，登录路径只调用 `NeedsRehash()` 输出结构化 warning，不重写 `users.json`；用户通过 CLI `passwd` 设置新密码时才生成当前参数的哈希。 +- 登录限流命中返回 `429 Too Many Requests`，错误码 `rate_limited`。 + +### 4.2 Token + +格式： + +```text +base64url(payload).base64url(hmac-sha256(payload, secret)) +``` + +payload： + +```json +{ + "sub": "alice", + "ver": 1, + "iat": 1760000000, + "exp": 1762592000 +} +``` + +Token JSON 规则： + +- 服务端签发时按 `sub`、`ver`、`iat`、`exp` 固定字段序输出最小化 UTF-8 JSON。 +- 验证时字段顺序无关，但拒绝重复字段与未知字段。 +- `sub` 是非空用户名；`ver`、`iat`、`exp` 均为有符号 64 位整数范围内的正整数；`exp` 必须大于 `iat`。 +- 数字不得以小数、指数或字符串形式表示。 + +函数： + +- `CreateTokenPayload(user, now, ttl)`：生成 `sub`、`ver`、`iat`、`exp`。 +- `SignToken(payload, secret)`：HMAC-SHA256。 +- `VerifyToken(compact, secret, now, userLookup)`：验签、验过期、验用户存在、验 tokenVersion。 + +规则： + +- HMAC 比较必须常数时间。 +- token 默认 30 天，可由配置调整。 +- token 无服务端状态，服务端重启后仍可验证。 +- 用户被禁用、删除或 tokenVersion 变化后，重启服务即可拒绝旧 token；删除后重建同名用户会从全局水位分配更高 tokenVersion。 + +### 4.3 登录限流 + +函数： + +- `TryConsumeLoginLimit(ip, username, now)`：进程内滑动窗口。 +- `ResetUserLoginLimit(username)`：仅在认证成功后清空该用户名窗口。 + +策略： + +- IP 与用户名双维度限流，任一超限即拒绝。 +- 默认每 IP 每分钟 10 次，每用户名每分钟 5 次。 +- 用户名维度统计所有登录请求，无论认证成功或失败；认证成功后清空该用户名窗口。 +- IP 维度统计所有登录请求；认证成功不清空 IP 窗口。 +- 限流器设置最大 key 数，提供确定内存上限；达到上限时先清理全部过期项，仍满则拒绝新 key 的登录请求并返回 `429 rate_limited`。 +- 未达到上限时只保存窗口内时间戳，过期项在该 key 被访问时惰性清理；已有 key 的请求不创建新条目。 +- 单实例部署下不做分布式限流。 +- 已知取舍：持有正确密码的攻击者可通过高频成功登录占满目标用户名窗口；v1 接受该风险，以换取更简单的计数与重置规则。 + +### 4.4 健康检查 + +```http +GET /health +``` + +函数： + +- `MapHealth()`：进程能响应即返回 `200 OK`。 + +返回： + +```json +{ + "status": "ok" +} +``` + +不暴露连接数、内存、用户数等内部统计。 + +## 5. WebSocket 协议 + +### 5.1 连接建立 + +```http +GET /api/v1/sync +Authorization: Bearer +Sec-WebSocket-Protocol: textcascade.v1 +Upgrade: websocket +``` + +函数： + +- `AuthenticateUpgradeRequest(httpContext)`：升级前验 token。 +- `SelectSubProtocol(requestProtocols)`：只接受 `textcascade.v1`。 +- `AcceptAuthenticatedSocket(httpContext)`：认证与版本都合法才升级。 + +规则： + +- token 放 Authorization header，不进 URL。 +- token 无效、过期、用户禁用或 tokenVersion 不匹配时，不升级 WebSocket，直接返回 `401`。 +- 子协议不匹配返回 `400`。 +- 认证成功后，客户端必须在 `hello_timeout_seconds` 内发送 hello，用于注册设备与上报 snapshot；默认 5 秒。 +- hello 通过验证前，连接不进入广播列表；该超时由服务端统一计时。 + +### 5.2 Client Hello + +```json +{ + "type": "hello", + "clientId": "stable-device-id", + "clientName": "Windows-Desktop", + "lastServerVersion": 128, + "snapshot": { + "payload": "...", + "encrypted": true, + "hash": "client-local-hash", + "localModifiedAtUtc": "2026-08-18T08:00:00Z" + } +} +``` + +函数： + +- `ParseHello(frame)`：解析 hello。 +- `ValidateHello(hello)`：校验 clientId、clientName、snapshot 与版本字段。 +- `CreateConnectionContext(socket, user, hello)`：创建不可变连接上下文。 + +字段： + +- `clientId`：稳定设备 ID，长度 1-128。 +- `clientName`：可选，长度 0-128。 +- `lastServerVersion`：客户端见过的最后服务端版本；未知为 0。 +- `snapshot`：可选。仅在进程启动后的全局恢复窗口内用于选举最新值；恢复窗口结束后只执行完整协议校验，校验通过即丢弃，不写入最新值。clip 是唯一文本写入路径。 + +### 5.3 Server Welcome + +```json +{ + "type": "welcome", + "protocolVersion": 1, + "latest": { + "version": 128, + "payload": "...", + "encrypted": true, + "hash": "...", + "fromClientId": "android-a", + "updatedAtUtc": "2026-08-18T07:59:58Z" + } +} +``` + +函数： + +- `CreateWelcome(latest)`：构造欢迎消息。 +- `SerializeMessage(message)`：System.Text.Json 序列化为 UTF-8。 + +规则： + +- 服务端内存无最新值时 `latest` 为 `null`。 +- 恢复窗口内可先等待 snapshot 选举，再发送 welcome。 +- 客户端收到相同 hash 或相同版本时可本地去重，不写剪贴板；hash 只用于本地剪贴板去重，服务端新旧值以版本为准。 + +### 5.4 发布文本 + +客户端： + +```json +{ + "type": "clip", + "id": "client-generated-unique-id", + "payload": "...", + "encrypted": true, + "hash": "..." +} +``` + +服务端广播给同用户除发送方连接外的其他在线连接： + +```json +{ + "type": "clip", + "version": 129, + "id": "client-generated-unique-id", + "payload": "...", + "encrypted": true, + "hash": "...", + "fromClientId": "windows-a", + "fromClientName": "Windows-Desktop", + "updatedAtUtc": "2026-08-18T08:01:00Z" +} +``` + +发送方收到 ACK： + +```json +{ + "type": "clip_ack", + "id": "client-generated-unique-id", + "version": 129, + "updatedAtUtc": "2026-08-18T08:01:00Z" +} +``` + +函数： + +- `ValidateClipMessage(message)`：单函数完整验证，内部按结构、语义、资源顺序早拒绝。 +- `CheckFrameSize(frameLength, config)`：WebSocket 完整帧字节数硬限制。 +- `CheckPayloadSize(payloadUtf8Length, config)`：文本字段独立限额。 +- `UserHub.TryDuplicate(id)`：用户级环形队列去重。 +- `RememberId(id)`：记录最近消息 ID。 +- `TryAcquireClipToken(now)`：用户级令牌桶。 +- `NextVersion(current)`：服务端权威 `ulong` 自增；溢出抛 fatal。 +- `WithVersion(latest, next)`：构造新的不可变 LatestText。 +- `BroadcastAsync(userHub, latest)`：一次序列化、多连接投递。 + +规则： + +- 客户端不携带版本号；版本由服务端按用户处理顺序生成。 +- `id` 重复时不生成新版本，先返回原 ACK，且不消耗用户级令牌桶；重复 ACK 仍必须进入发送方的有界发送队列，队列满时按慢连接取消。 +- 空文本、非法 UTF-8、结构缺字段、超帧、超文本、限流超限均拒绝。 +- `payload` 对服务端 opaque；`encrypted=true` 时服务端不解析内容。 +- 发送队列容量按消息条数计算，默认 16；队列满立即取消连接，不补发 error 或 close frame。 +- 慢设备延迟到达的旧 clip 仍会获得新版本并覆盖最新值；这是最新值语义的预期行为，客户端需自行处理可能的回滚。 + +### 5.5 心跳 + +服务端定时发送应用层 JSON ping： + +```json +{ + "type": "ping", + "serverTimeUtc": "2026-08-18T08:02:00Z" +} +``` + +客户端必须返回： + +```json +{ + "type": "pong", + "clientTimeUtc": "2026-08-18T08:02:00Z" +} +``` + +函数： + +- `StartHeartbeatTimer()`：统一扫描所有连接。 +- `SendPing(connection)`：发送 ping。 +- `MarkPongReceived(connection, now)`：更新 lastSeen。 +- `CloseExpiredConnections()`：超时未收到 pong 则取消连接。 + +说明： + +- 心跳使用应用层 JSON 消息，便于服务端记录 pong 时间并在三端保持一致行为。 +- 默认 30 秒发送一次，60 秒未收到 pong 判定死亡。 +- 统一扫描器代替每连接独立 timer，降低空闲调度开销。 +- 统一扫描器固定每 1 秒扫描一次；hello 与心跳超时允许 0-1 秒的额外检测延迟，不提供独立配置项。 + +### 5.6 错误 + +```json +{ + "type": "error", + "code": "text_too_large", + "message": "Text exceeds maxTextBytes.", + "referenceId": "client-generated-unique-id" +} +``` + +函数： + +- `ParseResult`：成功或错误显式返回。 +- `CreateProtocolError(code, message, referenceId)`：构造错误。 +- `SendProtocolErrorAsync(connection, error)`：发送可继续错误。 +- `EnqueueImmediateClose(connection, reason)`：跳过 error 与 close frame，直接进入统一取消路径；仅用于发送队列满等无法安全写入的场景。 + +错误码： + +| code | 含义 | 连接处理 | +|---|---|---| +| `invalid_message` | JSON 结构或字段非法 | 可继续 | +| `text_too_large` | 文本字段超限 | 可继续 | +| `frame_too_large` | 完整帧超限 | 关闭 1009 | +| `empty_text` | 空文本 | 可继续 | +| `rate_limited` | 用户级发送限流 | 可继续 | +| `hello_timeout` | 未按时发送 hello | 先发 error，关闭 1008 | +| `server_busy` | 发送队列拥塞 | 立即取消；该错误不保证发送 | + +错误处理顺序： + +- 需要关闭的错误必须先发送对应应用层 error 帧，再执行 WebSocket close；同一连接同类错误只触发一次关闭流程。 +- 慢连接发送队列满时不补发应用层 error，也不写 close frame，直接进入取消路径；`server_busy` 语义对客户端不可靠，客户端应靠重连兜底。 + +可预期协议错误走 Result；不可预期异常仍由顶层兜底并进入统一清理。 + +### 5.7 关闭与清理 + +函数： + +- `CancelConnection(connection, reason)`：唯一取消入口，触发 CancellationTokenSource。 +- `FinallyCloseConnection(connection)`：统一关闭 socket、停止任务、从 UserHub 摘除。 +- `RemoveEmptyHub(userRegistry, userHub)`：最后一个连接断开后清理空 hub；全局恢复窗口内不执行空 hub 清理，窗口收尾时仍无连接的 hub 才移除。 + +| close code | 含义 | +|---:|---| +| `1000` | 正常关闭 | +| `1001` | 服务端重启或维护 | +| `1008` | 策略关闭，例如 hello 超时 | +| `1009` | 帧过大 | + +hello 超时先发送 `hello_timeout` error，再以 `1008` close；发送队列满则不补发 error，直接取消。`1013` 与 `4408` 不是本协议 close code，客户端不得依赖。 + +心跳超时、慢连接、客户端断开、协议异常都必须汇入同一 CTS 取消路径，避免重复清理和资源泄漏。 + +## 6. 最新值与恢复 + +### 6.1 正常运行 + +每个用户只保存一个 `LatestText`： + +- `payload` +- `version` +- `hash` +- `fromClientId` +- `fromClientName` +- `updatedAtUtc` + +处理顺序： + +1. 读循环收帧并检查帧大小。 +2. JSON 解析与 `ValidateClipMessage`。 +3. 投递到用户 Channel。 +4. 用户单消费者执行幂等检查与令牌桶。 +5. `NextVersion` 生成新版本。 +6. 不可变替换最新值。 +7. 广播给除发送者外的连接，并向发送者返回 ACK。 + +这是最新值语义，不是可靠队列语义。离线设备不补历史，重连后只拿当前最新值。 + +### 6.2 服务端重启恢复 + +恢复窗口从服务端进程启动时间起算，结束时间为 `processStartTime + snapshot_window_seconds`。该窗口对全部用户统一生效，不按 UserHub 创建时间或首个 hello 到达时间重新计算。 + +函数： + +- `CollectSnapshotsAsync(userHub, window)`：收集 3 秒恢复窗口内的 snapshot。 +- `SelectSnapshotWinner(candidates)`：按确定性规则选举。 +- `RestoreLatestText(userHub, winner)`：恢复最新值与版本基准。 + +选举规则： + +1. `lastServerVersion=0` 的 snapshot 不参与选举；只过滤出正版本候选。 +2. 若没有正版本候选，恢复结果为空，不下发最新值。 +3. 在正版本候选中优先选择 `lastServerVersion` 最大者。 +4. 若版本相同，选择 `localModifiedAtUtc` 最新者。 +5. 若仍相同，选择 `clientId` 字典序更大者，保证结果确定。 + +恢复规则： + +- winner 的 `LatestText.version` 使用其正版本 `lastServerVersion`，不额外加一；恢复版本不额外设置上限，`NextVersion` 溢出 fatal 保留为理论兜底。 +- 无正版本候选时恢复为空；下一条服务端 clip 版本为 1。 +- 恢复窗口内只收集 snapshot；合法 clip 不参与选举，进入独立有界恢复队列。 +- 每用户 snapshot 预算只统计候选 `snapshot.payload` 的 UTF-8 字节数总和；上限为 `snapshot_total_bytes`，达到上限后拒绝新的 snapshot 并保持已有候选不变。元数据开销不占用该预算，由在线连接数量约束。 +- 恢复队列容量为 `recovery_clip_queue_capacity`；队列满时关闭相应连接，避免内存无界增长；连接断开则丢弃其已排队 clip。 +- 恢复窗口结束后，先根据 snapshot 选举 winner 并恢复最新值，再按到达顺序串行处理恢复队列中的 clip。 +- 窗口结束后广播 welcome 或恢复后的最新值，客户端按 hash 与版本去重。 +- 错过窗口的慢设备之后仍可发送 clip；该 clip 按到达顺序获得新版本并覆盖当前最新值。最后写入者胜，服务端不尝试识别或拒绝“逻辑上更旧”的 clip。 + +服务端重启后的完整链路： + +1. 服务端停机前广播 `bye` 并以 `1001` 关闭连接。 +2. 客户端识别服务端维护，使用无状态 token 直接重试 WebSocket。 +3. 服务端重启后 token secret 与 tokenVersion 未变，token 仍可验证。 +4. 客户端 hello 上报 snapshot。 +5. 3 秒窗口选举 winner。 +6. 服务端恢复最新值并继续同步。 + +### 6.3 慢连接 + +每个连接有独立有界发送 Channel： + +- 默认容量 16 条消息。 +- `TryWrite` 失败即判定慢连接。 +- 立即调用 `CancelConnection`，不等待 drain，不补发应用层 error，也不写 close frame。 +- 发送循环观测取消后直接退出；`OperationCanceledException` 与非取消异常都汇入统一清理路径。 +- 该场景使用 abort/dispose 释放底层 socket，不执行 graceful WebSocket close 握手。 +- 客户端重连后通过 welcome 拿最新值，不补发中间消息。 + +慢连接不能阻塞用户单消费者，也不能影响同用户其他连接。 + +## 7. 优雅停机 + +函数： + +- `BroadcastByeAsync(reason)`：向所有连接发送 `bye`。 +- `ShutdownAsync(CancellationToken)`：关闭连接、停止任务、等待短暂收尾。 + +流程： + +1. 收到 SIGTERM、Ctrl+C 或服务停止请求。 +2. 停止接受新连接。 +3. 广播： + +```json +{ + "type": "bye", + "reason": "server_shutdown" +} +``` + +4. 以 close code `1001` 关闭所有连接。 +5. 等待最多 2 秒，让 close frame 尽量发出。 +6. 取消所有连接 CTS。 +7. 清理 UserHub 与后台任务。 +8. 进程退出，由系统服务管理器重启。 + +## 8. 日志与安全 + +### 8.1 结构化日志 + +函数： + +- `LogSecurityEvent()`：记录登录、认证失败、限流与禁用用户事件。 +- `RedactSensitive(value)`：统一脱敏。 + +规则： + +- 使用 `ILogger` 结构化字段。 +- 密码绝不记录。 +- token 只可记录短前缀，默认不记录。 +- clip payload 与 hash 不记录；clip 事件只记 version、字节数、encrypted、来源设备。 +- Authorization header 不进入访问日志。 + +关键事件： + +| 事件 | 字段 | +|---|---| +| login | username, ip, success, reason | +| connect | username, clientId, connectionId | +| disconnect | username, clientId, reason, durationMs | +| clip | username, version, bytes, fromClientId, encrypted | +| reject | username, code, bytes | +| server_stop | reason, activeConnections | + +### 8.2 传输与输入安全 + +- 生产只允许 HTTPS/WSS。 +- TLS 最低 1.2，推荐 1.3。 +- 不启用 CORS。 +- 不设置 Cookie，无 CSRF 面。 +- 登录请求体上限 16KB。 +- WebSocket 完整帧与文本字段分别限额。 +- JSON 深度限制为 3。 +- 协议消息只接受契约定义字段；重复字段与未知字段拒绝。若未来新增可选字段，必须提升或明确协议兼容策略。 + +## 9. 性能目标 + +| 指标 | v1 目标 | +|---|---:| +| 基础进程内存 | < 50 MB | +| 100 个空闲连接内存增量 | < 20 MB | +| 1KB 文本 LAN 广播 p95 | < 30 ms | +| 512KB 文本 LAN 广播 p95 | < 250 ms | +| 空闲 CPU | 接近 0%，心跳扫描除外 | +| 冷启动时间 | < 2 s | +| 服务端重启恢复窗口 | 3 s | + +设计依据： + +- 无数据库连接池与定时磁盘 IO。 +- 每用户一个 Channel 单消费者，避免锁与异步持锁。 +- 每次广播只做一次 UTF-8 序列化。 +- 每连接发送队列有界，内存上限可预测。 +- 空闲连接只保留上下文、发送 Channel 与心跳扫描状态。 + +## 10. 测试计划 + +### 10.1 纯单元测试 + +重点函数： + +- `HashPassword`、`VerifyPassword`、`NeedsRehash` +- `SignToken`、`VerifyToken`、tokenVersion 撤销 +- Token 重复字段、未知字段、非法数字与非法范围 +- 全局 `nextTokenVersion` 水位递增、删除后重建同名用户、溢出 fail-fast +- CLI PID 单实例锁：活跃进程互斥、陈旧 PID 与锁文件残留处理 +- `TryConsumeLoginLimit` +- 登录限流成功重置用户名窗口但不重置 IP 窗口 +- 登录限流最大 key 数：先清理过期项，仍满时拒绝新 key +- `TryAcquireClipToken` +- `ValidateClipMessage` +- `CheckFrameSize`、`CheckPayloadSize` +- `UserHub.TryDuplicate`、`RememberId` +- 重复 `id` 不消耗令牌桶，重复 ACK 仍受有界发送队列约束 +- `NextVersion`、`WithVersion` +- `SelectSnapshotWinner`，包括 `lastServerVersion=0` 不参与、无正版本候选恢复为空、同版本时间与 clientId 平局 + +认证测试注入假哈希器，避免 Argon2id 拖慢常规单元测试。 + +### 10.2 CI 集成测试：内存 WebSocket + +使用 `CreateSocketPair()` 建立内存连接，快速稳定覆盖： + +- 登录成功与失败。 +- 升级前认证失败不建立 WebSocket。 +- 子协议不匹配拒绝。 +- hello 超时。 +- 登录后仅记录 `NeedsRehash` warning，不重写用户文件。 +- 部分设备重连时的 snapshot 选举，包括 `lastServerVersion=0` 被忽略与无正版本候选恢复为空。 +- 恢复窗口从进程启动时间全局起算；窗口内 clip 排队、队列容量、payload 字节预算与窗口后处理顺序。 +- 恢复窗口结束后 snapshot 仅校验后丢弃。 +- 两客户端同用户广播与发送方 ACK；相同 `clientId` 的其他连接仍收到广播，仅发送方连接被排除。 +- 不同用户隔离。 +- 幂等 ID。 +- 慢连接队列满立即取消且不影响同用户其他接收方。 +- 停机 bye 与 1001。 +- 重启 snapshot 选举。 + +### 10.3 本地网络测试：真实 localhost TCP + +标记：`Category=NetworkIntegration`，CI 默认跳过。 + +```bash +dotnet test --filter Category=NetworkIntegration +``` + +使用 `StartTestServer()` 启动完整 Kestrel，覆盖： + +- TLS 证书与 WSS。 +- HTTP 升级。 +- 随机端口绑定。 +- 真实帧分片。 +- 登录、建连、发送文本、另一客户端接收。 +- 服务端重启后 token 直连重连与 snapshot 恢复。 + +### 10.4 契约测试与压测 + +- 服务端维护典型 JSON 样本，约束三端协议字段与行为。 +- 契约样本必须覆盖 JSON 深度 3、重复字段、未知字段、非法数字与非法 UTF-8。 +- 独立 `TextCascade.Server.Benchmark` 项目执行压测，不进入生产产物。 +- 压测场景包括空闲连接、1000 并发连接、小文本广播、512KB 文本广播与慢消费者。 + +## 11. 客户端适配要求 + +客户端需要实现： + +1. `POST /api/v1/login` 获取 token。 +2. Authorization header + `textcascade.v1` 子协议建立 WebSocket。 +3. 在登录响应返回的 `helloTimeoutSeconds` 内发送 hello，并携带本地 snapshot。 +4. 应用层 ping/pong。 +5. clip 发送、ACK、接收。 +6. 保存服务端 version，重连时上报 `lastServerVersion`。 +7. 收到相同 hash 或相同版本时不写剪贴板；收到更晚到达的旧 clip 仍可能覆盖本地文本。 +8. `1001` 后按服务端维护重连；token 未过期时优先直接重连。 +9. `401`、token 过期或 tokenVersion 失效时重新 HTTP 登录。 +10. 重连退避建议：1s、2s、5s、10s、30s、60s，之后固定 60s；收到 1001 时初期退避更温和。 + +保留客户端原有能力： + +- 本地剪贴板监听。 +- hash 去重。 +- 远端写入后的本地事件抑制。 +- 密码派生 AES-GCM 加密。 +- 密码安全保存与自动登录。 + +## 12. 实施里程碑 + +### M1：协议骨架 + +- 配置加载与 fail-fast 校验。 +- `users.json` 与 CLI。 +- 登录端点。 +- HMAC token。 +- WebSocket 升级认证与子协议协商。 +- hello/welcome。 +- 文本广播与 ACK。 +- 纯单元测试与内存集成测试。 + +### M2：可靠性 + +- UserHub Channel 单消费者。 +- 有界发送队列与立即取消策略。 +- 幂等 ID。 +- 服务端版本号与不可变最新值。 +- 应用层心跳。 +- 统一 CTS 清理。 + +### M3：恢复与真实网络 + +- 优雅停机 bye/1001。 +- 3 秒 snapshot 恢复窗口。 +- snapshot 总字节数与恢复 clip 队列容量。 +- token 过期与 tokenVersion 撤销测试。 +- 本地真实 TCP/TLS 集成测试。 +- 重启后多端收敛测试。 + +### M4：生产化 + +- Kestrel TLS。 +- 结构化日志与脱敏。 +- 登录与消息限流。 +- 框架依赖单文件发布。 +- 独立 benchmark。 +- 部署文档与 Runtime 版本校验。 + +## 13. 版本与发布 + +- 产品版本采用 SemVer 2.0.0，从 `0.1.0` 开始，写入 `TextCascade.Server.csproj` 的 `Version`。 +- `protocolVersion` 只表示线协议版本，当前为 `1`，与产品版本独立演进。 +- 目标框架为 `net10.0`；目标机必须预装兼容的 .NET 10 Runtime。 +- 发布命令：`dotnet publish TextCascade.Server.csproj -c Release -p:PublishSingleFile=true`。 +- 本地编译命令：`dotnet build TextCascade.Server.csproj -c Release`。 + +## 14. 已关闭问题 + +| 问题 | 结论 | +|---|---| +| 最大文本默认值 | 512KB | +| 最新值磁盘持久化 | 不做，保持极简 | +| 用户配置热加载 | 不做，重启生效 | +| token 生命周期 | 长期 token + tokenVersion 撤销 | +| 删除后重建用户 | 全局 nextTokenVersion 水位 | +| metrics | v1 不启用 endpoint | +| 协议包 | 三端手写，服务端契约测试约束 | +| 关闭码 | 应用层 error + 标准 close code；不发送 1013/4408 | From eb0440ee55c9df5b4f62047d2af49ea19ba01d5d Mon Sep 17 00:00:00 2001 From: long45343 <1725334094@qq.com> Date: Sat, 22 Aug 2026 15:38:04 +0800 Subject: [PATCH 07/32] Fix sliding window login limiter queue cleanup --- TextCascade.Server.Tests/LoginLimiterTests.cs | 71 +++++++++++++++++++ TextCascade.Server/Core.cs | 66 ++++++++++++++--- 2 files changed, 127 insertions(+), 10 deletions(-) diff --git a/TextCascade.Server.Tests/LoginLimiterTests.cs b/TextCascade.Server.Tests/LoginLimiterTests.cs index 4e1f818..c9ccfd8 100644 --- a/TextCascade.Server.Tests/LoginLimiterTests.cs +++ b/TextCascade.Server.Tests/LoginLimiterTests.cs @@ -71,4 +71,75 @@ public void ExpiredEntriesAreLazilyRemoved() Assert.False(limiter.TryConsumeLoginLimit("1.1.1.1", "alice", now, config)); Assert.True(limiter.TryConsumeLoginLimit("1.1.1.1", "alice", later, config)); } + + [Fact] + public void RemoveExpiredKeepsUnexpiredRetryAfterOlderEntry() + { + var limiter = new SlidingWindowLoginLimiter(); + var t0 = DateTimeOffset.FromUnixTimeSeconds(1760000000); + var config = TextCascade.Server.Config.CreateDefaultConfig() with + { + RateLimit = new RateLimitConfig(3, 3, 10, 10, 2), + }; + + // Consume twice at t0 + Assert.True(limiter.TryConsumeLoginLimit("1.1.1.1", "alice", t0, config)); + Assert.True(limiter.TryConsumeLoginLimit("1.1.1.1", "alice", t0, config)); + + // Consume once at t0 + 70s (this purges entries <= t0 + 10s, but window here is at t0+70s, cutoff t0+10s) + var t70 = t0.AddSeconds(70); + Assert.True(limiter.TryConsumeLoginLimit("1.1.1.1", "alice", t70, config)); + + // In a non-monotonic test or another check: at t0 + 65s, let's test specific cutoff + // Let's test by direct consumption with another limiter to follow spec exactly: + var limiter2 = new SlidingWindowLoginLimiter(); + // limit=3; 同一 IP 在 t0 消费两次，t0+70s 消费一次。 + Assert.True(limiter2.TryConsumeLoginLimit("1.1.1.1", "alice", t0, config)); + Assert.True(limiter2.TryConsumeLoginLimit("1.1.1.1", "alice", t0, config)); + // Enqueue at t0+70s + Assert.True(limiter2.TryConsumeLoginLimit("1.1.1.1", "alice", t0.AddSeconds(70), config)); + // At t0+70s, the 2 t0 records expired, only 1 record (at 70s) remains. + // So we can consume 2 more times at t0+70s: + Assert.True(limiter2.TryConsumeLoginLimit("1.1.1.1", "alice", t0.AddSeconds(70), config)); + Assert.True(limiter2.TryConsumeLoginLimit("1.1.1.1", "alice", t0.AddSeconds(70), config)); + // Now 3 records at t0+70s exist -> next should fail + Assert.False(limiter2.TryConsumeLoginLimit("1.1.1.1", "alice", t0.AddSeconds(70), config)); + } + + [Fact] + public void RemoveExpiredDeletesKeyOnlyWhenQueueIsEmpty() + { + var limiter = new SlidingWindowLoginLimiter(); + var t0 = DateTimeOffset.FromUnixTimeSeconds(1760000000); + + limiter.EnqueueForTest("ip:1.1.1.1", t0); + limiter.EnqueueForTest("ip:1.1.1.1", t0.AddSeconds(40)); + + // Cutoff at t0 + 10s: first item expired, second item unexpired + limiter.RemoveExpiredForTest(t0.AddSeconds(10)); + Assert.True(limiter.HasWindowKey("ip:1.1.1.1")); + Assert.Equal(1, limiter.GetWindowCount("ip:1.1.1.1")); + + // Cutoff at t0 + 50s: all expired + limiter.RemoveExpiredForTest(t0.AddSeconds(50)); + Assert.False(limiter.HasWindowKey("ip:1.1.1.1")); + } + + [Fact] + public void LoginLimitsMustBePositive() + { + var baseConfig = TextCascade.Server.Config.CreateDefaultConfig() with { TokenSecret = new byte[32] }; + + var configZeroIp = baseConfig with + { + RateLimit = baseConfig.RateLimit with { LoginIpPerMinute = 0 }, + }; + Assert.Throws(() => TextCascade.Server.Config.ValidateConfig(configZeroIp)); + + var configZeroUser = baseConfig with + { + RateLimit = baseConfig.RateLimit with { LoginUserPerMinute = 0 }, + }; + Assert.Throws(() => TextCascade.Server.Config.ValidateConfig(configZeroUser)); + } } diff --git a/TextCascade.Server/Core.cs b/TextCascade.Server/Core.cs index ff61597..25828d0 100644 --- a/TextCascade.Server/Core.cs +++ b/TextCascade.Server/Core.cs @@ -29,6 +29,42 @@ public void ResetUserLoginLimit(string username) } } + internal int GetWindowCount(string key) + { + lock (gate) + { + return windows.TryGetValue(key, out var queue) ? queue.Count : 0; + } + } + + internal bool HasWindowKey(string key) + { + lock (gate) + { + return windows.ContainsKey(key); + } + } + + internal void RemoveExpiredForTest(DateTimeOffset cutoff) + { + lock (gate) + { + RemoveExpired(cutoff); + } + } + + internal void EnqueueForTest(string key, DateTimeOffset timestamp) + { + lock (gate) + { + if (!windows.TryGetValue(key, out var queue)) + { + windows[key] = queue = new Queue(); + } + queue.Enqueue(timestamp); + } + } + private bool TryConsume(string key, int limit, DateTimeOffset nowUtc, int maxKeys, bool allowNewKey) { var cutoff = nowUtc.AddMinutes(-1); @@ -57,22 +93,32 @@ private bool TryConsume(string key, int limit, DateTimeOffset nowUtc, int maxKey } queue.Enqueue(nowUtc); - if (queue.Count == 0) - { - windows.Remove(key); - } - return true; } private void RemoveExpired(DateTimeOffset cutoff) { - var stale = windows.Where(pair => pair.Value.Count == 0 || pair.Value.Peek() <= cutoff) - .Select(pair => pair.Key) - .ToList(); - foreach (var key in stale) + List? emptyKeys = null; + foreach (var pair in windows) { - windows.Remove(key); + var queue = pair.Value; + while (queue.Count > 0 && queue.Peek() <= cutoff) + { + queue.Dequeue(); + } + + if (queue.Count == 0) + { + (emptyKeys ??= new List()).Add(pair.Key); + } + } + + if (emptyKeys is not null) + { + foreach (var key in emptyKeys) + { + windows.Remove(key); + } } } } From 31b864643fbc8e8d3d1e8075fccc27734c6d6334 Mon Sep 17 00:00:00 2001 From: long45343 <1725334094@qq.com> Date: Sat, 22 Aug 2026 15:41:14 +0800 Subject: [PATCH 08/32] Add constant-time dummy hash verification for non-existent users --- .../AuthServiceTimingTests.cs | 179 ++++++++++++++++++ TextCascade.Server/AuthService.cs | 16 +- TextCascade.Server/SyncServer.cs | 6 + 3 files changed, 191 insertions(+), 10 deletions(-) create mode 100644 TextCascade.Server.Tests/AuthServiceTimingTests.cs diff --git a/TextCascade.Server.Tests/AuthServiceTimingTests.cs b/TextCascade.Server.Tests/AuthServiceTimingTests.cs new file mode 100644 index 0000000..4fb6092 --- /dev/null +++ b/TextCascade.Server.Tests/AuthServiceTimingTests.cs @@ -0,0 +1,179 @@ +using System.IO.Pipelines; +using System.Text; +using System.Text.Json; +using Isopoh.Cryptography.Argon2; +using Microsoft.AspNetCore.Http; +using Microsoft.Extensions.Logging; +using TextCascade.Server; + +namespace TextCascade.Server.Tests; + +public class AuthServiceTimingTests +{ + private sealed class RecordingHasher : IPasswordHasher + { + public List<(string Password, Argon2Config Config)> HashCalls { get; } = new(); + public List<(string Password, string EncodedHash)> VerifyCalls { get; } = new(); + + public string DummyHashReturn { get; set; } = "$argon2id$v=19$m=19456,t=2,p=1$dummy"; + + public string Hash(string password, Argon2Config config) + { + HashCalls.Add((password, config)); + return DummyHashReturn; + } + + public bool Verify(string password, string encodedHash) + { + VerifyCalls.Add((password, encodedHash)); + return string.Equals(password, "correct-password", StringComparison.Ordinal) + && string.Equals(encodedHash, "valid-hash", StringComparison.Ordinal); + } + + public bool NeedsRehash(string encodedHash, Argon2Config config) => false; + } + + private sealed class TestLogger : ILogger + { + public List LoggedMessages { get; } = new(); + + public IDisposable? BeginScope(TState state) where TState : notnull => null; + + public bool IsEnabled(LogLevel logLevel) => true; + + public void Log( + LogLevel logLevel, + EventId eventId, + TState state, + Exception? exception, + Func formatter) + { + LoggedMessages.Add(formatter(state, exception)); + } + } + + private static (DefaultHttpContext Context, MemoryStream ResponseBody) CreateHttpContext(string username, string password) + { + var context = new DefaultHttpContext(); + var json = JsonSerializer.Serialize(new { username, password }); + var bytes = Encoding.UTF8.GetBytes(json); + context.Request.Body = new MemoryStream(bytes); + context.Request.ContentLength = bytes.Length; + context.Request.ContentType = "application/json"; + + var responseBody = new MemoryStream(); + context.Response.Body = responseBody; + return (context, responseBody); + } + + [Fact] + public async Task LoginVerificationRunsForMissingUser() + { + var hasher = new RecordingHasher(); + var config = TextCascade.Server.Config.CreateDefaultConfig() with { TokenSecret = new byte[32] }; + var users = new UsersFile + { + Users = [new UserRecord("alice", "valid-hash", 1)], + NextTokenVersion = 2, + }; + var tempState = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString("N") + ".json"); + try + { + var stateStore = new RuntimeStateStore(tempState); + var server = new SyncServer(config, users, stateStore, hasher, new SystemClock(), Microsoft.Extensions.Logging.Abstractions.NullLogger.Instance); + + Assert.Equal(server.LoginDummyHash, hasher.DummyHashReturn); + + // 1. Missing user + var (missingContext, _) = CreateHttpContext("missing", "any-password"); + await AuthService.HandleLoginAsync(missingContext, config, server); + Assert.Equal(401, missingContext.Response.StatusCode); + Assert.Single(hasher.VerifyCalls); + Assert.Equal("missing", "missing"); + Assert.Equal(server.LoginDummyHash, hasher.VerifyCalls[0].EncodedHash); + Assert.Equal("any-password", hasher.VerifyCalls[0].Password); + + // 2. Existing user alice with wrong password + var (aliceContext, _) = CreateHttpContext("alice", "wrong-password"); + await AuthService.HandleLoginAsync(aliceContext, config, server); + Assert.Equal(401, aliceContext.Response.StatusCode); + Assert.Equal(2, hasher.VerifyCalls.Count); + Assert.Equal("valid-hash", hasher.VerifyCalls[1].EncodedHash); + Assert.Equal("wrong-password", hasher.VerifyCalls[1].Password); + } + finally + { + if (File.Exists(tempState)) File.Delete(tempState); + } + } + + [Fact] + public void LoginDummyHashUsesConfiguredArgon2Parameters() + { + var hasher = new RecordingHasher(); + var config = TextCascade.Server.Config.CreateDefaultConfig() with + { + TokenSecret = new byte[32], + Auth = new AuthConfig(30, "TEST_SECRET", 32768, 4, 2), + }; + var users = new UsersFile(); + var tempState = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString("N") + ".json"); + try + { + var stateStore = new RuntimeStateStore(tempState); + var server = new SyncServer(config, users, stateStore, hasher, new SystemClock(), Microsoft.Extensions.Logging.Abstractions.NullLogger.Instance); + + Assert.Single(hasher.HashCalls); + var call = hasher.HashCalls[0]; + Assert.Equal("textcascade-login-timing-dummy", call.Password); + Assert.Equal(32768, call.Config.MemoryCost); + Assert.Equal(4, call.Config.TimeCost); + Assert.Equal(2, call.Config.Threads); + Assert.Equal(server.LoginDummyHash, hasher.DummyHashReturn); + } + finally + { + if (File.Exists(tempState)) File.Delete(tempState); + } + } + + [Fact] + public async Task DisabledUserStillReturnsUnifiedInvalidCredentials() + { + var hasher = new RecordingHasher(); + var config = TextCascade.Server.Config.CreateDefaultConfig() with { TokenSecret = new byte[32] }; + var users = new UsersFile + { + Users = [new UserRecord("dave", "valid-hash", 1, Disabled: true)], + NextTokenVersion = 2, + }; + var tempState = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString("N") + ".json"); + var logger = new TestLogger(); + try + { + var stateStore = new RuntimeStateStore(tempState); + var server = new SyncServer(config, users, stateStore, hasher, new SystemClock(), Microsoft.Extensions.Logging.Abstractions.NullLogger.Instance); + + var (context, responseBody) = CreateHttpContext("dave", "correct-password"); + await AuthService.HandleLoginAsync(context, config, server, logger); + + Assert.Equal(401, context.Response.StatusCode); + var json = Encoding.UTF8.GetString(responseBody.ToArray()); + Assert.Contains("invalid_credentials", json); + + // Ensure logs do not mention "disabled" + Assert.NotEmpty(logger.LoggedMessages); + foreach (var log in logger.LoggedMessages) + { + Assert.DoesNotContain("reason=disabled", log, StringComparison.OrdinalIgnoreCase); + Assert.DoesNotContain("disabled", log, StringComparison.OrdinalIgnoreCase); + } + } + finally + { + if (File.Exists(tempState)) File.Delete(tempState); + } + } +} + + diff --git a/TextCascade.Server/AuthService.cs b/TextCascade.Server/AuthService.cs index d1688b8..da9304e 100644 --- a/TextCascade.Server/AuthService.cs +++ b/TextCascade.Server/AuthService.cs @@ -34,22 +34,18 @@ public static async Task HandleLoginAsync(HttpContext context, RuntimeConfig con var userLookup = syncServer.UserLookup; var found = userLookup.TryGetValue(request.Username, out var user); - var passwordOk = found && user is not null && syncServer.Hasher.Verify(request.Password, user.PasswordHash); - if (!passwordOk) + var passwordHash = found && user is not null + ? user.PasswordHash + : syncServer.LoginDummyHash; + var passwordOk = syncServer.Hasher.Verify(request.Password, passwordHash); + if (!found || !passwordOk || user is null || user.Disabled) { logger?.LogSecurityEvent("login", ("username", request.Username), ("ip", ip), ("success", false), ("reason", "invalid_credentials")); await WriteError(context, 401, "invalid_credentials", "Invalid username or password."); return; } - var authenticatedUser = user!; - if (authenticatedUser.Disabled) - { - logger?.LogSecurityEvent("login", ("username", request.Username), ("ip", ip), ("success", false), ("reason", "disabled")); - await WriteError(context, 401, "invalid_credentials", "Invalid username or password."); - return; - } - + var authenticatedUser = user; limiter.ResetUserLoginLimit(request.Username); logger?.LogSecurityEvent("login", ("username", request.Username), ("ip", ip), ("success", true)); diff --git a/TextCascade.Server/SyncServer.cs b/TextCascade.Server/SyncServer.cs index 1cb006c..44d058c 100644 --- a/TextCascade.Server/SyncServer.cs +++ b/TextCascade.Server/SyncServer.cs @@ -525,6 +525,7 @@ public sealed class SyncServer private readonly IClock clock; private readonly RuntimeStateStore runtimeStateStore; private readonly IReadOnlyDictionary userLookup; + private readonly string loginDummyHash; public UserRegistry Registry => registry; public IPasswordHasher Hasher => hasher; @@ -535,6 +536,7 @@ public sealed class SyncServer public DateTimeOffset ProcessStartTime { get; } public RuntimeConfig Config { get; } public RuntimeStateStore RuntimeStateStore => runtimeStateStore; + internal string LoginDummyHash => loginDummyHash; public SyncServer( RuntimeConfig config, @@ -551,6 +553,9 @@ public SyncServer( this.clock = clock; Logger = logger; ProcessStartTime = clock.UtcNow; + loginDummyHash = hasher.Hash( + "textcascade-login-timing-dummy", + Cli.CreateArgon2Config(config)); } public UserHub GetOrCreateHub(string username, RuntimeConfig runtimeConfig) @@ -1080,3 +1085,4 @@ private static async Task SendSafeAsync(ConnectionContext connection, byte[] pay internal sealed class FrameTooLargeException : Exception; internal sealed record ReceivedMessage(WebSocketMessageType MessageType, byte[] Payload); + From 24b59630a6a7bc02315fe8f515185593fc1a4a2f Mon Sep 17 00:00:00 2001 From: long45343 <1725334094@qq.com> Date: Sat, 22 Aug 2026 15:41:59 +0800 Subject: [PATCH 09/32] Optimize SeenIdRing with hash map index and FIFO circular queue --- TextCascade.Server.Tests/ClipAndCoreTests.cs | 65 +++++++++++++++++ TextCascade.Server/Core.cs | 73 ++++++++++---------- 2 files changed, 100 insertions(+), 38 deletions(-) diff --git a/TextCascade.Server.Tests/ClipAndCoreTests.cs b/TextCascade.Server.Tests/ClipAndCoreTests.cs index 6184282..2eac1ee 100644 --- a/TextCascade.Server.Tests/ClipAndCoreTests.cs +++ b/TextCascade.Server.Tests/ClipAndCoreTests.cs @@ -144,4 +144,69 @@ public void SelectSnapshotWinnerBreaksTiesByTimeThenClientId() var winner2 = CoreLogic.SelectSnapshotWinner(new[] { a2, b2 }); Assert.Equal("pb", winner2!.Snapshot.Payload); } + + [Fact] + public void SeenIdRingMaintainsFifoEvictionAfterRepeatedIds() + { + var ring = new SeenIdRing(2); + var result1 = new LatestText("p1", 1, "h1", false, "c1", "n1", DateTimeOffset.UtcNow); + var result2 = new LatestText("p2", 2, "h2", false, "c2", "n2", DateTimeOffset.UtcNow); + + ring.RememberId("a", result1); + ring.RememberId("b"); + ring.RememberId("a", result2); + ring.RememberId("c"); + + // Eviction order: slot0 had "a"(overwritten by "a"), slot1 had "b"(overwritten by "c") + // When inserting "c" into slot 1, slot1 ("b") is evicted. + // When slot 0 is overwritten by "a", slot0 ("a") was updated. + // Let's check: + // slot 0: was "a" (result1), replaced by "a" (result2). entries["a"] = result2 + // slot 1: was "b" (null), replaced by "c" (null). "b" was evicted! + // So "b" was evicted, "c" and "a" exist. + // If we now insert one more: + var result3 = new LatestText("p3", 3, "h3", false, "c3", "n3", DateTimeOffset.UtcNow); + ring.RememberId("d", result3); // overwrites slot 0 (which was "a"). Evicts "a"! + Assert.False(ring.TryGetResult("a", out _)); + Assert.True(ring.TryGetResult("d", out var resD)); + Assert.Equal(result3, resD); + } + + [Fact] + public void SeenIdRingRepeatedIdKeepsLatestResultBeforeEviction() + { + var ring = new SeenIdRing(4); + var old = new LatestText("old", 1, "h", false, "c", "n", DateTimeOffset.UtcNow); + var latest = new LatestText("new", 2, "h", false, "c", "n", DateTimeOffset.UtcNow); + + ring.RememberId("same", old); + ring.RememberId("same", latest); + + Assert.True(ring.TryGetResult("same", out var actual)); + Assert.Equal(latest, actual); + } + + [Fact] + public void SeenIdRingEvictsOldestWhenFull() + { + var ring = new SeenIdRing(3); + ring.RememberId("a"); + ring.RememberId("b"); + ring.RememberId("c"); + ring.RememberId("d"); + + Assert.False(ring.TryGetResult("a", out _)); + Assert.True(ring.TryGetResult("b", out _)); + Assert.True(ring.TryGetResult("c", out _)); + Assert.True(ring.TryGetResult("d", out _)); + } + + [Fact] + public void SeenIdRingUsesOrdinalComparison() + { + var ring = new SeenIdRing(4); + ring.RememberId("A"); + Assert.False(ring.TryDuplicate("a")); + Assert.True(ring.TryDuplicate("A")); + } } diff --git a/TextCascade.Server/Core.cs b/TextCascade.Server/Core.cs index 25828d0..d06229e 100644 --- a/TextCascade.Server/Core.cs +++ b/TextCascade.Server/Core.cs @@ -168,27 +168,31 @@ public bool TryAcquire(DateTimeOffset nowUtc) public sealed class SeenIdRing { private readonly object gate = new(); - private readonly string?[] ids; - private readonly LatestText?[] results; - private int next; + private readonly Dictionary entries; + private readonly string?[] insertionOrder; + private int nextInsertIndex; + + public int Capacity { get; } public SeenIdRing(int capacity) { - if (capacity <= 0) throw new ArgumentOutOfRangeException(nameof(capacity)); - ids = new string?[capacity]; - results = new LatestText?[capacity]; + if (capacity <= 0) + { + throw new ArgumentOutOfRangeException(nameof(capacity)); + } + + Capacity = capacity; + entries = new Dictionary(capacity, StringComparer.Ordinal); + insertionOrder = new string?[capacity]; } public bool TryDuplicate(string id) { lock (gate) { - for (var i = 0; i < ids.Length; i++) + if (entries.ContainsKey(id)) { - if (string.Equals(ids[i], id, StringComparison.Ordinal)) - { - return true; - } + return true; } RememberInternal(id, null); @@ -200,17 +204,7 @@ public bool TryGetResult(string id, out LatestText? result) { lock (gate) { - for (var i = 0; i < ids.Length; i++) - { - if (string.Equals(ids[i], id, StringComparison.Ordinal)) - { - result = results[i]; - return true; - } - } - - result = null; - return false; + return entries.TryGetValue(id, out result); } } @@ -228,30 +222,32 @@ public bool IsUnchangedDuplicate(string id, string payload, string hash, bool en { lock (gate) { - for (var index = 0; index < ids.Length; index++) + if (!entries.TryGetValue(id, out var remembered)) { - if (!string.Equals(ids[index], id, StringComparison.Ordinal)) - { - continue; - } - - latest = results[index]; - return latest is not null - && string.Equals(latest.Payload, payload, StringComparison.Ordinal) - && string.Equals(latest.Hash, hash, StringComparison.Ordinal) - && latest.Encrypted == encrypted; + latest = null; + return false; } - latest = null; - return false; + latest = remembered; + return remembered is not null + && string.Equals(remembered.Payload, payload, StringComparison.Ordinal) + && string.Equals(remembered.Hash, hash, StringComparison.Ordinal) + && remembered.Encrypted == encrypted; } } private void RememberInternal(string id, LatestText? result) { - ids[next] = id; - results[next] = result; - next = (next + 1) % ids.Length; + var evictedId = insertionOrder[nextInsertIndex]; + if (evictedId is not null + && !string.Equals(evictedId, id, StringComparison.Ordinal)) + { + entries.Remove(evictedId); + } + + insertionOrder[nextInsertIndex] = id; + nextInsertIndex = (nextInsertIndex + 1) % insertionOrder.Length; + entries[id] = result; } } @@ -286,3 +282,4 @@ public static LatestText WithVersion(LatestText latest, ulong next, DateTimeOffs .FirstOrDefault(); } } + From b7820093b03df1650e723dadf08e9efbaefde1c7 Mon Sep 17 00:00:00 2001 From: long45343 <1725334094@qq.com> Date: Sat, 22 Aug 2026 15:43:30 +0800 Subject: [PATCH 10/32] Use native .NET APIs for PEM certificate and key loading --- .../CertificateLoaderTests.cs | 143 ++++++++++++++++++ TextCascade.Server/ServerHost.cs | 71 +++------ 2 files changed, 167 insertions(+), 47 deletions(-) create mode 100644 TextCascade.Server.Tests/CertificateLoaderTests.cs diff --git a/TextCascade.Server.Tests/CertificateLoaderTests.cs b/TextCascade.Server.Tests/CertificateLoaderTests.cs new file mode 100644 index 0000000..649d6c6 --- /dev/null +++ b/TextCascade.Server.Tests/CertificateLoaderTests.cs @@ -0,0 +1,143 @@ +using System.Security.Cryptography; +using System.Security.Cryptography.X509Certificates; +using System.Text; +using TextCascade.Server; + +namespace TextCascade.Server.Tests; + +public class CertificateLoaderTests +{ + private static (string CertPem, string KeyPem) GenerateSelfSignedRsaPem() + { + using var rsa = RSA.Create(2048); + var request = new CertificateRequest("CN=localhost", rsa, HashAlgorithmName.SHA256, RSASignaturePadding.Pkcs1); + var notBefore = DateTimeOffset.UtcNow.AddMinutes(-5); + var notAfter = DateTimeOffset.UtcNow.AddDays(1); + using var cert = request.CreateSelfSigned(notBefore, notAfter); + + var certPem = cert.ExportCertificatePem(); + var keyPem = rsa.ExportPkcs8PrivateKeyPem(); + return (certPem, keyPem); + } + + private static (string CertPem, string KeyPem) GenerateSelfSignedEcdsaPem() + { + using var ecdsa = ECDsa.Create(ECCurve.NamedCurves.nistP256); + var request = new CertificateRequest("CN=localhost", ecdsa, HashAlgorithmName.SHA256); + var notBefore = DateTimeOffset.UtcNow.AddMinutes(-5); + var notAfter = DateTimeOffset.UtcNow.AddDays(1); + using var cert = request.CreateSelfSigned(notBefore, notAfter); + + var certPem = cert.ExportCertificatePem(); + var keyPem = ecdsa.ExportPkcs8PrivateKeyPem(); + return (certPem, keyPem); + } + + [Fact] + public void LoadPemCertificateSupportsRsaCertAndSeparateKey() + { + var tempDir = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString("N")); + Directory.CreateDirectory(tempDir); + try + { + var (certPem, keyPem) = GenerateSelfSignedRsaPem(); + var certPath = Path.Combine(tempDir, "server.crt"); + var keyPath = Path.Combine(tempDir, "server.key"); + File.WriteAllText(certPath, certPem, Encoding.UTF8); + File.WriteAllText(keyPath, keyPem, Encoding.UTF8); + + using var loaded = CertificateLoader.Load(certPath); + Assert.True(loaded.Certificate.HasPrivateKey); + Assert.NotEmpty(loaded.Chain); + Assert.Equal(loaded.Certificate.Thumbprint, loaded.Chain[0].Thumbprint); + } + finally + { + if (Directory.Exists(tempDir)) Directory.Delete(tempDir, true); + } + } + + [Fact] + public void LoadPemCertificateSupportsCombinedPem() + { + var tempDir = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString("N")); + Directory.CreateDirectory(tempDir); + try + { + var (certPem, keyPem) = GenerateSelfSignedRsaPem(); + var pemPath = Path.Combine(tempDir, "server.pem"); + File.WriteAllText(pemPath, certPem + "\n" + keyPem, Encoding.UTF8); + + using var loaded = CertificateLoader.Load(pemPath); + Assert.True(loaded.Certificate.HasPrivateKey); + Assert.NotEmpty(loaded.Chain); + } + finally + { + if (Directory.Exists(tempDir)) Directory.Delete(tempDir, true); + } + } + + [Fact] + public void LoadPemCertificateSupportsEcdsaCertAndSeparateKey() + { + var tempDir = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString("N")); + Directory.CreateDirectory(tempDir); + try + { + var (certPem, keyPem) = GenerateSelfSignedEcdsaPem(); + var certPath = Path.Combine(tempDir, "server.crt"); + var keyPath = Path.Combine(tempDir, "server.key"); + File.WriteAllText(certPath, certPem, Encoding.UTF8); + File.WriteAllText(keyPath, keyPem, Encoding.UTF8); + + using var loaded = CertificateLoader.Load(certPath); + Assert.True(loaded.Certificate.HasPrivateKey); + Assert.NotEmpty(loaded.Chain); + } + finally + { + if (Directory.Exists(tempDir)) Directory.Delete(tempDir, true); + } + } + + [Fact] + public void LoadPemCertificateWrapsMissingPrivateKey() + { + var tempDir = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString("N")); + Directory.CreateDirectory(tempDir); + try + { + var (certPem, _) = GenerateSelfSignedRsaPem(); + var certPath = Path.Combine(tempDir, "server.pem"); + File.WriteAllText(certPath, certPem, Encoding.UTF8); + + var ex = Assert.Throws(() => CertificateLoader.Load(certPath)); + Assert.NotNull(ex.InnerException); + Assert.Contains("Unable to load PEM certificate", ex.Message); + } + finally + { + if (Directory.Exists(tempDir)) Directory.Delete(tempDir, true); + } + } + + [Fact] + public void LoadPemCertificateRejectsNoCertificate() + { + var tempDir = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString("N")); + Directory.CreateDirectory(tempDir); + try + { + var certPath = Path.Combine(tempDir, "empty.pem"); + File.WriteAllText(certPath, "random garbage content", Encoding.UTF8); + + var ex = Assert.Throws(() => CertificateLoader.Load(certPath)); + Assert.Contains("Unable to load PEM certificate", ex.Message); + } + finally + { + if (Directory.Exists(tempDir)) Directory.Delete(tempDir, true); + } + } +} diff --git a/TextCascade.Server/ServerHost.cs b/TextCascade.Server/ServerHost.cs index de1e5c5..7643a0b 100644 --- a/TextCascade.Server/ServerHost.cs +++ b/TextCascade.Server/ServerHost.cs @@ -2,7 +2,6 @@ using System.Security.Cryptography; using System.Text; using System.Text.Json; -using System.Text.RegularExpressions; using System.Security.Cryptography.X509Certificates; using Microsoft.AspNetCore.Builder; using Microsoft.AspNetCore.Hosting; @@ -155,64 +154,41 @@ private static LoadedCertificate LoadPemCertificate(string certificatePath) var keyPath = File.Exists(Path.ChangeExtension(certificatePath, ".key")) ? Path.ChangeExtension(certificatePath, ".key") : certificatePath; + var chain = new X509Certificate2Collection(); try { - var certificatePem = File.ReadAllText(certificatePath, new UTF8Encoding(encoderShouldEmitUTF8Identifier: false, throwOnInvalidBytes: true)); - foreach (Match match in Regex.Matches(certificatePem, "-----BEGIN CERTIFICATE-----(?.*?)-----END CERTIFICATE-----", RegexOptions.Singleline)) - { - var base64 = Regex.Replace(match.Groups["data"].Value, "\\s", string.Empty); - var certificateBytes = Convert.FromBase64String(base64); - chain.Add(X509CertificateLoader.LoadCertificate(certificateBytes)); - } - + chain.ImportFromPemFile(certificatePath); if (chain.Count == 0) { throw new InvalidOperationException("PEM file does not contain a certificate."); } - } - catch (Exception exception) - { - DisposeChain(chain); - throw new InvalidOperationException($"Unable to parse PEM certificate '{certificatePath}': {exception.Message}", exception); - } - string keyPem; - try - { - keyPem = File.ReadAllText(keyPath, new UTF8Encoding(encoderShouldEmitUTF8Identifier: false, throwOnInvalidBytes: true)); - } - catch (Exception exception) - { - DisposeChain(chain); - throw new InvalidOperationException($"Unable to read PEM private key '{keyPath}': {exception.Message}", exception); - } - - try - { - using var rsa = RSA.Create(); - rsa.ImportFromPem(keyPem); - var certificateWithKey = chain[0].CopyWithPrivateKey(rsa); - chain[0].Dispose(); - chain[0] = certificateWithKey; - return new LoadedCertificate(certificateWithKey, chain); - } - catch (Exception rsaException) - { - try + var certificateWithKey = X509Certificate2.CreateFromPemFile(certificatePath, keyPath); + var originalLeaf = chain.Cast().FirstOrDefault(c => c.Equals(certificateWithKey)); + if (originalLeaf is not null) { - using var ecdsa = ECDsa.Create(); - ecdsa.ImportFromPem(keyPem); - var certificateWithKey = chain[0].CopyWithPrivateKey(ecdsa); - chain[0].Dispose(); - chain[0] = certificateWithKey; - return new LoadedCertificate(certificateWithKey, chain); + var originalIndex = chain.IndexOf(originalLeaf); + chain[originalIndex] = certificateWithKey; + originalLeaf.Dispose(); } - catch (Exception ecdsaException) + else { - DisposeChain(chain); - throw new InvalidOperationException($"Unable to bind PEM private key. RSA: {rsaException.Message}; ECDSA: {ecdsaException.Message}", ecdsaException); + chain.Insert(0, certificateWithKey); } + + return new LoadedCertificate(certificateWithKey, chain); + } + catch (Exception exception) when ( + exception is ArgumentException + or CryptographicException + or InvalidOperationException + or IOException) + { + DisposeChain(chain); + throw new InvalidOperationException( + $"Unable to load PEM certificate '{certificatePath}': {exception.Message}", + exception); } } @@ -292,3 +268,4 @@ public void Dispose() timer?.Dispose(); } } + From 30362ae0f013b1870788bee53c3189ee0f8680cf Mon Sep 17 00:00:00 2001 From: long45343 <1725334094@qq.com> Date: Sat, 22 Aug 2026 15:45:10 +0800 Subject: [PATCH 11/32] Relocate CLI single-instance lock adjacent to users.json file --- .../SingleInstanceLockTests.cs | 124 ++++++++++++++++++ TextCascade.Server/Cli.cs | 64 +++++++-- 2 files changed, 178 insertions(+), 10 deletions(-) create mode 100644 TextCascade.Server.Tests/SingleInstanceLockTests.cs diff --git a/TextCascade.Server.Tests/SingleInstanceLockTests.cs b/TextCascade.Server.Tests/SingleInstanceLockTests.cs new file mode 100644 index 0000000..48672f4 --- /dev/null +++ b/TextCascade.Server.Tests/SingleInstanceLockTests.cs @@ -0,0 +1,124 @@ +using System.Diagnostics; +using System.Globalization; +using System.Text; +using TextCascade.Server; + +namespace TextCascade.Server.Tests; + +public class SingleInstanceLockTests +{ + [Fact] + public void AcquireCreatesLockBesideUsersFile() + { + var tempDir = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString("N")); + Directory.CreateDirectory(tempDir); + try + { + var usersFile = Path.Combine(tempDir, "users.json"); + var lockPath = Cli.CreateLockPath(usersFile); + + Assert.EndsWith("users.json.lock", lockPath, StringComparison.Ordinal); + Assert.Equal(tempDir, Path.GetDirectoryName(lockPath)); + } + finally + { + if (Directory.Exists(tempDir)) Directory.Delete(tempDir, true); + } + } + + [Fact] + public void SecondProcessCannotAcquireSameUsersFileLock() + { + var tempDir = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString("N")); + Directory.CreateDirectory(tempDir); + try + { + var lockPath = Path.Combine(tempDir, "users.json.lock"); + using var handle1 = SingleInstanceLock.Acquire(lockPath, TimeSpan.FromMilliseconds(10)); + Assert.NotNull(handle1); + + using var handle2 = SingleInstanceLock.Acquire(lockPath, TimeSpan.FromMilliseconds(10)); + Assert.Null(handle2); + + handle1.Dispose(); + + using var handle3 = SingleInstanceLock.Acquire(lockPath, TimeSpan.FromMilliseconds(10)); + Assert.NotNull(handle3); + } + finally + { + if (Directory.Exists(tempDir)) Directory.Delete(tempDir, true); + } + } + + [Fact] + public void StaleLockIsRecovered() + { + var tempDir = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString("N")); + Directory.CreateDirectory(tempDir); + try + { + var lockPath = Path.Combine(tempDir, "users.json.lock"); + // Find a non-existent PID (e.g. 999999) + var deadPid = 999999; + File.WriteAllText(lockPath, deadPid.ToString(CultureInfo.InvariantCulture), Encoding.UTF8); + + using var handle = SingleInstanceLock.Acquire(lockPath, TimeSpan.FromMilliseconds(10)); + Assert.NotNull(handle); + } + finally + { + if (Directory.Exists(tempDir)) Directory.Delete(tempDir, true); + } + } + + [Fact] + public void LiveProcessLockIsNotRecovered() + { + var tempDir = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString("N")); + Directory.CreateDirectory(tempDir); + try + { + var lockPath = Path.Combine(tempDir, "users.json.lock"); + var currentPid = Environment.ProcessId; + File.WriteAllText(lockPath, currentPid.ToString(CultureInfo.InvariantCulture), Encoding.UTF8); + + using var handle = SingleInstanceLock.Acquire(lockPath, TimeSpan.FromMilliseconds(10)); + Assert.Null(handle); + Assert.True(File.Exists(lockPath)); + Assert.Equal(currentPid.ToString(CultureInfo.InvariantCulture), File.ReadAllText(lockPath).Trim()); + } + finally + { + if (Directory.Exists(tempDir)) Directory.Delete(tempDir, true); + } + } + + [Fact] + public void AcquireRejectsPathWithoutDirectory() + { + Assert.Throws(() => SingleInstanceLock.Acquire("users.json.lock")); + } + + [Fact] + public void DifferentUsersFilesCanLockIndependently() + { + var tempDir = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString("N")); + Directory.CreateDirectory(tempDir); + try + { + var lockPath1 = Path.Combine(tempDir, "users1.json.lock"); + var lockPath2 = Path.Combine(tempDir, "users2.json.lock"); + + using var handle1 = SingleInstanceLock.Acquire(lockPath1, TimeSpan.FromMilliseconds(10)); + using var handle2 = SingleInstanceLock.Acquire(lockPath2, TimeSpan.FromMilliseconds(10)); + + Assert.NotNull(handle1); + Assert.NotNull(handle2); + } + finally + { + if (Directory.Exists(tempDir)) Directory.Delete(tempDir, true); + } + } +} diff --git a/TextCascade.Server/Cli.cs b/TextCascade.Server/Cli.cs index 6c52614..18844b3 100644 --- a/TextCascade.Server/Cli.cs +++ b/TextCascade.Server/Cli.cs @@ -21,12 +21,6 @@ public static int RunCli(string[] args, IPasswordHasher? hasher = null) } hasher ??= new Argon2PasswordHasher(); - using var lockHandle = SingleInstanceLock.Acquire(); - if (lockHandle is null) - { - Console.Error.WriteLine("Another TextCascade CLI process is running."); - return Error; - } var rest = args.Skip(1).ToArray(); if (!TryExtractConfigOption(ref rest, out var configPath)) @@ -51,7 +45,27 @@ public static int RunCli(string[] args, IPasswordHasher? hasher = null) return Error; } - return rest switch + SingleInstanceLockHandle? lockHandle; + try + { + var lockPath = CreateLockPath(config.Files.UsersFile); + lockHandle = SingleInstanceLock.Acquire(lockPath); + } + catch (Exception exception) when (exception is IOException or UnauthorizedAccessException or DirectoryNotFoundException or ArgumentException or InvalidOperationException) + { + Console.Error.WriteLine($"Unable to acquire users file lock: {exception.Message}"); + return Error; + } + + using (lockHandle) + { + if (lockHandle is null) + { + Console.Error.WriteLine("Another TextCascade CLI process is running."); + return Error; + } + + return rest switch { { Length: > 0 } when rest[0] == "add" => CommandAddUser(rest, hasher, config), { Length: > 0 } when rest[0] == "passwd" => CommandPasswd(rest, hasher, config), @@ -63,6 +77,20 @@ public static int RunCli(string[] args, IPasswordHasher? hasher = null) { Length: > 0 } when rest[0] == "hash" => CommandHashPassword(rest, hasher, config), _ => PrintUsage(), }; + } + } + + internal static string CreateLockPath(string usersFile) + { + var fullUsersPath = Path.GetFullPath(usersFile); + var directory = Path.GetDirectoryName(fullUsersPath); + if (string.IsNullOrEmpty(directory)) + { + throw new InvalidOperationException("users.json path must include a parent directory."); + } + + var fileName = Path.GetFileName(fullUsersPath); + return Path.Combine(directory, $"{fileName}.lock"); } private static int PrintUsage() @@ -381,10 +409,24 @@ public void Dispose() public static class SingleInstanceLock { - public static SingleInstanceLockHandle? Acquire(TimeSpan? pollDelay = null) + public static SingleInstanceLockHandle? Acquire(string lockPath, TimeSpan? pollDelay = null) { - var directory = AppContext.BaseDirectory; - var lockPath = Path.Combine(directory, ".textcascade-cli.lock"); + if (string.IsNullOrWhiteSpace(lockPath)) + { + throw new ArgumentException("Lock path must not be empty.", nameof(lockPath)); + } + + var directory = Path.GetDirectoryName(lockPath); + if (string.IsNullOrEmpty(directory)) + { + throw new ArgumentException("Lock path must include a directory.", nameof(lockPath)); + } + + if (!Directory.Exists(directory)) + { + throw new DirectoryNotFoundException($"Directory '{directory}' does not exist."); + } + var delay = pollDelay ?? TimeSpan.FromMilliseconds(100); for (var attempt = 0; attempt < 3; attempt++) @@ -469,3 +511,5 @@ private static bool IsProcessAlive(int pid) } } } + + From bedbca74628664e906b2c262bc599afb5061cb6b Mon Sep 17 00:00:00 2001 From: long45343 <1725334094@qq.com> Date: Sat, 22 Aug 2026 15:50:03 +0800 Subject: [PATCH 12/32] Split SyncServer into domain-oriented files and folders --- .../Hosting/ConnectionHandler.cs | 269 ++++++ .../Hosting/HeartbeatScannerService.cs | 50 ++ TextCascade.Server/Hosting/SyncEndpoint.cs | 53 ++ TextCascade.Server/Hub/UserHub.cs | 355 ++++++++ TextCascade.Server/Hub/UserJobs.cs | 21 + TextCascade.Server/Hub/UserRegistry.cs | 29 + .../Models/ConnectionContext.cs | 26 + .../Models/ConnectionStateBag.cs | 88 ++ TextCascade.Server/Models/ReceivedMessage.cs | 5 + TextCascade.Server/ServerHost.cs | 47 - TextCascade.Server/SyncServer.cs | 815 +----------------- 11 files changed, 897 insertions(+), 861 deletions(-) create mode 100644 TextCascade.Server/Hosting/ConnectionHandler.cs create mode 100644 TextCascade.Server/Hosting/HeartbeatScannerService.cs create mode 100644 TextCascade.Server/Hosting/SyncEndpoint.cs create mode 100644 TextCascade.Server/Hub/UserHub.cs create mode 100644 TextCascade.Server/Hub/UserJobs.cs create mode 100644 TextCascade.Server/Hub/UserRegistry.cs create mode 100644 TextCascade.Server/Models/ConnectionContext.cs create mode 100644 TextCascade.Server/Models/ConnectionStateBag.cs create mode 100644 TextCascade.Server/Models/ReceivedMessage.cs diff --git a/TextCascade.Server/Hosting/ConnectionHandler.cs b/TextCascade.Server/Hosting/ConnectionHandler.cs new file mode 100644 index 0000000..de94052 --- /dev/null +++ b/TextCascade.Server/Hosting/ConnectionHandler.cs @@ -0,0 +1,269 @@ +using System.Globalization; +using System.Net.WebSockets; +using System.Text; +using Microsoft.Extensions.Logging; + +namespace TextCascade.Server; + +public static class ConnectionHandler +{ + public static async Task RunAsync(ConnectionContext provisional, TokenPayload payload, RuntimeConfig config, SyncServer server) + { + server.RegisterPendingHello(provisional); + ClientHello hello; + try + { + var received = await ReceiveFrameAsync(provisional, config.Limits.MaxFrameBytes, provisional.State.Cts.Token); + if (received.MessageType == WebSocketMessageType.Close) + { + await provisional.Socket.CloseOutputAsync( + WebSocketCloseStatus.NormalClosure, + "client_closed", + CancellationToken.None); + server.CancelConnection(provisional, "closed"); + return; + } + + var parse = Protocol.ParseClientMessage(received.Payload, config); + if (!parse.IsSuccess || parse.Kind != MessageKind.Hello) + { + var error = Protocol.SerializeProtocolError(new ProtocolError( + ProtocolErrorCode.InvalidMessage, + "Expected a valid hello message.", + parse.Error?.ReferenceId)); + await SendAndClosePreHelloAsync( + provisional, + error, + WebSocketCloseStatus.PolicyViolation, + "invalid_hello", + server); + return; + } + + hello = (ClientHello)parse.Message!; + } + catch (FrameTooLargeException) + { + var error = Protocol.SerializeProtocolError(new ProtocolError(ProtocolErrorCode.FrameTooLarge, "frame_too_large", null)); + await SendAndClosePreHelloAsync(provisional, error, WebSocketCloseStatus.MessageTooBig, "frame_too_large", server); + return; + } + catch (OperationCanceledException) + { + // Hello timeout is owned by the unified heartbeat scanner; here the socket was + // cancelled for another reason (e.g. shutdown). Fall through to unified cleanup. + server.CancelConnection(provisional, "cancelled"); + return; + } + catch (WebSocketException) + { + server.CancelConnection(provisional, "socket_error"); + return; + } + + var hub = server.GetOrCreateHub(payload.Subject, config); + var connection = new ConnectionContext( + provisional.ConnectionId, + payload.Subject, + hello.ClientId, + hello.ClientName, + provisional.Socket, + hub, + config); + hub.AddConnection(connection); + server.Logger.LogSecurityEvent("connect", + ("username", connection.Username), + ("clientId", connection.ClientId), + ("connectionId", connection.ConnectionId)); + connection.State.HelloReceived = true; + connection.State.LastSeen = DateTimeOffset.UtcNow; + server.UnregisterPendingHello(provisional); + if (!hub.TryWriteJob(new HelloJob(connection, hello))) + { + server.CancelConnection(connection, "user_loop_unavailable"); + return; + } + + var sendTask = ConnectionSendLoopAsync(connection); + var readTask = ReadLoopAsync(connection, config, server); + await Task.WhenAll(sendTask, readTask); + server.CancelConnection(connection, "disconnected"); + } + + private static async Task ReceiveFrameAsync( + ConnectionContext connection, + int maxBytes, + CancellationToken cancellationToken) + { + using var stream = new MemoryStream(Math.Min(maxBytes, 16 * 1024)); + var buffer = new byte[Math.Min(maxBytes, 16 * 1024)]; + while (true) + { + var received = await connection.Socket.ReceiveAsync(new ArraySegment(buffer), cancellationToken); + if (received.Count > maxBytes - stream.Length) + { + throw new FrameTooLargeException(); + } + + stream.Write(buffer, 0, received.Count); + if (received.EndOfMessage) + { + return new ReceivedMessage(received.MessageType, stream.ToArray()); + } + } + } + + private static async Task SendAndClosePreHelloAsync( + ConnectionContext connection, + byte[] error, + WebSocketCloseStatus status, + string reason, + SyncServer server) + { + try + { + if (connection.Socket.State == WebSocketState.Open) + { + await connection.Socket.SendAsync(error, WebSocketMessageType.Text, true, CancellationToken.None); + await connection.Socket.CloseAsync(status, reason, CancellationToken.None); + } + } + catch (Exception) + { + server.EnqueueImmediateClose(connection, "server_busy"); + } + finally + { + server.CancelConnection(connection, reason); + } + } + + private static async Task ReadLoopAsync(ConnectionContext connection, RuntimeConfig config, SyncServer server) + { + try + { + while (connection.State.Cts.IsCancellationRequested == false && connection.Socket.State == WebSocketState.Open) + { + ReceivedMessage received; + try + { + received = await ReceiveFrameAsync(connection, config.Limits.MaxFrameBytes, connection.State.Cts.Token); + } + catch (FrameTooLargeException) + { + var oversized = Protocol.SerializeProtocolError(new ProtocolError(ProtocolErrorCode.FrameTooLarge, "frame_too_large", null)); + await SendSafeAsync(connection, oversized, server); + await Task.Delay(100, connection.State.Cts.Token); + if (connection.Socket.State == WebSocketState.Open) + { + await connection.Socket.CloseAsync(WebSocketCloseStatus.MessageTooBig, "frame_too_large", CancellationToken.None); + } + server.CancelConnection(connection, "frame_too_large"); + break; + } + + if (received.MessageType == WebSocketMessageType.Close) + { + try + { + await connection.Socket.CloseOutputAsync( + WebSocketCloseStatus.NormalClosure, + "client_closed", + CancellationToken.None); + } + catch (WebSocketException) { } + break; + } + + if (!Protocol.CheckFrameSize(received.Payload.Length, config)) + { + var error = Protocol.SerializeProtocolError(new ProtocolError(ProtocolErrorCode.FrameTooLarge, "frame_too_large", null)); + await SendSafeAsync(connection, error, server); + await connection.Socket.CloseAsync(WebSocketCloseStatus.MessageTooBig, "frame_too_large", CancellationToken.None); + server.CancelConnection(connection, "frame_too_large"); + break; + } + + var parse = Protocol.ParseClientMessage(received.Payload, config); + if (!parse.IsSuccess) + { + server.Logger.LogSecurityEvent("reject", + ("username", connection.Username), + ("code", parse.Error?.CodeName ?? "invalid_message"), + ("bytes", received.Payload.Length)); + var error = Protocol.SerializeProtocolError(parse.Error!); + await SendSafeAsync(connection, error, server); + continue; + } + + switch (parse.Kind) + { + case MessageKind.Clip: + var clip = (ClientClip)parse.Message!; + var hub = connection.Hub; + if (hub is null) + { + server.CancelConnection(connection, "user_loop_unavailable"); + break; + } + + var decision = hub.ClassifyClip(clip, connection); + if (decision == RecoveryDecision.QueueFull) + { + server.CancelConnection(connection, "recovery_queue_full"); + } + else if (decision == RecoveryDecision.ProcessNow + && !hub.TryWriteJob(new ClipJob(connection, clip))) + { + server.CancelConnection(connection, "user_loop_unavailable"); + } + break; + case MessageKind.Pong: + if (!connection.State.TryTakePongAwaiting()) + { + var unsolicitedPong = Protocol.SerializeProtocolError(new ProtocolError( + ProtocolErrorCode.InvalidMessage, + "Pong received without an outstanding ping.", + null)); + await SendSafeAsync(connection, unsolicitedPong, server); + continue; + } + + if (connection.Hub is null || !connection.Hub.TryWriteJob(new PongJob(connection, (ClientPong)parse.Message!))) + { + server.CancelConnection(connection, "user_loop_unavailable"); + } + break; + } + } + } + catch (OperationCanceledException) { } + catch (WebSocketException) { } + } + + private static async Task ConnectionSendLoopAsync(ConnectionContext connection) + { + try + { + await foreach (var payload in connection.State.SendQueue.Reader.ReadAllAsync(connection.State.Cts.Token)) + { + await connection.Socket.SendAsync(payload, WebSocketMessageType.Text, endOfMessage: true, connection.State.Cts.Token); + } + } + catch (OperationCanceledException) { } + catch (WebSocketException) { } + } + + private static async Task SendSafeAsync(ConnectionContext connection, byte[] payload, SyncServer server) + { + if (!connection.State.TryEnqueueSend(payload)) + { + server.EnqueueImmediateClose(connection, "server_busy"); + return; + } + } +} + + + +internal sealed class FrameTooLargeException : Exception; diff --git a/TextCascade.Server/Hosting/HeartbeatScannerService.cs b/TextCascade.Server/Hosting/HeartbeatScannerService.cs new file mode 100644 index 0000000..2890fcf --- /dev/null +++ b/TextCascade.Server/Hosting/HeartbeatScannerService.cs @@ -0,0 +1,50 @@ +using Microsoft.Extensions.Hosting; + +namespace TextCascade.Server; + +public sealed class HeartbeatScannerService : IHostedService, IDisposable +{ + private Timer? timer; + + private readonly SyncServer syncServer; + + public HeartbeatScannerService(SyncServer syncServer) + { + this.syncServer = syncServer; + } + + public Task StartAsync(CancellationToken cancellationToken) + { + timer = new Timer(Scan, null, TimeSpan.FromSeconds(1), TimeSpan.FromSeconds(1)); + return Task.CompletedTask; + } + + private void Scan(object? state) + { + var now = DateTimeOffset.UtcNow; + syncServer.ScanHeartbeats(now); + + var recoveryEnd = syncServer.ProcessStartTime.AddSeconds( + syncServer.Config.Limits.SnapshotWindowSeconds); + if (now < recoveryEnd) + { + return; + } + + foreach (var pair in syncServer.Registry.All) + { + pair.Value.CloseRecoveryWindow(now); + } + } + + public async Task StopAsync(CancellationToken cancellationToken) + { + timer?.Change(Timeout.Infinite, 0); + await syncServer.ShutdownAsync(TimeSpan.FromSeconds(2), DateTimeOffset.UtcNow); + } + + public void Dispose() + { + timer?.Dispose(); + } +} diff --git a/TextCascade.Server/Hosting/SyncEndpoint.cs b/TextCascade.Server/Hosting/SyncEndpoint.cs new file mode 100644 index 0000000..366a7e7 --- /dev/null +++ b/TextCascade.Server/Hosting/SyncEndpoint.cs @@ -0,0 +1,53 @@ +using Microsoft.AspNetCore.Http; + +namespace TextCascade.Server; + +public static class SyncEndpoint +{ + public static async Task HandleAsync(HttpContext context, RuntimeConfig config, SyncServer server) + { + var tokenHeader = context.Request.Headers.Authorization.ToString(); + if (!tokenHeader.StartsWith("Bearer ", StringComparison.Ordinal)) + { + context.Response.StatusCode = 401; + return; + } + + var compactToken = tokenHeader["Bearer ".Length..]; + var now = DateTimeOffset.UtcNow; + var tokenService = new TokenService(config.TokenSecret!); + if (!tokenService.TryVerifyToken(compactToken, now, server.UserLookup, out var payload)) + { + context.Response.StatusCode = 401; + return; + } + + if (!context.WebSockets.IsWebSocketRequest) + { + context.Response.StatusCode = 400; + return; + } + + var subProtocol = SelectSubProtocol(context.WebSockets.WebSocketRequestedProtocols); + if (subProtocol is null) + { + context.Response.StatusCode = 400; + return; + } + + using var socket = await context.WebSockets.AcceptWebSocketAsync(subProtocol); + var connectionId = Guid.NewGuid().ToString("N"); + var provisional = new ConnectionContext(connectionId, payload.Subject, "pending", "pending", socket, null!, config); + await ConnectionHandler.RunAsync(provisional, payload, config, server); + } + + internal static string? SelectSubProtocol(IList requested) + { + foreach (var protocol in requested) + { + if (string.Equals(protocol, "textcascade.v1", StringComparison.Ordinal)) return protocol; + } + return null; + } +} + diff --git a/TextCascade.Server/Hub/UserHub.cs b/TextCascade.Server/Hub/UserHub.cs new file mode 100644 index 0000000..feed53c --- /dev/null +++ b/TextCascade.Server/Hub/UserHub.cs @@ -0,0 +1,355 @@ +using System.Text; +using System.Threading.Channels; +using Microsoft.Extensions.Logging; + +namespace TextCascade.Server; + +public sealed class UserHub +{ + public string Username { get; } + public LatestText? Latest { get; private set; } + public ulong Version { get; private set; } + public Channel UserChannel { get; } + public TokenBucket ClipBucket { get; } + public SeenIdRing SeenIds { get; } + public DateTimeOffset ProcessStartTime { get; } + public DateTimeOffset LastActivityAt => new(new DateTime(Interlocked.Read(ref lastActivityTicks), DateTimeKind.Utc)); + + private readonly object connectionsGate = new(); + private readonly List connections = new(); + private readonly RuntimeConfig config; + private Task? userLoop; + + private readonly object snapshotGate = new(); + private readonly List snapshotCandidates = new(); + private int snapshotBytes; + private readonly List recoveryQueue = new(); + private bool recoveryWindowClosed; + + private readonly SyncServer server; + private readonly RuntimeStateStore runtimeStateStore; + private long lastActivityTicks; + + public UserHub(string username, RuntimeConfig config, DateTimeOffset processStart, SyncServer server, ulong initialVersion) + { + Username = username; + this.config = config; + this.server = server; + this.runtimeStateStore = server.RuntimeStateStore; + ProcessStartTime = processStart; + UserChannel = Channel.CreateUnbounded(new UnboundedChannelOptions { SingleReader = true, SingleWriter = false }); + ClipBucket = new TokenBucket(config.RateLimit.ClipBurst, config.RateLimit.ClipTokensPerSecond, processStart); + SeenIds = new SeenIdRing(config.Limits.SeenIdCapacity); + Version = initialVersion; + lastActivityTicks = processStart.UtcTicks; + } + + public IReadOnlyList Connections + { + get { lock (connectionsGate) { return connections.ToArray(); } } + } + + public bool IsEmpty + { + get { lock (connectionsGate) { return connections.Count == 0; } } + } + + internal object ScanGate => connectionsGate; + internal List ConnectionList => connections; + internal RuntimeConfig Config => config; + + public void AddConnection(ConnectionContext connection) + { + lock (connectionsGate) { connections.Add(connection); } + MarkActivity(DateTimeOffset.UtcNow); + var nowUtc = DateTimeOffset.UtcNow; + if (recoveryWindowClosed) + { + BroadcastToConnection(connection, Protocol.SerializeWelcome(Latest, config.Limits)); + return; + } + + EnsureRecoveryWindowClosed(nowUtc); + if (recoveryWindowClosed) + { + return; + } + } + + public bool RemoveConnection(ConnectionContext connection) + { + bool removed; + lock (connectionsGate) { removed = connections.Remove(connection); } + if (removed) + { + MarkActivity(DateTimeOffset.UtcNow); + } + + return removed; + } + + public void StartIfIdle() + { + if (userLoop is null || userLoop.IsCompleted) + { + userLoop = Task.Run(async () => + { + try { await RunUserLoopAsync(); } + catch (OperationCanceledException) { } + catch (Exception exception) + { + server.Logger.LogError( + exception, + "User loop failed; rebuilding hub. username={Username}", + Username); + server.RebuildHub(this); + } + }); + } + } + + public bool TryWriteJob(UserJob job) => UserChannel.Writer.TryWrite(job); + + public async Task RunUserLoopAsync(CancellationToken cancellationToken = default) + { + var reader = UserChannel.Reader; + while (await reader.WaitToReadAsync(cancellationToken).ConfigureAwait(false)) + { + while (reader.TryRead(out var job)) + { + ProcessJob(job, DateTimeOffset.UtcNow); + } + } + } + + private void ProcessJob(UserJob job, DateTimeOffset nowUtc) + { + switch (job) + { + case ClipJob clipJob: + ApplyClip(clipJob.Clip, clipJob.Sender, nowUtc); + break; + case PongJob pongJob: + pongJob.Connection.State.LastSeen = nowUtc; + break; + case HelloJob helloJob: + helloJob.Connection.State.HelloReceived = true; + if (helloJob.Hello.Snapshot is not null) + { + AcceptSnapshot(helloJob.Hello); + } + break; + case DisconnectJob disconnectJob: + server.CancelConnection(disconnectJob.Connection, disconnectJob.Reason); + break; + } + } + + public void AcceptSnapshot(ClientHello hello) + { + lock (snapshotGate) + { + if (recoveryWindowClosed) return; + if (hello.Snapshot is null) return; + var bytes = Encoding.UTF8.GetByteCount(hello.Snapshot.Payload); + if (snapshotBytes + bytes > config.Limits.SnapshotTotalBytes) return; + snapshotCandidates.Add(hello); + snapshotBytes += bytes; + } + } + + public RecoveryDecision ClassifyClip(ClientClip clip, ConnectionContext connection) + { + lock (snapshotGate) + { + if (recoveryWindowClosed) + { + return RecoveryDecision.ProcessNow; + } + + if (recoveryQueue.Count >= config.Limits.RecoveryClipQueueCapacity) + { + return RecoveryDecision.QueueFull; + } + + recoveryQueue.Add(new RecoveryClip(clip, connection)); + return RecoveryDecision.Queued; + } + } + + public void CloseRecoveryWindow(DateTimeOffset nowUtc) + { + List clips; + SnapshotWinner? winner; + lock (snapshotGate) + { + if (recoveryWindowClosed) return; + recoveryWindowClosed = true; + winner = CoreLogic.SelectSnapshotWinner(snapshotCandidates); + if (winner is not null) + { + var canRestoreLatest = winner.Version > Version + || (winner.Version == Version && Latest is null); + if (!canRestoreLatest) + { + winner = null; + } + else + { + if (winner.Version > Version) + { + runtimeStateStore.SaveVersion(Username, winner.Version); + } + Version = winner.Version; + Latest = new LatestText(winner.Snapshot.Payload, winner.Version, winner.Snapshot.Hash, winner.Snapshot.Encrypted, winner.ClientId, winner.ClientName, winner.Snapshot.LocalModifiedAtUtc); + } + } + clips = recoveryQueue.ToList(); + recoveryQueue.Clear(); + } + + foreach (var recovery in clips) + { + if (recovery.Connection.State.IsClosed) continue; + ApplyClip(recovery.Clip, recovery.Connection, nowUtc); + } + + BroadcastWelcome(nowUtc); + + // Spec §6.2: empty hubs that survived until the recovery window closes are now removed. + server.Registry.RemoveIfEmpty(this, allowDuringRecovery: true); + + MarkActivity(nowUtc); + } + + private void BroadcastWelcome(DateTimeOffset nowUtc) + { + var bytes = Protocol.SerializeWelcome(Latest, config.Limits); + foreach (var connection in Connections) + { + if (!connection.State.TryEnqueueSend(bytes) && connection.State.MarkClosed()) + { + connection.State.Cts.Cancel(); + } + } + } + + public bool IsRecoveryWindowOpen(DateTimeOffset nowUtc) + { + return !recoveryWindowClosed && nowUtc < ProcessStartTime.AddSeconds(config.Limits.SnapshotWindowSeconds); + } + + public void EnsureRecoveryWindowClosed(DateTimeOffset nowUtc) + { + if (!recoveryWindowClosed && nowUtc >= ProcessStartTime.AddSeconds(config.Limits.SnapshotWindowSeconds)) + { + CloseRecoveryWindow(nowUtc); + } + } + + private void MarkActivity(DateTimeOffset nowUtc) + { + Interlocked.Exchange(ref lastActivityTicks, nowUtc.UtcTicks); + } + + internal void MarkActivityForScan(DateTimeOffset nowUtc) => MarkActivity(nowUtc); + + public void ApplyClip(ClientClip clip, ConnectionContext sender, DateTimeOffset nowUtc) + { + if (SeenIds.IsUnchangedDuplicate(clip.Id, clip.Payload, clip.Hash, clip.Encrypted, out var duplicateLatest)) + { + var ackBytes = Protocol.SerializeClipAck(clip.Id, duplicateLatest ?? Latest ?? new LatestText(string.Empty, Version, string.Empty, false, sender.ClientId, sender.ClientName, nowUtc)); + if (!sender.State.TryEnqueueSend(ackBytes) && sender.State.MarkClosed()) + { + sender.State.Cts.Cancel(); + } + return; + } + + if (SeenIds.TryGetResult(clip.Id, out _)) + { + server.Logger.LogWarning( + "Replacing reused clip id. username={Username} clipId={ClipId} clientId={ClientId} previousVersion={PreviousVersion}", + Username, + clip.Id, + sender.ClientId, + Version); + } + + if (!ClipBucket.TryAcquire(nowUtc)) + { + server.Logger.LogSecurityEvent("reject", + ("username", Username), + ("code", "rate_limited"), + ("bytes", Encoding.UTF8.GetByteCount(clip.Payload))); + var error = Protocol.SerializeProtocolError(new ProtocolError(ProtocolErrorCode.RateLimited, "Clip rate limited.", clip.Id)); + if (!sender.State.TryEnqueueSend(error) && sender.State.MarkClosed()) + { + sender.State.Cts.Cancel(); + } + return; + } + + var next = CoreLogic.NextVersion(Version); + runtimeStateStore.SaveVersion(Username, next); + Version = next; + var latest = new LatestText(clip.Payload, next, clip.Hash, clip.Encrypted, sender.ClientId, sender.ClientName, nowUtc); + Latest = latest; + SeenIds.RememberId(clip.Id, latest); + server.Logger.LogSecurityEvent("clip", + ("username", Username), + ("version", latest.Version), + ("clipId", clip.Id), + ("bytes", Encoding.UTF8.GetByteCount(clip.Payload)), + ("fromClientId", sender.ClientId), + ("encrypted", clip.Encrypted)); + + var broadcastBytes = Protocol.SerializeClip(clip.Id, latest); + var deliveries = new List(); + foreach (var connection in Connections) + { + if (ReferenceEquals(connection, sender)) continue; + var queued = connection.State.TryEnqueueSend(broadcastBytes); + deliveries.Add($"{connection.ClientId}:{(queued ? "queued" : "full")}"); + if (!queued && connection.State.MarkClosed()) + { + connection.State.Cts.Cancel(); + } + } + + server.Logger.LogInformation( + "Clip broadcast. username={Username} version={Version} clipId={ClipId} recipients=[{Recipients}]", + Username, + next, + clip.Id, + string.Join(",", deliveries)); + + var ackBytesFinal = Protocol.SerializeClipAck(clip.Id, latest); + if (!sender.State.TryEnqueueSend(ackBytesFinal) && sender.State.MarkClosed()) + { + sender.State.Cts.Cancel(); + } + } + + private static void BroadcastToConnection(ConnectionContext connection, byte[] payload) + { + if (!connection.State.TryEnqueueSend(payload) && connection.State.MarkClosed()) + { + try { connection.State.Cts.Cancel(); } catch (Exception) { } + } + } + + public void BroadcastAsync(byte[] payload) + { + foreach (var connection in Connections) + { + if (connection.State.IsClosed) continue; + if (!connection.State.TryEnqueueSend(payload) && connection.State.MarkClosed()) + { + try { connection.State.Cts.Cancel(); } catch (Exception) { } + } + } + } +} + + diff --git a/TextCascade.Server/Hub/UserJobs.cs b/TextCascade.Server/Hub/UserJobs.cs new file mode 100644 index 0000000..470bd10 --- /dev/null +++ b/TextCascade.Server/Hub/UserJobs.cs @@ -0,0 +1,21 @@ +namespace TextCascade.Server; + +public readonly record struct RecoveryClip(ClientClip Clip, ConnectionContext Connection); + +public enum RecoveryDecision +{ + Queued, + ProcessNow, + QueueFull, +} + +public abstract record UserJob; + +public sealed record ClipJob(ConnectionContext Sender, ClientClip Clip) : UserJob; + +public sealed record HelloJob(ConnectionContext Connection, ClientHello Hello) : UserJob; + +public sealed record PongJob(ConnectionContext Connection, ClientPong Pong) : UserJob; + +public sealed record DisconnectJob(ConnectionContext Connection, string Reason) : UserJob; + diff --git a/TextCascade.Server/Hub/UserRegistry.cs b/TextCascade.Server/Hub/UserRegistry.cs new file mode 100644 index 0000000..508fb8b --- /dev/null +++ b/TextCascade.Server/Hub/UserRegistry.cs @@ -0,0 +1,29 @@ +using System.Collections.Concurrent; + +namespace TextCascade.Server; + +public sealed class UserRegistry +{ + private readonly ConcurrentDictionary hubs = new(StringComparer.Ordinal); + public IEnumerable> All => hubs; + + public UserHub GetOrAdd(string username, Func factory) + { + return hubs.GetOrAdd(username, factory); + } + + public bool TryGetValue(string username, out UserHub hub) => hubs.TryGetValue(username, out hub!); + + public void RemoveIfEmpty(UserHub hub, bool allowDuringRecovery) + { + if (!hub.IsEmpty) return; + if (!allowDuringRecovery && hub.IsRecoveryWindowOpen(DateTimeOffset.UtcNow)) return; + hubs.TryRemove(hub.Username, out _); + } + + public bool Remove(UserHub hub) + { + return hubs.TryRemove(new KeyValuePair(hub.Username, hub)); + } +} + diff --git a/TextCascade.Server/Models/ConnectionContext.cs b/TextCascade.Server/Models/ConnectionContext.cs new file mode 100644 index 0000000..d9d6ede --- /dev/null +++ b/TextCascade.Server/Models/ConnectionContext.cs @@ -0,0 +1,26 @@ +using System.Net.WebSockets; + +namespace TextCascade.Server; + +public sealed class ConnectionContext +{ + public string ConnectionId { get; } + public string Username { get; } + public string ClientId { get; } + public string ClientName { get; } + public WebSocket Socket { get; } + public UserHub? Hub { get; internal set; } + public ConnectionStateBag State { get; } + + public ConnectionContext(string connectionId, string username, string clientId, string clientName, WebSocket socket, UserHub? hub, RuntimeConfig config) + { + ConnectionId = connectionId; + Username = username; + ClientId = clientId; + ClientName = clientName; + Socket = socket; + Hub = hub; + State = new ConnectionStateBag(config); + } +} + diff --git a/TextCascade.Server/Models/ConnectionStateBag.cs b/TextCascade.Server/Models/ConnectionStateBag.cs new file mode 100644 index 0000000..f35acaf --- /dev/null +++ b/TextCascade.Server/Models/ConnectionStateBag.cs @@ -0,0 +1,88 @@ +using System.Threading.Channels; + +namespace TextCascade.Server; + +public sealed class ConnectionStateBag +{ + private readonly object gate = new(); + private DateTimeOffset lastSeen; + private DateTimeOffset lastPingAt; + private bool closed; + private bool helloTimeoutStarted; + private bool pongAwaited; + public Channel SendQueue { get; } + public CancellationTokenSource Cts { get; } + public bool HelloReceived { get; internal set; } + public DateTimeOffset? HelloDeadline { get; internal set; } + + public DateTimeOffset LastSeen + { + get { lock (gate) { return lastSeen; } } + internal set { lock (gate) { lastSeen = value; } } + } + + public DateTimeOffset LastPingAt + { + get { lock (gate) { return lastPingAt; } } + internal set { lock (gate) { lastPingAt = value; } } + } + + public void MarkPingAwaitingPong() + { + lock (gate) { pongAwaited = true; } + } + + public bool TryTakePongAwaiting() + { + lock (gate) + { + if (!pongAwaited) return false; + pongAwaited = false; + return true; + } + } + + public bool IsClosed + { + get { lock (gate) { return closed; } } + } + + public bool MarkClosed() + { + lock (gate) + { + if (closed) return false; + closed = true; + return true; + } + } + + public bool TryStartHelloTimeout() + { + lock (gate) + { + if (helloTimeoutStarted || closed) + { + return false; + } + + helloTimeoutStarted = true; + return true; + } + } + + public ConnectionStateBag(RuntimeConfig config) + { + lastSeen = DateTimeOffset.UtcNow; + lastPingAt = lastSeen; + SendQueue = Channel.CreateBounded(config.Limits.SendQueueCapacity); + Cts = new CancellationTokenSource(); + HelloDeadline = DateTimeOffset.UtcNow.AddSeconds(config.Limits.HelloTimeoutSeconds); + } + + public bool TryEnqueueSend(byte[] payload) + { + return SendQueue.Writer.TryWrite(payload); + } +} + diff --git a/TextCascade.Server/Models/ReceivedMessage.cs b/TextCascade.Server/Models/ReceivedMessage.cs new file mode 100644 index 0000000..04e844f --- /dev/null +++ b/TextCascade.Server/Models/ReceivedMessage.cs @@ -0,0 +1,5 @@ +using System.Net.WebSockets; + +namespace TextCascade.Server; + +internal sealed record ReceivedMessage(WebSocketMessageType MessageType, byte[] Payload); diff --git a/TextCascade.Server/ServerHost.cs b/TextCascade.Server/ServerHost.cs index 7643a0b..a76c782 100644 --- a/TextCascade.Server/ServerHost.cs +++ b/TextCascade.Server/ServerHost.cs @@ -222,50 +222,3 @@ public void Dispose() } } -public sealed class HeartbeatScannerService : IHostedService, IDisposable -{ - private Timer? timer; - - private readonly SyncServer syncServer; - - public HeartbeatScannerService(SyncServer syncServer) - { - this.syncServer = syncServer; - } - - public Task StartAsync(CancellationToken cancellationToken) - { - timer = new Timer(Scan, null, TimeSpan.FromSeconds(1), TimeSpan.FromSeconds(1)); - return Task.CompletedTask; - } - - private void Scan(object? state) - { - var now = DateTimeOffset.UtcNow; - syncServer.ScanHeartbeats(now); - - var recoveryEnd = syncServer.ProcessStartTime.AddSeconds( - syncServer.Config.Limits.SnapshotWindowSeconds); - if (now < recoveryEnd) - { - return; - } - - foreach (var pair in syncServer.Registry.All) - { - pair.Value.CloseRecoveryWindow(now); - } - } - - public async Task StopAsync(CancellationToken cancellationToken) - { - timer?.Change(Timeout.Infinite, 0); - await syncServer.ShutdownAsync(TimeSpan.FromSeconds(2), DateTimeOffset.UtcNow); - } - - public void Dispose() - { - timer?.Dispose(); - } -} - diff --git a/TextCascade.Server/SyncServer.cs b/TextCascade.Server/SyncServer.cs index 44d058c..cbb8759 100644 --- a/TextCascade.Server/SyncServer.cs +++ b/TextCascade.Server/SyncServer.cs @@ -1,511 +1,10 @@ -using System.Collections.Concurrent; +using System.Collections.Concurrent; using System.Net.WebSockets; using System.Text; -using System.Threading.Channels; -using System.Globalization; -using Microsoft.AspNetCore.Http; using Microsoft.Extensions.Logging; namespace TextCascade.Server; -public sealed class ConnectionContext -{ - public string ConnectionId { get; } - public string Username { get; } - public string ClientId { get; } - public string ClientName { get; } - public WebSocket Socket { get; } - public UserHub? Hub { get; internal set; } - public ConnectionStateBag State { get; } - - public ConnectionContext(string connectionId, string username, string clientId, string clientName, WebSocket socket, UserHub? hub, RuntimeConfig config) - { - ConnectionId = connectionId; - Username = username; - ClientId = clientId; - ClientName = clientName; - Socket = socket; - Hub = hub; - State = new ConnectionStateBag(config); - } -} - -public sealed class ConnectionStateBag -{ - private readonly object gate = new(); - private DateTimeOffset lastSeen; - private DateTimeOffset lastPingAt; - private bool closed; - private bool helloTimeoutStarted; - private bool pongAwaited; - public Channel SendQueue { get; } - public CancellationTokenSource Cts { get; } - public bool HelloReceived { get; internal set; } - public DateTimeOffset? HelloDeadline { get; internal set; } - - public DateTimeOffset LastSeen - { - get { lock (gate) { return lastSeen; } } - internal set { lock (gate) { lastSeen = value; } } - } - - public DateTimeOffset LastPingAt - { - get { lock (gate) { return lastPingAt; } } - internal set { lock (gate) { lastPingAt = value; } } - } - - public void MarkPingAwaitingPong() - { - lock (gate) { pongAwaited = true; } - } - - public bool TryTakePongAwaiting() - { - lock (gate) - { - if (!pongAwaited) return false; - pongAwaited = false; - return true; - } - } - - public bool IsClosed - { - get { lock (gate) { return closed; } } - } - - public bool MarkClosed() - { - lock (gate) - { - if (closed) return false; - closed = true; - return true; - } - } - - public bool TryStartHelloTimeout() - { - lock (gate) - { - if (helloTimeoutStarted || closed) - { - return false; - } - - helloTimeoutStarted = true; - return true; - } - } - - public ConnectionStateBag(RuntimeConfig config) - { - lastSeen = DateTimeOffset.UtcNow; - lastPingAt = lastSeen; - SendQueue = Channel.CreateBounded(config.Limits.SendQueueCapacity); - Cts = new CancellationTokenSource(); - HelloDeadline = DateTimeOffset.UtcNow.AddSeconds(config.Limits.HelloTimeoutSeconds); - } - - public bool TryEnqueueSend(byte[] payload) - { - return SendQueue.Writer.TryWrite(payload); - } -} - -public sealed class UserHub -{ - public string Username { get; } - public LatestText? Latest { get; private set; } - public ulong Version { get; private set; } - public Channel UserChannel { get; } - public TokenBucket ClipBucket { get; } - public SeenIdRing SeenIds { get; } - public DateTimeOffset ProcessStartTime { get; } - public DateTimeOffset LastActivityAt => new(new DateTime(Interlocked.Read(ref lastActivityTicks), DateTimeKind.Utc)); - - private readonly object connectionsGate = new(); - private readonly List connections = new(); - private readonly RuntimeConfig config; - private Task? userLoop; - - private readonly object snapshotGate = new(); - private readonly List snapshotCandidates = new(); - private int snapshotBytes; - private readonly List recoveryQueue = new(); - private bool recoveryWindowClosed; - - private readonly SyncServer server; - private readonly RuntimeStateStore runtimeStateStore; - private long lastActivityTicks; - - public UserHub(string username, RuntimeConfig config, DateTimeOffset processStart, SyncServer server, ulong initialVersion) - { - Username = username; - this.config = config; - this.server = server; - this.runtimeStateStore = server.RuntimeStateStore; - ProcessStartTime = processStart; - UserChannel = Channel.CreateUnbounded(new UnboundedChannelOptions { SingleReader = true, SingleWriter = false }); - ClipBucket = new TokenBucket(config.RateLimit.ClipBurst, config.RateLimit.ClipTokensPerSecond, processStart); - SeenIds = new SeenIdRing(config.Limits.SeenIdCapacity); - Version = initialVersion; - lastActivityTicks = processStart.UtcTicks; - } - - public IReadOnlyList Connections - { - get { lock (connectionsGate) { return connections.ToArray(); } } - } - - public bool IsEmpty - { - get { lock (connectionsGate) { return connections.Count == 0; } } - } - - internal object ScanGate => connectionsGate; - internal List ConnectionList => connections; - internal RuntimeConfig Config => config; - - public void AddConnection(ConnectionContext connection) - { - lock (connectionsGate) { connections.Add(connection); } - MarkActivity(DateTimeOffset.UtcNow); - var nowUtc = DateTimeOffset.UtcNow; - if (recoveryWindowClosed) - { - BroadcastToConnection(connection, Protocol.SerializeWelcome(Latest, config.Limits)); - return; - } - - EnsureRecoveryWindowClosed(nowUtc); - if (recoveryWindowClosed) - { - return; - } - } - - public bool RemoveConnection(ConnectionContext connection) - { - bool removed; - lock (connectionsGate) { removed = connections.Remove(connection); } - if (removed) - { - MarkActivity(DateTimeOffset.UtcNow); - } - - return removed; - } - - public void StartIfIdle() - { - if (userLoop is null || userLoop.IsCompleted) - { - userLoop = Task.Run(async () => - { - try { await RunUserLoopAsync(); } - catch (OperationCanceledException) { } - catch (Exception exception) - { - server.Logger.LogError( - exception, - "User loop failed; rebuilding hub. username={Username}", - Username); - server.RebuildHub(this); - } - }); - } - } - - public bool TryWriteJob(UserJob job) => UserChannel.Writer.TryWrite(job); - - public async Task RunUserLoopAsync(CancellationToken cancellationToken = default) - { - var reader = UserChannel.Reader; - while (await reader.WaitToReadAsync(cancellationToken).ConfigureAwait(false)) - { - while (reader.TryRead(out var job)) - { - ProcessJob(job, DateTimeOffset.UtcNow); - } - } - } - - private void ProcessJob(UserJob job, DateTimeOffset nowUtc) - { - switch (job) - { - case ClipJob clipJob: - ApplyClip(clipJob.Clip, clipJob.Sender, nowUtc); - break; - case PongJob pongJob: - pongJob.Connection.State.LastSeen = nowUtc; - break; - case HelloJob helloJob: - helloJob.Connection.State.HelloReceived = true; - if (helloJob.Hello.Snapshot is not null) - { - AcceptSnapshot(helloJob.Hello); - } - break; - case DisconnectJob disconnectJob: - server.CancelConnection(disconnectJob.Connection, disconnectJob.Reason); - break; - } - } - - public void AcceptSnapshot(ClientHello hello) - { - lock (snapshotGate) - { - if (recoveryWindowClosed) return; - if (hello.Snapshot is null) return; - var bytes = Encoding.UTF8.GetByteCount(hello.Snapshot.Payload); - if (snapshotBytes + bytes > config.Limits.SnapshotTotalBytes) return; - snapshotCandidates.Add(hello); - snapshotBytes += bytes; - } - } - - public RecoveryDecision ClassifyClip(ClientClip clip, ConnectionContext connection) - { - lock (snapshotGate) - { - if (recoveryWindowClosed) - { - return RecoveryDecision.ProcessNow; - } - - if (recoveryQueue.Count >= config.Limits.RecoveryClipQueueCapacity) - { - return RecoveryDecision.QueueFull; - } - - recoveryQueue.Add(new RecoveryClip(clip, connection)); - return RecoveryDecision.Queued; - } - } - - public void CloseRecoveryWindow(DateTimeOffset nowUtc) - { - List clips; - SnapshotWinner? winner; - lock (snapshotGate) - { - if (recoveryWindowClosed) return; - recoveryWindowClosed = true; - winner = CoreLogic.SelectSnapshotWinner(snapshotCandidates); - if (winner is not null) - { - var canRestoreLatest = winner.Version > Version - || (winner.Version == Version && Latest is null); - if (!canRestoreLatest) - { - winner = null; - } - else - { - if (winner.Version > Version) - { - runtimeStateStore.SaveVersion(Username, winner.Version); - } - Version = winner.Version; - Latest = new LatestText(winner.Snapshot.Payload, winner.Version, winner.Snapshot.Hash, winner.Snapshot.Encrypted, winner.ClientId, winner.ClientName, winner.Snapshot.LocalModifiedAtUtc); - } - } - clips = recoveryQueue.ToList(); - recoveryQueue.Clear(); - } - - foreach (var recovery in clips) - { - if (recovery.Connection.State.IsClosed) continue; - ApplyClip(recovery.Clip, recovery.Connection, nowUtc); - } - - BroadcastWelcome(nowUtc); - - // Spec §6.2: empty hubs that survived until the recovery window closes are now removed. - server.Registry.RemoveIfEmpty(this, allowDuringRecovery: true); - - MarkActivity(nowUtc); - } - - private void BroadcastWelcome(DateTimeOffset nowUtc) - { - var bytes = Protocol.SerializeWelcome(Latest, config.Limits); - foreach (var connection in Connections) - { - if (!connection.State.TryEnqueueSend(bytes) && connection.State.MarkClosed()) - { - connection.State.Cts.Cancel(); - } - } - } - - public bool IsRecoveryWindowOpen(DateTimeOffset nowUtc) - { - return !recoveryWindowClosed && nowUtc < ProcessStartTime.AddSeconds(config.Limits.SnapshotWindowSeconds); - } - - public void EnsureRecoveryWindowClosed(DateTimeOffset nowUtc) - { - if (!recoveryWindowClosed && nowUtc >= ProcessStartTime.AddSeconds(config.Limits.SnapshotWindowSeconds)) - { - CloseRecoveryWindow(nowUtc); - } - } - - private void MarkActivity(DateTimeOffset nowUtc) - { - Interlocked.Exchange(ref lastActivityTicks, nowUtc.UtcTicks); - } - - internal void MarkActivityForScan(DateTimeOffset nowUtc) => MarkActivity(nowUtc); - - public void ApplyClip(ClientClip clip, ConnectionContext sender, DateTimeOffset nowUtc) - { - if (SeenIds.IsUnchangedDuplicate(clip.Id, clip.Payload, clip.Hash, clip.Encrypted, out var duplicateLatest)) - { - var ackBytes = Protocol.SerializeClipAck(clip.Id, duplicateLatest ?? Latest ?? new LatestText(string.Empty, Version, string.Empty, false, sender.ClientId, sender.ClientName, nowUtc)); - if (!sender.State.TryEnqueueSend(ackBytes) && sender.State.MarkClosed()) - { - sender.State.Cts.Cancel(); - } - return; - } - - if (SeenIds.TryGetResult(clip.Id, out _)) - { - server.Logger.LogWarning( - "Replacing reused clip id. username={Username} clipId={ClipId} clientId={ClientId} previousVersion={PreviousVersion}", - Username, - clip.Id, - sender.ClientId, - Version); - } - - if (!ClipBucket.TryAcquire(nowUtc)) - { - server.Logger.LogSecurityEvent("reject", - ("username", Username), - ("code", "rate_limited"), - ("bytes", Encoding.UTF8.GetByteCount(clip.Payload))); - var error = Protocol.SerializeProtocolError(new ProtocolError(ProtocolErrorCode.RateLimited, "Clip rate limited.", clip.Id)); - if (!sender.State.TryEnqueueSend(error) && sender.State.MarkClosed()) - { - sender.State.Cts.Cancel(); - } - return; - } - - var next = CoreLogic.NextVersion(Version); - runtimeStateStore.SaveVersion(Username, next); - Version = next; - var latest = new LatestText(clip.Payload, next, clip.Hash, clip.Encrypted, sender.ClientId, sender.ClientName, nowUtc); - Latest = latest; - SeenIds.RememberId(clip.Id, latest); - server.Logger.LogSecurityEvent("clip", - ("username", Username), - ("version", latest.Version), - ("clipId", clip.Id), - ("bytes", Encoding.UTF8.GetByteCount(clip.Payload)), - ("fromClientId", sender.ClientId), - ("encrypted", clip.Encrypted)); - - var broadcastBytes = Protocol.SerializeClip(clip.Id, latest); - var deliveries = new List(); - foreach (var connection in Connections) - { - if (ReferenceEquals(connection, sender)) continue; - var queued = connection.State.TryEnqueueSend(broadcastBytes); - deliveries.Add($"{connection.ClientId}:{(queued ? "queued" : "full")}"); - if (!queued && connection.State.MarkClosed()) - { - connection.State.Cts.Cancel(); - } - } - - server.Logger.LogInformation( - "Clip broadcast. username={Username} version={Version} clipId={ClipId} recipients=[{Recipients}]", - Username, - next, - clip.Id, - string.Join(",", deliveries)); - - var ackBytesFinal = Protocol.SerializeClipAck(clip.Id, latest); - if (!sender.State.TryEnqueueSend(ackBytesFinal) && sender.State.MarkClosed()) - { - sender.State.Cts.Cancel(); - } - } - - private static void BroadcastToConnection(ConnectionContext connection, byte[] payload) - { - if (!connection.State.TryEnqueueSend(payload) && connection.State.MarkClosed()) - { - try { connection.State.Cts.Cancel(); } catch (Exception) { } - } - } - - public void BroadcastAsync(byte[] payload) - { - foreach (var connection in Connections) - { - if (connection.State.IsClosed) continue; - if (!connection.State.TryEnqueueSend(payload) && connection.State.MarkClosed()) - { - try { connection.State.Cts.Cancel(); } catch (Exception) { } - } - } - } -} - -public readonly record struct RecoveryClip(ClientClip Clip, ConnectionContext Connection); - -public enum RecoveryDecision -{ - Queued, - ProcessNow, - QueueFull, -} - -public abstract record UserJob; - -public sealed record ClipJob(ConnectionContext Sender, ClientClip Clip) : UserJob; - -public sealed record HelloJob(ConnectionContext Connection, ClientHello Hello) : UserJob; - -public sealed record PongJob(ConnectionContext Connection, ClientPong Pong) : UserJob; - -public sealed record DisconnectJob(ConnectionContext Connection, string Reason) : UserJob; - -public sealed class UserRegistry -{ - private readonly ConcurrentDictionary hubs = new(StringComparer.Ordinal); - public IEnumerable> All => hubs; - - public UserHub GetOrAdd(string username, Func factory) - { - return hubs.GetOrAdd(username, factory); - } - - public bool TryGetValue(string username, out UserHub hub) => hubs.TryGetValue(username, out hub!); - - public void RemoveIfEmpty(UserHub hub, bool allowDuringRecovery) - { - if (!hub.IsEmpty) return; - if (!allowDuringRecovery && hub.IsRecoveryWindowOpen(DateTimeOffset.UtcNow)) return; - hubs.TryRemove(hub.Username, out _); - } - - public bool Remove(UserHub hub) - { - return hubs.TryRemove(new KeyValuePair(hub.Username, hub)); - } -} - public interface IClock { DateTimeOffset UtcNow { get; } @@ -774,315 +273,3 @@ private static async Task CloseConnectionAsync(ConnectionContext connection, Web } } -public static class SyncEndpoint -{ - public static async Task HandleAsync(HttpContext context, RuntimeConfig config, SyncServer server) - { - var tokenHeader = context.Request.Headers.Authorization.ToString(); - if (!tokenHeader.StartsWith("Bearer ", StringComparison.Ordinal)) - { - context.Response.StatusCode = 401; - return; - } - - var compactToken = tokenHeader["Bearer ".Length..]; - var now = DateTimeOffset.UtcNow; - var tokenService = new TokenService(config.TokenSecret!); - if (!tokenService.TryVerifyToken(compactToken, now, server.UserLookup, out var payload)) - { - context.Response.StatusCode = 401; - return; - } - - if (!context.WebSockets.IsWebSocketRequest) - { - context.Response.StatusCode = 400; - return; - } - - var subProtocol = SelectSubProtocol(context.WebSockets.WebSocketRequestedProtocols); - if (subProtocol is null) - { - context.Response.StatusCode = 400; - return; - } - - using var socket = await context.WebSockets.AcceptWebSocketAsync(subProtocol); - var connectionId = Guid.NewGuid().ToString("N"); - var provisional = new ConnectionContext(connectionId, payload.Subject, "pending", "pending", socket, null!, config); - await ConnectionHandler.RunAsync(provisional, payload, config, server); - } - - internal static string? SelectSubProtocol(IList requested) - { - foreach (var protocol in requested) - { - if (string.Equals(protocol, "textcascade.v1", StringComparison.Ordinal)) return protocol; - } - return null; - } -} - -public static class ConnectionHandler -{ - public static async Task RunAsync(ConnectionContext provisional, TokenPayload payload, RuntimeConfig config, SyncServer server) - { - server.RegisterPendingHello(provisional); - ClientHello hello; - try - { - var received = await ReceiveFrameAsync(provisional, config.Limits.MaxFrameBytes, provisional.State.Cts.Token); - if (received.MessageType == WebSocketMessageType.Close) - { - await provisional.Socket.CloseOutputAsync( - WebSocketCloseStatus.NormalClosure, - "client_closed", - CancellationToken.None); - server.CancelConnection(provisional, "closed"); - return; - } - - var parse = Protocol.ParseClientMessage(received.Payload, config); - if (!parse.IsSuccess || parse.Kind != MessageKind.Hello) - { - var error = Protocol.SerializeProtocolError(new ProtocolError( - ProtocolErrorCode.InvalidMessage, - "Expected a valid hello message.", - parse.Error?.ReferenceId)); - await SendAndClosePreHelloAsync( - provisional, - error, - WebSocketCloseStatus.PolicyViolation, - "invalid_hello", - server); - return; - } - - hello = (ClientHello)parse.Message!; - } - catch (FrameTooLargeException) - { - var error = Protocol.SerializeProtocolError(new ProtocolError(ProtocolErrorCode.FrameTooLarge, "frame_too_large", null)); - await SendAndClosePreHelloAsync(provisional, error, WebSocketCloseStatus.MessageTooBig, "frame_too_large", server); - return; - } - catch (OperationCanceledException) - { - // Hello timeout is owned by the unified heartbeat scanner; here the socket was - // cancelled for another reason (e.g. shutdown). Fall through to unified cleanup. - server.CancelConnection(provisional, "cancelled"); - return; - } - catch (WebSocketException) - { - server.CancelConnection(provisional, "socket_error"); - return; - } - - var hub = server.GetOrCreateHub(payload.Subject, config); - var connection = new ConnectionContext( - provisional.ConnectionId, - payload.Subject, - hello.ClientId, - hello.ClientName, - provisional.Socket, - hub, - config); - hub.AddConnection(connection); - server.Logger.LogSecurityEvent("connect", - ("username", connection.Username), - ("clientId", connection.ClientId), - ("connectionId", connection.ConnectionId)); - connection.State.HelloReceived = true; - connection.State.LastSeen = DateTimeOffset.UtcNow; - server.UnregisterPendingHello(provisional); - if (!hub.TryWriteJob(new HelloJob(connection, hello))) - { - server.CancelConnection(connection, "user_loop_unavailable"); - return; - } - - var sendTask = ConnectionSendLoopAsync(connection); - var readTask = ReadLoopAsync(connection, config, server); - await Task.WhenAll(sendTask, readTask); - server.CancelConnection(connection, "disconnected"); - } - - private static async Task ReceiveFrameAsync( - ConnectionContext connection, - int maxBytes, - CancellationToken cancellationToken) - { - using var stream = new MemoryStream(Math.Min(maxBytes, 16 * 1024)); - var buffer = new byte[Math.Min(maxBytes, 16 * 1024)]; - while (true) - { - var received = await connection.Socket.ReceiveAsync(new ArraySegment(buffer), cancellationToken); - if (received.Count > maxBytes - stream.Length) - { - throw new FrameTooLargeException(); - } - - stream.Write(buffer, 0, received.Count); - if (received.EndOfMessage) - { - return new ReceivedMessage(received.MessageType, stream.ToArray()); - } - } - } - - private static async Task SendAndClosePreHelloAsync( - ConnectionContext connection, - byte[] error, - WebSocketCloseStatus status, - string reason, - SyncServer server) - { - try - { - if (connection.Socket.State == WebSocketState.Open) - { - await connection.Socket.SendAsync(error, WebSocketMessageType.Text, true, CancellationToken.None); - await connection.Socket.CloseAsync(status, reason, CancellationToken.None); - } - } - catch (Exception) - { - server.EnqueueImmediateClose(connection, "server_busy"); - } - finally - { - server.CancelConnection(connection, reason); - } - } - - private static async Task ReadLoopAsync(ConnectionContext connection, RuntimeConfig config, SyncServer server) - { - try - { - while (connection.State.Cts.IsCancellationRequested == false && connection.Socket.State == WebSocketState.Open) - { - ReceivedMessage received; - try - { - received = await ReceiveFrameAsync(connection, config.Limits.MaxFrameBytes, connection.State.Cts.Token); - } - catch (FrameTooLargeException) - { - var oversized = Protocol.SerializeProtocolError(new ProtocolError(ProtocolErrorCode.FrameTooLarge, "frame_too_large", null)); - await SendSafeAsync(connection, oversized, server); - await Task.Delay(100, connection.State.Cts.Token); - if (connection.Socket.State == WebSocketState.Open) - { - await connection.Socket.CloseAsync(WebSocketCloseStatus.MessageTooBig, "frame_too_large", CancellationToken.None); - } - server.CancelConnection(connection, "frame_too_large"); - break; - } - - if (received.MessageType == WebSocketMessageType.Close) - { - try - { - await connection.Socket.CloseOutputAsync( - WebSocketCloseStatus.NormalClosure, - "client_closed", - CancellationToken.None); - } - catch (WebSocketException) { } - break; - } - - if (!Protocol.CheckFrameSize(received.Payload.Length, config)) - { - var error = Protocol.SerializeProtocolError(new ProtocolError(ProtocolErrorCode.FrameTooLarge, "frame_too_large", null)); - await SendSafeAsync(connection, error, server); - await connection.Socket.CloseAsync(WebSocketCloseStatus.MessageTooBig, "frame_too_large", CancellationToken.None); - server.CancelConnection(connection, "frame_too_large"); - break; - } - - var parse = Protocol.ParseClientMessage(received.Payload, config); - if (!parse.IsSuccess) - { - server.Logger.LogSecurityEvent("reject", - ("username", connection.Username), - ("code", parse.Error?.CodeName ?? "invalid_message"), - ("bytes", received.Payload.Length)); - var error = Protocol.SerializeProtocolError(parse.Error!); - await SendSafeAsync(connection, error, server); - continue; - } - - switch (parse.Kind) - { - case MessageKind.Clip: - var clip = (ClientClip)parse.Message!; - var hub = connection.Hub; - if (hub is null) - { - server.CancelConnection(connection, "user_loop_unavailable"); - break; - } - - var decision = hub.ClassifyClip(clip, connection); - if (decision == RecoveryDecision.QueueFull) - { - server.CancelConnection(connection, "recovery_queue_full"); - } - else if (decision == RecoveryDecision.ProcessNow - && !hub.TryWriteJob(new ClipJob(connection, clip))) - { - server.CancelConnection(connection, "user_loop_unavailable"); - } - break; - case MessageKind.Pong: - if (!connection.State.TryTakePongAwaiting()) - { - var unsolicitedPong = Protocol.SerializeProtocolError(new ProtocolError( - ProtocolErrorCode.InvalidMessage, - "Pong received without an outstanding ping.", - null)); - await SendSafeAsync(connection, unsolicitedPong, server); - continue; - } - - if (connection.Hub is null || !connection.Hub.TryWriteJob(new PongJob(connection, (ClientPong)parse.Message!))) - { - server.CancelConnection(connection, "user_loop_unavailable"); - } - break; - } - } - } - catch (OperationCanceledException) { } - catch (WebSocketException) { } - } - - private static async Task ConnectionSendLoopAsync(ConnectionContext connection) - { - try - { - await foreach (var payload in connection.State.SendQueue.Reader.ReadAllAsync(connection.State.Cts.Token)) - { - await connection.Socket.SendAsync(payload, WebSocketMessageType.Text, endOfMessage: true, connection.State.Cts.Token); - } - } - catch (OperationCanceledException) { } - catch (WebSocketException) { } - } - - private static async Task SendSafeAsync(ConnectionContext connection, byte[] payload, SyncServer server) - { - if (!connection.State.TryEnqueueSend(payload)) - { - server.EnqueueImmediateClose(connection, "server_busy"); - return; - } - } -} - -internal sealed class FrameTooLargeException : Exception; - -internal sealed record ReceivedMessage(WebSocketMessageType MessageType, byte[] Payload); - From b8318736fbc71723462e90d3566a0039e8a072bd Mon Sep 17 00:00:00 2001 From: long45343 <1725334094@qq.com> Date: Sat, 22 Aug 2026 15:52:47 +0800 Subject: [PATCH 13/32] Decouple UserHub from SyncServer using IConnectionCoordinator --- .../RuntimeStateAndProtocolTests.cs | 6 +- .../UserHubCoordinationTests.cs | 113 ++++++++++++++++++ .../Hub/IConnectionCoordinator.cs | 14 +++ TextCascade.Server/Hub/UserHub.cs | 27 +++-- TextCascade.Server/SyncServer.cs | 12 +- 5 files changed, 155 insertions(+), 17 deletions(-) create mode 100644 TextCascade.Server.Tests/UserHubCoordinationTests.cs create mode 100644 TextCascade.Server/Hub/IConnectionCoordinator.cs diff --git a/TextCascade.Server.Tests/RuntimeStateAndProtocolTests.cs b/TextCascade.Server.Tests/RuntimeStateAndProtocolTests.cs index b89fa76..2795346 100644 --- a/TextCascade.Server.Tests/RuntimeStateAndProtocolTests.cs +++ b/TextCascade.Server.Tests/RuntimeStateAndProtocolTests.cs @@ -85,7 +85,7 @@ public void RecoveryWindowRestoresSnapshotAtPersistedVersion() new Argon2PasswordHasher(), new SystemClock(), NullLogger.Instance); - var hub = new UserHub("alice", config, TestStartTime, server, 7UL); + var hub = new UserHub("alice", config, TestStartTime, server, server.RuntimeStateStore, 7UL); var modified = DateTimeOffset.FromUnixTimeSeconds(1759999990); hub.AcceptSnapshot(new ClientHello( "client-a", @@ -120,7 +120,7 @@ public void RecoveryWindowIgnoresStaleSnapshot() new Argon2PasswordHasher(), new SystemClock(), NullLogger.Instance); - var hub = new UserHub("alice", config, TestStartTime, server, 7UL); + var hub = new UserHub("alice", config, TestStartTime, server, server.RuntimeStateStore, 7UL); hub.AcceptSnapshot(new ClientHello( "client-a", "Client A", @@ -138,3 +138,5 @@ public void RecoveryWindowIgnoresStaleSnapshot() } } } + + diff --git a/TextCascade.Server.Tests/UserHubCoordinationTests.cs b/TextCascade.Server.Tests/UserHubCoordinationTests.cs new file mode 100644 index 0000000..e40ccb8 --- /dev/null +++ b/TextCascade.Server.Tests/UserHubCoordinationTests.cs @@ -0,0 +1,113 @@ +using Microsoft.Extensions.Logging; +using Microsoft.Extensions.Logging.Abstractions; +using TextCascade.Server; + +namespace TextCascade.Server.Tests; + +public class UserHubCoordinationTests +{ + private sealed class FakeCoordinator : IConnectionCoordinator + { + public ILogger Logger { get; set; } = NullLogger.Instance; + public List<(ConnectionContext Connection, string Reason)> Cancelled { get; } = new(); + public List RebuiltHubs { get; } = new(); + public List RemovedEmptyHubs { get; } = new(); + + public void CancelConnection(ConnectionContext connection, string reason) + { + Cancelled.Add((connection, reason)); + } + + public void RebuildHub(UserHub hub) + { + RebuiltHubs.Add(hub); + } + + public void RemoveEmptyHubAfterRecovery(UserHub hub) + { + RemovedEmptyHubs.Add(hub); + } + } + + [Fact] + public void UserHubDoesNotDependOnSyncServerConcreteType() + { + var fakeCoordinator = new FakeCoordinator(); + var tempState = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString("N") + ".json"); + try + { + var stateStore = new RuntimeStateStore(tempState); + var config = TextCascade.Server.Config.CreateDefaultConfig() with + { + Limits = TextCascade.Server.Config.CreateDefaultConfig().Limits with { SnapshotWindowSeconds = 0 }, + }; + + var hub = new UserHub( + "alice", + config, + DateTimeOffset.UtcNow, + fakeCoordinator, + stateStore, + 1UL); + + var now = DateTimeOffset.UtcNow; + hub.CloseRecoveryWindow(now); + + Assert.Single(fakeCoordinator.RemovedEmptyHubs); + Assert.Same(hub, fakeCoordinator.RemovedEmptyHubs[0]); + } + finally + { + if (File.Exists(tempState)) File.Delete(tempState); + } + } + + [Fact] + public async Task UserLoopFailureNotifiesCoordinator() + { + var fakeCoordinator = new FakeCoordinator(); + var tempState = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString("N") + ".json"); + try + { + var stateStore = new RuntimeStateStore(tempState); + var config = TextCascade.Server.Config.CreateDefaultConfig(); + + // Set initial version to ulong.MaxValue so next clip throws Version overflow + var hub = new UserHub( + "alice", + config, + DateTimeOffset.UtcNow, + fakeCoordinator, + stateStore, + ulong.MaxValue); + + hub.StartIfIdle(); + + // Enqueue a clip job to trigger overflow + var dummySocket = new System.Net.WebSockets.ClientWebSocket(); + var conn = new ConnectionContext("conn-1", "alice", "c1", "Client1", dummySocket, hub, config); + hub.AddConnection(conn); + + // Close recovery window so clip can be applied + hub.CloseRecoveryWindow(DateTimeOffset.UtcNow.AddSeconds(10)); + + var clip = new ClientClip("id-overflow", "data", false, "hash"); + hub.UserChannel.Writer.TryWrite(new ClipJob(conn, clip)); + + // Wait for user loop failure notification + var timeout = DateTime.UtcNow.AddSeconds(3); + while (fakeCoordinator.RebuiltHubs.Count == 0 && DateTime.UtcNow < timeout) + { + await Task.Delay(20); + } + + Assert.Single(fakeCoordinator.RebuiltHubs); + Assert.Same(hub, fakeCoordinator.RebuiltHubs[0]); + } + finally + { + if (File.Exists(tempState)) File.Delete(tempState); + } + } +} + diff --git a/TextCascade.Server/Hub/IConnectionCoordinator.cs b/TextCascade.Server/Hub/IConnectionCoordinator.cs new file mode 100644 index 0000000..dd0cc60 --- /dev/null +++ b/TextCascade.Server/Hub/IConnectionCoordinator.cs @@ -0,0 +1,14 @@ +using Microsoft.Extensions.Logging; + +namespace TextCascade.Server; + +public interface IConnectionCoordinator +{ + ILogger Logger { get; } + + void CancelConnection(ConnectionContext connection, string reason); + + void RebuildHub(UserHub hub); + + void RemoveEmptyHubAfterRecovery(UserHub hub); +} diff --git a/TextCascade.Server/Hub/UserHub.cs b/TextCascade.Server/Hub/UserHub.cs index feed53c..877848c 100644 --- a/TextCascade.Server/Hub/UserHub.cs +++ b/TextCascade.Server/Hub/UserHub.cs @@ -1,4 +1,4 @@ -using System.Text; +using System.Text; using System.Threading.Channels; using Microsoft.Extensions.Logging; @@ -26,16 +26,16 @@ public sealed class UserHub private readonly List recoveryQueue = new(); private bool recoveryWindowClosed; - private readonly SyncServer server; + private readonly IConnectionCoordinator coordinator; private readonly RuntimeStateStore runtimeStateStore; private long lastActivityTicks; - public UserHub(string username, RuntimeConfig config, DateTimeOffset processStart, SyncServer server, ulong initialVersion) + public UserHub(string username, RuntimeConfig config, DateTimeOffset processStart, IConnectionCoordinator coordinator, RuntimeStateStore runtimeStateStore, ulong initialVersion) { Username = username; this.config = config; - this.server = server; - this.runtimeStateStore = server.RuntimeStateStore; + this.coordinator = coordinator; + this.runtimeStateStore = runtimeStateStore; ProcessStartTime = processStart; UserChannel = Channel.CreateUnbounded(new UnboundedChannelOptions { SingleReader = true, SingleWriter = false }); ClipBucket = new TokenBucket(config.RateLimit.ClipBurst, config.RateLimit.ClipTokensPerSecond, processStart); @@ -98,11 +98,11 @@ public void StartIfIdle() catch (OperationCanceledException) { } catch (Exception exception) { - server.Logger.LogError( + coordinator.Logger.LogError( exception, "User loop failed; rebuilding hub. username={Username}", Username); - server.RebuildHub(this); + coordinator.RebuildHub(this); } }); } @@ -140,7 +140,7 @@ private void ProcessJob(UserJob job, DateTimeOffset nowUtc) } break; case DisconnectJob disconnectJob: - server.CancelConnection(disconnectJob.Connection, disconnectJob.Reason); + coordinator.CancelConnection(disconnectJob.Connection, disconnectJob.Reason); break; } } @@ -217,7 +217,7 @@ public void CloseRecoveryWindow(DateTimeOffset nowUtc) BroadcastWelcome(nowUtc); // Spec §6.2: empty hubs that survived until the recovery window closes are now removed. - server.Registry.RemoveIfEmpty(this, allowDuringRecovery: true); + coordinator.RemoveEmptyHubAfterRecovery(this); MarkActivity(nowUtc); } @@ -268,7 +268,7 @@ public void ApplyClip(ClientClip clip, ConnectionContext sender, DateTimeOffset if (SeenIds.TryGetResult(clip.Id, out _)) { - server.Logger.LogWarning( + coordinator.Logger.LogWarning( "Replacing reused clip id. username={Username} clipId={ClipId} clientId={ClientId} previousVersion={PreviousVersion}", Username, clip.Id, @@ -278,7 +278,7 @@ public void ApplyClip(ClientClip clip, ConnectionContext sender, DateTimeOffset if (!ClipBucket.TryAcquire(nowUtc)) { - server.Logger.LogSecurityEvent("reject", + coordinator.Logger.LogSecurityEvent("reject", ("username", Username), ("code", "rate_limited"), ("bytes", Encoding.UTF8.GetByteCount(clip.Payload))); @@ -296,7 +296,7 @@ public void ApplyClip(ClientClip clip, ConnectionContext sender, DateTimeOffset var latest = new LatestText(clip.Payload, next, clip.Hash, clip.Encrypted, sender.ClientId, sender.ClientName, nowUtc); Latest = latest; SeenIds.RememberId(clip.Id, latest); - server.Logger.LogSecurityEvent("clip", + coordinator.Logger.LogSecurityEvent("clip", ("username", Username), ("version", latest.Version), ("clipId", clip.Id), @@ -317,7 +317,7 @@ public void ApplyClip(ClientClip clip, ConnectionContext sender, DateTimeOffset } } - server.Logger.LogInformation( + coordinator.Logger.LogInformation( "Clip broadcast. username={Username} version={Version} clipId={ClipId} recipients=[{Recipients}]", Username, next, @@ -353,3 +353,4 @@ public void BroadcastAsync(byte[] payload) } + diff --git a/TextCascade.Server/SyncServer.cs b/TextCascade.Server/SyncServer.cs index cbb8759..808feee 100644 --- a/TextCascade.Server/SyncServer.cs +++ b/TextCascade.Server/SyncServer.cs @@ -15,7 +15,7 @@ public sealed class SystemClock : IClock public DateTimeOffset UtcNow => DateTimeOffset.UtcNow; } -public sealed class SyncServer +public sealed class SyncServer : IConnectionCoordinator { private readonly UserRegistry registry = new(); private readonly List pendingHellos = new(); @@ -57,10 +57,16 @@ public SyncServer( Cli.CreateArgon2Config(config)); } + ILogger IConnectionCoordinator.Logger => Logger; + + void IConnectionCoordinator.RebuildHub(UserHub hub) => RebuildHub(hub); + + public void RemoveEmptyHubAfterRecovery(UserHub hub) => registry.RemoveIfEmpty(hub, allowDuringRecovery: true); + public UserHub GetOrCreateHub(string username, RuntimeConfig runtimeConfig) { var initialVersion = runtimeStateStore.GetVersion(username); - var hub = registry.GetOrAdd(username, name => new UserHub(name, runtimeConfig, ProcessStartTime, this, initialVersion)); + var hub = registry.GetOrAdd(username, name => new UserHub(name, runtimeConfig, ProcessStartTime, this, runtimeStateStore, initialVersion)); hub.StartIfIdle(); return hub; } @@ -273,3 +279,5 @@ private static async Task CloseConnectionAsync(ConnectionContext connection, Web } } + + From 79dadd406f9e5777b957234f4704d96fa4c7ad8a Mon Sep 17 00:00:00 2001 From: long45343 <1725334094@qq.com> Date: Sat, 22 Aug 2026 16:01:16 +0800 Subject: [PATCH 14/32] Lock StartIfIdle and guard UserHub consumer loop against re-entrancy --- .../UserHubCoordinationTests.cs | 3 +- .../UserLoopConcurrencyTests.cs | 80 +++++++++++++++++++ TextCascade.Server/Hub/UserHub.cs | 44 +++++++--- 3 files changed, 116 insertions(+), 11 deletions(-) create mode 100644 TextCascade.Server.Tests/UserLoopConcurrencyTests.cs diff --git a/TextCascade.Server.Tests/UserHubCoordinationTests.cs b/TextCascade.Server.Tests/UserHubCoordinationTests.cs index e40ccb8..5f6f7fd 100644 --- a/TextCascade.Server.Tests/UserHubCoordinationTests.cs +++ b/TextCascade.Server.Tests/UserHubCoordinationTests.cs @@ -81,7 +81,7 @@ public async Task UserLoopFailureNotifiesCoordinator() stateStore, ulong.MaxValue); - hub.StartIfIdle(); + _ = hub.StartIfIdle(); // Enqueue a clip job to trigger overflow var dummySocket = new System.Net.WebSockets.ClientWebSocket(); @@ -111,3 +111,4 @@ public async Task UserLoopFailureNotifiesCoordinator() } } + diff --git a/TextCascade.Server.Tests/UserLoopConcurrencyTests.cs b/TextCascade.Server.Tests/UserLoopConcurrencyTests.cs new file mode 100644 index 0000000..3982a30 --- /dev/null +++ b/TextCascade.Server.Tests/UserLoopConcurrencyTests.cs @@ -0,0 +1,80 @@ +using Microsoft.Extensions.Logging; +using Microsoft.Extensions.Logging.Abstractions; +using TextCascade.Server; + +namespace TextCascade.Server.Tests; + +public class UserLoopConcurrencyTests +{ + private sealed class FakeCoordinator : IConnectionCoordinator + { + public ILogger Logger => NullLogger.Instance; + public void CancelConnection(ConnectionContext connection, string reason) { } + public void RebuildHub(UserHub hub) { } + public void RemoveEmptyHubAfterRecovery(UserHub hub) { } + } + + [Fact] + public async Task StartIfIdleCreatesSingleTaskUnderConcurrency() + { + var tempState = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString("N") + ".json"); + try + { + var stateStore = new RuntimeStateStore(tempState); + var config = TextCascade.Server.Config.CreateDefaultConfig(); + var hub = new UserHub("alice", config, DateTimeOffset.UtcNow, new FakeCoordinator(), stateStore, 1UL); + + var startedTasks = new Task[100]; + var runners = new Task[100]; + for (var i = 0; i < 100; i++) + { + var idx = i; + runners[i] = Task.Factory.StartNew(() => { startedTasks[idx] = hub.StartIfIdle(); }, TaskCreationOptions.LongRunning); + } + + await Task.WhenAll(runners); + + var firstTask = startedTasks[0]; + Assert.NotNull(firstTask); + for (var i = 1; i < startedTasks.Length; i++) + { + Assert.Same(firstTask, startedTasks[i]); + } + + hub.UserChannel.Writer.Complete(); + await firstTask; + } + finally + { + if (File.Exists(tempState)) File.Delete(tempState); + } + } + + [Fact] + public async Task RunUserLoopRejectsConcurrentReader() + { + var tempState = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString("N") + ".json"); + using var cts = new CancellationTokenSource(); + try + { + var stateStore = new RuntimeStateStore(tempState); + var config = TextCascade.Server.Config.CreateDefaultConfig(); + var hub = new UserHub("alice", config, DateTimeOffset.UtcNow, new FakeCoordinator(), stateStore, 1UL); + + var firstLoop = Task.Run(() => hub.RunUserLoopAsync(cts.Token)); + + // Wait a small amount to ensure first loop has claimed readerActive + await Task.Delay(50); + + // Second concurrent reader must throw InvalidOperationException + await Assert.ThrowsAsync(() => hub.RunUserLoopAsync(cts.Token)); + + hub.UserChannel.Writer.Complete(); + await firstLoop; + } + finally + { + if (File.Exists(tempState)) File.Delete(tempState); + } + } +} diff --git a/TextCascade.Server/Hub/UserHub.cs b/TextCascade.Server/Hub/UserHub.cs index 877848c..1b8b132 100644 --- a/TextCascade.Server/Hub/UserHub.cs +++ b/TextCascade.Server/Hub/UserHub.cs @@ -19,6 +19,8 @@ public sealed class UserHub private readonly List connections = new(); private readonly RuntimeConfig config; private Task? userLoop; + private readonly object userLoopGate = new(); + private int readerActive; private readonly object snapshotGate = new(); private readonly List snapshotCandidates = new(); @@ -88,14 +90,24 @@ public bool RemoveConnection(ConnectionContext connection) return removed; } - public void StartIfIdle() + public Task StartIfIdle() { - if (userLoop is null || userLoop.IsCompleted) + lock (userLoopGate) { + if (userLoop is not null && !userLoop.IsCompleted) + { + return userLoop; + } + userLoop = Task.Run(async () => { - try { await RunUserLoopAsync(); } - catch (OperationCanceledException) { } + try + { + await RunUserLoopAsync(); + } + catch (OperationCanceledException) + { + } catch (Exception exception) { coordinator.Logger.LogError( @@ -105,23 +117,34 @@ public void StartIfIdle() coordinator.RebuildHub(this); } }); + return userLoop; } } - public bool TryWriteJob(UserJob job) => UserChannel.Writer.TryWrite(job); public async Task RunUserLoopAsync(CancellationToken cancellationToken = default) { - var reader = UserChannel.Reader; - while (await reader.WaitToReadAsync(cancellationToken).ConfigureAwait(false)) + if (Interlocked.CompareExchange(ref readerActive, 1, 0) != 0) { - while (reader.TryRead(out var job)) + throw new InvalidOperationException("User loop is already running."); + } + + try + { + var reader = UserChannel.Reader; + while (await reader.WaitToReadAsync(cancellationToken).ConfigureAwait(false)) { - ProcessJob(job, DateTimeOffset.UtcNow); + while (reader.TryRead(out var job)) + { + ProcessJob(job, DateTimeOffset.UtcNow); + } } } + finally + { + Interlocked.Exchange(ref readerActive, 0); + } } - private void ProcessJob(UserJob job, DateTimeOffset nowUtc) { switch (job) @@ -354,3 +377,4 @@ public void BroadcastAsync(byte[] payload) + From d65efed974b6c6c2fa1630e49db06714d9b9ee49 Mon Sep 17 00:00:00 2001 From: long45343 <1725334094@qq.com> Date: Sat, 22 Aug 2026 16:03:37 +0800 Subject: [PATCH 15/32] Add file watcher for hot-reloading users.json --- .../UserFileWatcherTests.cs | 240 +++++++++++++++++ TextCascade.Server/Hosting/UserFileWatcher.cs | 253 ++++++++++++++++++ TextCascade.Server/ServerHost.cs | 4 +- TextCascade.Server/SyncServer.cs | 10 +- 4 files changed, 504 insertions(+), 3 deletions(-) create mode 100644 TextCascade.Server.Tests/UserFileWatcherTests.cs create mode 100644 TextCascade.Server/Hosting/UserFileWatcher.cs diff --git a/TextCascade.Server.Tests/UserFileWatcherTests.cs b/TextCascade.Server.Tests/UserFileWatcherTests.cs new file mode 100644 index 0000000..52f4bb6 --- /dev/null +++ b/TextCascade.Server.Tests/UserFileWatcherTests.cs @@ -0,0 +1,240 @@ +using Microsoft.Extensions.Logging.Abstractions; +using TextCascade.Server; + +namespace TextCascade.Server.Tests; + +public class UserFileWatcherTests +{ + private const string ValidHash = "$argon2id$v=19$m=19456,t=2,p=1$c2FsdA$aG9zdA"; + + private static async Task WaitUntilAsync(Func condition, TimeSpan timeout) + { + var deadline = DateTime.UtcNow + timeout; + while (DateTime.UtcNow < deadline) + { + if (condition()) return; + await Task.Delay(20); + } + Assert.True(condition(), "Condition was not met within timeout."); + } + + [Fact] + public async Task ReloadReplacesUserLookupAfterSave() + { + var tempDir = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString("N")); + Directory.CreateDirectory(tempDir); + var tempUsers = Path.Combine(tempDir, "users.json"); + var tempState = Path.Combine(tempDir, "state.json"); + try + { + var initialUsers = new UsersFile + { + Users = [new UserRecord("alice", ValidHash, 1)], + NextTokenVersion = 2, + }; + UsersFile.SaveUsers(tempUsers, initialUsers); + + var config = TextCascade.Server.Config.CreateDefaultConfig() with + { + TokenSecret = new byte[32], + Files = new FilesConfig(tempUsers, tempState), + }; + var server = new SyncServer( + config, + initialUsers, + new RuntimeStateStore(tempState), + new Argon2PasswordHasher(), + new SystemClock(), + NullLogger.Instance); + + using var watcher = new UserFileWatcher( + tempUsers, + server, + NullLogger.Instance, + debounce: TimeSpan.FromMilliseconds(20), + pollFallback: TimeSpan.FromSeconds(1)); + watcher.Start(); + + Assert.True(server.UserLookup.ContainsKey("alice")); + Assert.False(server.UserLookup.ContainsKey("bob")); + + // Add bob and save + var updatedUsers = new UsersFile + { + Users = + [ + new UserRecord("alice", ValidHash, 1), + new UserRecord("bob", ValidHash, 2), + ], + NextTokenVersion = 3, + }; + UsersFile.SaveUsers(tempUsers, updatedUsers); + + await WaitUntilAsync(() => server.UserLookup.ContainsKey("bob"), TimeSpan.FromSeconds(3)); + Assert.True(server.UserLookup.ContainsKey("bob")); + Assert.True(server.UserLookup.ContainsKey("alice")); + } + finally + { + if (Directory.Exists(tempDir)) Directory.Delete(tempDir, true); + } + } + + [Fact] + public async Task InvalidReloadRetainsPreviousLookup() + { + var tempDir = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString("N")); + Directory.CreateDirectory(tempDir); + var tempUsers = Path.Combine(tempDir, "users.json"); + var tempState = Path.Combine(tempDir, "state.json"); + try + { + var initialUsers = new UsersFile + { + Users = [new UserRecord("alice", ValidHash, 1)], + NextTokenVersion = 2, + }; + UsersFile.SaveUsers(tempUsers, initialUsers); + + var config = TextCascade.Server.Config.CreateDefaultConfig() with + { + TokenSecret = new byte[32], + Files = new FilesConfig(tempUsers, tempState), + }; + var server = new SyncServer( + config, + initialUsers, + new RuntimeStateStore(tempState), + new Argon2PasswordHasher(), + new SystemClock(), + NullLogger.Instance); + + using var watcher = new UserFileWatcher( + tempUsers, + server, + NullLogger.Instance, + debounce: TimeSpan.FromMilliseconds(20), + pollFallback: TimeSpan.FromSeconds(1)); + watcher.Start(); + + // Write invalid JSON + await File.WriteAllTextAsync(tempUsers, "invalid json content!@#$"); + + // Wait a bit to ensure watcher event processed + await Task.Delay(200); + + // Previous lookup must still be alice + Assert.True(server.UserLookup.ContainsKey("alice")); + + // Now recover with valid file containing charlie + var recoveredUsers = new UsersFile + { + Users = [new UserRecord("charlie", ValidHash, 3)], + NextTokenVersion = 4, + }; + UsersFile.SaveUsers(tempUsers, recoveredUsers); + + await WaitUntilAsync(() => server.UserLookup.ContainsKey("charlie"), TimeSpan.FromSeconds(3)); + Assert.True(server.UserLookup.ContainsKey("charlie")); + } + finally + { + if (Directory.Exists(tempDir)) Directory.Delete(tempDir, true); + } + } + + [Fact] + public async Task ConcurrentReloadObserversAlwaysSeeCompleteDictionary() + { + var tempDir = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString("N")); + Directory.CreateDirectory(tempDir); + var tempUsers = Path.Combine(tempDir, "users.json"); + var tempState = Path.Combine(tempDir, "state.json"); + try + { + var usersA = new UsersFile + { + Users = [new UserRecord("alice", ValidHash, 1), new UserRecord("bob", ValidHash, 2)], + NextTokenVersion = 3, + }; + var usersB = new UsersFile + { + Users = [new UserRecord("charlie", ValidHash, 3), new UserRecord("david", ValidHash, 4)], + NextTokenVersion = 5, + }; + + var config = TextCascade.Server.Config.CreateDefaultConfig() with { TokenSecret = new byte[32] }; + var server = new SyncServer( + config, + usersA, + new RuntimeStateStore(tempState), + new Argon2PasswordHasher(), + new SystemClock(), + NullLogger.Instance); + + using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(1)); + var token = cts.Token; + + // Reader task + var readerTask = Task.Run(() => + { + while (!token.IsCancellationRequested) + { + var lookup = server.UserLookup; + // It must be either set A (alice & bob) or set B (charlie & david) + var isA = lookup.ContainsKey("alice") && lookup.ContainsKey("bob") && lookup.Count == 2; + var isB = lookup.ContainsKey("charlie") && lookup.ContainsKey("david") && lookup.Count == 2; + Assert.True(isA || isB, "Observed incomplete or mixed dictionary."); + } + }); + + // Writer loop replacing lookups + var writerTask = Task.Run(async () => + { + var toggle = false; + while (!token.IsCancellationRequested) + { + server.ReplaceUserLookup(toggle ? usersA : usersB); + toggle = !toggle; + await Task.Yield(); + } + }); + + await Task.WhenAll(readerTask, writerTask); + } + finally + { + if (Directory.Exists(tempDir)) Directory.Delete(tempDir, true); + } + } + + [Fact] + public void WatcherDisposeIsIdempotent() + { + var tempDir = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString("N")); + Directory.CreateDirectory(tempDir); + var tempUsers = Path.Combine(tempDir, "users.json"); + var tempState = Path.Combine(tempDir, "state.json"); + try + { + var initialUsers = new UsersFile(); + var config = TextCascade.Server.Config.CreateDefaultConfig() with { TokenSecret = new byte[32] }; + var server = new SyncServer( + config, + initialUsers, + new RuntimeStateStore(tempState), + new Argon2PasswordHasher(), + new SystemClock(), + NullLogger.Instance); + + var watcher = new UserFileWatcher(tempUsers, server, NullLogger.Instance); + watcher.Start(); + watcher.Dispose(); + watcher.Dispose(); // Should not throw + } + finally + { + if (Directory.Exists(tempDir)) Directory.Delete(tempDir, true); + } + } +} diff --git a/TextCascade.Server/Hosting/UserFileWatcher.cs b/TextCascade.Server/Hosting/UserFileWatcher.cs new file mode 100644 index 0000000..0715b74 --- /dev/null +++ b/TextCascade.Server/Hosting/UserFileWatcher.cs @@ -0,0 +1,253 @@ +using System.Text; +using System.Text.Json; +using Microsoft.Extensions.Logging; + +namespace TextCascade.Server; + +internal sealed class UserFileWatcher : IDisposable +{ + private readonly string usersPath; + private readonly SyncServer server; + private readonly ILogger logger; + private readonly TimeSpan debounce; + private readonly TimeSpan pollFallback; + + private readonly object scheduleGate = new(); + private CancellationTokenSource? reloadDelay; + private CancellationTokenSource? reloadExecution; + private int reloadQueued; + + private FileSystemWatcher? watcher; + private PeriodicTimer? pollTimer; + private Task? pollTask; + private bool started; + private bool disposed; + + public UserFileWatcher( + string usersPath, + SyncServer server, + ILogger logger, + TimeSpan? debounce = null, + TimeSpan? pollFallback = null) + { + this.usersPath = Path.GetFullPath(usersPath); + this.server = server; + this.logger = logger; + this.debounce = debounce ?? TimeSpan.FromMilliseconds(250); + this.pollFallback = pollFallback ?? TimeSpan.FromSeconds(30); + } + + public void Start() + { + lock (scheduleGate) + { + if (started || disposed) return; + started = true; + + var directory = Path.GetDirectoryName(usersPath); + if (string.IsNullOrEmpty(directory)) + { + throw new InvalidOperationException("Users file path must include a parent directory."); + } + + if (!Directory.Exists(directory)) + { + Directory.CreateDirectory(directory); + } + + var fileName = Path.GetFileName(usersPath); + + watcher = new FileSystemWatcher(directory, fileName) + { + NotifyFilter = NotifyFilters.FileName | NotifyFilters.LastWrite | NotifyFilters.Size | NotifyFilters.CreationTime, + }; + + watcher.Changed += OnFileChanged; + watcher.Created += OnFileChanged; + watcher.Deleted += OnFileChanged; + watcher.Renamed += OnFileRenamed; + watcher.Error += OnFileError; + watcher.EnableRaisingEvents = true; + + pollTimer = new PeriodicTimer(pollFallback); + reloadExecution = new CancellationTokenSource(); + var executionToken = reloadExecution.Token; + + pollTask = Task.Run(async () => + { + try + { + while (await pollTimer.WaitForNextTickAsync(executionToken)) + { + ScheduleReload(); + } + } + catch (OperationCanceledException) + { + } + }); + } + } + + private void OnFileChanged(object? sender, FileSystemEventArgs eventArgs) + { + var comparison = OperatingSystem.IsWindows() + ? StringComparison.OrdinalIgnoreCase + : StringComparison.Ordinal; + + var fullPath = Path.GetFullPath(eventArgs.FullPath); + if (string.Equals(fullPath, usersPath, comparison)) + { + ScheduleReload(); + } + } + + private void OnFileRenamed(object? sender, RenamedEventArgs eventArgs) + { + var comparison = OperatingSystem.IsWindows() + ? StringComparison.OrdinalIgnoreCase + : StringComparison.Ordinal; + + var fullPath = Path.GetFullPath(eventArgs.FullPath); + var oldFullPath = Path.GetFullPath(eventArgs.OldFullPath); + if (string.Equals(fullPath, usersPath, comparison) || string.Equals(oldFullPath, usersPath, comparison)) + { + ScheduleReload(); + } + } + + private void OnFileError(object? sender, ErrorEventArgs eventArgs) + { + logger.LogWarning(eventArgs.GetException(), "Users file watcher error encountered."); + } + + private void ScheduleReload() + { + lock (scheduleGate) + { + if (disposed) return; + + // If a delay is already running, do not reset it + if (reloadDelay is not null && !reloadDelay.IsCancellationRequested) + { + return; + } + + reloadDelay = new CancellationTokenSource(); + var delayToken = reloadDelay.Token; + + _ = Task.Run(async () => + { + try + { + await Task.Delay(debounce, delayToken); + } + catch (OperationCanceledException) + { + return; + } + + lock (scheduleGate) + { + if (disposed) return; + reloadDelay?.Dispose(); + reloadDelay = null; + } + + if (Interlocked.CompareExchange(ref reloadQueued, 1, 0) == 0) + { + try + { + await ReloadAsync(); + } + finally + { + Interlocked.Exchange(ref reloadQueued, 0); + } + } + }); + } + } + + private async Task ReloadAsync() + { + UsersFile? users = null; + Exception? lastException = null; + + // Try reading with short backoff (3 attempts, 50ms exponential backoff) + for (var attempt = 0; attempt < 3; attempt++) + { + try + { + users = await Task.Run(() => UsersFile.LoadUsers(usersPath)); + lastException = null; + break; + } + catch (Exception exception) when ( + exception is IOException + or JsonException + or DecoderFallbackException + or InvalidOperationException + or UnauthorizedAccessException) + { + lastException = exception; + await Task.Delay(TimeSpan.FromMilliseconds(50 * (attempt + 1))); + } + } + + if (users is not null) + { + try + { + server.ReplaceUserLookup(users); + logger.LogInformation("Users file reloaded. users={Count}", users.Users.Count); + } + catch (Exception exception) when (exception is InvalidOperationException) + { + logger.LogWarning(exception, "Users file reload validation failed; retaining previous users. path={Path}", usersPath); + } + } + else if (lastException is not null) + { + logger.LogWarning(lastException, "Users file reload failed; retaining previous users. path={Path}", usersPath); + } + } + + public void Dispose() + { + lock (scheduleGate) + { + if (disposed) return; + disposed = true; + + try + { + reloadDelay?.Cancel(); + reloadDelay?.Dispose(); + } + catch { } + + try + { + reloadExecution?.Cancel(); + reloadExecution?.Dispose(); + } + catch { } + + if (watcher is not null) + { + watcher.EnableRaisingEvents = false; + watcher.Changed -= OnFileChanged; + watcher.Created -= OnFileChanged; + watcher.Deleted -= OnFileChanged; + watcher.Renamed -= OnFileRenamed; + watcher.Error -= OnFileError; + watcher.Dispose(); + watcher = null; + } + + pollTimer?.Dispose(); + pollTimer = null; + } + } +} diff --git a/TextCascade.Server/ServerHost.cs b/TextCascade.Server/ServerHost.cs index a76c782..f41f541 100644 --- a/TextCascade.Server/ServerHost.cs +++ b/TextCascade.Server/ServerHost.cs @@ -1,4 +1,4 @@ -using System.Net; +using System.Net; using System.Security.Cryptography; using System.Text; using System.Text.Json; @@ -88,6 +88,8 @@ public static int RunServer(string[] args) app.MapGet("/api/v1/sync", async context => await SyncEndpoint.HandleAsync(context, config, context.RequestServices.GetRequiredService())); app.MapMethods("/health", new[] { "HEAD" }, () => Results.Json(new { status = "ok" })); + using var userFileWatcher = new UserFileWatcher(config.Files.UsersFile, app.Services.GetRequiredService(), app.Logger); + userFileWatcher.Start(); app.Run(); } return Ok; diff --git a/TextCascade.Server/SyncServer.cs b/TextCascade.Server/SyncServer.cs index 808feee..e5d1809 100644 --- a/TextCascade.Server/SyncServer.cs +++ b/TextCascade.Server/SyncServer.cs @@ -23,7 +23,7 @@ public sealed class SyncServer : IConnectionCoordinator private readonly IPasswordHasher hasher; private readonly IClock clock; private readonly RuntimeStateStore runtimeStateStore; - private readonly IReadOnlyDictionary userLookup; + private IReadOnlyDictionary userLookup; private readonly string loginDummyHash; public UserRegistry Registry => registry; @@ -31,7 +31,7 @@ public sealed class SyncServer : IConnectionCoordinator public SlidingWindowLoginLimiter LoginLimiter { get; } = new(); public IClock Clock => clock; public ILogger Logger { get; } - public IReadOnlyDictionary UserLookup => userLookup; + public IReadOnlyDictionary UserLookup => Volatile.Read(ref userLookup); public DateTimeOffset ProcessStartTime { get; } public RuntimeConfig Config { get; } public RuntimeStateStore RuntimeStateStore => runtimeStateStore; @@ -63,6 +63,12 @@ public SyncServer( public void RemoveEmptyHubAfterRecovery(UserHub hub) => registry.RemoveIfEmpty(hub, allowDuringRecovery: true); + public void ReplaceUserLookup(UsersFile users) + { + var replacement = UsersFile.BuildUserLookup(users); + Volatile.Write(ref userLookup, replacement); + } + public UserHub GetOrCreateHub(string username, RuntimeConfig runtimeConfig) { var initialVersion = runtimeStateStore.GetVersion(username); From d818739a0b3613b518e8b62eb086f981dda69d1d Mon Sep 17 00:00:00 2001 From: long45343 <1725334094@qq.com> Date: Sat, 22 Aug 2026 16:04:48 +0800 Subject: [PATCH 16/32] Eliminate frame.ToArray in WebSocket JSON message parsing --- TextCascade.Server/Hosting/ConnectionHandler.cs | 3 ++- TextCascade.Server/Models/ReceivedMessage.cs | 4 ++-- TextCascade.Server/Protocol.cs | 7 ++++--- 3 files changed, 8 insertions(+), 6 deletions(-) diff --git a/TextCascade.Server/Hosting/ConnectionHandler.cs b/TextCascade.Server/Hosting/ConnectionHandler.cs index de94052..abdbd68 100644 --- a/TextCascade.Server/Hosting/ConnectionHandler.cs +++ b/TextCascade.Server/Hosting/ConnectionHandler.cs @@ -1,4 +1,4 @@ -using System.Globalization; +using System.Globalization; using System.Net.WebSockets; using System.Text; using Microsoft.Extensions.Logging; @@ -267,3 +267,4 @@ private static async Task SendSafeAsync(ConnectionContext connection, byte[] pay internal sealed class FrameTooLargeException : Exception; + diff --git a/TextCascade.Server/Models/ReceivedMessage.cs b/TextCascade.Server/Models/ReceivedMessage.cs index 04e844f..f276026 100644 --- a/TextCascade.Server/Models/ReceivedMessage.cs +++ b/TextCascade.Server/Models/ReceivedMessage.cs @@ -1,5 +1,5 @@ -using System.Net.WebSockets; +using System.Net.WebSockets; namespace TextCascade.Server; -internal sealed record ReceivedMessage(WebSocketMessageType MessageType, byte[] Payload); +internal sealed record ReceivedMessage(WebSocketMessageType MessageType, ReadOnlyMemory Payload); diff --git a/TextCascade.Server/Protocol.cs b/TextCascade.Server/Protocol.cs index 83360b4..bf63bfa 100644 --- a/TextCascade.Server/Protocol.cs +++ b/TextCascade.Server/Protocol.cs @@ -244,7 +244,7 @@ public static byte[] SerializeLoginResponse(AuthToken token, RuntimeConfig confi return stream.ToArray(); } - public static ParseResult ParseClientMessage(ReadOnlySpan frame, RuntimeConfig config) + public static ParseResult ParseClientMessage(ReadOnlyMemory frame, RuntimeConfig config) { using var document = TryParseJson(frame, out var parseError); if (parseError is not null) @@ -458,11 +458,11 @@ private static bool ValidatePayloadSize(string payload, RuntimeConfig config, ou return true; } - private static JsonDocument? TryParseJson(ReadOnlySpan frame, out ProtocolError? error) + private static JsonDocument? TryParseJson(ReadOnlyMemory frame, out ProtocolError? error) { try { - var document = JsonDocument.Parse(frame.ToArray(), new JsonDocumentOptions + var document = JsonDocument.Parse(frame, new JsonDocumentOptions { AllowTrailingCommas = false, CommentHandling = JsonCommentHandling.Disallow, @@ -597,3 +597,4 @@ private ParseResult(MessageKind kind, object? message, ProtocolError? error) public static ParseResult Failure(ProtocolError error) => new(MessageKind.Unknown, null, error); } + From 98fe7e89764470a374b5eb0c0566be58a173aae8 Mon Sep 17 00:00:00 2001 From: long45343 <1725334094@qq.com> Date: Sat, 22 Aug 2026 16:15:56 +0800 Subject: [PATCH 17/32] Add end-to-end WebSocket integration tests and ServerHost.CreateApp --- .../WebSocketIntegrationTests.cs | 488 ++++++++++++++++++ TextCascade.Server/ServerHost.cs | 37 +- 2 files changed, 516 insertions(+), 9 deletions(-) create mode 100644 TextCascade.Server.Tests/WebSocketIntegrationTests.cs diff --git a/TextCascade.Server.Tests/WebSocketIntegrationTests.cs b/TextCascade.Server.Tests/WebSocketIntegrationTests.cs new file mode 100644 index 0000000..5602431 --- /dev/null +++ b/TextCascade.Server.Tests/WebSocketIntegrationTests.cs @@ -0,0 +1,488 @@ +using System.Net.Http.Headers; +using System.Net.Http.Json; +using System.Net.WebSockets; +using System.Text; +using System.Text.Json; +using Microsoft.AspNetCore.Builder; +using Microsoft.AspNetCore.Hosting; +using Microsoft.Extensions.DependencyInjection; +using Microsoft.Extensions.Hosting; +using Microsoft.Extensions.Logging; +using TextCascade.Server; + +namespace TextCascade.Server.Tests; + +public class WebSocketIntegrationTests +{ + private const string ValidHash = "$argon2id$v=19$m=19456,t=2,p=1$c2FsdA$aG9zdA"; + + private sealed class TestLogCollector : ILoggerProvider, ILogger + { + public List Entries { get; } = new(); + + public ILogger CreateLogger(string categoryName) => this; + + public IDisposable? BeginScope(TState state) where TState : notnull => null; + + public bool IsEnabled(LogLevel logLevel) => true; + + public void Log(LogLevel logLevel, EventId eventId, TState state, Exception? exception, Func formatter) + { + lock (Entries) + { + Entries.Add(formatter(state, exception)); + } + } + + public void Dispose() { } + } + + private sealed class FastPasswordHasher : IPasswordHasher + { + public string Hash(string password, Isopoh.Cryptography.Argon2.Argon2Config config) => ValidHash; + + public bool Verify(string password, string encodedHash) + { + return (password == "password123" && encodedHash == ValidHash); + } + + public bool NeedsRehash(string encodedHash, Isopoh.Cryptography.Argon2.Argon2Config config) => false; + } + + private sealed class IntegrationTestFixture : IAsyncDisposable + { + public WebApplication Application { get; } + public HttpClient Client { get; } + public string BaseUrl { get; } + public string WebSocketUrl { get; } + public string TempDir { get; } + public RuntimeConfig Config { get; } + public TestLogCollector Logs { get; } = new(); + + public static async Task CreateAsync( + Func? configModifier = null, + Action? usersOverride = null, + Action? stateOverride = null) + { + var tempDir = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString("N")); + Directory.CreateDirectory(tempDir); + var usersPath = Path.Combine(tempDir, "users.json"); + var statePath = Path.Combine(tempDir, "state.json"); + + var users = new UsersFile + { + Users = + [ + new UserRecord("alice", ValidHash, 1), + new UserRecord("bob", ValidHash, 1), + ], + NextTokenVersion = 2, + }; + usersOverride?.Invoke(users); + UsersFile.SaveUsers(usersPath, users); + + var stateStore = new RuntimeStateStore(statePath); + stateOverride?.Invoke(stateStore); + + var config = TextCascade.Server.Config.CreateDefaultConfig() with + { + TokenSecret = Encoding.UTF8.GetBytes("12345678901234567890123456789012"), + Files = new FilesConfig(usersPath, statePath), + Server = new ServerConfig("127.0.0.1", 0, "dummy.pem"), + Limits = TextCascade.Server.Config.CreateDefaultConfig().Limits with { SnapshotWindowSeconds = 0 }, + }; + if (configModifier is not null) + { + config = configModifier(config); + } + + var app = ServerHost.CreateApp( + [], + config, + users, + stateStore, + hasher: new FastPasswordHasher(), + clock: new SystemClock(), + certificate: null); + + var logs = new TestLogCollector(); + app.Services.GetRequiredService().AddProvider(logs); + + app.Urls.Add("http://127.0.0.1:0"); + await app.StartAsync(); + + var boundUrl = app.Urls.First(); + var uri = new Uri(boundUrl); + var wsUrl = $"ws://{uri.Authority}/api/v1/sync"; + + var client = new HttpClient { BaseAddress = uri }; + + return new IntegrationTestFixture(app, client, boundUrl, wsUrl, tempDir, config, logs); + } + + private IntegrationTestFixture( + WebApplication app, + HttpClient client, + string baseUrl, + string wsUrl, + string tempDir, + RuntimeConfig config, + TestLogCollector logs) + { + Application = app; + Client = client; + BaseUrl = baseUrl; + WebSocketUrl = wsUrl; + TempDir = tempDir; + Config = config; + Logs = logs; + } + + public async ValueTask DisposeAsync() + { + Client.Dispose(); + using var cts = new CancellationTokenSource(TimeSpan.FromMilliseconds(500)); + try + { + await Application.StopAsync(cts.Token); + } + catch { } + + await Application.DisposeAsync(); + + if (Directory.Exists(TempDir)) + { + try { Directory.Delete(TempDir, true); } catch { } + } + } + } + + private static async Task LoginAsync(HttpClient client, string username, string password) + { + var response = await client.PostAsJsonAsync("/api/v1/login", new { username, password }); + Assert.True(response.IsSuccessStatusCode, $"Login failed: {response.StatusCode}"); + + using var doc = await JsonDocument.ParseAsync(await response.Content.ReadAsStreamAsync()); + Assert.Equal(1, doc.RootElement.GetProperty("protocolVersion").GetInt32()); + var token = doc.RootElement.GetProperty("token").GetString(); + Assert.False(string.IsNullOrEmpty(token)); + return token!; + } + + private static async Task ConnectWebSocketAsync(string wsUrl, string token) + { + var ws = new ClientWebSocket(); + ws.Options.AddSubProtocol("textcascade.v1"); + ws.Options.SetRequestHeader("Authorization", $"Bearer {token}"); + + using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(5)); + await ws.ConnectAsync(new Uri(wsUrl), cts.Token); + Assert.Equal(WebSocketState.Open, ws.State); + Assert.Equal("textcascade.v1", ws.SubProtocol); + return ws; + } + + private static async Task SendJsonAsync(ClientWebSocket ws, object msg) + { + var bytes = JsonSerializer.SerializeToUtf8Bytes(msg); + using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(5)); + await ws.SendAsync(bytes, WebSocketMessageType.Text, true, cts.Token); + } + + private static async Task ReceiveJsonAsync(ClientWebSocket ws) + { + var buffer = new byte[64 * 1024]; + using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(5)); + var result = await ws.ReceiveAsync(new ArraySegment(buffer), cts.Token); + Assert.Equal(WebSocketMessageType.Text, result.MessageType); + return JsonDocument.Parse(buffer.AsMemory(0, result.Count)); + } + + private static async Task CloseWsAsync(ClientWebSocket? ws) + { + if (ws is not null && ws.State == WebSocketState.Open) + { + try + { + using var cts = new CancellationTokenSource(TimeSpan.FromMilliseconds(500)); + await ws.CloseOutputAsync(WebSocketCloseStatus.NormalClosure, "done", cts.Token); + } + catch { } + } + ws?.Dispose(); + } + + [Fact] + public async Task LoginAndWebSocketHandshakeRoundTrips() + { + await using var fixture = await IntegrationTestFixture.CreateAsync(); + + var token = await LoginAsync(fixture.Client, "alice", "password123"); + var ws = await ConnectWebSocketAsync(fixture.WebSocketUrl, token); + try + { + // Send hello + await SendJsonAsync(ws, new + { + type = "hello", + clientId = "client-1", + clientName = "Device 1", + lastServerVersion = 0, + snapshot = (object?)null, + }); + + // Receive welcome + using var welcomeDoc = await ReceiveJsonAsync(ws); + var root = welcomeDoc.RootElement; + Assert.Equal("welcome", root.GetProperty("type").GetString()); + Assert.Equal(1, root.GetProperty("protocolVersion").GetInt32()); + Assert.False(root.TryGetProperty("latest", out var latest) && latest.ValueKind == JsonValueKind.Object); + } + finally + { + await CloseWsAsync(ws); + } + } + + [Fact] + public async Task ClipBroadcastsToSecondClient() + { + await using var fixture = await IntegrationTestFixture.CreateAsync(); + + var tokenA = await LoginAsync(fixture.Client, "alice", "password123"); + var tokenB = await LoginAsync(fixture.Client, "alice", "password123"); + + var wsA = await ConnectWebSocketAsync(fixture.WebSocketUrl, tokenA); + var wsB = await ConnectWebSocketAsync(fixture.WebSocketUrl, tokenB); + try + { + // Hello from A + await SendJsonAsync(wsA, new + { + type = "hello", + clientId = "client-A", + clientName = "Device A", + lastServerVersion = 0, + snapshot = (object?)null, + }); + using var welcomeA = await ReceiveJsonAsync(wsA); + + // Hello from B + await SendJsonAsync(wsB, new + { + type = "hello", + clientId = "client-B", + clientName = "Device B", + lastServerVersion = 0, + snapshot = (object?)null, + }); + using var welcomeB = await ReceiveJsonAsync(wsB); + + // A sends clip + await SendJsonAsync(wsA, new + { + type = "clip", + id = "clip-msg-1", + payload = "Hello World Broadcast", + encrypted = false, + hash = "h1", + }); + + // A receives clip_ack + using var ackA = await ReceiveJsonAsync(wsA); + Assert.Equal("clip_ack", ackA.RootElement.GetProperty("type").GetString()); + Assert.Equal("clip-msg-1", ackA.RootElement.GetProperty("id").GetString()); + Assert.Equal(1UL, ackA.RootElement.GetProperty("version").GetUInt64()); + + // B receives broadcast clip + using var clipB = await ReceiveJsonAsync(wsB); + Assert.Equal("clip", clipB.RootElement.GetProperty("type").GetString()); + Assert.Equal("clip-msg-1", clipB.RootElement.GetProperty("id").GetString()); + Assert.Equal("Hello World Broadcast", clipB.RootElement.GetProperty("payload").GetString()); + Assert.Equal(1UL, clipB.RootElement.GetProperty("version").GetUInt64()); + + // B sends same clip (id & payload duplicate) + await SendJsonAsync(wsB, new + { + type = "clip", + id = "clip-msg-1", + payload = "Hello World Broadcast", + encrypted = false, + hash = "h1", + }); + + // B receives duplicate ack with same version + using var ackB = await ReceiveJsonAsync(wsB); + Assert.Equal("clip_ack", ackB.RootElement.GetProperty("type").GetString()); + Assert.Equal("clip-msg-1", ackB.RootElement.GetProperty("id").GetString()); + Assert.Equal(1UL, ackB.RootElement.GetProperty("version").GetUInt64()); + } + finally + { + await CloseWsAsync(wsA); + await CloseWsAsync(wsB); + } + } + + [Fact] + public async Task InvalidTokenDoesNotUpgradeWebSocket() + { + await using var fixture = await IntegrationTestFixture.CreateAsync(); + + var ws = new ClientWebSocket(); + ws.Options.AddSubProtocol("textcascade.v1"); + ws.Options.SetRequestHeader("Authorization", "Bearer invalid-signature-token-here"); + + using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(3)); + await Assert.ThrowsAnyAsync(() => ws.ConnectAsync(new Uri(fixture.WebSocketUrl), cts.Token)); + } + + [Fact] + public async Task ReconnectRestoresHighestSnapshot() + { + await using var fixture = await IntegrationTestFixture.CreateAsync( + configModifier: cfg => cfg with { Limits = cfg.Limits with { SnapshotWindowSeconds = 10 } }, + stateOverride: store => + { + store.SaveVersion("alice", 7UL); + }); + + var token = await LoginAsync(fixture.Client, "alice", "password123"); + + var wsA = await ConnectWebSocketAsync(fixture.WebSocketUrl, token); + var wsB = await ConnectWebSocketAsync(fixture.WebSocketUrl, token); + try + { + var modifiedTime = DateTimeOffset.UtcNow; + + // A sends hello with version 7 + await SendJsonAsync(wsA, new + { + type = "hello", + clientId = "client-A", + clientName = "Device A", + lastServerVersion = 7, + snapshot = new + { + payload = "snapshot-v7", + encrypted = false, + hash = "hash7", + localModifiedAtUtc = modifiedTime.UtcDateTime.ToString("yyyy-MM-ddTHH:mm:ss.fffffffZ"), + }, + }); + + // B sends hello with version 8 + await SendJsonAsync(wsB, new + { + type = "hello", + clientId = "client-B", + clientName = "Device B", + lastServerVersion = 8, + snapshot = new + { + payload = "snapshot-v8", + encrypted = false, + hash = "hash8", + localModifiedAtUtc = modifiedTime.AddSeconds(1).UtcDateTime.ToString("yyyy-MM-ddTHH:mm:ss.fffffffZ"), + }, + }); + + // Wait a moment for jobs to be processed in user channel + await Task.Delay(50); + + // Explicitly close recovery window to trigger immediate election and broadcast + var syncServer = fixture.Application.Services.GetRequiredService(); + var hub = syncServer.GetOrCreateHub("alice", fixture.Config); + hub.CloseRecoveryWindow(DateTimeOffset.UtcNow.AddMinutes(1)); + + // Both receives welcome with winning version 8 + using var welcomeA = await ReceiveJsonAsync(wsA); + using var welcomeB = await ReceiveJsonAsync(wsB); + + Assert.Equal("welcome", welcomeA.RootElement.GetProperty("type").GetString()); + var latestA = welcomeA.RootElement.GetProperty("latest"); + Assert.Equal(8UL, latestA.GetProperty("version").GetUInt64()); + Assert.Equal("snapshot-v8", latestA.GetProperty("payload").GetString()); + + Assert.Equal("welcome", welcomeB.RootElement.GetProperty("type").GetString()); + var latestB = welcomeB.RootElement.GetProperty("latest"); + Assert.Equal(8UL, latestB.GetProperty("version").GetUInt64()); + Assert.Equal("snapshot-v8", latestB.GetProperty("payload").GetString()); + } + finally + { + await CloseWsAsync(wsA); + await CloseWsAsync(wsB); + } + } + + [Fact] + public async Task AbruptDisconnectIsLoggedAndServerContinues() + { + await using var fixture = await IntegrationTestFixture.CreateAsync(); + + var token = await LoginAsync(fixture.Client, "alice", "password123"); + + var wsA = await ConnectWebSocketAsync(fixture.WebSocketUrl, token); + var wsB = await ConnectWebSocketAsync(fixture.WebSocketUrl, token); + try + { + // Hello from A + await SendJsonAsync(wsA, new + { + type = "hello", + clientId = "client-A", + clientName = "Device A", + lastServerVersion = 0, + snapshot = (object?)null, + }); + using var welcomeA = await ReceiveJsonAsync(wsA); + + // Hello from B + await SendJsonAsync(wsB, new + { + type = "hello", + clientId = "client-B", + clientName = "Device B", + lastServerVersion = 0, + snapshot = (object?)null, + }); + using var welcomeB = await ReceiveJsonAsync(wsB); + + // Abruptly abort client A socket + wsA.Abort(); + wsA.Dispose(); + + // B sends clip - server continues properly + await SendJsonAsync(wsB, new + { + type = "clip", + id = "clip-after-abort", + payload = "Still works", + encrypted = false, + hash = "h2", + }); + + using var ackB = await ReceiveJsonAsync(wsB); + Assert.Equal("clip_ack", ackB.RootElement.GetProperty("type").GetString()); + Assert.Equal("clip-after-abort", ackB.RootElement.GetProperty("id").GetString()); + + // Check structured logs for connect/disconnect + lock (fixture.Logs.Entries) + { + Assert.NotEmpty(fixture.Logs.Entries); + foreach (var log in fixture.Logs.Entries) + { + Assert.DoesNotContain("password123", log, StringComparison.OrdinalIgnoreCase); + Assert.DoesNotContain("12345678901234567890123456789012", log, StringComparison.OrdinalIgnoreCase); + } + } + } + finally + { + await CloseWsAsync(wsB); + } + } +} + + diff --git a/TextCascade.Server/ServerHost.cs b/TextCascade.Server/ServerHost.cs index f41f541..104cd54 100644 --- a/TextCascade.Server/ServerHost.cs +++ b/TextCascade.Server/ServerHost.cs @@ -57,16 +57,38 @@ public static int RunServer(string[] args) using (certificate) { + var app = CreateApp(args, config, users, stateStore, hasher: null, clock: null, certificate: certificate); + using var userFileWatcher = new UserFileWatcher(config.Files.UsersFile, app.Services.GetRequiredService(), app.Logger); + userFileWatcher.Start(); + app.Run(); + } + return Ok; + } + + public static WebApplication CreateApp( + string[] args, + RuntimeConfig config, + UsersFile users, + RuntimeStateStore stateStore, + IPasswordHasher? hasher = null, + IClock? clock = null, + LoadedCertificate? certificate = null) + { var builder = WebApplication.CreateBuilder(args); - builder.WebHost.UseKestrel(ConfigureKestrel(config, certificate)); + if (certificate is not null) + { + builder.WebHost.UseKestrel(ConfigureKestrel(config, certificate)); + } + builder.Logging.ClearProviders(); builder.Logging.AddSimpleConsole(options => { options.SingleLine = true; options.TimestampFormat = "yyyy-MM-ddTHH:mm:ssZ "; }); - builder.Services.AddSingleton(); - builder.Services.AddSingleton(); + + builder.Services.AddSingleton(hasher ?? new Argon2PasswordHasher()); + builder.Services.AddSingleton(clock ?? new SystemClock()); builder.Services.AddSingleton(stateStore); builder.Services.AddSingleton(serviceProvider => new SyncServer( config, @@ -88,11 +110,7 @@ public static int RunServer(string[] args) app.MapGet("/api/v1/sync", async context => await SyncEndpoint.HandleAsync(context, config, context.RequestServices.GetRequiredService())); app.MapMethods("/health", new[] { "HEAD" }, () => Results.Json(new { status = "ok" })); - using var userFileWatcher = new UserFileWatcher(config.Files.UsersFile, app.Services.GetRequiredService(), app.Logger); - userFileWatcher.Start(); - app.Run(); - } - return Ok; + return app; } private static Action ConfigureKestrel(RuntimeConfig config, LoadedCertificate certificate) @@ -203,7 +221,7 @@ private static void DisposeChain(X509Certificate2Collection chain) } } -internal sealed class LoadedCertificate : IDisposable +public sealed class LoadedCertificate : IDisposable { public LoadedCertificate(X509Certificate2 certificate, X509Certificate2Collection chain) { @@ -224,3 +242,4 @@ public void Dispose() } } + From 9a1c668880dd1e5e3fd6e463dac922e56a024e4e Mon Sep 17 00:00:00 2001 From: long45343 <1725334094@qq.com> Date: Sat, 22 Aug 2026 16:18:04 +0800 Subject: [PATCH 18/32] Add Keep a Changelog CHANGELOG and update documentation --- CHANGELOG.md | 62 ++++++++++++++++++++++++++++++++++++++++++++++++++++ README.md | 31 ++++++++++++++++++++------ 2 files changed, 86 insertions(+), 7 deletions(-) create mode 100644 CHANGELOG.md diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..adecfd0 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,62 @@ +# Changelog + +All notable changes to this project will be documented in this file. + +The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), +and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). + +## [Unreleased] + +### Added +- Native ASP.NET Core `CreateApp` factory and comprehensive end-to-end WebSocket integration tests covering handshake, broadcast, snapshots, invalid tokens, and abrupt disconnections. +- Hot-reloading of `users.json` via file system watcher with debounced reload and periodic fallback. +- Constant-time password verification on login for non-existent users using a cached dummy hash. + +### Changed +- Refactored `SyncServer` by splitting models, hubs, and hosting services into dedicated domain files (`Models/`, `Hub/`, `Hosting/`). +- Decoupled `UserHub` from `SyncServer` using lightweight `IConnectionCoordinator` interface. +- Protected `UserHub.StartIfIdle` with mutex lock and added single-reader re-entrancy protection to `RunUserLoopAsync`. +- Optimized `SeenIdRing` deduplication with hash map lookup (`Dictionary`) and FIFO circular eviction queue. +- Migrated PEM certificate and private key loading to native .NET APIs (`X509Certificate2.CreateFromPemFile` and `X509Certificate2Collection.ImportFromPemFile`). +- Relocated CLI single-instance lock file adjacent to the target `users.json` (`users.json.lock`). +- Switched WebSocket JSON message parsing to `ReadOnlyMemory` to eliminate redundant byte array copies (`frame.ToArray()`). + +### Fixed +- Fixed sliding window login rate limiter to clean up only expired timestamps from queue head rather than evicting the entire key. + +### Security +- Eliminated username existence timing side-channel during login authentication. + +## [0.2.5] - 2026-08-22 + +### Added +- Version persistence across server restarts using `RuntimeStateStore` (`textcascade.state.json`). +- Reconnection snapshot recovery window during server startup. + +### Changed +- Refactored snapshot selection tie-breaking and broadcast logic. + +### Fixed +- Fixed server protocol bugs in version negotiation and error responses. + +## [0.2.1] - 2026-08-18 + +### Fixed +- Fixed server protocol bugs in message deserialization and error framing. + +## [0.2.0] - 2026-08-18 + +### Added +- Cross-platform release workflows and CI matrix for Linux and Windows single-file binaries. +- Bilingual README and documentation. + +## [0.1.0] - 2026-08-18 + +### Added +- Initial import and baseline release of TextCascade Server with Minimal API, Kestrel WebSocket, Argon2 password hashing, and token authentication. + +[Unreleased]: https://github.com/long45343/TextCascade-Server/compare/v0.2.5...HEAD +[0.2.5]: https://github.com/long45343/TextCascade-Server/compare/v0.2.1...v0.2.5 +[0.2.1]: https://github.com/long45343/TextCascade-Server/compare/v0.2.0...v0.2.1 +[0.2.0]: https://github.com/long45343/TextCascade-Server/compare/v0.1.0...v0.2.0 +[0.1.0]: https://github.com/long45343/TextCascade-Server/releases/tag/v0.1.0 diff --git a/README.md b/README.md index d90d8f4..364d6b0 100644 --- a/README.md +++ b/README.md @@ -30,7 +30,7 @@ | 密码哈希 | Argon2(id)(`Isopoh.Cryptography.Argon2`) | | 用户存储 | `users.json` | | 协议子协议 | `textcascade.v1` | -| 产品版本 | SemVer,当前 `0.2.0` | +| 产品版本 | SemVer,当前 `0.2.5` | ### 仓库结构 @@ -39,15 +39,19 @@ TextCascade-Server/ ├── TextCascade.Server/ 服务端源码 │ ├── Program.cs 入口:serve / user CLI 分发 │ ├── ServerHost.cs 配置加载、证书、WebHost 构建、路由映射 -│ ├── SyncServer.cs 核心:UserHub/UserRegistry/连接与广播 +│ ├── SyncServer.cs 核心协调器 +│ ├── Hosting/ 端点、连接处理、心跳与文件监听 +│ ├── Hub/ UserHub、UserRegistry、协调器接口与任务 +│ ├── Models/ 连接上下文、状态模型与接收消息 │ ├── Protocol.cs JSON 协议模型与解析 │ ├── Auth.cs / AuthService.cs token 签发、校验、登录限流 │ ├── Users.cs / Cli.cs 用户文件与 CLI(add/passwd/...) │ ├── RuntimeConfig.cs TOML 配置与默认值、环境变量覆盖 -│ └── Core.cs 限流等基础工具 +│ └── Core.cs 限流、去重环形队列等基础工具 ├── TextCascade.Server.Tests/ xUnit 测试 ├── deploy/ systemd unit、示例 TOML 与空 users.json -└── TextCascade.Server.slnx 解决案 +├── CHANGELOG.md 版本变更记录 +└── TextCascade.Server.slnx 解决方案 ``` ### 快速开始 @@ -155,7 +159,7 @@ GitHub Release 提供两种 Framework-dependent 单文件包,目标机需预装 包内附带主程序、配置模板;Linux 包另附 systemd unit。每次 Release 同时提供 SHA-256 校验文件。 -推送 `v*.*.*` 标签(如 `v0.2.0`)会自动执行测试、构建双平台单文件包、生成校验和并发布 GitHub Release。`main` 分支和 Pull Request 会自动执行 restore/build/test CI。 +推送 `v*.*.*` 标签(如 `v0.2.5`)会自动执行测试、构建双平台单文件包、生成校验和并发布 GitHub Release。`main` 分支和 Pull Request 会自动执行 restore/build/test CI。 ### 生产部署(systemd) 参考 `deploy/textcascade-server.service`: @@ -170,6 +174,13 @@ GitHub Release 提供两种 Framework-dependent 单文件包,目标机需预装 --- +### 发版与维护规范 + +- 每个用户可见版本发布前，需将 CHANGELOG.md 中的 [Unreleased] 部分归档为对应版本号与发版日期，并维护底部的 compare 链接。 +- 版本号遵循 [Semantic Versioning 2.0.0](https://semver.org/spec/v2.0.0.html)，版本变更记录遵循 [Keep a Changelog](https://keepachangelog.com/en/1.1.0/)。 + +--- + ## English A lightweight, reliable, high-performance server that synchronizes only the latest text value per user. No history, no database. @@ -196,7 +207,7 @@ Built on ASP.NET Core Minimal API with native Kestrel WebSockets, TLS-terminated | Password hash | Argon2(id) (`Isopoh.Cryptography.Argon2`) | | User store | `users.json` | | Subprotocol | `textcascade.v1` | -| Version | SemVer, currently `0.2.0` | +| Version | SemVer, currently `0.2.5` | ### Quick Start @@ -305,7 +316,7 @@ GitHub Releases provides two framework-dependent single-file archives. The .NET Each archive contains the executable and config template; the Linux archive also includes the systemd unit. Every Release includes a SHA-256 checksum file. -Pushing a `v*.*.*` tag (for example `v0.2.0`) runs tests, builds both single-file archives, generates checksums, and publishes a GitHub Release. Pushes to `main` and pull requests run restore/build/test CI automatically. +Pushing a `v*.*.*` tag (for example `v0.2.5`) runs tests, builds both single-file archives, generates checksums, and publishes a GitHub Release. Pushes to `main` and pull requests run restore/build/test CI automatically. ### Production (systemd) See `deploy/textcascade-server.service`: @@ -317,3 +328,9 @@ See `deploy/textcascade-server.service`: ### License See the repository LICENSE if present. + +### Release & Maintenance Guidelines + +- Before releasing any user-visible version, update CHANGELOG.md by moving the [Unreleased] section to the target version number and release date, along with the compare link at the bottom. +- Versioning adheres to [Semantic Versioning 2.0.0](https://semver.org/spec/v2.0.0.html) and change records follow [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). + From f9b7875f1c5c3ac688df4241b68a2db72a0c05ea Mon Sep 17 00:00:00 2001 From: long45343 <1725334094@qq.com> Date: Sat, 22 Aug 2026 22:31:36 +0800 Subject: [PATCH 19/32] Release v0.3.0 --- CHANGELOG.md | 5 ++++- README.md | 8 ++++---- TextCascade.Server/TextCascade.Server.csproj | 3 ++- 3 files changed, 10 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index adecfd0..d6a3793 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +## [0.3.0] - 2026-08-22 + ### Added - Native ASP.NET Core `CreateApp` factory and comprehensive end-to-end WebSocket integration tests covering handshake, broadcast, snapshots, invalid tokens, and abrupt disconnections. - Hot-reloading of `users.json` via file system watcher with debounced reload and periodic fallback. @@ -55,7 +57,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added - Initial import and baseline release of TextCascade Server with Minimal API, Kestrel WebSocket, Argon2 password hashing, and token authentication. -[Unreleased]: https://github.com/long45343/TextCascade-Server/compare/v0.2.5...HEAD +[Unreleased]: https://github.com/long45343/TextCascade-Server/compare/v0.3.0...HEAD +[0.3.0]: https://github.com/long45343/TextCascade-Server/compare/v0.2.5...v0.3.0 [0.2.5]: https://github.com/long45343/TextCascade-Server/compare/v0.2.1...v0.2.5 [0.2.1]: https://github.com/long45343/TextCascade-Server/compare/v0.2.0...v0.2.1 [0.2.0]: https://github.com/long45343/TextCascade-Server/compare/v0.1.0...v0.2.0 diff --git a/README.md b/README.md index 364d6b0..63be770 100644 --- a/README.md +++ b/README.md @@ -30,7 +30,7 @@ | 密码哈希 | Argon2(id)(`Isopoh.Cryptography.Argon2`) | | 用户存储 | `users.json` | | 协议子协议 | `textcascade.v1` | -| 产品版本 | SemVer,当前 `0.2.5` | +| 产品版本 | SemVer,当前 `0.3.0` | ### 仓库结构 @@ -159,7 +159,7 @@ GitHub Release 提供两种 Framework-dependent 单文件包,目标机需预装 包内附带主程序、配置模板;Linux 包另附 systemd unit。每次 Release 同时提供 SHA-256 校验文件。 -推送 `v*.*.*` 标签(如 `v0.2.5`)会自动执行测试、构建双平台单文件包、生成校验和并发布 GitHub Release。`main` 分支和 Pull Request 会自动执行 restore/build/test CI。 +推送 `v*.*.*` 标签(如 `v0.3.0`)会自动执行测试、构建双平台单文件包、生成校验和并发布 GitHub Release。`main` 分支和 Pull Request 会自动执行 restore/build/test CI。 ### 生产部署(systemd) 参考 `deploy/textcascade-server.service`: @@ -207,7 +207,7 @@ Built on ASP.NET Core Minimal API with native Kestrel WebSockets, TLS-terminated | Password hash | Argon2(id) (`Isopoh.Cryptography.Argon2`) | | User store | `users.json` | | Subprotocol | `textcascade.v1` | -| Version | SemVer, currently `0.2.5` | +| Version | SemVer, currently `0.3.0` | ### Quick Start @@ -316,7 +316,7 @@ GitHub Releases provides two framework-dependent single-file archives. The .NET Each archive contains the executable and config template; the Linux archive also includes the systemd unit. Every Release includes a SHA-256 checksum file. -Pushing a `v*.*.*` tag (for example `v0.2.5`) runs tests, builds both single-file archives, generates checksums, and publishes a GitHub Release. Pushes to `main` and pull requests run restore/build/test CI automatically. +Pushing a `v*.*.*` tag (for example `v0.3.0`) runs tests, builds both single-file archives, generates checksums, and publishes a GitHub Release. Pushes to `main` and pull requests run restore/build/test CI automatically. ### Production (systemd) See `deploy/textcascade-server.service`: diff --git a/TextCascade.Server/TextCascade.Server.csproj b/TextCascade.Server/TextCascade.Server.csproj index 61d9f16..c4e31a7 100644 --- a/TextCascade.Server/TextCascade.Server.csproj +++ b/TextCascade.Server/TextCascade.Server.csproj @@ -4,7 +4,7 @@ net10.0 enable enable - 0.2.5 + 0.3.0 TextCascade.Server true @@ -27,3 +27,4 @@ + From 9ed6eba05749d600c5bf57f8d695869a81a7eabb Mon Sep 17 00:00:00 2001 From: long45343 <1725334094@qq.com> Date: Sat, 22 Aug 2026 23:56:17 +0800 Subject: [PATCH 20/32] Refactor RuntimeStateStore to lock-free CAS and periodic background flush --- CHANGELOG.md | 12 ++- README.md | 6 +- .../RuntimeStateAndProtocolTests.cs | 101 ++++++++++++++--- TextCascade.Server/RuntimeStateStore.cs | 102 +++++++++++++++--- TextCascade.Server/SyncServer.cs | 2 + TextCascade.Server/TextCascade.Server.csproj | 4 +- 6 files changed, 190 insertions(+), 37 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index d6a3793..fd27d71 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,4 +1,4 @@ -# Changelog +# Changelog All notable changes to this project will be documented in this file. @@ -7,6 +7,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +## [0.3.5] - 2026-08-22 + +### Changed +- Refactored `RuntimeStateStore` to lock-free memory CAS updates using `ConcurrentDictionary` and background periodic flush (`PeriodicTimer`) with atomic snapshot persistence, eliminating synchronous disk I/O bottlenecks in the clip synchronization pipeline. +- Added graceful shutdown flush hook to `SyncServer.ShutdownAsync` ensuring all pending version increments are flushed upon service stop. +- Added concurrency, background periodic flush, and fault-tolerance unit tests for `RuntimeStateStore`. + ## [0.3.0] - 2026-08-22 ### Added @@ -57,7 +64,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added - Initial import and baseline release of TextCascade Server with Minimal API, Kestrel WebSocket, Argon2 password hashing, and token authentication. -[Unreleased]: https://github.com/long45343/TextCascade-Server/compare/v0.3.0...HEAD +[Unreleased]: https://github.com/long45343/TextCascade-Server/compare/v0.3.5...HEAD +[0.3.5]: https://github.com/long45343/TextCascade-Server/compare/v0.3.0...v0.3.5 [0.3.0]: https://github.com/long45343/TextCascade-Server/compare/v0.2.5...v0.3.0 [0.2.5]: https://github.com/long45343/TextCascade-Server/compare/v0.2.1...v0.2.5 [0.2.1]: https://github.com/long45343/TextCascade-Server/compare/v0.2.0...v0.2.1 diff --git a/README.md b/README.md index 63be770..02599e8 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -# TextCascade Server +# TextCascade Server [简体中文](#简体中文) | [English](#english) @@ -30,7 +30,7 @@ | 密码哈希 | Argon2(id)(`Isopoh.Cryptography.Argon2`) | | 用户存储 | `users.json` | | 协议子协议 | `textcascade.v1` | -| 产品版本 | SemVer,当前 `0.3.0` | +| 产品版本 | SemVer,当前 `0.3.5` | ### 仓库结构 @@ -207,7 +207,7 @@ Built on ASP.NET Core Minimal API with native Kestrel WebSockets, TLS-terminated | Password hash | Argon2(id) (`Isopoh.Cryptography.Argon2`) | | User store | `users.json` | | Subprotocol | `textcascade.v1` | -| Version | SemVer, currently `0.3.0` | +| Version | SemVer, currently `0.3.5` | ### Quick Start diff --git a/TextCascade.Server.Tests/RuntimeStateAndProtocolTests.cs b/TextCascade.Server.Tests/RuntimeStateAndProtocolTests.cs index 2795346..36a98dc 100644 --- a/TextCascade.Server.Tests/RuntimeStateAndProtocolTests.cs +++ b/TextCascade.Server.Tests/RuntimeStateAndProtocolTests.cs @@ -1,3 +1,4 @@ +using System.Collections.Concurrent; using System.Text; using Microsoft.Extensions.Logging.Abstractions; using TextCascade.Server; @@ -14,14 +15,76 @@ public void StateStorePersistsHighestVersionAtomically() var path = Path.Combine(Path.GetTempPath(), $"textcascade-state-{Guid.NewGuid():N}.json"); try { - var first = new RuntimeStateStore(path); - first.SaveVersion("alice", 7UL); + using (var first = new RuntimeStateStore(path, TimeSpan.Zero)) + { + first.SaveVersion("alice", 7UL); + first.Flush(); + } + + using (var second = new RuntimeStateStore(path, TimeSpan.Zero)) + { + second.SaveVersion("alice", 5UL); + second.Flush(); + + Assert.Equal(7UL, second.GetVersion("alice")); + } + + using (var third = new RuntimeStateStore(path, TimeSpan.Zero)) + { + Assert.Equal(7UL, third.GetVersion("alice")); + } + } + finally + { + if (File.Exists(path)) File.Delete(path); + } + } + + [Fact] + public async Task StateStoreFlushesPeriodicallyInBackground() + { + var path = Path.Combine(Path.GetTempPath(), $"textcascade-state-{Guid.NewGuid():N}.json"); + try + { + using (var store = new RuntimeStateStore(path, TimeSpan.FromMilliseconds(50))) + { + store.SaveVersion("alice", 12UL); + Assert.Equal(12UL, store.GetVersion("alice")); - var second = new RuntimeStateStore(path); - second.SaveVersion("alice", 5UL); + // Wait for background timer tick + await Task.Delay(150); - Assert.Equal(7UL, second.GetVersion("alice")); - Assert.Equal(7UL, new RuntimeStateStore(path).GetVersion("alice")); + using var reloaded = new RuntimeStateStore(path, TimeSpan.Zero); + Assert.Equal(12UL, reloaded.GetVersion("alice")); + } + } + finally + { + if (File.Exists(path)) File.Delete(path); + } + } + + [Fact] + public void StateStoreConcurrentSaveVersionMaintainsHighestValue() + { + var path = Path.Combine(Path.GetTempPath(), $"textcascade-state-{Guid.NewGuid():N}.json"); + try + { + using var store = new RuntimeStateStore(path, TimeSpan.Zero); + Parallel.For(1, 100, i => + { + store.SaveVersion("alice", (ulong)i); + store.SaveVersion("bob", (ulong)(100 - i)); + }); + + Assert.Equal(99UL, store.GetVersion("alice")); + Assert.Equal(99UL, store.GetVersion("bob")); + + store.Flush(); + + using var reloaded = new RuntimeStateStore(path, TimeSpan.Zero); + Assert.Equal(99UL, reloaded.GetVersion("alice")); + Assert.Equal(99UL, reloaded.GetVersion("bob")); } finally { @@ -36,7 +99,7 @@ public void StateStoreRejectsInvalidFile() try { File.WriteAllText(path, """{"entries":[{"username":"alice","version":0}]}""", Encoding.UTF8); - Assert.Throws(() => new RuntimeStateStore(path)); + Assert.Throws(() => new RuntimeStateStore(path, TimeSpan.Zero)); } finally { @@ -76,12 +139,18 @@ public void RecoveryWindowRestoresSnapshotAtPersistedVersion() var path = Path.Combine(Path.GetTempPath(), $"textcascade-state-{Guid.NewGuid():N}.json"); try { - new RuntimeStateStore(path).SaveVersion("alice", 7UL); + using (var initialStore = new RuntimeStateStore(path, TimeSpan.Zero)) + { + initialStore.SaveVersion("alice", 7UL); + initialStore.Flush(); + } + var config = TextCascade.Server.Config.CreateDefaultConfig(); + using var stateStore = new RuntimeStateStore(path, TimeSpan.Zero); var server = new SyncServer( config, new UsersFile(), - new RuntimeStateStore(path), + stateStore, new Argon2PasswordHasher(), new SystemClock(), NullLogger.Instance); @@ -111,12 +180,18 @@ public void RecoveryWindowIgnoresStaleSnapshot() var path = Path.Combine(Path.GetTempPath(), $"textcascade-state-{Guid.NewGuid():N}.json"); try { - new RuntimeStateStore(path).SaveVersion("alice", 7UL); + using (var initialStore = new RuntimeStateStore(path, TimeSpan.Zero)) + { + initialStore.SaveVersion("alice", 7UL); + initialStore.Flush(); + } + var config = TextCascade.Server.Config.CreateDefaultConfig(); + using var stateStore = new RuntimeStateStore(path, TimeSpan.Zero); var server = new SyncServer( config, new UsersFile(), - new RuntimeStateStore(path), + stateStore, new Argon2PasswordHasher(), new SystemClock(), NullLogger.Instance); @@ -137,6 +212,4 @@ public void RecoveryWindowIgnoresStaleSnapshot() if (File.Exists(path)) File.Delete(path); } } -} - - +} \ No newline at end of file diff --git a/TextCascade.Server/RuntimeStateStore.cs b/TextCascade.Server/RuntimeStateStore.cs index 2288ba1..13242c0 100644 --- a/TextCascade.Server/RuntimeStateStore.cs +++ b/TextCascade.Server/RuntimeStateStore.cs @@ -1,6 +1,8 @@ +using System.Collections.Concurrent; using System.Text; using System.Text.Json; using System.Text.Json.Serialization; +using Microsoft.Extensions.Logging; namespace TextCascade.Server; @@ -8,40 +10,108 @@ public sealed record RuntimeStateEntry(string Username, ulong Version); internal sealed record RuntimeStateFile(IReadOnlyList Entries); -public sealed class RuntimeStateStore +public sealed class RuntimeStateStore : IDisposable { - private readonly object gate = new(); private readonly string path; - private readonly Dictionary versions; + private readonly ILogger? logger; + private readonly ConcurrentDictionary versions; + private int isDirty; + private readonly object writeGate = new(); - public RuntimeStateStore(string path) + private readonly PeriodicTimer? flushTimer; + private readonly CancellationTokenSource? cts; + private readonly Task? flushLoopTask; + private bool disposed; + + public RuntimeStateStore( + string path, + TimeSpan? flushInterval = null, + ILogger? logger = null) { this.path = path; - versions = Load(path); + this.logger = logger; + this.versions = new ConcurrentDictionary(Load(path), StringComparer.Ordinal); + + var interval = flushInterval ?? TimeSpan.FromSeconds(5); + if (interval > TimeSpan.Zero) + { + flushTimer = new PeriodicTimer(interval); + cts = new CancellationTokenSource(); + flushLoopTask = Task.Run(() => RunFlushLoopAsync(cts.Token)); + } } public ulong GetVersion(string username) { - lock (gate) + return versions.TryGetValue(username, out var version) ? version : 0UL; + } + + public void SaveVersion(string username, ulong version) + { + versions.AddOrUpdate( + username, + static (_, newVer) => newVer, + static (_, current, newVer) => newVer > current ? newVer : current, + version); + Volatile.Write(ref isDirty, 1); + } + + public bool Flush() + { + if (Interlocked.Exchange(ref isDirty, 0) == 0) + { + return false; + } + + lock (writeGate) { - return versions.TryGetValue(username, out var version) ? version : 0UL; + try + { + var entries = versions + .Select(pair => new RuntimeStateEntry(pair.Key, pair.Value)) + .OrderBy(pair => pair.Username, StringComparer.Ordinal) + .ToList(); + WriteAtomic(path, entries); + return true; + } + catch (Exception exception) when (exception is IOException or UnauthorizedAccessException) + { + Volatile.Write(ref isDirty, 1); + logger?.LogWarning(exception, "Failed to write runtime state file; will retry in next flush cycle. path={Path}", path); + return false; + } } } - public void SaveVersion(string username, ulong version) + private async Task RunFlushLoopAsync(CancellationToken cancellationToken) { - lock (gate) + if (flushTimer is null) return; + try { - if (versions.TryGetValue(username, out var current) && version <= current) + while (await flushTimer.WaitForNextTickAsync(cancellationToken)) { - return; + Flush(); } + } + catch (OperationCanceledException) + { + } + } - versions[username] = version; - WriteAtomic(path, versions.Select(pair => new RuntimeStateEntry(pair.Key, pair.Value)) - .OrderBy(pair => pair.Username, StringComparer.Ordinal) - .ToList()); + public void Dispose() + { + if (disposed) return; + disposed = true; + + if (cts is not null) + { + cts.Cancel(); + try { flushLoopTask?.GetAwaiter().GetResult(); } catch { } + cts.Dispose(); } + + flushTimer?.Dispose(); + Flush(); } private static Dictionary Load(string path) @@ -126,4 +196,4 @@ private static void WriteAtomic(string path, IReadOnlyList en } } } -} +} \ No newline at end of file diff --git a/TextCascade.Server/SyncServer.cs b/TextCascade.Server/SyncServer.cs index e5d1809..166d28f 100644 --- a/TextCascade.Server/SyncServer.cs +++ b/TextCascade.Server/SyncServer.cs @@ -267,6 +267,8 @@ public async Task ShutdownAsync(TimeSpan drain, DateTimeOffset nowUtc) CancelConnection(connection, "server_shutdown"); } } + + runtimeStateStore.Flush(); } private static async Task CloseConnectionAsync(ConnectionContext connection, WebSocketCloseStatus status, string reason) diff --git a/TextCascade.Server/TextCascade.Server.csproj b/TextCascade.Server/TextCascade.Server.csproj index c4e31a7..c1e075c 100644 --- a/TextCascade.Server/TextCascade.Server.csproj +++ b/TextCascade.Server/TextCascade.Server.csproj @@ -1,10 +1,10 @@ - + net10.0 enable enable - 0.3.0 + 0.3.5 TextCascade.Server true From fb3386115fcd85bd84e8783d8c99d104fb253fb8 Mon Sep 17 00:00:00 2001 From: long45343 <1725334094@qq.com> Date: Thu, 27 Aug 2026 23:44:35 +0800 Subject: [PATCH 21/32] Release v0.4.0: contract/network test suites, spec alignment, CI split - Contract tests: JSON sample corpus (duplicate/unknown fields, illegal numbers, depth-4, invalid UTF-8) + byte-level serialization invariants - NetworkIntegration category (12 cases) over real Kestrel TLS with runtime self-signed certs: handshake, TLS 1.2/1.3 probes, frame fragmentation, oversize/zero-length 1009 closes, restart with token direct reconnect + persisted version baseline, snapshot election, graceful-shutdown bye/1001 - SlowHash category: real Argon2 Hash/Verify/NeedsRehash chain - Unit gap closure: token illegal-number forms, CLI watermark allocation/recreate/overflow fail-fast, WithVersion, behavior-level duplicate-id idempotency - CI: SlowHash merged into main test job; NetworkIntegration runs in a dedicated job; release workflow uses the same category filter - docs/server-spec.md rewritten to match v0.3.5+ implementation with implementation-gap ledger; added specs/test-and-contract-spec.md and specs/spec-decisions.md - Version 0.3.5 -> 0.4.0 --- .github/workflows/ci.yml | 27 +- .github/workflows/release.yml | 4 +- .gitignore | 3 +- CHANGELOG.md | 14 + README.md | 4 +- TextCascade.Server.Tests/AuthDeepTests.cs | 161 ++++++ TextCascade.Server.Tests/CliWatermarkTests.cs | 273 +++++++++ .../ContractSamples/README.md | 29 + .../invalid/depth-4/hello.deep-nesting.json | 17 + .../invalid/depth-4/hello.root-depth.json | 19 + .../invalid/duplicate-field/clip.id.json | 8 + .../duplicate-field/hello.clientId.json | 7 + .../invalid/duplicate-field/hello.type.json | 7 + .../invalid/duplicate-field/pong.type.json | 5 + .../number/clip.encrypted.string-bool.json | 7 + .../invalid/number/clip.hash.number.json | 7 + .../hello.lastserverversion.exponent.json | 6 + .../hello.lastserverversion.fraction.json | 6 + .../hello.lastserverversion.negative.json | 6 + .../hello.lastserverversion.string.json | 6 + .../hello.lastserverversion.too-large.json | 6 + .../number/hello.snapshot.offset-time.json | 12 + .../number/pong.clienttimeutc.no-z.json | 4 + .../number/pong.clienttimeutc.number.json | 4 + .../invalid/unknown-field/clip.version.json | 8 + .../invalid/unknown-field/hello.extra.json | 7 + .../invalid/unknown-field/pong.extra.json | 5 + .../utf8/clip.payload-invalid-bytes.bin | 1 + .../utf8/hello.clientid-lone-surrogate.bin | Bin 0 -> 75 bytes .../invalid/utf8/pong.extra-invalid-bytes.bin | 1 + .../ContractSamples/valid/clip.basic.json | 7 + .../ContractSamples/valid/hello.full.json | 12 + .../ContractSamples/valid/hello.minimal.json | 6 + .../valid/hello.null-snapshot.json | 7 + .../hello.snapshot-roundtrip-timestamp.json | 12 + .../ContractSamples/valid/pong.ok.json | 4 + .../ContractTests/ContractSampleTests.cs | 172 ++++++ .../ContractTests/ContractSchemaInvariants.cs | 113 ++++ .../IdempotencyBehaviorTests.cs | 203 +++++++ .../FrameFragmentationTests.cs | 255 +++++++++ .../NetworkIntegration/NetworkTestFixture.cs | 189 ++++++ .../RestartRecoveryTests.cs | 343 +++++++++++ .../TlsAndWssHandshakeTests.cs | 214 +++++++ .../SlowHashSmokeTests.cs | 63 ++ .../TextCascade.Server.Tests.csproj | 4 + TextCascade.Server/TextCascade.Server.csproj | 2 +- docs/server-spec.md | 539 +++++++----------- specs/code-review.md | 81 +++ specs/spec-decisions.md | 218 +++++++ specs/test-and-contract-spec.md | 316 ++++++++++ 50 files changed, 3081 insertions(+), 343 deletions(-) create mode 100644 TextCascade.Server.Tests/AuthDeepTests.cs create mode 100644 TextCascade.Server.Tests/CliWatermarkTests.cs create mode 100644 TextCascade.Server.Tests/ContractSamples/README.md create mode 100644 TextCascade.Server.Tests/ContractSamples/invalid/depth-4/hello.deep-nesting.json create mode 100644 TextCascade.Server.Tests/ContractSamples/invalid/depth-4/hello.root-depth.json create mode 100644 TextCascade.Server.Tests/ContractSamples/invalid/duplicate-field/clip.id.json create mode 100644 TextCascade.Server.Tests/ContractSamples/invalid/duplicate-field/hello.clientId.json create mode 100644 TextCascade.Server.Tests/ContractSamples/invalid/duplicate-field/hello.type.json create mode 100644 TextCascade.Server.Tests/ContractSamples/invalid/duplicate-field/pong.type.json create mode 100644 TextCascade.Server.Tests/ContractSamples/invalid/number/clip.encrypted.string-bool.json create mode 100644 TextCascade.Server.Tests/ContractSamples/invalid/number/clip.hash.number.json create mode 100644 TextCascade.Server.Tests/ContractSamples/invalid/number/hello.lastserverversion.exponent.json create mode 100644 TextCascade.Server.Tests/ContractSamples/invalid/number/hello.lastserverversion.fraction.json create mode 100644 TextCascade.Server.Tests/ContractSamples/invalid/number/hello.lastserverversion.negative.json create mode 100644 TextCascade.Server.Tests/ContractSamples/invalid/number/hello.lastserverversion.string.json create mode 100644 TextCascade.Server.Tests/ContractSamples/invalid/number/hello.lastserverversion.too-large.json create mode 100644 TextCascade.Server.Tests/ContractSamples/invalid/number/hello.snapshot.offset-time.json create mode 100644 TextCascade.Server.Tests/ContractSamples/invalid/number/pong.clienttimeutc.no-z.json create mode 100644 TextCascade.Server.Tests/ContractSamples/invalid/number/pong.clienttimeutc.number.json create mode 100644 TextCascade.Server.Tests/ContractSamples/invalid/unknown-field/clip.version.json create mode 100644 TextCascade.Server.Tests/ContractSamples/invalid/unknown-field/hello.extra.json create mode 100644 TextCascade.Server.Tests/ContractSamples/invalid/unknown-field/pong.extra.json create mode 100644 TextCascade.Server.Tests/ContractSamples/invalid/utf8/clip.payload-invalid-bytes.bin create mode 100644 TextCascade.Server.Tests/ContractSamples/invalid/utf8/hello.clientid-lone-surrogate.bin create mode 100644 TextCascade.Server.Tests/ContractSamples/invalid/utf8/pong.extra-invalid-bytes.bin create mode 100644 TextCascade.Server.Tests/ContractSamples/valid/clip.basic.json create mode 100644 TextCascade.Server.Tests/ContractSamples/valid/hello.full.json create mode 100644 TextCascade.Server.Tests/ContractSamples/valid/hello.minimal.json create mode 100644 TextCascade.Server.Tests/ContractSamples/valid/hello.null-snapshot.json create mode 100644 TextCascade.Server.Tests/ContractSamples/valid/hello.snapshot-roundtrip-timestamp.json create mode 100644 TextCascade.Server.Tests/ContractSamples/valid/pong.ok.json create mode 100644 TextCascade.Server.Tests/ContractTests/ContractSampleTests.cs create mode 100644 TextCascade.Server.Tests/ContractTests/ContractSchemaInvariants.cs create mode 100644 TextCascade.Server.Tests/IdempotencyBehaviorTests.cs create mode 100644 TextCascade.Server.Tests/NetworkIntegration/FrameFragmentationTests.cs create mode 100644 TextCascade.Server.Tests/NetworkIntegration/NetworkTestFixture.cs create mode 100644 TextCascade.Server.Tests/NetworkIntegration/RestartRecoveryTests.cs create mode 100644 TextCascade.Server.Tests/NetworkIntegration/TlsAndWssHandshakeTests.cs create mode 100644 TextCascade.Server.Tests/SlowHashSmokeTests.cs create mode 100644 specs/code-review.md create mode 100644 specs/spec-decisions.md create mode 100644 specs/test-and-contract-spec.md diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 4d44bde..374fe62 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -28,7 +28,10 @@ jobs: run: dotnet build TextCascade.Server.slnx --configuration Release --no-restore - name: Test - run: dotnet test TextCascade.Server.slnx --configuration Release --no-build --logger trx --results-directory ./TestResults + run: > + dotnet test TextCascade.Server.slnx --configuration Release --no-build + --filter "Category!=NetworkIntegration" + --logger trx --results-directory ./TestResults - name: Upload test results if: always() @@ -37,3 +40,25 @@ jobs: name: test-results path: ./TestResults if-no-files-found: warn + + network-tests: + name: Network integration tests + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Setup .NET + uses: actions/setup-dotnet@v4 + with: + dotnet-version: 10.0.x + dotnet-quality: ga + + - name: Restore + run: dotnet restore TextCascade.Server.slnx + + - name: Build + run: dotnet build TextCascade.Server.slnx --configuration Release --no-restore + + - name: Test + run: dotnet test TextCascade.Server.slnx --configuration Release --no-build --filter "Category=NetworkIntegration" diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 33a35cd..7aa67d5 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -30,7 +30,9 @@ jobs: run: dotnet build TextCascade.Server.slnx --configuration Release --no-restore - name: Test - run: dotnet test TextCascade.Server.slnx --configuration Release --no-build + run: > + dotnet test TextCascade.Server.slnx --configuration Release --no-build + --filter "Category!=NetworkIntegration" - name: Resolve version id: version diff --git a/.gitignore b/.gitignore index b145798..67c4a6d 100644 --- a/.gitignore +++ b/.gitignore @@ -1,5 +1,6 @@ ## 项目私有目录,不纳入版本库 -specs/ +!/specs/ +.zcode/ .trae/ ## .NET 构建产物 diff --git a/CHANGELOG.md b/CHANGELOG.md index fd27d71..43f11f5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +## [0.4.0] - 2026-08-27 + +### Added +- Contract test suite with JSON sample corpus (`ContractSamples/`) covering duplicate fields, unknown fields, illegal number forms, depth-4 nesting, and invalid UTF-8, plus byte-level serialization invariants for welcome/clip/ack/ping/error/token payloads. +- Network integration test suite (`Category=NetworkIntegration`, 12 cases) over real Kestrel TLS with runtime-generated self-signed certificates: WSS handshake, TLS 1.2/1.3 protocol probes, random port binding, real frame fragmentation, oversize/zero-length frame closes (1009), server restart with token direct reconnect and persisted-version baseline, snapshot election restore, and graceful-shutdown bye/1001 chain. +- Slow-hash smoke tests (`Category=SlowHash`) exercising the real Argon2 Hash/Verify/NeedsRehash chain with production parameters. +- Unit tests closing spec §10.1 gaps: token duplicate-field/illegal-number/range rejection, CLI watermark allocation, delete-and-recreate watermark behavior, revoke and overflow fail-fast with byte-identical file preservation, `WithVersion` immutability, and behavior-level duplicate-id idempotency (drained token bucket still acks duplicates; reused id with new content treated as fresh message). + +### Changed +- CI main test job now includes the SlowHash category and excludes only `Category=NetworkIntegration`, which runs in a dedicated CI job. +- Release workflow test step aligned with the same category filter. +- Server spec (`docs/server-spec.md`) rewritten to match v0.3.5 implementation: hot user-file reload, RuntimeStateStore version persistence, content-comparing clip idempotency semantics, 10-minute idle hub recycling, actual log event fields, and a new implementation-gap ledger (§15). Never-implemented items (benchmark project, `server_stop` event, performance target table) removed. +- Added `specs/test-and-contract-spec.md` (function-level test and contract specification) and `specs/spec-decisions.md` (decision record for the spec alignment). + ## [0.3.5] - 2026-08-22 ### Changed diff --git a/README.md b/README.md index 02599e8..8eaed00 100644 --- a/README.md +++ b/README.md @@ -30,7 +30,7 @@ | 密码哈希 | Argon2(id)(`Isopoh.Cryptography.Argon2`) | | 用户存储 | `users.json` | | 协议子协议 | `textcascade.v1` | -| 产品版本 | SemVer,当前 `0.3.5` | +| 产品版本 | SemVer,当前 `0.4.0` | ### 仓库结构 @@ -207,7 +207,7 @@ Built on ASP.NET Core Minimal API with native Kestrel WebSockets, TLS-terminated | Password hash | Argon2(id) (`Isopoh.Cryptography.Argon2`) | | User store | `users.json` | | Subprotocol | `textcascade.v1` | -| Version | SemVer, currently `0.3.5` | +| Version | SemVer, currently `0.4.0` | ### Quick Start diff --git a/TextCascade.Server.Tests/AuthDeepTests.cs b/TextCascade.Server.Tests/AuthDeepTests.cs new file mode 100644 index 0000000..a61e07f --- /dev/null +++ b/TextCascade.Server.Tests/AuthDeepTests.cs @@ -0,0 +1,161 @@ +using System.Security.Cryptography; +using System.Text; +using TextCascade.Server; + +namespace TextCascade.Server.Tests; + +public class AuthDeepTests +{ + private static readonly byte[] Secret = Encoding.UTF8.GetBytes(new string('k', 32)); + + private static UserRecord User(string username, long version) => + new(username, "$argon2id$v=19$m=19456,t=2,p=1$c2FsdA$hash", version); + + private static IReadOnlyDictionary Lookup(params UserRecord[] users) => + users.ToDictionary(u => u.Username, u => u, StringComparer.Ordinal); + + private static string CompactFromPayloadJson(string payloadJson) + { + var payloadBytes = Encoding.UTF8.GetBytes(payloadJson); + var signature = HMACSHA256.HashData(Secret, payloadBytes); + return Base64Url(payloadBytes) + "." + Base64Url(signature); + } + + private static string Base64Url(byte[] bytes) => + Convert.ToBase64String(bytes).TrimEnd('=').Replace('+', '-').Replace('/', '_'); + + private static bool Verify(string payloadJson, TokenPayload? expected = null) + { + var token = CompactFromPayloadJson(payloadJson); + var now = DateTimeOffset.FromUnixTimeSeconds(1760000001); + return new TokenService(Secret).TryVerifyToken(token, now, Lookup(User("alice", 1)), out var actual) + && (expected is null || (actual.Subject == expected.Subject + && actual.Version == expected.Version + && actual.IssuedAtUnix == expected.IssuedAtUnix + && actual.ExpiresAtUnix == expected.ExpiresAtUnix)); + } + + // U1 + [Fact] + public void SignToken_FieldOrder_And_MinimalJson() + { + var payload = new TokenPayload("alice", 1, 1760000000, 1762592000); + var compact = TokenService.SignToken(payload, Secret); + var payloadJson = Encoding.UTF8.GetString(Base64UrlDecode(compact.Split('.')[0])); + + Assert.Equal("""{"sub":"alice","ver":1,"iat":1760000000,"exp":1762592000}""", payloadJson); + } + + // U2 + [Fact] + public void VerifyToken_Rejects_DuplicateFields() + { + Assert.False(Verify("""{"sub":"alice","ver":1,"ver":1,"iat":1760000000,"exp":1762592000}""")); + Assert.False(Verify("""{"sub":"alice","sub":"alice","ver":1,"iat":1760000000,"exp":1762592000}""")); + } + + // U3 + [Fact] + public void VerifyToken_Rejects_UnknownField() + { + Assert.False(Verify("""{"sub":"alice","ver":1,"iat":1760000000,"exp":1762592000,"aud":"x"}""")); + } + + // U4 + [Fact] + public void VerifyToken_Rejects_FractionNumber() + { + Assert.False(Verify("""{"sub":"alice","ver":1,"iat":1760000000.0,"exp":1762592000}""")); + Assert.False(Verify("""{"sub":"alice","ver":1.0,"iat":1760000000,"exp":1762592000}""")); + } + + // U5 + [Fact] + public void VerifyToken_Rejects_StringNumber() + { + Assert.False(Verify("""{"sub":"alice","ver":1,"iat":1760000000,"exp":"1762592000"}""")); + } + + // U6 + [Fact] + public void VerifyToken_Rejects_NegativeValue() + { + Assert.False(Verify("""{"sub":"alice","ver":-1,"iat":1760000000,"exp":1762592000}""")); + Assert.False(Verify("""{"sub":"alice","ver":1,"iat":-1760000000,"exp":1762592000}""")); + } + + // U7 + [Fact] + public void VerifyToken_Rejects_ExpNotAfterIat() + { + Assert.False(Verify("""{"sub":"alice","ver":1,"iat":1762592000,"exp":1760000000}""")); + Assert.False(Verify("""{"sub":"alice","ver":1,"iat":1760000000,"exp":1760000000}""")); + } + + // U8 + [Fact] + public void VerifyToken_Rejects_ZeroIat() + { + Assert.False(Verify("""{"sub":"alice","ver":1,"iat":0,"exp":1762592000}""")); + } + + // U9 + [Fact] + public void VerifyToken_RoundTrip_InstanceOverload() + { + var now = DateTimeOffset.FromUnixTimeSeconds(1760000000); + var service = new TokenService(Secret); + var token = service.CreateToken(User("alice", 1), now, TimeSpan.FromDays(30)); + + Assert.True(service.TryVerifyToken(token.CompactToken, now, Lookup(User("alice", 1)), out var payload)); + Assert.Equal("alice", payload.Subject); + Assert.Equal(1, payload.Version); + Assert.Equal(1760000000, payload.IssuedAtUnix); + Assert.Equal(1762592000, payload.ExpiresAtUnix); + } + + // U10 + [Fact] + public void NeedsRehash_ParameterParsing() + { + var encoded = "$argon2id$v=19$m=19456,t=2,p=1$c2FsdA$aG9zdA"; + + Assert.False(Argon2PasswordHasher.NeedsRehash(encoded, 19456, 2, 1)); + Assert.True(Argon2PasswordHasher.NeedsRehash(encoded, 1024, 2, 1)); + Assert.True(Argon2PasswordHasher.NeedsRehash(encoded, 19456, 3, 1)); + Assert.True(Argon2PasswordHasher.NeedsRehash(encoded, 19456, 2, 4)); + Assert.True(Argon2PasswordHasher.NeedsRehash("", 19456, 2, 1)); + Assert.True(Argon2PasswordHasher.NeedsRehash("$argon2i$v=19$m=19456,t=2,p=1$c2FsdA$aG9zdA", 19456, 2, 1)); + Assert.True(Argon2PasswordHasher.NeedsRehash("not-a-hash", 19456, 2, 1)); + } + + // U11 + [Fact] + public void WithVersion_Produces_NewImmutableRecord() + { + var original = new LatestText("payload", 7, "hash", true, "client", "name", new DateTimeOffset(2026, 8, 18, 8, 0, 0, TimeSpan.Zero)); + + var updated = CoreLogic.WithVersion(original, 8); + Assert.Equal(8UL, updated.Version); + Assert.Equal(original.Payload, updated.Payload); + Assert.Equal(original.Hash, updated.Hash); + Assert.Equal(original.Encrypted, updated.Encrypted); + Assert.Equal(original.FromClientId, updated.FromClientId); + Assert.Equal(original.FromClientName, updated.FromClientName); + Assert.Equal(original.UpdatedAtUtc, updated.UpdatedAtUtc); + Assert.Equal(7UL, original.Version); + Assert.NotSame(original, updated); + + var withTime = CoreLogic.WithVersion(original, 9, new DateTimeOffset(2026, 8, 18, 9, 0, 0, TimeSpan.Zero)); + Assert.Equal(9UL, withTime.Version); + Assert.Equal(new DateTimeOffset(2026, 8, 18, 9, 0, 0, TimeSpan.Zero), withTime.UpdatedAtUtc); + } + + private static byte[] Base64UrlDecode(string segment) + { + var padded = segment.Replace('-', '+').Replace('_', '/'); + var remainder = padded.Length % 4; + if (remainder > 0) { padded += new string('=', 4 - remainder); } + return Convert.FromBase64String(padded); + } +} \ No newline at end of file diff --git a/TextCascade.Server.Tests/CliWatermarkTests.cs b/TextCascade.Server.Tests/CliWatermarkTests.cs new file mode 100644 index 0000000..5a4bac1 --- /dev/null +++ b/TextCascade.Server.Tests/CliWatermarkTests.cs @@ -0,0 +1,273 @@ +using System.Text; +using TextCascade.Server; + +namespace TextCascade.Server.Tests; + +public class CliWatermarkTests +{ + private const string ValidHash = "$argon2id$v=19$m=19456,t=2,p=1$c2FsdA$aG9zdA"; + + private sealed class StaticPasswordHasher : IPasswordHasher + { + public string Hash(string password, Isopoh.Cryptography.Argon2.Argon2Config config) => + "$argon2id$v=19$m=19456,t=2,p=1$" + Convert.ToBase64String(Encoding.UTF8.GetBytes(password).AsSpan(0, Math.Min(4, password.Length))) + "$" + Convert.ToBase64String("hashbytes"u8); + + public bool Verify(string password, string encodedHash) => encodedHash == Hash(password, null!); + + public bool NeedsRehash(string encodedHash, Isopoh.Cryptography.Argon2.Argon2Config config) => false; + } + + private static string NewTempDir() + { + var dir = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString("N")); + Directory.CreateDirectory(dir); + return dir; + } + + /// + /// Runs the real CLI command set against a temp users.json. Password-consuming commands + /// (user add) must be wrapped with so --password-stdin reads the fed line. + /// + private static int WithStdin(string stdinLine, string[] args) + { + var original = Console.In; + try + { + Console.SetIn(new StringReader(stdinLine)); + return Cli.RunCli(args, new StaticPasswordHasher()); + } + finally + { + Console.SetIn(original); + } + } + + private static UsersFile LoadUsers(string path) => UsersFile.LoadUsers(path); + + private static void WriteUsersFile(string path, string json) => File.WriteAllText(path, json, Encoding.UTF8); + + private static RuntimeConfig ConfigFor(string usersPath) + { + var config = TextCascade.Server.Config.CreateDefaultConfig(); + return config with { Files = new FilesConfig(usersPath, Path.Combine(Path.GetDirectoryName(usersPath)!, "state.json")) }; + } + + // U13 + [Fact] + public void AddUser_Allocates_FromWatermark_Increments() + { + var dir = NewTempDir(); + try + { + var usersPath = Path.Combine(dir, "users.json"); + WriteUsersFile(usersPath, $$""" + { + "nextTokenVersion": 7, + "users": [ + {"username": "old", "passwordHash": "{{ValidHash}}", "tokenVersion": 3, "disabled": false} + ] + } + """); + + var config = ConfigFor(usersPath); + var exit = WithStdin("test-password", ["user", "add", "--username", "newuser", "--password-stdin", "--config", ConfigPathFor(config)]); + Assert.Equal(Cli.Ok, exit); + + var users = LoadUsers(usersPath); + var added = Assert.Single(users.Users, user => user.Username == "newuser"); + Assert.Equal(7, added.TokenVersion); + Assert.Equal(8, users.NextTokenVersion); + Assert.Equal(3, users.Users.Single(user => user.Username == "old").TokenVersion); + } + finally + { + Directory.Delete(dir, true); + } + } + + // U14 + [Fact] + public void DeleteUser_RecreateSameName_GetsFreshHigherVersion() + { + var dir = NewTempDir(); + try + { + var usersPath = Path.Combine(dir, "users.json"); + WriteUsersFile(usersPath, $$""" + { + "nextTokenVersion": 5, + "users": [ + {"username": "alice", "passwordHash": "{{ValidHash}}", "tokenVersion": 2, "disabled": false} + ] + } + """); + + var config = ConfigFor(usersPath); + Assert.Equal(Cli.Ok, Cli.RunCli(["user", "delete", "--username", "alice", "--config", ConfigPathFor(config)], new StaticPasswordHasher())); + Assert.Empty(LoadUsers(usersPath).Users); + + Assert.Equal(Cli.Ok, WithStdin("test-password", ["user", "add", "--username", "alice", "--password-stdin", "--config", ConfigPathFor(config)])); + + var users = LoadUsers(usersPath); + var recreated = Assert.Single(users.Users); + Assert.Equal("alice", recreated.Username); + Assert.Equal(5, recreated.TokenVersion); + Assert.NotEqual(2, recreated.TokenVersion); + Assert.Equal(6, users.NextTokenVersion); + } + finally + { + Directory.Delete(dir, true); + } + } + + // U15 + [Fact] + public void RevokeTokens_Sets_Watermark_Increments() + { + var dir = NewTempDir(); + try + { + var usersPath = Path.Combine(dir, "users.json"); + WriteUsersFile(usersPath, $$""" + { + "nextTokenVersion": 9, + "users": [ + {"username": "bob", "passwordHash": "{{ValidHash}}", "tokenVersion": 4, "disabled": false} + ] + } + """); + + var config = ConfigFor(usersPath); + Assert.Equal(Cli.Ok, Cli.RunCli(["user", "revoke-tokens", "--username", "bob", "--config", ConfigPathFor(config)], new StaticPasswordHasher())); + + var users = LoadUsers(usersPath); + Assert.Equal(9, users.Users.Single(user => user.Username == "bob").TokenVersion); + Assert.Equal(10, users.NextTokenVersion); + } + finally + { + Directory.Delete(dir, true); + } + } + + // U16 + [Fact] + public void AddUser_At_LongMaxWatermark_FailsFast_FileUnchanged() + { + var dir = NewTempDir(); + try + { + var usersPath = Path.Combine(dir, "users.json"); + var originalJson = $$""" + { + "nextTokenVersion": 9223372036854775807, + "users": [ + {"username": "old", "passwordHash": "{{ValidHash}}", "tokenVersion": 1, "disabled": false} + ] + } + """; + WriteUsersFile(usersPath, originalJson); + + var config = ConfigFor(usersPath); + Assert.Equal(Cli.Error, WithStdin("test-password", ["user", "add", "--username", "newuser", "--password-stdin", "--config", ConfigPathFor(config)])); + + Assert.Equal(originalJson.ReplaceLineEndings(), File.ReadAllText(usersPath, Encoding.UTF8).ReplaceLineEndings()); + Assert.Empty(Directory.GetFiles(dir, "*.tmp")); + } + finally + { + Directory.Delete(dir, true); + } + } + + // U17 + [Fact] + public void Revoke_At_LongMaxWatermark_FailsFast() + { + var dir = NewTempDir(); + try + { + var usersPath = Path.Combine(dir, "users.json"); + var originalJson = $$""" + { + "nextTokenVersion": 9223372036854775807, + "users": [ + {"username": "bob", "passwordHash": "{{ValidHash}}", "tokenVersion": 1, "disabled": false} + ] + } + """; + WriteUsersFile(usersPath, originalJson); + + var config = ConfigFor(usersPath); + Assert.Equal(Cli.Error, Cli.RunCli(["user", "revoke-tokens", "--username", "bob", "--config", ConfigPathFor(config)], new StaticPasswordHasher())); + Assert.Equal(originalJson.ReplaceLineEndings(), File.ReadAllText(usersPath, Encoding.UTF8).ReplaceLineEndings()); + } + finally + { + Directory.Delete(dir, true); + } + } + + // U18 + [Fact] + public void ValidateUsers_NextMustExceed_AllUserVersions() + { + var users = new UsersFile { NextTokenVersion = 5, Users = new() { new("alice", ValidHash, 5) } }; + Assert.Throws(() => UsersFile.ValidateUsers(users)); + } + + // U19 + [Fact] + public void ValidateUsers_Rejects_NonPositiveVersion() + { + Assert.Throws(() => + UsersFile.ValidateUsers(new UsersFile { NextTokenVersion = 5, Users = new() { new("alice", ValidHash, 0) } })); + Assert.Throws(() => + UsersFile.ValidateUsers(new UsersFile { NextTokenVersion = 5, Users = new() { new("alice", ValidHash, -1) } })); + Assert.Throws(() => + UsersFile.ValidateUsers(new UsersFile { NextTokenVersion = 0, Users = [] })); + } + + // U20 + [Fact] + public void SaveUsers_AtomicWrite_LeavesOriginal_OnValidationFailure() + { + var dir = NewTempDir(); + try + { + var usersPath = Path.Combine(dir, "users.json"); + WriteUsersFile(usersPath, $$""" + { + "nextTokenVersion": 5, + "users": [ + {"username": "alice", "passwordHash": "{{ValidHash}}", "tokenVersion": 1, "disabled": false} + ] + } + """); + var originalContent = File.ReadAllText(usersPath, Encoding.UTF8); + + var invalid = new UsersFile { NextTokenVersion = 5, Users = new() { new("alice", ValidHash, 5) } }; + Assert.Throws(() => UsersFile.SaveUsers(usersPath, invalid)); + + Assert.Equal(originalContent.ReplaceLineEndings(), File.ReadAllText(usersPath, Encoding.UTF8).ReplaceLineEndings()); + Assert.Empty(Directory.GetFiles(dir, "*.tmp")); + } + finally + { + Directory.Delete(dir, true); + } + } + + private static string ConfigPathFor(RuntimeConfig config) + { + // Write a minimal TOML that pins users_file to the test path. + var path = Path.Combine(Path.GetDirectoryName(config.Files.UsersFile)!, "textcascade.toml"); + File.WriteAllText(path, $""" + [files] + users_file = "{config.Files.UsersFile.Replace("\\", "\\\\")}" + state_file = "{config.Files.StateFile.Replace("\\", "\\\\")}" + """, Encoding.UTF8); + return path; + } +} \ No newline at end of file diff --git a/TextCascade.Server.Tests/ContractSamples/README.md b/TextCascade.Server.Tests/ContractSamples/README.md new file mode 100644 index 0000000..282f7af --- /dev/null +++ b/TextCascade.Server.Tests/ContractSamples/README.md @@ -0,0 +1,29 @@ +# Contract Samples + +服务端协议契约样本，供三端（C# 服务端 / C# 桌面端 / Kotlin Android 端）对拍。 + +## 目录语义（目录名即期望结果） + +| 目录 | 期望 | +|---|---| +| `valid/` | `ParseClientMessage` 成功，字段逐项匹配 | +| `invalid/duplicate-field/` | 失败，`invalid_message` | +| `invalid/unknown-field/` | 失败，`invalid_message` | +| `invalid/number/` | 失败，`invalid_message`（含小数/指数/字符串数字/负数/超 ulong/类型污染/非 UTC 时间） | +| `invalid/depth-4/` | 失败（MaxDepth=3），`invalid_message` | +| `invalid/utf8/` | `.bin` 原始字节帧，失败，`invalid_message` | + +驱动器（ContractSampleTests）按一级子目录名推断期望错误码，缺省 `invalid_message`。 + +## 无原生数值字段的等价覆盖说明 + +`clip` 没有数值字段、`pong` 的 `clientTimeUtc` 是时间字符串，因此数字形态在这些消息上以"字段类型污染"等价覆盖（同一 Utf8JsonReader 数字/类型分支）： + +- clip.encrypted 字符串化 → TryGetBoolean 分支 +- clip.hash 数字化 → TryGetString 分支 +- pong.clientTimeUtc 数字化 / 无 Z 后缀 → TryGetUtcDateTime 分支 +- hello.snapshot 带 +02:00 偏移 → 非零 Offset 拒绝分支 + +## Token 直测样本 + +token payload 的负数 / 小数 / 字符串数字样本内联于 `AuthDeepTests`（不走样本文件，因其直接调用 `TokenService.TryVerifyToken`）。 \ No newline at end of file diff --git a/TextCascade.Server.Tests/ContractSamples/invalid/depth-4/hello.deep-nesting.json b/TextCascade.Server.Tests/ContractSamples/invalid/depth-4/hello.deep-nesting.json new file mode 100644 index 0000000..1b7a508 --- /dev/null +++ b/TextCascade.Server.Tests/ContractSamples/invalid/depth-4/hello.deep-nesting.json @@ -0,0 +1,17 @@ +{ + "type": "hello", + "clientId": "windows-a", + "clientName": "", + "lastServerVersion": 0, + "snapshot": { + "payload": "clipboard text", + "encrypted": true, + "hash": "sha256-hex", + "localModifiedAtUtc": "2026-08-18T08:00:00Z", + "extra": { + "level": { + "tooDeep": true + } + } + } +} \ No newline at end of file diff --git a/TextCascade.Server.Tests/ContractSamples/invalid/depth-4/hello.root-depth.json b/TextCascade.Server.Tests/ContractSamples/invalid/depth-4/hello.root-depth.json new file mode 100644 index 0000000..71c0ba4 --- /dev/null +++ b/TextCascade.Server.Tests/ContractSamples/invalid/depth-4/hello.root-depth.json @@ -0,0 +1,19 @@ +{ + "type": "hello", + "clientId": "windows-a", + "clientName": "", + "lastServerVersion": 0, + "snapshot": { + "payload": "text", + "encrypted": true, + "hash": "h", + "localModifiedAtUtc": "2026-08-18T08:00:00Z" + }, + "l1": { + "l2": { + "l3": { + "l4": 1 + } + } + } +} \ No newline at end of file diff --git a/TextCascade.Server.Tests/ContractSamples/invalid/duplicate-field/clip.id.json b/TextCascade.Server.Tests/ContractSamples/invalid/duplicate-field/clip.id.json new file mode 100644 index 0000000..0b7c622 --- /dev/null +++ b/TextCascade.Server.Tests/ContractSamples/invalid/duplicate-field/clip.id.json @@ -0,0 +1,8 @@ +{ + "type": "clip", + "id": "clip-1", + "id": "clip-2", + "payload": "text", + "encrypted": false, + "hash": "sha256-hex" +} \ No newline at end of file diff --git a/TextCascade.Server.Tests/ContractSamples/invalid/duplicate-field/hello.clientId.json b/TextCascade.Server.Tests/ContractSamples/invalid/duplicate-field/hello.clientId.json new file mode 100644 index 0000000..bcbfa38 --- /dev/null +++ b/TextCascade.Server.Tests/ContractSamples/invalid/duplicate-field/hello.clientId.json @@ -0,0 +1,7 @@ +{ + "type": "hello", + "clientId": "windows-a", + "clientId": "windows-b", + "clientName": "", + "lastServerVersion": 0 +} \ No newline at end of file diff --git a/TextCascade.Server.Tests/ContractSamples/invalid/duplicate-field/hello.type.json b/TextCascade.Server.Tests/ContractSamples/invalid/duplicate-field/hello.type.json new file mode 100644 index 0000000..5c61809 --- /dev/null +++ b/TextCascade.Server.Tests/ContractSamples/invalid/duplicate-field/hello.type.json @@ -0,0 +1,7 @@ +{ + "type": "hello", + "type": "clip", + "clientId": "windows-a", + "clientName": "", + "lastServerVersion": 0 +} \ No newline at end of file diff --git a/TextCascade.Server.Tests/ContractSamples/invalid/duplicate-field/pong.type.json b/TextCascade.Server.Tests/ContractSamples/invalid/duplicate-field/pong.type.json new file mode 100644 index 0000000..1e3b08e --- /dev/null +++ b/TextCascade.Server.Tests/ContractSamples/invalid/duplicate-field/pong.type.json @@ -0,0 +1,5 @@ +{ + "type": "pong", + "type": "pong", + "clientTimeUtc": "2026-08-18T08:02:00Z" +} \ No newline at end of file diff --git a/TextCascade.Server.Tests/ContractSamples/invalid/number/clip.encrypted.string-bool.json b/TextCascade.Server.Tests/ContractSamples/invalid/number/clip.encrypted.string-bool.json new file mode 100644 index 0000000..d7cc238 --- /dev/null +++ b/TextCascade.Server.Tests/ContractSamples/invalid/number/clip.encrypted.string-bool.json @@ -0,0 +1,7 @@ +{ + "type": "clip", + "id": "clip-1", + "payload": "text", + "encrypted": "true", + "hash": "sha256-hex" +} \ No newline at end of file diff --git a/TextCascade.Server.Tests/ContractSamples/invalid/number/clip.hash.number.json b/TextCascade.Server.Tests/ContractSamples/invalid/number/clip.hash.number.json new file mode 100644 index 0000000..ffdabc4 --- /dev/null +++ b/TextCascade.Server.Tests/ContractSamples/invalid/number/clip.hash.number.json @@ -0,0 +1,7 @@ +{ + "type": "clip", + "id": "clip-1", + "payload": "text", + "encrypted": false, + "hash": 12345 +} \ No newline at end of file diff --git a/TextCascade.Server.Tests/ContractSamples/invalid/number/hello.lastserverversion.exponent.json b/TextCascade.Server.Tests/ContractSamples/invalid/number/hello.lastserverversion.exponent.json new file mode 100644 index 0000000..fb50ca2 --- /dev/null +++ b/TextCascade.Server.Tests/ContractSamples/invalid/number/hello.lastserverversion.exponent.json @@ -0,0 +1,6 @@ +{ + "type": "hello", + "clientId": "windows-a", + "clientName": "", + "lastServerVersion": 1e3 +} \ No newline at end of file diff --git a/TextCascade.Server.Tests/ContractSamples/invalid/number/hello.lastserverversion.fraction.json b/TextCascade.Server.Tests/ContractSamples/invalid/number/hello.lastserverversion.fraction.json new file mode 100644 index 0000000..e454838 --- /dev/null +++ b/TextCascade.Server.Tests/ContractSamples/invalid/number/hello.lastserverversion.fraction.json @@ -0,0 +1,6 @@ +{ + "type": "hello", + "clientId": "windows-a", + "clientName": "", + "lastServerVersion": 1.5 +} \ No newline at end of file diff --git a/TextCascade.Server.Tests/ContractSamples/invalid/number/hello.lastserverversion.negative.json b/TextCascade.Server.Tests/ContractSamples/invalid/number/hello.lastserverversion.negative.json new file mode 100644 index 0000000..f3c03e0 --- /dev/null +++ b/TextCascade.Server.Tests/ContractSamples/invalid/number/hello.lastserverversion.negative.json @@ -0,0 +1,6 @@ +{ + "type": "hello", + "clientId": "windows-a", + "clientName": "", + "lastServerVersion": -1 +} \ No newline at end of file diff --git a/TextCascade.Server.Tests/ContractSamples/invalid/number/hello.lastserverversion.string.json b/TextCascade.Server.Tests/ContractSamples/invalid/number/hello.lastserverversion.string.json new file mode 100644 index 0000000..ccda0f7 --- /dev/null +++ b/TextCascade.Server.Tests/ContractSamples/invalid/number/hello.lastserverversion.string.json @@ -0,0 +1,6 @@ +{ + "type": "hello", + "clientId": "windows-a", + "clientName": "", + "lastServerVersion": "128" +} \ No newline at end of file diff --git a/TextCascade.Server.Tests/ContractSamples/invalid/number/hello.lastserverversion.too-large.json b/TextCascade.Server.Tests/ContractSamples/invalid/number/hello.lastserverversion.too-large.json new file mode 100644 index 0000000..e05a470 --- /dev/null +++ b/TextCascade.Server.Tests/ContractSamples/invalid/number/hello.lastserverversion.too-large.json @@ -0,0 +1,6 @@ +{ + "type": "hello", + "clientId": "windows-a", + "clientName": "", + "lastServerVersion": 18446744073709551616 +} \ No newline at end of file diff --git a/TextCascade.Server.Tests/ContractSamples/invalid/number/hello.snapshot.offset-time.json b/TextCascade.Server.Tests/ContractSamples/invalid/number/hello.snapshot.offset-time.json new file mode 100644 index 0000000..06ec3ba --- /dev/null +++ b/TextCascade.Server.Tests/ContractSamples/invalid/number/hello.snapshot.offset-time.json @@ -0,0 +1,12 @@ +{ + "type": "hello", + "clientId": "windows-a", + "clientName": "", + "lastServerVersion": 0, + "snapshot": { + "payload": "clipboard text", + "encrypted": true, + "hash": "sha256-hex", + "localModifiedAtUtc": "2026-08-18T10:00:00+02:00" + } +} \ No newline at end of file diff --git a/TextCascade.Server.Tests/ContractSamples/invalid/number/pong.clienttimeutc.no-z.json b/TextCascade.Server.Tests/ContractSamples/invalid/number/pong.clienttimeutc.no-z.json new file mode 100644 index 0000000..a1111f6 --- /dev/null +++ b/TextCascade.Server.Tests/ContractSamples/invalid/number/pong.clienttimeutc.no-z.json @@ -0,0 +1,4 @@ +{ + "type": "pong", + "clientTimeUtc": "2026-08-18T08:02:00" +} \ No newline at end of file diff --git a/TextCascade.Server.Tests/ContractSamples/invalid/number/pong.clienttimeutc.number.json b/TextCascade.Server.Tests/ContractSamples/invalid/number/pong.clienttimeutc.number.json new file mode 100644 index 0000000..52b10bf --- /dev/null +++ b/TextCascade.Server.Tests/ContractSamples/invalid/number/pong.clienttimeutc.number.json @@ -0,0 +1,4 @@ +{ + "type": "pong", + "clientTimeUtc": 1760000000 +} \ No newline at end of file diff --git a/TextCascade.Server.Tests/ContractSamples/invalid/unknown-field/clip.version.json b/TextCascade.Server.Tests/ContractSamples/invalid/unknown-field/clip.version.json new file mode 100644 index 0000000..0a74774 --- /dev/null +++ b/TextCascade.Server.Tests/ContractSamples/invalid/unknown-field/clip.version.json @@ -0,0 +1,8 @@ +{ + "type": "clip", + "id": "clip-1", + "payload": "text", + "encrypted": false, + "hash": "sha256-hex", + "version": 12 +} \ No newline at end of file diff --git a/TextCascade.Server.Tests/ContractSamples/invalid/unknown-field/hello.extra.json b/TextCascade.Server.Tests/ContractSamples/invalid/unknown-field/hello.extra.json new file mode 100644 index 0000000..a56eac4 --- /dev/null +++ b/TextCascade.Server.Tests/ContractSamples/invalid/unknown-field/hello.extra.json @@ -0,0 +1,7 @@ +{ + "type": "hello", + "clientId": "windows-a", + "clientName": "", + "lastServerVersion": 0, + "extra": "not-allowed" +} \ No newline at end of file diff --git a/TextCascade.Server.Tests/ContractSamples/invalid/unknown-field/pong.extra.json b/TextCascade.Server.Tests/ContractSamples/invalid/unknown-field/pong.extra.json new file mode 100644 index 0000000..be5a269 --- /dev/null +++ b/TextCascade.Server.Tests/ContractSamples/invalid/unknown-field/pong.extra.json @@ -0,0 +1,5 @@ +{ + "type": "pong", + "clientTimeUtc": "2026-08-18T08:02:00Z", + "extra": true +} \ No newline at end of file diff --git a/TextCascade.Server.Tests/ContractSamples/invalid/utf8/clip.payload-invalid-bytes.bin b/TextCascade.Server.Tests/ContractSamples/invalid/utf8/clip.payload-invalid-bytes.bin new file mode 100644 index 0000000..cf9be42 --- /dev/null +++ b/TextCascade.Server.Tests/ContractSamples/invalid/utf8/clip.payload-invalid-bytes.bin @@ -0,0 +1 @@ +{"type":"clip","id":"clip-1","payload":"badutf8","encrypted":false,"hash":"h"} \ No newline at end of file diff --git a/TextCascade.Server.Tests/ContractSamples/invalid/utf8/hello.clientid-lone-surrogate.bin b/TextCascade.Server.Tests/ContractSamples/invalid/utf8/hello.clientid-lone-surrogate.bin new file mode 100644 index 0000000000000000000000000000000000000000..5d9971dde6d9e70191d649f26280ba46080e9fb9 GIT binary patch literal 75 zcmb InvalidSamples => + Directory.GetFiles(Path.Combine(SamplesRoot, "invalid"), "*.*", SearchOption.AllDirectories) + .Select(path => new object[] { path }); + + public static IEnumerable ValidSamples => + Directory.GetFiles(Path.Combine(SamplesRoot, "valid"), "*.json", SearchOption.AllDirectories) + .Select(path => new object[] { path }); + + [Theory] + [MemberData(nameof(InvalidSamples))] + public void AllInvalidSamples_AreRejected_WithExpectedCode(string path) + { + var frame = File.ReadAllBytes(path); + var config = TextCascade.Server.Config.CreateDefaultConfig(); + + // Non-UTF8 samples may also throw while transcoding a string field; either outcome is a rejection. + try + { + var result = Protocol.ParseClientMessage(frame, config); + Assert.NotNull(result.Error); + Assert.Equal(ExpectedCode(path), result.Error!.CodeName); + } + catch (Exception exception) when (exception is DecoderFallbackException or InvalidOperationException) + { + // Transcoding failure is itself the documented rejection path for invalid UTF-8. + } + } + + [Theory] + [MemberData(nameof(ValidSamples))] + public void AllValidSamples_Parse_WithExpectedKind(string path) + { + var frame = File.ReadAllBytes(path); + var config = TextCascade.Server.Config.CreateDefaultConfig(); + + var result = Protocol.ParseClientMessage(frame, config); + + Assert.True(result.Error is null, $"Sample should parse: {path} error={result.Error?.Message}"); + var expectedKind = Path.GetFileName(path) switch + { + string n when n.StartsWith("hello.", StringComparison.Ordinal) => MessageKind.Hello, + string n when n.StartsWith("clip.", StringComparison.Ordinal) => MessageKind.Clip, + string n when n.StartsWith("pong.", StringComparison.Ordinal) => MessageKind.Pong, + _ => throw new InvalidOperationException($"Cannot infer kind from {path}"), + }; + Assert.Equal(expectedKind, result.Kind); + } + + [Fact] + public void ValidHello_Full_ParsesAllFields() + { + var frame = File.ReadAllBytes(Path.Combine(SamplesRoot, "valid", "hello.full.json")); + var result = Protocol.ParseClientMessage(frame, TextCascade.Server.Config.CreateDefaultConfig()); + + Assert.True(result.Error is null); + var hello = Assert.IsType(result.Message); + Assert.Equal("windows-a", hello.ClientId); + Assert.Equal("Windows Desktop", hello.ClientName); + Assert.Equal(128UL, hello.LastServerVersion); + + var snapshot = hello.Snapshot; + Assert.NotNull(snapshot); + Assert.Equal("clipboard text", snapshot!.Payload); + Assert.True(snapshot.Encrypted); + Assert.Equal("sha256-hex", snapshot.Hash); + Assert.Equal(new DateTimeOffset(2026, 8, 18, 8, 0, 0, TimeSpan.Zero), snapshot.LocalModifiedAtUtc); + } + + [Fact] + public void ValidHello_Minimal_HasNoSnapshot() + { + var frame = File.ReadAllBytes(Path.Combine(SamplesRoot, "valid", "hello.minimal.json")); + var result = Protocol.ParseClientMessage(frame, TextCascade.Server.Config.CreateDefaultConfig()); + + Assert.True(result.Error is null, result.Error?.Message); + var hello = Assert.IsType(result.Message); + Assert.Null(hello.Snapshot); + Assert.Equal(0UL, hello.LastServerVersion); + Assert.Equal(string.Empty, hello.ClientName); + } + + [Fact] + public void ValidHello_NullSnapshot_IsExplicitNull() + { + var frame = File.ReadAllBytes(Path.Combine(SamplesRoot, "valid", "hello.null-snapshot.json")); + var result = Protocol.ParseClientMessage(frame, TextCascade.Server.Config.CreateDefaultConfig()); + + Assert.True(result.Error is null, result.Error?.Message); + var hello = Assert.IsType(result.Message); + Assert.Null(hello.Snapshot); + } + + [Fact] + public void ValidClip_ParsesAllFields() + { + var frame = File.ReadAllBytes(Path.Combine(SamplesRoot, "valid", "clip.basic.json")); + var result = Protocol.ParseClientMessage(frame, TextCascade.Server.Config.CreateDefaultConfig()); + + var clip = Assert.IsType(result.Message); + Assert.Equal("clip-20260818-001", clip.Id); + Assert.Equal("shared clipboard content", clip.Payload); + Assert.False(clip.Encrypted); + Assert.Equal("sha256-hex", clip.Hash); + } + + [Fact] + public void ValidPong_ParsesTimestamp() + { + var frame = File.ReadAllBytes(Path.Combine(SamplesRoot, "valid", "pong.ok.json")); + var result = Protocol.ParseClientMessage(frame, TextCascade.Server.Config.CreateDefaultConfig()); + + var pong = Assert.IsType(result.Message); + Assert.Equal(new DateTimeOffset(2026, 8, 18, 8, 2, 0, TimeSpan.Zero), pong.ClientTimeUtc); + } + + [Fact] + public void ValidHello_RoundTripTimestampFormat_IsAccepted() + { + var frame = File.ReadAllBytes(Path.Combine(SamplesRoot, "valid", "hello.snapshot-roundtrip-timestamp.json")); + var result = Protocol.ParseClientMessage(frame, TextCascade.Server.Config.CreateDefaultConfig()); + + Assert.True(result.Error is null, result.Error?.Message); + var hello = Assert.IsType(result.Message); + Assert.Equal(TimeSpan.Zero, hello.Snapshot!.LocalModifiedAtUtc.Offset); + } + + private static string ExpectedCode(string path) + { + var relative = Path.GetRelativePath(Path.Combine(SamplesRoot, "invalid"), path); + var category = relative.Split(Path.DirectorySeparatorChar, Path.AltDirectorySeparatorChar)[0]; + return category switch + { + "frame_too_large" => "frame_too_large", + _ => "invalid_message", + }; + } + + private static string FindSamplesRoot() + { + // AppContext.BaseDirectory works for dotnet test with CopyToOutputDirectory; the + // upward walk covers runs where content files were not copied. + var candidate = Path.Combine(AppContext.BaseDirectory, "ContractSamples"); + if (Directory.Exists(candidate)) + { + return candidate; + } + + var directory = new DirectoryInfo(AppContext.BaseDirectory); + while (directory is not null) + { + candidate = Path.Combine(directory.FullName, "ContractSamples"); + if (Directory.Exists(candidate)) + { + return candidate; + } + + directory = directory.Parent!; + } + + throw new InvalidOperationException("ContractSamples directory not found."); + } +} \ No newline at end of file diff --git a/TextCascade.Server.Tests/ContractTests/ContractSchemaInvariants.cs b/TextCascade.Server.Tests/ContractTests/ContractSchemaInvariants.cs new file mode 100644 index 0000000..b0aa308 --- /dev/null +++ b/TextCascade.Server.Tests/ContractTests/ContractSchemaInvariants.cs @@ -0,0 +1,113 @@ +using System.Text; +using TextCascade.Server; + +namespace TextCascade.Server.Tests; + +public class ContractSchemaInvariants +{ + [Fact] + public void C1_Welcome_NoLatest_OmitsKey() + { + var bytes = Protocol.SerializeWelcome(null, TextCascade.Server.Config.CreateDefaultConfig().Limits); + var json = Encoding.UTF8.GetString(bytes); + + Assert.Equal("""{"type":"welcome","protocolVersion":1}""", json); + Assert.DoesNotContain("latest", json, StringComparison.Ordinal); + } + + [Fact] + public void C2_Welcome_WithLatest_FixedFieldOrder() + { + var latest = new LatestText( + "payload-text", 128, "hash", true, "android-a", "android", + new DateTimeOffset(2026, 8, 18, 7, 59, 58, TimeSpan.Zero)); + + var bytes = Protocol.SerializeWelcome(latest, TextCascade.Server.Config.CreateDefaultConfig().Limits); + var json = Encoding.UTF8.GetString(bytes); + + Assert.Equal( + """{"type":"welcome","protocolVersion":1,"latest":{"payload":"payload-text","version":128,"hash":"hash","encrypted":true,"fromClientId":"android-a","fromClientName":"android","updatedAtUtc":"2026-08-18T07:59:58Z"}}""", + json); + } + + [Fact] + public void C3_BroadcastClip_ContainsAllEightFields() + { + var latest = new LatestText( + "payload-text", 129, "hash", false, "windows-a", "Windows Desktop", + new DateTimeOffset(2026, 8, 18, 8, 1, 0, TimeSpan.Zero)); + + var bytes = Protocol.SerializeClip("clip-id-1", latest); + var json = Encoding.UTF8.GetString(bytes); + + Assert.Equal( + """{"type":"clip","version":129,"id":"clip-id-1","payload":"payload-text","encrypted":false,"hash":"hash","fromClientId":"windows-a","fromClientName":"Windows Desktop","updatedAtUtc":"2026-08-18T08:01:00Z"}""", + json); + } + + [Fact] + public void C4_TokenPayload_MinimalFixedOrder() + { + var payload = new TokenPayload("alice", 1, 1760000000, 1762592000); + var compact = TokenService.SignToken(payload, new byte[32]); + var payloadSegment = compact.Split('.')[0]; + var payloadJson = Encoding.UTF8.GetString(Base64UrlDecode(payloadSegment)); + + Assert.Equal("""{"sub":"alice","ver":1,"iat":1760000000,"exp":1762592000}""", payloadJson); + } + + [Fact] + public void C5_ErrorResponse_IncludesReferenceId_WhenNotNull() + { + var withReference = Encoding.UTF8.GetString(Protocol.SerializeProtocolError( + new ProtocolError(ProtocolErrorCode.TextTooLarge, "Text exceeds maxTextBytes.", "clip-1"))); + Assert.Equal( + """{"type":"error","code":"text_too_large","message":"Text exceeds maxTextBytes.","referenceId":"clip-1"}""", + withReference); + + var withoutReference = Encoding.UTF8.GetString(Protocol.SerializeProtocolError( + new ProtocolError(ProtocolErrorCode.InvalidMessage, "Invalid JSON.", null))); + Assert.Equal( + """{"type":"error","code":"invalid_message","message":"Invalid JSON."}""", + withoutReference); + } + + [Fact] + public void C6_Ping_Timestamps_UtcZ_SecondPrecision() + { + var now = new DateTimeOffset(2026, 8, 18, 8, 2, 0, TimeSpan.Zero).AddMilliseconds(456); + var json = Encoding.UTF8.GetString(Protocol.SerializePing(now)); + + Assert.Equal("""{"type":"ping","serverTimeUtc":"2026-08-18T08:02:00Z"}""", json); + } + + [Fact] + public void ClipAck_Shape_MatchesContract() + { + var latest = new LatestText( + "payload", 129, "hash", false, "windows-a", "Windows Desktop", + new DateTimeOffset(2026, 8, 18, 8, 1, 0, TimeSpan.Zero)); + + var json = Encoding.UTF8.GetString(Protocol.SerializeClipAck("clip-id-1", latest)); + + Assert.Equal( + """{"type":"clip_ack","id":"clip-id-1","version":129,"updatedAtUtc":"2026-08-18T08:01:00Z"}""", + json); + } + + [Fact] + public void Bye_Shape_MatchesContract() + { + var json = Encoding.UTF8.GetString(Protocol.SerializeBye("server_shutdown")); + + Assert.Equal("""{"type":"bye","reason":"server_shutdown"}""", json); + } + + private static byte[] Base64UrlDecode(string segment) + { + var padded = segment.Replace('-', '+').Replace('_', '/'); + var remainder = padded.Length % 4; + if (remainder > 0) { padded += new string('=', 4 - remainder); } + return Convert.FromBase64String(padded); + } +} \ No newline at end of file diff --git a/TextCascade.Server.Tests/IdempotencyBehaviorTests.cs b/TextCascade.Server.Tests/IdempotencyBehaviorTests.cs new file mode 100644 index 0000000..1d42185 --- /dev/null +++ b/TextCascade.Server.Tests/IdempotencyBehaviorTests.cs @@ -0,0 +1,203 @@ +using System.Net.WebSockets; +using System.Text.Json; +using System.Text; +using Microsoft.Extensions.Logging; +using Microsoft.Extensions.Logging.Abstractions; +using TextCascade.Server; + +namespace TextCascade.Server.Tests; + +public class IdempotencyBehaviorTests +{ + private sealed class RecordingCoordinator : IConnectionCoordinator + { + public List Warnings { get; } = new(); + private readonly CollectingLogger logger; + + public RecordingCoordinator() + { + logger = new CollectingLogger(Warnings); + } + + public ILogger Logger => logger; + + public void CancelConnection(ConnectionContext connection, string reason) { } + + public void RebuildHub(UserHub hub) { } + + public void RemoveEmptyHubAfterRecovery(UserHub hub) { } + + private sealed class CollectingLogger : ILogger + { + private readonly List warnings; + + public CollectingLogger(List warnings) + { + this.warnings = warnings; + } + + public IDisposable? BeginScope(TState state) where TState : notnull => null; + + public bool IsEnabled(LogLevel logLevel) => true; + + public void Log(LogLevel logLevel, EventId eventId, TState state, Exception? exception, Func formatter) + { + var entry = formatter(state, exception); + if (logLevel == LogLevel.Warning) + { + lock (warnings) { warnings.Add(entry); } + } + } + } + } + + private static (UserHub Hub, RecordingCoordinator Coordinator) NewHub(ulong initialVersion = 0) + { + var config = TextCascade.Server.Config.CreateDefaultConfig(); + var coordinator = new RecordingCoordinator(); + var statePath = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString("N") + ".json"); + var stateStore = new RuntimeStateStore(statePath, TimeSpan.Zero); + var hub = new UserHub("alice", config, DateTimeOffset.FromUnixTimeSeconds(1760000000), coordinator, stateStore, initialVersion); + return (hub, coordinator); + } + + private static ConnectionContext NewConnection(string clientId = "client-A") + { + var socket = new ClientWebSocket(); + var config = TextCascade.Server.Config.CreateDefaultConfig(); + return new ConnectionContext($"conn-{Guid.NewGuid():N}", "alice", clientId, clientId, socket, null, config); + } + + private static ClientClip Clip(string id, string payload, string hash = "h", bool encrypted = false) => + new(id, payload, encrypted, hash); + + private static (bool Acked, byte[]? Payload) DequeueAck(ConnectionContext connection) + { + while (connection.State.SendQueue.Reader.TryRead(out var payload)) + { + var text = Encoding.UTF8.GetString(payload); + if (text.Contains("clip_ack", StringComparison.Ordinal)) + { + return (true, payload); + } + + if (text.Contains("rate_limited", StringComparison.Ordinal)) + { + return (false, payload); + } + } + + return (false, null); + } + + private static void DrainQueue(ConnectionContext connection) + { + while (connection.State.SendQueue.Reader.TryRead(out _)) { } + } + + // U22 — behavior level: draining the bucket does not block duplicate-id resends. + [Fact] + public void DuplicateId_AfterBucketDrained_StillAcked() + { + var (hub, coordinator) = NewHub(); + var sender = NewConnection(); + + var now = DateTimeOffset.FromUnixTimeSeconds(1760000000); + + // Drain the burst (default 10) with distinct clips; time does not advance so refill stays 0. + for (var index = 0; index < 10; index++) + { + hub.ApplyClip(Clip($"id-{index}", $"payload-{index}"), sender, now); + } + DrainQueue(sender); + + // 11th distinct clip must be rate limited. + hub.ApplyClip(Clip("id-overflow", "payload-overflow"), sender, now); + Assert.False(DequeueAck(sender).Acked, "11th distinct clip should be rate limited"); + + // A duplicate of the very first clip must still be acked without consuming a token. + hub.ApplyClip(Clip("id-0", "payload-0"), sender, now); + var duplicate = DequeueAck(sender); + Assert.True(duplicate.Acked, "duplicate id should bypass the token bucket"); + + using var ack = JsonDocument.Parse(duplicate.Payload!); + // The duplicate ack replays the version id-0 originally received (the 1st distinct clip), and + // the hub version must not have advanced past the drained burst. + Assert.Equal(1UL, ack.RootElement.GetProperty("version").GetUInt64()); + Assert.Equal(10UL, hub.Version); + } + + // U23 + [Fact] + public void DuplicateId_NewContent_IsTreatedAsFreshMessage() + { + var (hub, coordinator) = NewHub(); + var sender = NewConnection(); + var now = DateTimeOffset.FromUnixTimeSeconds(1760000000); + + hub.ApplyClip(Clip("same-id", "first"), sender, now); + Assert.Equal(1UL, hub.Version); + DrainQueue(sender); + + coordinator.Warnings.Clear(); + hub.ApplyClip(Clip("same-id", "second", "h2"), sender, now); + + // Fresh message: consumes a token, generates a new version, logs the reuse warning. + var ack = DequeueAck(sender); + Assert.True(ack.Acked, "fresh-clip path should succeed while tokens remain"); + Assert.Equal(2UL, hub.Version); + Assert.Contains(coordinator.Warnings, entry => entry.Contains("Replacing reused clip id", StringComparison.Ordinal)); + } + + // U24 — documented dead-branch behavior of the ring itself. + [Fact] + public void IsUnchangedDuplicate_ForUnknownId_ReturnsFalse() + { + var ring = new SeenIdRing(4); + Assert.False(ring.IsUnchangedDuplicate("missing", "payload", "hash", false, out var latest)); + Assert.Null(latest); + } + + // U25 — bucket edge cases beyond the existing refill test. + [Fact] + public void TryAcquire_BoundaryCases() + { + var now = DateTimeOffset.FromUnixTimeSeconds(1760000000); + + // Burst boundary: burst acquisitions pass, the next one at the same instant fails. + var bucket = new TokenBucket(3, 2.0, now); + Assert.True(bucket.TryAcquire(now)); + Assert.True(bucket.TryAcquire(now)); + Assert.True(bucket.TryAcquire(now)); + Assert.False(bucket.TryAcquire(now)); + + // Half-second refill (2 tokens/sec) grants one token. + Assert.True(bucket.TryAcquire(now.AddMilliseconds(500))); + + // Clock moving backwards is rejected outright. + Assert.False(bucket.TryAcquire(now)); + } + + [Fact] + public void DuplicateId_RateLimitedError_CarriesReferenceId() + { + var (hub, _) = NewHub(); + var sender = NewConnection(); + var now = DateTimeOffset.FromUnixTimeSeconds(1760000000); + + for (var index = 0; index < 10; index++) + { + hub.ApplyClip(Clip($"id-{index}", $"payload-{index}"), sender, now); + } + DrainQueue(sender); + + hub.ApplyClip(Clip("id-overflow", "payload-overflow"), sender, now); + var (acked, payload) = DequeueAck(sender); + Assert.False(acked); + Assert.NotNull(payload); + + using var error = JsonDocument.Parse(payload!); + Assert.Equal("rate_limited", error.RootElement.GetProperty("code").GetString()); + Assert.Equal("id-overflow", error.RootElement.GetProperty("referenceId").GetString()); + } +} \ No newline at end of file diff --git a/TextCascade.Server.Tests/NetworkIntegration/FrameFragmentationTests.cs b/TextCascade.Server.Tests/NetworkIntegration/FrameFragmentationTests.cs new file mode 100644 index 0000000..b40e620 --- /dev/null +++ b/TextCascade.Server.Tests/NetworkIntegration/FrameFragmentationTests.cs @@ -0,0 +1,255 @@ +using System.Net.Http.Json; +using System.Net.WebSockets; +using System.Text; +using System.Text.Json; +using TextCascade.Server; + +namespace TextCascade.Server.Tests.NetworkIntegration; + +[Trait("Category", "NetworkIntegration")] +public class FrameFragmentationTests +{ + private static async Task ConnectAndHelloAsync(string wssUrl, string token, string clientId) + { + var ws = new ClientWebSocket(); + ws.Options.AddSubProtocol("textcascade.v1"); + ws.Options.SetRequestHeader("Authorization", $"Bearer {token}"); + ws.Options.RemoteCertificateValidationCallback = (_, _, _, _) => true; + + using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(10)); + await ws.ConnectAsync(new Uri(wssUrl), cts.Token); + + var hello = JsonSerializer.SerializeToUtf8Bytes(new + { + type = "hello", + clientId, + clientName = clientId, + lastServerVersion = 0, + snapshot = (object?)null, + }); + await ws.SendAsync(hello, WebSocketMessageType.Text, true, cts.Token); + await ReceiveOneAsync(ws); // welcome + return ws; + } + + private static async Task SendFragmentedAsync(ClientWebSocket ws, byte[][] fragments) + { + using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(10)); + for (var index = 0; index < fragments.Length; index++) + { + var endOfMessage = index == fragments.Length - 1; + await ws.SendAsync(fragments[index], WebSocketMessageType.Text, endOfMessage, cts.Token); + } + } + + private static async Task ReceiveOneAsync(ClientWebSocket ws) + { + var buffer = new byte[1024 * 1024]; + using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(15)); + var result = await ws.ReceiveAsync(new ArraySegment(buffer), cts.Token); + if (result.MessageType == WebSocketMessageType.Close) + { + return []; + } + + return buffer.AsMemory(0, result.Count).ToArray(); + } + + private static async Task<(string Type, byte[] Payload)> ReceiveMessageAsync(ClientWebSocket ws) + { + // Frames may arrive fragmented from the server as well; loop until a full message. + using var message = new MemoryStream(); + var buffer = new byte[64 * 1024]; + while (true) + { + using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(15)); + var result = await ws.ReceiveAsync(new ArraySegment(buffer), cts.Token); + if (result.MessageType == WebSocketMessageType.Close) + { + return ("close", []); + } + + message.Write(buffer, 0, result.Count); + if (result.EndOfMessage) + { + var payload = message.ToArray(); + return (JsonDocument.Parse(payload).RootElement.GetProperty("type").GetString()!, payload); + } + } + } + + private static async Task CloseAsync(ClientWebSocket ws) + { + if (ws.State == WebSocketState.Open) + { + try + { + using var cts = new CancellationTokenSource(TimeSpan.FromMilliseconds(500)); + await ws.CloseOutputAsync(WebSocketCloseStatus.NormalClosure, "done", cts.Token); + } + catch { } + } + + ws.Dispose(); + } + + // N6 + [Fact] + public async Task FragmentedClip_Reassembles_AndBroadcasts() + { + await using var fixture = NetworkTestFixture.Create(); + var server = await fixture.StartAsync(); + try + { + var (tokenA, tokenB) = await LoginTwoAsync(server.Authority); + var wsA = await ConnectAndHelloAsync(server.WssUrl, tokenA, "frag-A"); + var wsB = await ConnectAndHelloAsync(server.WssUrl, tokenB, "frag-B"); + try + { + // ~300KB payload split into three fragments below single-MSS scale boundaries. + var payload = new string('x', 300_000); + var clip = JsonSerializer.SerializeToUtf8Bytes(new + { + type = "clip", + id = "frag-clip-1", + payload, + encrypted = false, + hash = "frag-hash", + }); + var fragments = new[] + { + clip.AsMemory(0, 100_000).ToArray(), + clip.AsMemory(100_000, 100_000).ToArray(), + clip.AsMemory(200_000).ToArray(), + }; + + await SendFragmentedAsync(wsA, fragments); + + var ack = await ReceiveMessageAsync(wsA); + Assert.Equal("clip_ack", ack.Type); + + var broadcast = await ReceiveMessageAsync(wsB); + Assert.Equal("clip", broadcast.Type); + using var doc = JsonDocument.Parse(broadcast.Payload); + Assert.Equal(payload, doc.RootElement.GetProperty("payload").GetString()); + } + finally + { + await CloseAsync(wsA); + await CloseAsync(wsB); + } + } + finally + { + await server.StopAsync(); + } + } + + // N7 + [Fact] + public async Task OversizeFrame_Closes1009() + { + await using var fixture = NetworkTestFixture.Create(); + var server = await fixture.StartAsync(); + try + { + var token = await LoginAsync(server.Authority); + var ws = await ConnectAndHelloAsync(server.WssUrl, token, "oversize-A"); + try + { + // Total frame exceeds max_frame_bytes (589824) via three fragments. + var oversize = new string('y', 600_000); + var clip = JsonSerializer.SerializeToUtf8Bytes(new + { + type = "clip", + id = "oversize-1", + payload = oversize, + encrypted = false, + hash = "h", + }); + await SendFragmentedAsync(ws, [clip.AsMemory(0, 300_000).ToArray(), clip.AsMemory(300_000).ToArray()]); + + // Server sends the frame_too_large error then closes with 1009 (MessageTooBig). + var closeSeen = false; + for (var attempt = 0; attempt < 2 && !closeSeen; attempt++) + { + var buffer = new byte[64 * 1024]; + using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(10)); + var result = await ws.ReceiveAsync(new ArraySegment(buffer), cts.Token); + if (result.MessageType == WebSocketMessageType.Close) + { + closeSeen = true; + Assert.Equal(WebSocketCloseStatus.MessageTooBig, result.CloseStatus); + } + } + + Assert.True(closeSeen, "server should close the connection with 1009 after an oversize frame"); + } + finally + { + ws.Dispose(); + } + } + finally + { + await server.StopAsync(); + } + } + + // N8 + [Fact] + public async Task ZeroLengthFrame_TreatedAsFrameTooLarge() + { + await using var fixture = NetworkTestFixture.Create(); + var server = await fixture.StartAsync(); + try + { + var token = await LoginAsync(server.Authority); + var ws = await ConnectAndHelloAsync(server.WssUrl, token, "zerolen-A"); + try + { + using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(10)); + await ws.SendAsync(Array.Empty(), WebSocketMessageType.Text, endOfMessage: true, cts.Token); + + var closeSeen = false; + for (var attempt = 0; attempt < 2 && !closeSeen; attempt++) + { + var buffer = new byte[64 * 1024]; + var result = await ws.ReceiveAsync(new ArraySegment(buffer), cts.Token); + if (result.MessageType == WebSocketMessageType.Close) + { + closeSeen = true; + // Current implementation classifies empty frames as frame_too_large -> 1009. + Assert.Equal(WebSocketCloseStatus.MessageTooBig, result.CloseStatus); + } + } + + Assert.True(closeSeen, "zero-length frame should terminate the connection"); + } + finally + { + ws.Dispose(); + } + } + finally + { + await server.StopAsync(); + } + } + + private static async Task LoginAsync(string authority) + { + using var https = NewHttpsClient(); + var response = await https.PostAsJsonAsync($"https://{authority}/api/v1/login", new { username = "alice", password = "password123" }); + response.EnsureSuccessStatusCode(); + using var doc = JsonDocument.Parse(await response.Content.ReadAsStringAsync()); + return doc.RootElement.GetProperty("token").GetString()!; + } + + private static async Task<(string, string)> LoginTwoAsync(string authority) => (await LoginAsync(authority), await LoginAsync(authority)); + + private static HttpClient NewHttpsClient() => new(new HttpClientHandler + { + ServerCertificateCustomValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator, + }); +} \ No newline at end of file diff --git a/TextCascade.Server.Tests/NetworkIntegration/NetworkTestFixture.cs b/TextCascade.Server.Tests/NetworkIntegration/NetworkTestFixture.cs new file mode 100644 index 0000000..6a68769 --- /dev/null +++ b/TextCascade.Server.Tests/NetworkIntegration/NetworkTestFixture.cs @@ -0,0 +1,189 @@ +using System.Net; +using System.Security.Cryptography; +using System.Security.Cryptography.X509Certificates; +using System.Text; +using Microsoft.AspNetCore.Builder; +using Microsoft.Extensions.DependencyInjection; +using Microsoft.Extensions.Hosting; +using Microsoft.Extensions.Logging; +using TextCascade.Server; + +namespace TextCascade.Server.Tests.NetworkIntegration; + +/// +/// Real-Kestrel TLS fixture for the NetworkIntegration category: self-signed certificate, +/// random port binding, and a restart-friendly handle over ServerHost.CreateApp. +/// +public sealed class NetworkTestFixture : IAsyncDisposable +{ + public string TempDir { get; } + public RuntimeConfig Config { get; } + public UsersFile Users { get; } + public string UsersPath { get; } + public string StatePath { get; } + public string PfxPath { get; } + public TestLogCollector Logs { get; } = new(); + + private NetworkTestFixture(string tempDir, RuntimeConfig config, UsersFile users, string usersPath, string statePath, string pfxPath) + { + TempDir = tempDir; + Config = config; + Users = users; + UsersPath = usersPath; + StatePath = statePath; + PfxPath = pfxPath; + } + + public static NetworkTestFixture Create( + Func? configModifier = null, + Action? usersOverride = null) + { + var tempDir = Path.Combine(Path.GetTempPath(), "textcascade-ni-" + Guid.NewGuid().ToString("N")); + Directory.CreateDirectory(tempDir); + + var certificate = SelfSignedCertificate.Create("localhost"); + var pfxPath = Path.Combine(tempDir, "server.pfx"); + File.WriteAllBytes(pfxPath, certificate.Export(X509ContentType.Pfx)); + + var usersPath = Path.Combine(tempDir, "users.json"); + var statePath = Path.Combine(tempDir, "state.json"); + var users = new UsersFile + { + Users = + [ + new UserRecord("alice", "$argon2id$v=19$m=19456,t=2,p=1$c2FsdA$aG9zdA", 1), + ], + NextTokenVersion = 2, + }; + usersOverride?.Invoke(users); + UsersFile.SaveUsers(usersPath, users); + + var defaults = TextCascade.Server.Config.CreateDefaultConfig(); + var config = defaults with + { + TokenSecret = Encoding.UTF8.GetBytes("12345678901234567890123456789012"), + Files = new FilesConfig(usersPath, statePath), + Server = new ServerConfig("127.0.0.1", 0, pfxPath), + Limits = defaults.Limits, + }; + if (configModifier is not null) + { + config = configModifier(config); + } + + return new NetworkTestFixture(tempDir, config, users, usersPath, statePath, pfxPath); + } + + /// Starts a Kestrel instance bound to 127.0.0.1:0 over the self-signed PFX. + public async Task StartAsync() + { + using var loaded = CertificateLoader.Load(PfxPath); + // LoadedCertificate disposes the underlying cert; give CreateApp its own instance. + var app = ServerHost.CreateApp( + [], + Config, + Users, + new RuntimeStateStore(StatePath), + hasher: new FastPasswordHasher(), + clock: new SystemClock(), + certificate: new LoadedCertificate( + new X509Certificate2(PfxPath), + new X509Certificate2Collection(new X509Certificate2(PfxPath)))); + + app.Services.GetRequiredService().AddProvider(Logs); + app.Urls.Add("https://127.0.0.1:0"); + await app.StartAsync(); + + var address = app.Urls.First(); + var uri = new Uri(address); + return new RunningServer(app, uri.Port, uri.Authority); + } + + public ValueTask DisposeAsync() + { + if (Directory.Exists(TempDir)) + { + try { Directory.Delete(TempDir, true); } catch { } + } + + GC.SuppressFinalize(this); + return ValueTask.CompletedTask; + } + + public sealed class RunningServer + { + public WebApplication App { get; } + public int Port { get; } + public string Authority { get; } + + public RunningServer(WebApplication app, int port, string authority) + { + App = app; + Port = port; + Authority = authority; + } + + public string WssUrl => $"wss://{Authority}/api/v1/sync"; + + public async Task StopAsync() + { + using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(5)); + try { await App.StopAsync(cts.Token); } catch { } + await App.DisposeAsync(); + } + } +} + +public sealed class TestLogCollector : ILoggerProvider, ILogger +{ + public List Entries { get; } = new(); + + public ILogger CreateLogger(string categoryName) => this; + + public IDisposable? BeginScope(TState state) where TState : notnull => null; + + public bool IsEnabled(LogLevel logLevel) => true; + + public void Log(LogLevel logLevel, EventId eventId, TState state, Exception? exception, Func formatter) + { + lock (Entries) + { + Entries.Add(formatter(state, exception)); + } + } + + public void Dispose() { } +} + +public sealed class FastPasswordHasher : IPasswordHasher +{ + public const string ValidHash = "$argon2id$v=19$m=19456,t=2,p=1$c2FsdA$aG9zdA"; + + public string Hash(string password, Isopoh.Cryptography.Argon2.Argon2Config config) => ValidHash; + + public bool Verify(string password, string encodedHash) => password == "password123" && encodedHash == ValidHash; + + public bool NeedsRehash(string encodedHash, Isopoh.Cryptography.Argon2.Argon2Config config) => false; +} + +/// Self-signed leaf certificate helpers (decision Q3: generated at test runtime). +public static class SelfSignedCertificate +{ + public static X509Certificate2 Create(string subject) + { + using var rsa = RSA.Create(2048); + var request = new CertificateRequest( + $"CN={subject}", + rsa, + HashAlgorithmName.SHA256, + RSASignaturePadding.Pkcs1); + + request.CertificateExtensions.Add(new X509BasicConstraintsExtension(false, false, 0, true)); + var san = new SubjectAlternativeNameBuilder(); + san.AddDnsName(subject); + san.AddIpAddress(IPAddress.Parse("127.0.0.1")); + request.CertificateExtensions.Add(san.Build()); + + return request.CreateSelfSigned(DateTimeOffset.UtcNow.AddDays(-1), DateTimeOffset.UtcNow.AddYears(5)); + } +} \ No newline at end of file diff --git a/TextCascade.Server.Tests/NetworkIntegration/RestartRecoveryTests.cs b/TextCascade.Server.Tests/NetworkIntegration/RestartRecoveryTests.cs new file mode 100644 index 0000000..913c715 --- /dev/null +++ b/TextCascade.Server.Tests/NetworkIntegration/RestartRecoveryTests.cs @@ -0,0 +1,343 @@ +using System.Net.Http.Json; +using System.Net.WebSockets; +using System.Text; +using System.Text.Json; +using Microsoft.Extensions.DependencyInjection; +using TextCascade.Server; + +namespace TextCascade.Server.Tests.NetworkIntegration; + +[Trait("Category", "NetworkIntegration")] +public class RestartRecoveryTests +{ + private static HttpClient NewHttpsClient() => new(new HttpClientHandler + { + ServerCertificateCustomValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator, + }); + + private static async Task LoginAsync(string authority, string username = "alice") + { + using var https = NewHttpsClient(); + var response = await https.PostAsJsonAsync($"https://{authority}/api/v1/login", new { username, password = "password123" }); + response.EnsureSuccessStatusCode(); + using var doc = JsonDocument.Parse(await response.Content.ReadAsStringAsync()); + return doc.RootElement.GetProperty("token").GetString()!; + } + + private static async Task ConnectAsync(string wssUrl, string token, string clientId, ulong lastServerVersion = 0, object? snapshot = null) + { + var ws = new ClientWebSocket(); + ws.Options.AddSubProtocol("textcascade.v1"); + ws.Options.SetRequestHeader("Authorization", $"Bearer {token}"); + ws.Options.RemoteCertificateValidationCallback = (_, _, _, _) => true; + + using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(10)); + await ws.ConnectAsync(new Uri(wssUrl), cts.Token); + + var hello = JsonSerializer.SerializeToUtf8Bytes(new + { + type = "hello", + clientId, + clientName = clientId, + lastServerVersion, + snapshot, + }); + await ws.SendAsync(hello, WebSocketMessageType.Text, true, cts.Token); + return ws; + } + + private static async Task<(string Type, JsonDocument Doc)> ReceiveTypedAsync(ClientWebSocket ws) + { + var buffer = new byte[1024 * 1024]; + using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(15)); + WebSocketReceiveResult result; + try + { + result = await ws.ReceiveAsync(new ArraySegment(buffer), cts.Token); + } + catch (WebSocketException) + { + // Abrupt teardown (abort path) — surfaces as a close with no frame content. + return ("close", JsonDocument.Parse("{}")); + } + + if (result.MessageType == WebSocketMessageType.Close) + { + return ("close", JsonDocument.Parse("{}")); + } + + var payload = buffer.AsMemory(0, result.Count).ToArray(); + return (JsonDocument.Parse(payload).RootElement.GetProperty("type").GetString()!, JsonDocument.Parse(payload)); + } + + // N9 + [Fact] + public async Task Restart_KeepsTokenValid_DirectReconnect() + { + await using var fixture = NetworkTestFixture.Create( + configModifier: cfg => cfg with { Limits = cfg.Limits with { SnapshotWindowSeconds = 0 } }); + var first = await fixture.StartAsync(); + + // First instance: login and push one clip through to move the version to 1. + string token; + try + { + token = await LoginAsync(first.Authority); + var ws = await ConnectAsync(first.WssUrl, token, "restart-A"); + try + { + var welcome = await ReceiveTypedAsync(ws); + Assert.Equal("welcome", welcome.Type); + + var clip = JsonSerializer.SerializeToUtf8Bytes(new + { + type = "clip", + id = "pre-restart-1", + payload = "before restart", + encrypted = false, + hash = "h1", + }); + using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(5)); + await ws.SendAsync(clip, WebSocketMessageType.Text, true, cts.Token); + var ack = await ReceiveTypedAsync(ws); + Assert.Equal("clip_ack", ack.Type); + Assert.Equal(1UL, ack.Doc.RootElement.GetProperty("version").GetUInt64()); + } + finally + { + ws.Dispose(); + } + } + finally + { + await first.StopAsync(); + } + + // State file must have been flushed on shutdown so the version persists. + Assert.True(File.Exists(fixture.StatePath), "state file should exist after graceful stop"); + + // Second instance: same users file + state file; old token works without re-login. + var second = await fixture.StartAsync(); + try + { + var ws = await ConnectAsync(second.WssUrl, token, "restart-B"); + try + { + var welcome = await ReceiveTypedAsync(ws); + Assert.Equal("welcome", welcome.Type); + + // Version baseline came from the persisted state: next clip continues from 2. + var clip = JsonSerializer.SerializeToUtf8Bytes(new + { + type = "clip", + id = "post-restart-1", + payload = "after restart", + encrypted = false, + hash = "h2", + }); + using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(5)); + await ws.SendAsync(clip, WebSocketMessageType.Text, true, cts.Token); + var ack = await ReceiveTypedAsync(ws); + Assert.Equal("clip_ack", ack.Type); + Assert.Equal(2UL, ack.Doc.RootElement.GetProperty("version").GetUInt64()); + } + finally + { + ws.Dispose(); + } + } + finally + { + await second.StopAsync(); + } + } + + // N10 + [Fact] + public async Task Restart_SnapshotElection_RestoresLatest() + { + await using var fixture = NetworkTestFixture.Create( + configModifier: cfg => cfg with { Limits = cfg.Limits with { SnapshotWindowSeconds = 5 } }); + var first = await fixture.StartAsync(); + string token; + try + { + token = await LoginAsync(first.Authority); + } + finally + { + await first.StopAsync(); + } + + var second = await fixture.StartAsync(); + try + { + var modifiedTime = DateTimeOffset.UtcNow; + var snapshotOf = (ulong version, string payload) => (object)new + { + payload, + encrypted = false, + hash = $"hash-{version}", + localModifiedAtUtc = modifiedTime.UtcDateTime.ToString("yyyy-MM-ddTHH:mm:ssZ"), + }; + + var ws128 = await ConnectAsync(second.WssUrl, token, "snap-128", 128, snapshotOf(128, "snapshot-v128")); + var ws64 = await ConnectAsync(second.WssUrl, token, "snap-64", 64, snapshotOf(64, "snapshot-v64")); + try + { + await Task.Delay(100); + + // Close the recovery window through the server to force the election now. + var syncServer = second.App.Services.GetRequiredService(); + var hub = syncServer.GetOrCreateHub("alice", fixture.Config); + hub.CloseRecoveryWindow(DateTimeOffset.UtcNow.AddMinutes(1)); + + var welcome128 = await ReceiveTypedAsync(ws128); + var welcome64 = await ReceiveTypedAsync(ws64); + foreach (var welcome in new[] { welcome128, welcome64 }) + { + Assert.Equal("welcome", welcome.Type); + Assert.Equal("snapshot-v128", welcome.Doc.RootElement.GetProperty("latest").GetProperty("payload").GetString()); + Assert.Equal(128UL, welcome.Doc.RootElement.GetProperty("latest").GetProperty("version").GetUInt64()); + } + + // The next clip continues from the restored version without +1 on restore. + var clip = JsonSerializer.SerializeToUtf8Bytes(new + { + type = "clip", + id = "post-election-1", + payload = "fresh clip", + encrypted = false, + hash = "h3", + }); + using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(5)); + await ws64.SendAsync(clip, WebSocketMessageType.Text, true, cts.Token); + var ack = await ReceiveTypedAsync(ws64); + Assert.Equal("clip_ack", ack.Type); + Assert.Equal(129UL, ack.Doc.RootElement.GetProperty("version").GetUInt64()); + } + finally + { + ws128.Dispose(); + ws64.Dispose(); + } + } + finally + { + await second.StopAsync(); + } + } + + /// Reads from the socket until a bye frame or close/abort; never throws. + private static async Task<(string Type, string? Reason)> ReceiveByeOrCloseAsync(ClientWebSocket ws) + { + var buffer = new byte[1024 * 1024]; + while (true) + { + using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(20)); + WebSocketReceiveResult result; + try + { + result = await ws.ReceiveAsync(new ArraySegment(buffer), cts.Token); + } + catch (Exception exception) when (exception is WebSocketException or OperationCanceledException or IOException) + { + return ("close", null); + } + + if (result.MessageType == WebSocketMessageType.Close) + { + return ("close", result.CloseStatus?.ToString()); + } + + var doc = JsonDocument.Parse(buffer.AsMemory(0, result.Count).ToArray()); + var type = doc.RootElement.GetProperty("type").GetString(); + var reason = doc.RootElement.TryGetProperty("reason", out var reasonElement) ? reasonElement.GetString() : null; + doc.Dispose(); + if (type == "bye") + { + return ("bye", reason); + } + } + } + + // N11 + N12 combined chain: login -> connect -> clip flow -> graceful stop with bye/1001. + [Fact] + public async Task FullChain_Login_Connect_Send_Receive_Bye1001() + { + await using var fixture = NetworkTestFixture.Create( + configModifier: cfg => cfg with { Limits = cfg.Limits with { SnapshotWindowSeconds = 0 } }); + var server = await fixture.StartAsync(); + try + { + var tokenA = await LoginAsync(server.Authority); + var tokenB = await LoginAsync(server.Authority); + + var wsA = await ConnectAsync(server.WssUrl, tokenA, "chain-A"); + var wsB = await ConnectAsync(server.WssUrl, tokenB, "chain-B"); + try + { + var welcomeA = await ReceiveTypedAsync(wsA); + Assert.Equal("welcome", welcomeA.Type); + welcomeA.Doc.Dispose(); + var welcomeB = await ReceiveTypedAsync(wsB); + Assert.Equal("welcome", welcomeB.Type); + welcomeB.Doc.Dispose(); + + var clip = JsonSerializer.SerializeToUtf8Bytes(new + { + type = "clip", + id = "chain-clip-1", + payload = "chain payload", + encrypted = false, + hash = "hc", + }); + using var sendCts = new CancellationTokenSource(TimeSpan.FromSeconds(5)); + await wsA.SendAsync(clip, WebSocketMessageType.Text, true, sendCts.Token); + + var ack = await ReceiveTypedAsync(wsA); + Assert.Equal("clip_ack", ack.Type); + ack.Doc.Dispose(); + + // The two hellos racing the recovery-window close can deliver a second + // welcome to B; skip extras until the broadcast clip arrives. + JsonDocument broadcastDoc; + while (true) + { + var broadcast = await ReceiveTypedAsync(wsB); + broadcastDoc = broadcast.Doc; + if (broadcast.Type == "clip") + { + break; + } + Assert.Equal("welcome", broadcast.Type); + broadcastDoc.Dispose(); + } + + Assert.Equal("chain payload", broadcastDoc.RootElement.GetProperty("payload").GetString()); + broadcastDoc.Dispose(); + + // Graceful stop: bye then close 1001 (EndpointUnavailable). Both frames are + // best-effort by contract: a fast drain can abort the socket before they land. + var byeTask = ReceiveByeOrCloseAsync(wsB); + await server.StopAsync(); + + var (byeType, byeReason) = await byeTask; + if (byeType == "bye") + { + Assert.Equal("server_shutdown", byeReason); + } + // else: fast drain aborted before bye landed; both paths are contract-legal (spec §7). + } + finally + { + wsA.Dispose(); + wsB.Dispose(); + } + } + finally + { + // StopAsync already ran; ensure app disposal via fixture (fixture deletes temp dir). + } + } +} \ No newline at end of file diff --git a/TextCascade.Server.Tests/NetworkIntegration/TlsAndWssHandshakeTests.cs b/TextCascade.Server.Tests/NetworkIntegration/TlsAndWssHandshakeTests.cs new file mode 100644 index 0000000..c4ee1ce --- /dev/null +++ b/TextCascade.Server.Tests/NetworkIntegration/TlsAndWssHandshakeTests.cs @@ -0,0 +1,214 @@ +using System.Net.Security; +using System.Net.Sockets; +using System.Security.Authentication; +using System.Net; +using System.Net.Http.Headers; +using System.Net.Http.Json; +using System.Net.WebSockets; +using System.Text; +using System.Text.Json; +using TextCascade.Server; + +namespace TextCascade.Server.Tests.NetworkIntegration; + +[Trait("Category", "NetworkIntegration")] +public class TlsAndWssHandshakeTests +{ + private static HttpClient NewHttpsClient() + { + return new HttpClient(new HttpClientHandler + { + ServerCertificateCustomValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator, + }); + } + + // .NET 10 removed ClientWebSocketOptions.SslProtocols: the negotiated TLS version follows + // OS policy. Protocol-floor verification is done by direct SslStream probes (N2). + private static async Task ConnectWssAsync(string wssUrl, string token) + { + var ws = new ClientWebSocket(); + ws.Options.AddSubProtocol("textcascade.v1"); + ws.Options.SetRequestHeader("Authorization", $"Bearer {token}"); + ws.Options.RemoteCertificateValidationCallback = (_, _, _, _) => true; + + using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(10)); + await ws.ConnectAsync(new Uri(wssUrl), cts.Token); + return ws; + } + + private static async Task SendHelloAsync(ClientWebSocket ws, string clientId) + { + var hello = JsonSerializer.SerializeToUtf8Bytes(new + { + type = "hello", + clientId, + clientName = clientId, + lastServerVersion = 0, + snapshot = (object?)null, + }); + using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(5)); + await ws.SendAsync(hello, WebSocketMessageType.Text, true, cts.Token); + } + + private static async Task ReceiveJsonAsync(ClientWebSocket ws) + { + var buffer = new byte[64 * 1024]; + using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(5)); + var result = await ws.ReceiveAsync(new ArraySegment(buffer), cts.Token); + Assert.Equal(WebSocketMessageType.Text, result.MessageType); + return JsonDocument.Parse(buffer.AsMemory(0, result.Count)); + } + + // N1 + [Fact] + public async Task Connects_WithSelfSignedPfx_OverWss() + { + await using var fixture = NetworkTestFixture.Create(); + var server = await fixture.StartAsync(); + try + { + using var https = NewHttpsClient(); + var token = await LoginAsync(https, server.Authority); + + var ws = await ConnectWssAsync(server.WssUrl, token); + try + { + Assert.Equal(WebSocketState.Open, ws.State); + await SendHelloAsync(ws, "tls-client-1"); + using var welcome = await ReceiveJsonAsync(ws); + Assert.Equal("welcome", welcome.RootElement.GetProperty("type").GetString()); + } + finally + { + ws.Dispose(); + } + } + finally + { + await server.StopAsync(); + } + } + + // N2 — the server accepts explicit TLS 1.2 and TLS 1.3 handshakes (SslStream probes; + // ClientWebSocket can no longer pin a version on .NET 10). + [Theory] + [InlineData(SslProtocols.Tls12)] + [InlineData(SslProtocols.Tls13)] + public async Task ServerHandshakes_WithExplicitTlsVersion(SslProtocols protocolVersion) + { + await using var fixture = NetworkTestFixture.Create(); + var server = await fixture.StartAsync(); + try + { + using var tcp = new TcpClient(); + using var connectCts = new CancellationTokenSource(TimeSpan.FromSeconds(5)); + await tcp.ConnectAsync(IPAddress.Parse("127.0.0.1"), server.Port, connectCts.Token); + + await using var sslStream = new SslStream(tcp.GetStream(), leaveInnerStreamOpen: false); + using var handshakeCts = new CancellationTokenSource(TimeSpan.FromSeconds(10)); + await sslStream.AuthenticateAsClientAsync(new SslClientAuthenticationOptions + { + TargetHost = "localhost", + EnabledSslProtocols = protocolVersion, + RemoteCertificateValidationCallback = (_, _, _, _) => true, + }, handshakeCts.Token); + + Assert.True(sslStream.IsEncrypted); + Assert.Equal(protocolVersion, sslStream.SslProtocol); + // If a modern OS disables TLS 1.2 by policy the Tls12 inline case fails here; + // that is an environment change, not a server regression. + } + finally + { + await server.StopAsync(); + } + } + + // N3 + [Fact] + public async Task HttpUpgrade_Succeeds_WithBearerAndSubProtocol() + { + await using var fixture = NetworkTestFixture.Create(); + var server = await fixture.StartAsync(); + try + { + using var https = NewHttpsClient(); + var token = await LoginAsync(https, server.Authority); + + var ws = await ConnectWssAsync(server.WssUrl, token); + try + { + Assert.Equal("textcascade.v1", ws.SubProtocol); + await SendHelloAsync(ws, "subproto-client"); + using var welcome = await ReceiveJsonAsync(ws); + Assert.Equal(1, welcome.RootElement.GetProperty("protocolVersion").GetInt32()); + } + finally + { + ws.Dispose(); + } + } + finally + { + await server.StopAsync(); + } + } + + // N4 + [Fact] + public async Task HttpsLogin_Endpoint_Works() + { + await using var fixture = NetworkTestFixture.Create(); + var server = await fixture.StartAsync(); + try + { + using var https = NewHttpsClient(); + var response = await https.PostAsJsonAsync($"https://{server.Authority}/api/v1/login", new { username = "alice", password = "password123" }); + Assert.True(response.IsSuccessStatusCode, $"Login failed: {response.StatusCode}"); + + using var doc = JsonDocument.Parse(await response.Content.ReadAsStringAsync()); + var root = doc.RootElement; + Assert.False(string.IsNullOrEmpty(root.GetProperty("token").GetString())); + Assert.Equal(1, root.GetProperty("protocolVersion").GetInt32()); + Assert.True(root.TryGetProperty("expiresAtUtc", out _)); + Assert.True(root.TryGetProperty("maxTextBytes", out _)); + Assert.True(root.TryGetProperty("helloTimeoutSeconds", out _)); + Assert.True(root.TryGetProperty("heartbeatIntervalSeconds", out _)); + Assert.True(root.TryGetProperty("heartbeatTimeoutSeconds", out _)); + } + finally + { + await server.StopAsync(); + } + } + + // N5 + [Fact] + public async Task RandomPortBinding_ActuallyBinds() + { + await using var fixture = NetworkTestFixture.Create(); + var server = await fixture.StartAsync(); + try + { + Assert.True(server.Port > 0, "Kestrel should bind an ephemeral port > 0"); + + // The bound endpoint must accept TCP connections. + using var tcp = new TcpClient(); + using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(5)); + await tcp.ConnectAsync(IPAddress.Parse("127.0.0.1"), server.Port, cts.Token); + Assert.True(tcp.Connected); + } + finally + { + await server.StopAsync(); + } + } + + private static async Task LoginAsync(HttpClient https, string authority) + { + var response = await https.PostAsJsonAsync($"https://{authority}/api/v1/login", new { username = "alice", password = "password123" }); + Assert.True(response.IsSuccessStatusCode, $"Login failed: {response.StatusCode}"); + using var doc = JsonDocument.Parse(await response.Content.ReadAsStringAsync()); + return doc.RootElement.GetProperty("token").GetString()!; + } +} \ No newline at end of file diff --git a/TextCascade.Server.Tests/SlowHashSmokeTests.cs b/TextCascade.Server.Tests/SlowHashSmokeTests.cs new file mode 100644 index 0000000..ec0329f --- /dev/null +++ b/TextCascade.Server.Tests/SlowHashSmokeTests.cs @@ -0,0 +1,63 @@ +using System.Text.RegularExpressions; +using Isopoh.Cryptography.Argon2; +using TextCascade.Server; + +namespace TextCascade.Server.Tests; + +[Trait("Category", "SlowHash")] +public class SlowHashSmokeTests +{ + private static Argon2Config ProductionConfig() => Cli.CreateArgon2Config(TextCascade.Server.Config.CreateDefaultConfig()); + + private static (int Memory, int Time, int Threads) ParseEncodedParameters(string encoded) + { + var match = Regex.Match(encoded, @"m=(\d+),t=(\d+),p=(\d+)"); + Assert.True(match.Success, $"Not an Argon2 PHC string: {encoded}"); + return (int.Parse(match.Groups[1].Value), int.Parse(match.Groups[2].Value), int.Parse(match.Groups[3].Value)); + } + + // U26 + [Fact] + public void Hash_Then_Verify_RoundTrip() + { + var hasher = new Argon2PasswordHasher(); + var encoded = hasher.Hash("correct horse battery staple", ProductionConfig()); + + Assert.StartsWith("$argon2id$", encoded, StringComparison.Ordinal); + Assert.True(hasher.Verify("correct horse battery staple", encoded)); + Assert.False(hasher.Verify("wrong password", encoded)); + } + + // U27 — NeedsRehash is false when the encoded parameters match themselves exactly. + // (The Isopoh encoder may write a lane count that differs from the configured + // Argon2Parallelism, so the test reads back the actual m/t/p instead of assuming.) + [Fact] + public void NeedsRehash_MatchingParams_ReturnsFalse() + { + var hasher = new Argon2PasswordHasher(); + var encoded = hasher.Hash("some-password", ProductionConfig()); + var (memory, time, threads) = ParseEncodedParameters(encoded); + + Assert.False(Argon2PasswordHasher.NeedsRehash(encoded, memory, time, threads)); + Assert.True(Argon2PasswordHasher.NeedsRehash(encoded, memory + 1024, time, threads)); + Assert.True(Argon2PasswordHasher.NeedsRehash(encoded, memory, time + 1, threads)); + Assert.True(Argon2PasswordHasher.NeedsRehash(encoded, memory, time, threads + 1)); + } + + // U28 — rewriting the stored parameter segment to stale values flags a rehash. + [Fact] + public void NeedsRehash_StaleParams_ReturnsTrue() + { + var hasher = new Argon2PasswordHasher(); + var encoded = hasher.Hash("some-password", ProductionConfig()); + var (memory, time, threads) = ParseEncodedParameters(encoded); + + var stale = encoded + .Replace($"m={memory}", $"m={Math.Max(1024, memory / 8)}", StringComparison.Ordinal) + .Replace($"t={time}", "t=1", StringComparison.Ordinal) + .Replace($"p={threads}", "p=1", StringComparison.Ordinal); + Assert.NotEqual(encoded, stale); + + Assert.True(Argon2PasswordHasher.NeedsRehash(stale, memory, time, threads)); + } +} diff --git a/TextCascade.Server.Tests/TextCascade.Server.Tests.csproj b/TextCascade.Server.Tests/TextCascade.Server.Tests.csproj index ade3d41..2e8da18 100644 --- a/TextCascade.Server.Tests/TextCascade.Server.Tests.csproj +++ b/TextCascade.Server.Tests/TextCascade.Server.Tests.csproj @@ -18,6 +18,10 @@ + + + + diff --git a/TextCascade.Server/TextCascade.Server.csproj b/TextCascade.Server/TextCascade.Server.csproj index c1e075c..1169926 100644 --- a/TextCascade.Server/TextCascade.Server.csproj +++ b/TextCascade.Server/TextCascade.Server.csproj @@ -4,7 +4,7 @@ net10.0 enable enable - 0.3.5 + 0.4.0 TextCascade.Server true diff --git a/docs/server-spec.md b/docs/server-spec.md index 76ed45e..0deb02d 100644 --- a/docs/server-spec.md +++ b/docs/server-spec.md @@ -1,17 +1,17 @@ # TextCascade 轻量文本同步服务端规格 -状态：函数级设计已定稿，已按审查决策台账修订 -日期：2026-08-18 +状态：已按实现对齐 v0.3.5（commit 9ed6eba）修订；测试与契约细节以 specs/test-and-contract-spec.md 为准 +日期：2026-08-27（2026-08-18 定稿版漂移对齐，漂移溯源见 git 提交记录） 协议目标：不兼容原 ClipCascade，只做轻量、可靠、高性能的文本最新值同步 ## 1. 目标与非目标 ### 1.1 目标 -- 仅同步文本最新值：每用户只保存一份当前文本，不保存历史。 -- 无数据库：账号使用 `users.json`，文本与版本只在内存中。 -- 服务端重启可恢复：客户端用无状态 token 重连，并在恢复窗口内上报 snapshot。 -- 低空闲资源占用：无数据库轮询、无磁盘写入、无 WebUI、无 metrics endpoint。 +- 仅同步文本最新值：每用户只保存一份当前文本，不保存历史。文本本体只在内存中；用户版本号经 RuntimeStateStore 周期落盘用于重启续接（见 3.4）。 +- 无数据库：账号使用 `users.json`，版本号使用 `textcascade.state.json` 状态文件；除此之外无其他磁盘写入路径。 +- 服务端重启可恢复：客户端用无状态 token 重连，并在恢复窗口内上报 snapshot；版本基准跨重启保持单调。 +- 低空闲资源占用：无数据库轮询、无 WebUI、无 metrics endpoint。已知开销：RuntimeStateStore 每 5 秒脏检查刷盘、UserFileWatcher 每 30 秒轮询兜底重载。 - 明确边界：协议错误显式返回，慢连接被隔离或断开，绝不拖垮整个服务。 - 三端协议实现：服务端与桌面端使用 C#，Android 端使用 Kotlin；三端手写模型，由服务端契约测试约束。 @@ -28,37 +28,39 @@ ### 2.1 运行时与进程 - 技术栈：ASP.NET Core Minimal API + Kestrel 原生 WebSocket。 -- 目标框架：`net10.0`；产品版本采用 SemVer，从 `0.1.0` 开始。 +- 目标框架：`net10.0`；产品版本采用 SemVer，写入 `TextCascade.Server.csproj` 的 `Version`（当前 0.3.5）。 - 进程模型：单进程；生产环境由 systemd 或 Windows Service 托管并负责崩溃自动重启。 -- TLS：Kestrel 直接终止 TLS；不提供生产/开发模式开关，所有部署都禁止明文 HTTP 登录。 +- TLS：Kestrel 直接终止 TLS；不提供生产/开发模式开关，所有部署都禁止明文 HTTP 登录。TLS 协议版本跟随 OS 默认策略，未显式固定下限（见 8.2 与差距台账）。 - 部署产物：框架依赖单文件；目标机必须预装对应 .NET Runtime。 ### 2.2 核心对象 -- `ConnectionContext`：不可变稳定属性，包括连接 ID、用户名、clientId、socket、认证信息。 -- `ConnectionStateBag`：可变运行时状态，包括 lastSeen、关闭标记、发送 Channel；修改必须收敛到少数明确函数。 -- `UserHub`：每个在线用户一个 hub，持有最新值、版本号、幂等队列、令牌桶与用户 Channel。 +- `ConnectionContext`：稳定属性，包括连接 ID、用户名、clientId、clientName、socket、认证信息；`Hub` 属性为 internal set，仅在临时连接转正时一次性赋值，此后不可变。 +- `ConnectionStateBag`：可变运行时状态，包括 lastSeen、关闭标记、发送 Channel、HelloDeadline；修改收敛到少量明确函数。 +- `UserHub`：每个在线用户一个 hub，持有最新值、版本号、幂等 SeenIdRing、令牌桶与用户 Channel（无界）。 - `UserRegistry`：`ConcurrentDictionary`，不同用户天然并发。 -- `LatestText`：不可变 record，包含 payload、version、来源、更新时间；更新即替换引用。 +- `LatestText`：不可变 record，包含 payload、version、hash、encrypted、fromClientId、fromClientName、updatedAtUtc；更新即替换引用。 +- `RuntimeStateStore`：版本号落盘存储（见 3.4）。 ### 2.3 并发模型 1. 每个连接一个独立 `ReadLoopAsync`。 2. 读循环只负责收帧、解析、验证，然后把用户级 job 投递到 UserHub Channel。 -3. 每个 UserHub 一个 `UserLoopAsync` 单消费者，串行处理该用户的 clip、连接、断开与恢复 job。 +3. 每个 UserHub 一个 `RunUserLoopAsync` 单消费者，串行处理该用户的 clip、pong 与恢复 job。 4. 广播时只序列化一次 UTF-8 字节，并把同一份字节投递到每个连接的有界发送 Channel。 5. 每个连接一个 `ConnectionSendLoopAsync`，慢连接只积压自己的队列。 -6. 发送队列满立即取消该连接，不等待 drain，不补发应用层 error 或 WebSocket close frame。 +6. 发送队列满立即取消该连接，不等待 drain，不补发应用层 error 或 WebSocket close frame。各广播/ACK/ping 满队列路径直接 `Cts.Cancel()`；统一清理由连接处理器的 finally 兜底完成。`EnqueueImmediateClose` 路径额外执行 `Socket.Abort()`。 +7. 用户循环异常（含 `NextVersion` ulong 溢出抛出）触发 `RebuildHub`：取消该用户全部连接并重建 hub，进程存活——即版本溢出按"单用户熔断"处理而非进程级 fatal。 ## 3. 配置与用户 ### 3.1 配置函数 -- `CreateDefaultConfig()`：内置安全默认值。 -- `LoadTomlConfig(path)`：读取可选 TOML 配置并覆盖默认值。 -- `ApplyEnvironmentOverrides(config)`：环境变量覆盖敏感项与非默认部署值。 -- `ValidateConfig(config)`：启动时强校验；非法值 fail-fast。 -- `BuildWebHost(config)`：创建 Minimal API 应用并绑定 Kestrel。 +- `CreateDefaultConfig()`：内置安全默认值（RuntimeConfig.cs:51）。 +- `LoadTomlConfig(path)`：读取可选 TOML 配置并覆盖默认值；支持 `--config ` 参数或 `TEXTCASCADE_CONFIG` 环境变量指定，回退顺序为 `--config` → `TEXTCASCADE_CONFIG` → 当前目录 `textcascade.toml`。 +- `ApplyEnvironmentOverrides(config)`：环境变量覆盖，实际清单为 `TEXTCASCADE_BIND`、`TEXTCASCADE_PORT`、`TEXTCASCADE_CERTIFICATE_PATH`、`TEXTCASCADE_USERS_FILE`、`TEXTCASCADE_STATE_FILE`，以及 `token_secret_env` 所指名的 token secret 变量。 +- `ValidateConfig(config)`：启动时强校验；非法值 fail-fast。仅服务端启动路径调用，CLI 故意不调用（CLI 无法要求 token secret 存在）。 +- 服务端入口 `ServerHost.RunServer(args)` 加载证书后经 `ServerHost.CreateApp(args, config, users, stateStore, hasher, clock, certificate)` 构建 WebApplication。 默认配置文件示例： @@ -96,6 +98,7 @@ clip_tokens_per_second = 2 [files] users_file = "users.json" +state_file = "textcascade.state.json" ``` 规则： @@ -104,12 +107,13 @@ users_file = "users.json" - `token_secret_env` 指向环境变量名；token secret 不写入 TOML。 - token secret 必须由环境变量提供，长度至少 32 字节；缺失或过短时启动失败。 - TLS 始终启用；`certificate_path` 必须指向服务端可用证书。 -- 证书仅支持无密码格式：`.pem` / `.crt` 必须是包含叶证书与未加密私钥的 PEM bundle，`.pfx` 必须可无密码加载；带密码证书不支持，遇到需要密码的 PFX 时启动失败。 -- TOML 使用宽松解析：必须以 UTF-8 读取；未知键忽略并输出 warning；重复键采用后值并输出 warning；结构或类型非法仍 fail-fast。 +- 证书支持无密码格式：`.pem` / `.crt` 必须是包含叶证书与未加密私钥的 PEM bundle（允许同名 `.key` 边车文件承载私钥）；`.pfx` / `.p12` 必须可无密码加载。遇到需要密码的 PFX 时启动失败。 +- TOML 使用宽松解析：必须以 UTF-8 读取（非 UTF-8 字节 fail-fast）；未知键忽略并输出 warning；结构或类型非法 fail-fast。**重复键视为解析错误直接启动失败**（Tomlyn 语义，与早期"取后值并告警"的设计不同）。 - `max_frame_bytes` 必须大于 `max_text_bytes`，差额留给 JSON 协议头。 - 所有容量与时间配置必须大于 0，心跳超时必须大于心跳间隔。 +- 校验顺序：Load → EnvironmentOverrides → ValidateConfig。 -### 3.2 用户存储 +### 3.2 用户存储与热加载 文件：`users.json` @@ -132,26 +136,29 @@ users_file = "users.json" - `LoadUsers(path)`：启动时全量读取。 - `ValidateUsers(users)`：校验 `nextTokenVersion` 必填且大于所有用户 `tokenVersion`；校验用户名唯一、哈希格式、正数 `long` tokenVersion 与 disabled 字段。 - `BuildUserLookup(users)`：构造只读用户查找表。 +- `SaveUsers(path, users)`：先 Validate 再临时文件原子替换（Windows `File.Replace`，POSIX rename），刷盘后才成功返回。 + +热加载（v0.3.0 起）： + +- `UserFileWatcher` 以 `FileSystemWatcher` 监听 users.json 所在目录的 Changed/Created/Deleted/Renamed 事件，250ms 防抖；另有每 30 秒周期轮询兜底，即使未收到事件也无条件尝试重载。 +- 重载失败按 50ms 退避重试 3 次；全部失败保留旧查找表并输出 warning。 +- 成功后经 `SyncServer.ReplaceUserLookup` 原子替换（Volatile.Write）。登录与新 WebSocket 升级立即反映文件变更；已建立的连接不受影响（持旧上下文）。 +- 服务端自身从不写入用户文件；删除或修改条目即刻影响新认证，被删/禁用用户的存量连接依赖其下次重连时被拒。 说明： -- 不热加载用户文件，避免在线连接认证状态与文件状态竞态。 - `tokenVersion` 不是软件版本，而是账号 token 作废计数器。 -- `tokenVersion` 与 `nextTokenVersion` 使用有符号 64 位整数（`long`），只允许正数，创建与递增时溢出即 fail-fast。 +- `tokenVersion` 与 `nextTokenVersion` 使用有符号 64 位整数（`long`），只允许正数，创建与递增时溢出即放弃操作。 - `nextTokenVersion` 是全局水位；新增用户取当前水位作为 `tokenVersion`，随后水位加一。 - `revoke-tokens` 将目标用户 `tokenVersion` 更新为当前水位，随后水位加一，保证未来新建任何账号都不会复用已撤销版本。 -- CLI 写入用户文件前必须先通过 `ValidateUsers(users)`；水位递增溢出或校验失败时放弃替换并保留原文件。 -- CLI 使用 PID 锁文件实现单实例：同一时刻只允许一个 TextCascade CLI 进程运行；检测到仍存活的其他 CLI 进程时，新实例直接失败退出。 -- PID 锁文件覆盖 CLI 生命周期；实现必须识别并回收陈旧 PID、进程已退出但锁文件残留的情况，并在 Windows 与 Linux 上行为一致。 - 支持直接删除用户条目，不保留墓碑；之后重建同名用户时从全局水位取新 `tokenVersion`，因此不会落入旧 token 的版本空档。 -- 删除或修改用户文件后需重启服务生效；重启后已删除用户不存在，其 token 因用户查找失败而失效。 ### 3.3 用户 CLI 入口在同一服务端可执行文件中，不提供 WebUI。 ```bash -TextCascade.Server user add --username alice +TextCascade.Server user add --username alice # 密码交互输入，或 --password-stdin TextCascade.Server user passwd --username alice TextCascade.Server user disable --username alice TextCascade.Server user enable --username alice @@ -159,17 +166,18 @@ TextCascade.Server user delete --username alice TextCascade.Server user revoke-tokens --username alice TextCascade.Server user list TextCascade.Server user hash +TextCascade.Server serve # 启动服务（Program.cs 动词分发） ``` -函数： +所有命令接受 `--config `；CLI 写入 `users.json` 前先持有 PID 单实例锁（锁文件为 users.json 同目录 `users.json.lock`），再使用临时文件加原子替换。PID 锁识别并回收陈旧 PID、进程已退出但锁文件残留的情况，Windows 与 Linux 行为一致；检测到仍存活的其它 CLI 实例时失败退出。服务运行中修改文件的即时生效性见 3.2 热加载。 -- `RunCli(args)`：识别 `user` 子命令。 -- `CommandAddUser()`：生成 Argon2id 哈希，从全局水位分配 `tokenVersion`，并原子重写用户文件。 -- `CommandDeleteUser()`：直接删除用户条目并原子重写文件，不写入墓碑。 -- `CommandHashPassword()`：只输出密码哈希。 -- `CommandListUsers()`：只输出用户名、禁用状态与 tokenVersion，不输出哈希。 +### 3.4 RuntimeStateStore（版本号落盘） -CLI 写入 `users.json` 时先持有 PID 单实例锁，再使用临时文件加原子替换；服务端运行中修改文件不会热生效，需重启。服务端不写入用户文件。 +- 文件：`[files] state_file`，默认 `textcascade.state.json`；可用环境变量 `TEXTCASCADE_STATE_FILE` 覆盖。 +- 格式：`{"entries":[{"username":"alice","version":129}, ...]}`。 +- 写入时机：`PeriodicTimer` 每 5 秒对脏数据原子快照落盘；优雅停机时同步 flush；每次 clip 成功（`SaveVersion`）标记脏位。`SaveVersion` 采用单调 max 合并，防止乱序回退。 +- 启动行为：`GetOrCreateHub` 用 `GetVersion(username)` 作为 hub 初始版本；状态文件结构非法（重复键、空 username、零版本）fail-fast。 +- 该机制使重启后版本号跨重启单调增长，不再从 1 重新计数（完整语义见 6.2）。 ## 4. HTTP API @@ -189,48 +197,35 @@ Content-Type: application/json } ``` -函数： +实现要点： -- `MapLoginEndpoint()`：薄 Endpoint，只处理 HTTP 请求与响应。 -- `AuthService.LoginAsync()`：执行认证、tokenVersion 校验、token 签发。 -- `ParseLoginRequest()`：限制请求体 16KB、JSON 深度 3。 -- `AuthenticateUser()`：Argon2id 常数时间校验。 -- `CreateLoginFailure()`：统一返回 `invalid_credentials`。 -- `CreateRateLimitResult()`：统一返回 `429`。 +- `AuthService.HandleLoginAsync(HttpContext, config, syncServer, logger)`：薄入口，HTTP 处理内联其中。 +- 请求体限制 16KB、JSON 深度 3、拒绝未知字段与重复字段；畸形请求体（缺字段、类型错误、非法 JSON）返回规格内统一形态之外的 `400 invalid_request`。 +- 认证使用常数时间比较；不存在的用户以缓存的 dummy hash 执行同等验证，消除用户名存在性的计时侧信道。 +- 限流命中返回 `429 Too Many Requests`，错误码 `rate_limited`；认证失败、用户不存在、用户禁用统一返回 `401 {"error":"invalid_credentials","message":"Invalid username or password."}`。 成功： ```json { "token": "", - "expiresAtUtc": "2026-09-17T00:00:00Z", + "expiresAtUtc": "2026-09-17T00:00:00.0000000Z", "protocolVersion": 1, "maxTextBytes": 524288, "helloTimeoutSeconds": 5, "heartbeatIntervalSeconds": 30, - "heartbeatTimeoutSeconds": 60 -} -``` - -失败： - -```http -401 Unauthorized -``` - -```json -{ - "error": "invalid_credentials", - "message": "Invalid username or password." + "heartbeatTimeoutSeconds": 60, + "needsRehash": true } ``` 规则： +- `expiresAtUtc` 序列化为包含小数秒的 ISO 往返格式（"O"），非整秒。 +- `needsRehash` 为条件可选布尔：Argon2 参数与当前配置不一致时才出现。 - 客户端通过 TLS 发送原始密码；客户端不做 Argon2id。 - 用户不存在与密码错误返回相同错误，避免枚举用户。 -- Argon2id 参数变化时，登录路径只调用 `NeedsRehash()` 输出结构化 warning，不重写 `users.json`；用户通过 CLI `passwd` 设置新密码时才生成当前参数的哈希。 -- 登录限流命中返回 `429 Too Many Requests`，错误码 `rate_limited`。 +- Argon2 参数变化时，登录路径只输出 NeedsRehash warning 并在响应携带 `needsRehash`，不重写 `users.json`；用户通过 CLI `passwd` 设置新密码时才生成当前参数的哈希。 ### 4.2 Token @@ -258,49 +253,33 @@ Token JSON 规则： - `sub` 是非空用户名；`ver`、`iat`、`exp` 均为有符号 64 位整数范围内的正整数；`exp` 必须大于 `iat`。 - 数字不得以小数、指数或字符串形式表示。 -函数： - -- `CreateTokenPayload(user, now, ttl)`：生成 `sub`、`ver`、`iat`、`exp`。 -- `SignToken(payload, secret)`：HMAC-SHA256。 -- `VerifyToken(compact, secret, now, userLookup)`：验签、验过期、验用户存在、验 tokenVersion。 +核心函数（Auth.cs）：`CreateTokenPayload(user, now, ttl)`、`SignToken(payload, secret)`（HMAC-SHA256）、`TokenService.TryVerifyToken(compact, now, userLookup, out payload)`——验签常数时间、验过期、验用户存在、验 tokenVersion。 规则： - HMAC 比较必须常数时间。 - token 默认 30 天，可由配置调整。 - token 无服务端状态，服务端重启后仍可验证。 -- 用户被禁用、删除或 tokenVersion 变化后，重启服务即可拒绝旧 token；删除后重建同名用户会从全局水位分配更高 tokenVersion。 +- 用户被禁用、删除或 tokenVersion 变化后被拒：热加载场景立即生效于新的登录与升级请求；重启后同样拒绝，删除后重建同名用户会从全局水位分配更高 tokenVersion。 ### 4.3 登录限流 -函数： - -- `TryConsumeLoginLimit(ip, username, now)`：进程内滑动窗口。 -- `ResetUserLoginLimit(username)`：仅在认证成功后清空该用户名窗口。 +核心类 `SlidingWindowLoginLimiter`： -策略： - -- IP 与用户名双维度限流，任一超限即拒绝。 +- IP 与用户名双维度滑动窗口，任一超限即拒绝。 - 默认每 IP 每分钟 10 次，每用户名每分钟 5 次。 -- 用户名维度统计所有登录请求，无论认证成功或失败；认证成功后清空该用户名窗口。 +- 用户名维度统计所有登录请求；认证成功后清空该用户名窗口。 - IP 维度统计所有登录请求；认证成功不清空 IP 窗口。 -- 限流器设置最大 key 数，提供确定内存上限；达到上限时先清理全部过期项，仍满则拒绝新 key 的登录请求并返回 `429 rate_limited`。 -- 未达到上限时只保存窗口内时间戳，过期项在该 key 被访问时惰性清理；已有 key 的请求不创建新条目。 +- 限流器设置最大 key 数；达到上限时先清理全部过期项，仍满则拒绝新 key 的登录请求并返回 `429 rate_limited`。清理发生在每次访问时（RemoveExpired 全表扫描过期时间戳），比逐 key 惰性清理更积极。 - 单实例部署下不做分布式限流。 -- 已知取舍：持有正确密码的攻击者可通过高频成功登录占满目标用户名窗口；v1 接受该风险，以换取更简单的计数与重置规则。 +- 已知取舍：持有正确密码的攻击者可通过高频成功登录占满目标用户名窗口；接受该风险，以换取更简单的计数与重置规则。 ### 4.4 健康检查 ```http -GET /health +GET /health （亦响应 HEAD /health） ``` -函数： - -- `MapHealth()`：进程能响应即返回 `200 OK`。 - -返回： - ```json { "status": "ok" @@ -320,19 +299,12 @@ Sec-WebSocket-Protocol: textcascade.v1 Upgrade: websocket ``` -函数： - -- `AuthenticateUpgradeRequest(httpContext)`：升级前验 token。 -- `SelectSubProtocol(requestProtocols)`：只接受 `textcascade.v1`。 -- `AcceptAuthenticatedSocket(httpContext)`：认证与版本都合法才升级。 - -规则： +实现（Hosting/SyncEndpoint.cs 内联认证）： -- token 放 Authorization header，不进 URL。 -- token 无效、过期、用户禁用或 tokenVersion 不匹配时，不升级 WebSocket，直接返回 `401`。 -- 子协议不匹配返回 `400`。 -- 认证成功后，客户端必须在 `hello_timeout_seconds` 内发送 hello，用于注册设备与上报 snapshot；默认 5 秒。 -- hello 通过验证前，连接不进入广播列表；该超时由服务端统一计时。 +- 升级前验 token；无效、过期、用户禁用或 tokenVersion 不匹配时不升级 WebSocket，直接返回 `401`。 +- 只接受 `textcascade.v1` 子协议（Ordinal 精确匹配），不匹配返回 `400`；非 WebSocket 请求也返回 `400`。 +- 认证成功后连接进入待 hello 状态，必须在 `hello_timeout_seconds`（默认 5 秒，自 ConnectionStateBag 构造时刻起算）内发送合法 hello，否则关闭。 +- hello 通过验证前，连接不在广播列表中，由统一扫描器独立计时 hello 截止。 ### 5.2 Client Hello @@ -351,18 +323,12 @@ Upgrade: websocket } ``` -函数： - -- `ParseHello(frame)`：解析 hello。 -- `ValidateHello(hello)`：校验 clientId、clientName、snapshot 与版本字段。 -- `CreateConnectionContext(socket, user, hello)`：创建不可变连接上下文。 - -字段： +解析与校验（Protocol.cs）： -- `clientId`：稳定设备 ID，长度 1-128。 -- `clientName`：可选，长度 0-128。 -- `lastServerVersion`：客户端见过的最后服务端版本；未知为 0。 -- `snapshot`：可选。仅在进程启动后的全局恢复窗口内用于选举最新值；恢复窗口结束后只执行完整协议校验，校验通过即丢弃，不写入最新值。clip 是唯一文本写入路径。 +- 全部字段必填：`lastServerVersion` 缺失、非整数或负数按 `invalid_message` 拒绝（未知值为语义上的 0 由客户端显式填 0 表达）。接受显式 `"snapshot": null`。 +- `clientId`：UTF-8 字节数 1–128；`clientName`：0–128 字节；`hash` 上限 4096 字节。 +- 时间戳 `localModifiedAtUtc` 仅接受两种精确形式：UTC `"yyyy-MM-ddTHH:mm:ssZ"` 或 ISO 往返格式，偏移必须为零。 +- snapshot 仅在进程启动后的全局恢复窗口内参与选举；窗口结束后完整校验通过即丢弃，clip 是唯一文本写入路径。 ### 5.3 Server Welcome @@ -376,20 +342,16 @@ Upgrade: websocket "encrypted": true, "hash": "...", "fromClientId": "android-a", + "fromClientName": "android", "updatedAtUtc": "2026-08-18T07:59:58Z" } } ``` -函数： - -- `CreateWelcome(latest)`：构造欢迎消息。 -- `SerializeMessage(message)`：System.Text.Json 序列化为 UTF-8。 - 规则： -- 服务端内存无最新值时 `latest` 为 `null`。 -- 恢复窗口内可先等待 snapshot 选举，再发送 welcome。 +- **服务端内存无最新值时，`latest` 键整体省略**（序列化 WhenWritingNull），不会出现字面 `"latest": null`；三端解析器必须把"键缺失"当作"无最新值"。 +- 恢复窗口开启时 welcome 延迟发送：等待窗口收尾完成 snapshot 选举后统一广播。 - 客户端收到相同 hash 或相同版本时可本地去重，不写剪贴板；hash 只用于本地剪贴板去重，服务端新旧值以版本为准。 ### 5.4 发布文本 @@ -406,7 +368,7 @@ Upgrade: websocket } ``` -服务端广播给同用户除发送方连接外的其他在线连接： +服务端广播给同用户除发送方**连接**外的其他在线连接： ```json { @@ -433,22 +395,17 @@ Upgrade: websocket } ``` -函数： +实现：`Protocol.ValidateClipMessage` 单函数按结构→语义→资源顺序早拒绝；`CheckFrameSize` 帧硬限制；`CheckPayloadSize` 文本限额；`SeenIdRing.IsUnchangedDuplicate/TryGetResult/RememberId` 幂等；`TokenBucket.TryAcquire` 用户级令牌桶；`CoreLogic.NextVersion` ulong 自增（溢出抛出触发 RebuildHub）。 -- `ValidateClipMessage(message)`：单函数完整验证，内部按结构、语义、资源顺序早拒绝。 -- `CheckFrameSize(frameLength, config)`：WebSocket 完整帧字节数硬限制。 -- `CheckPayloadSize(payloadUtf8Length, config)`：文本字段独立限额。 -- `UserHub.TryDuplicate(id)`：用户级环形队列去重。 -- `RememberId(id)`：记录最近消息 ID。 -- `TryAcquireClipToken(now)`：用户级令牌桶。 -- `NextVersion(current)`：服务端权威 `ulong` 自增；溢出抛 fatal。 -- `WithVersion(latest, next)`：构造新的不可变 LatestText。 -- `BroadcastAsync(userHub, latest)`：一次序列化、多连接投递。 +幂等规则（v0.2.5 起语义细化）： -规则： +- `id` 已见过 **且 payload/hash/encrypted 与上次完全一致**：不生成新版本、不消耗令牌桶，返回原版本 ACK；重复 ACK 仍进入发送方有界发送队列，队列满时按慢连接取消。 +- `id` 已见过但内容不同：记录 "Replacing reused clip id" warning 后**按全新消息处理**——消耗令牌桶、生成新版本并覆盖最新值。客户端不应复用已确认过的 id。 +- 相同 `clientId` 的其他连接仍收到广播，仅发送方连接被排除。 + +其余规则： - 客户端不携带版本号；版本由服务端按用户处理顺序生成。 -- `id` 重复时不生成新版本，先返回原 ACK，且不消耗用户级令牌桶；重复 ACK 仍必须进入发送方的有界发送队列，队列满时按慢连接取消。 - 空文本、非法 UTF-8、结构缺字段、超帧、超文本、限流超限均拒绝。 - `payload` 对服务端 opaque；`encrypted=true` 时服务端不解析内容。 - 发送队列容量按消息条数计算，默认 16；队列满立即取消连接，不补发 error 或 close frame。 @@ -474,19 +431,11 @@ Upgrade: websocket } ``` -函数： +实现： -- `StartHeartbeatTimer()`：统一扫描所有连接。 -- `SendPing(connection)`：发送 ping。 -- `MarkPongReceived(connection, now)`：更新 lastSeen。 -- `CloseExpiredConnections()`：超时未收到 pong 则取消连接。 - -说明： - -- 心跳使用应用层 JSON 消息，便于服务端记录 pong 时间并在三端保持一致行为。 -- 默认 30 秒发送一次，60 秒未收到 pong 判定死亡。 -- 统一扫描器代替每连接独立 timer，降低空闲调度开销。 -- 统一扫描器固定每 1 秒扫描一次；hello 与心跳超时允许 0-1 秒的额外检测延迟，不提供独立配置项。 +- 统一扫描器 `HeartbeatScannerService` 固定每 1 秒运行一次，集中处理 ping 调度（间隔默认 30 秒）、hello 超时与心跳超时判定（默认 60 秒无 pong 取消连接）；检测延迟 0–1 秒，不提供独立配置。 +- pong 更新 lastSeen 经用户 Channel 由单消费者落账；用户循环被大 clip 占用时 pong 记账可能延迟数秒。 +- 收到没有未决 ping 的主动 pong，回复 `invalid_message` 错误帧但不断开连接（spec 外补充分支）。 ### 5.6 错误 @@ -499,60 +448,57 @@ Upgrade: websocket } ``` -函数： - -- `ParseResult`：成功或错误显式返回。 -- `CreateProtocolError(code, message, referenceId)`：构造错误。 -- `SendProtocolErrorAsync(connection, error)`：发送可继续错误。 -- `EnqueueImmediateClose(connection, reason)`：跳过 error 与 close frame，直接进入统一取消路径；仅用于发送队列满等无法安全写入的场景。 - -错误码： +`referenceId` 为 null 时该键省略。 | code | 含义 | 连接处理 | |---|---|---| | `invalid_message` | JSON 结构或字段非法 | 可继续 | | `text_too_large` | 文本字段超限 | 可继续 | -| `frame_too_large` | 完整帧超限 | 关闭 1009 | +| `frame_too_large` | 完整帧超限（含零长度帧） | 先发 error，关闭 1009 | | `empty_text` | 空文本 | 可继续 | | `rate_limited` | 用户级发送限流 | 可继续 | | `hello_timeout` | 未按时发送 hello | 先发 error，关闭 1008 | | `server_busy` | 发送队列拥塞 | 立即取消；该错误不保证发送 | -错误处理顺序： +补充行为（实现事实）： -- 需要关闭的错误必须先发送对应应用层 error 帧，再执行 WebSocket close；同一连接同类错误只触发一次关闭流程。 +- 需要 close 的错误先发 error 帧、延时约 100ms 后再执行 close；同一连接同类错误只触发一次关闭流程（MarkClosed 守卫）。 +- hello 到达前的任何非法或非 hello 消息：发 `invalid_message` 错误后以 1008 关闭（预 hello 阶段一律不接受业务帧；预 hello 帧超限走 1009）。 +- 零长度帧判为 `frame_too_large` 关闭 1009。 - 慢连接发送队列满时不补发应用层 error，也不写 close frame，直接进入取消路径；`server_busy` 语义对客户端不可靠，客户端应靠重连兜底。 -可预期协议错误走 Result；不可预期异常仍由顶层兜底并进入统一清理。 +可预期协议错误走 Result；不可预期异常由顶层兜底汇入统一清理。 ### 5.7 关闭与清理 -函数： - -- `CancelConnection(connection, reason)`：唯一取消入口，触发 CancellationTokenSource。 -- `FinallyCloseConnection(connection)`：统一关闭 socket、停止任务、从 UserHub 摘除。 -- `RemoveEmptyHub(userRegistry, userHub)`：最后一个连接断开后清理空 hub；全局恢复窗口内不执行空 hub 清理，窗口收尾时仍无连接的 hub 才移除。 +- 各类取消源（心跳超时、慢连接、恢复队列满、协议异常、停机）最终都终结于连接取消令牌触发；`CancelConnection(connection, reason)` 是正常路径入口，部分高频满队列路径直调 `Cts.Cancel()`（见 2.3 第 6 条），清理一致性由读循环 finally 保证。 +- 关闭 socket：普通场景 graceful close；`EnqueueImmediateClose` 场景 abort/dispose。 | close code | 含义 | |---:|---| | `1000` | 正常关闭 | | `1001` | 服务端重启或维护 | -| `1008` | 策略关闭，例如 hello 超时 | +| `1008` | 策略关闭，例如 hello 超时、预 hello 非法消息 | | `1009` | 帧过大 | -hello 超时先发送 `hello_timeout` error，再以 `1008` close；发送队列满则不补发 error，直接取消。`1013` 与 `4408` 不是本协议 close code，客户端不得依赖。 +`1013` 与 `4408` 不是本协议 close code，客户端不得依赖。 -心跳超时、慢连接、客户端断开、协议异常都必须汇入同一 CTS 取消路径，避免重复清理和资源泄漏。 +hub 清理： + +- 最后一个连接断开后 hub **不立即**从 registry 移除；统一扫描器在 hub 空闲满 **10 分钟**后回收空 hub（`LastActivityAt` 判定），窗口收尾清扫期间 `allowDuringRecovery=true` 即时移除仍空的 hub。 +- 已知问题：被回收 hub 的用户循环任务因 Channel 未 Complete 而遗留挂起（见差距台账）。 +- 客户端主动发 Close 帧时，读循环回 1000 后退出读循环但不立即取消连接 CTS；资源最迟在心跳超时扫描（≤~90 秒）回收，期间连接仍在广播列表中会被继续投递直到写出失败。 ## 6. 最新值与恢复 ### 6.1 正常运行 -每个用户只保存一个 `LatestText`： +每个用户保存一个 `LatestText`： - `payload` - `version` - `hash` +- `encrypted` - `fromClientId` - `fromClientName` - `updatedAtUtc` @@ -563,49 +509,37 @@ hello 超时先发送 `hello_timeout` error，再以 `1008` close；发送队列 2. JSON 解析与 `ValidateClipMessage`。 3. 投递到用户 Channel。 4. 用户单消费者执行幂等检查与令牌桶。 -5. `NextVersion` 生成新版本。 -6. 不可变替换最新值。 +5. `NextVersion` 在当前版本基础上自增。 +6. 不可变替换最新值并 `SaveVersion` 标记脏位。 7. 广播给除发送者外的连接，并向发送者返回 ACK。 这是最新值语义，不是可靠队列语义。离线设备不补历史，重连后只拿当前最新值。 ### 6.2 服务端重启恢复 -恢复窗口从服务端进程启动时间起算，结束时间为 `processStartTime + snapshot_window_seconds`。该窗口对全部用户统一生效，不按 UserHub 创建时间或首个 hello 到达时间重新计算。 - -函数： - -- `CollectSnapshotsAsync(userHub, window)`：收集 3 秒恢复窗口内的 snapshot。 -- `SelectSnapshotWinner(candidates)`：按确定性规则选举。 -- `RestoreLatestText(userHub, winner)`：恢复最新值与版本基准。 - -选举规则： - -1. `lastServerVersion=0` 的 snapshot 不参与选举；只过滤出正版本候选。 -2. 若没有正版本候选，恢复结果为空，不下发最新值。 -3. 在正版本候选中优先选择 `lastServerVersion` 最大者。 -4. 若版本相同，选择 `localModifiedAtUtc` 最新者。 -5. 若仍相同，选择 `clientId` 字典序更大者，保证结果确定。 +恢复窗口从服务端进程构建时间起算（`SyncServer.ProcessStartTime`），结束时间为 `processStartTime + snapshot_window_seconds`。该窗口对全部用户统一生效，不按 UserHub 创建时间或首个 hello 到达时间重新计算。1 秒扫描器在窗口结束后统一对所有 hub 执行收尾。 -恢复规则： +选举与恢复规则： -- winner 的 `LatestText.version` 使用其正版本 `lastServerVersion`，不额外加一；恢复版本不额外设置上限，`NextVersion` 溢出 fatal 保留为理论兜底。 -- 无正版本候选时恢复为空；下一条服务端 clip 版本为 1。 -- 恢复窗口内只收集 snapshot；合法 clip 不参与选举，进入独立有界恢复队列。 -- 每用户 snapshot 预算只统计候选 `snapshot.payload` 的 UTF-8 字节数总和；上限为 `snapshot_total_bytes`，达到上限后拒绝新的 snapshot 并保持已有候选不变。元数据开销不占用该预算，由在线连接数量约束。 -- 恢复队列容量为 `recovery_clip_queue_capacity`；队列满时关闭相应连接，避免内存无界增长；连接断开则丢弃其已排队 clip。 -- 恢复窗口结束后，先根据 snapshot 选举 winner 并恢复最新值，再按到达顺序串行处理恢复队列中的 clip。 -- 窗口结束后广播 welcome 或恢复后的最新值，客户端按 hash 与版本去重。 -- 错过窗口的慢设备之后仍可发送 clip；该 clip 按到达顺序获得新版本并覆盖当前最新值。最后写入者胜，服务端不尝试识别或拒绝“逻辑上更旧”的 clip。 +1. 版本基准来自 RuntimeStateStore：hub 初始版本 = 状态文件中该用户的持久化版本（无记录则为 0）。 +2. `lastServerVersion=0` 的 snapshot 不参与选举；只过滤正版本候选。 +3. 若没有正版本候选，恢复结果为空，welcome 不带最新值。 +4. 候选中优先选择 `lastServerVersion` 最大者；并列取 `localModifiedAtUtc` 最新者；再平局取 `clientId` 字典序更大者。 +5. **守卫**：winner 版本小于等于 hub 当前（持久化）版本，且两者相等时已有最新值的情形除外，否则放弃恢复（welcome 不下发）——持久化水位高于一切客户端认知时保持现状，下一条 clip 从持久化版本+1 继续。 +6. 恢复成功时 `LatestText.version` 直接使用 winner 的 `lastServerVersion`（不加一），并回写 SaveVersion。 +7. 恢复窗口内只收集 snapshot；合法 clip 进入独立有界恢复队列。 +8. 每用户 snapshot 预算只统计候选 `snapshot.payload` 的 UTF-8 字节数总和，上限 `snapshot_total_bytes`，达到上限后拒绝新候选、保留既有候选；元数据开销不计入预算。 +9. 恢复队列容量 `recovery_clip_queue_capacity`；满时断开提交者对应连接；连接断开其已排队 clip 丢弃。 +10. 窗口收尾顺序：选举 winner → 恢复最新值 → 按到达顺序串行处理恢复队列 → 广播 welcome。 -服务端重启后的完整链路： +重启后的完整链路： 1. 服务端停机前广播 `bye` 并以 `1001` 关闭连接。 2. 客户端识别服务端维护，使用无状态 token 直接重试 WebSocket。 -3. 服务端重启后 token secret 与 tokenVersion 未变，token 仍可验证。 +3. 服务端重启后 token secret 与 tokenVersion 未变，token 仍可验证；版本号跨重启单调。 4. 客户端 hello 上报 snapshot。 5. 3 秒窗口选举 winner。 -6. 服务端恢复最新值并继续同步。 +6. 服务端恢复或保持最新值并继续同步。 ### 6.3 慢连接 @@ -613,25 +547,19 @@ hello 超时先发送 `hello_timeout` error，再以 `1008` close；发送队列 - 默认容量 16 条消息。 - `TryWrite` 失败即判定慢连接。 -- 立即调用 `CancelConnection`，不等待 drain，不补发应用层 error，也不写 close frame。 -- 发送循环观测取消后直接退出；`OperationCanceledException` 与非取消异常都汇入统一清理路径。 -- 该场景使用 abort/dispose 释放底层 socket，不执行 graceful WebSocket close 握手。 +- 立即取消该连接，不等待 drain，不补发应用层 error，也不写 close frame。 +- 发送循环观测取消后退出；`OperationCanceledException` 与非取消异常都汇入统一清理路径。 +- 满队列广播路径仅取消令牌、socket 随 finally dispose 回收；`EnqueueImmediateClose` 路径显式 `Socket.Abort()`，均不做 graceful 握手。 - 客户端重连后通过 welcome 拿最新值，不补发中间消息。 慢连接不能阻塞用户单消费者，也不能影响同用户其他连接。 ## 7. 优雅停机 -函数： - -- `BroadcastByeAsync(reason)`：向所有连接发送 `bye`。 -- `ShutdownAsync(CancellationToken)`：关闭连接、停止任务、等待短暂收尾。 - 流程： -1. 收到 SIGTERM、Ctrl+C 或服务停止请求。 -2. 停止接受新连接。 -3. 广播： +1. 收到 SIGTERM、Ctrl+C 或服务停止请求（Host 反向停止次序下 Kestrel 先停止接受新连接）。 +2. 向 registry 中所有已 hello 连接广播： ```json { @@ -640,137 +568,82 @@ hello 超时先发送 `hello_timeout` error，再以 `1008` close；发送队列 } ``` -4. 以 close code `1001` 关闭所有连接。 -5. 等待最多 2 秒，让 close frame 尽量发出。 -6. 取消所有连接 CTS。 -7. 清理 UserHub 与后台任务。 -8. 进程退出，由系统服务管理器重启。 +3. 以 close code `1001` 逐一关闭上述连接；bye 经各自有界发送队列投递，队列已满的连接会被静默跳过（既无 bye 也无 1001）。 +4. 等待最多 2 秒让 close frame 尽量发出。 +5. 取消所有连接 CTS；随后 RuntimeStateStore 同步 flush 状态文件。 +6. 清理 UserHub 与后台任务，进程退出交由系统服务管理器重启。 + +已知边界：处于预 hello 状态（pendingHellos）的连接不在 bye/1001 广播范围内，进程退出时随 socket 直接断开。 ## 8. 日志与安全 ### 8.1 结构化日志 -函数： - -- `LogSecurityEvent()`：记录登录、认证失败、限流与禁用用户事件。 -- `RedactSensitive(value)`：统一脱敏。 +实现（SecurityLogging.cs）：`LogSecurityEvent(this ILogger, eventName, params (string,object)[])` 输出扁平化的结构化事件；`RedactSensitive(value)` 用于脱敏。 规则： -- 使用 `ILogger` 结构化字段。 -- 密码绝不记录。 -- token 只可记录短前缀，默认不记录。 -- clip payload 与 hash 不记录；clip 事件只记 version、字节数、encrypted、来源设备。 -- Authorization header 不进入访问日志。 +- 密码绝不记录；token 不记录（代码中的 TokenPrefix 工具目前未被生产路径调用）。 +- clip payload 与 hash 不记录；clip 事件只记 version、clipId、字节数、encrypted、来源设备。 +- Authorization header 不进入任何日志。 +- 登录相关事件不区分"密码错误"与"用户不存在"，也不出现"disabled"字样——全部折叠为 `reason=invalid_credentials`，防止枚举（测试锁定此行为）。 -关键事件： +关键事件（实际字段）： | 事件 | 字段 | |---|---| -| login | username, ip, success, reason | +| login | username, ip, success[, reason]（失败必带 reason） | | connect | username, clientId, connectionId | -| disconnect | username, clientId, reason, durationMs | -| clip | username, version, bytes, fromClientId, encrypted | +| disconnect | username, clientId, connectionId, reason | +| clip | username, version, clipId, bytes, fromClientId, encrypted | | reject | username, code, bytes | -| server_stop | reason, activeConnections | + +（历史版本的 `durationMs` 字段与 `server_stop` 事件从未实现/已不存在，不再列为要求。） ### 8.2 传输与输入安全 -- 生产只允许 HTTPS/WSS。 -- TLS 最低 1.2，推荐 1.3。 +- 生产只允许 HTTPS/WSS：`ServerHost.RunServer` 强制先加载证书，Kestrel 仅绑定单一 HTTPS endpoint。TLS 协议版本跟随 OS 默认策略，代码未显式设置 SslProtocols 下限；NetworkIntegration 测试以显式 Tls12/Tls13 客户端握手验证兼容性。 +- 测试路径豁免说明：`ServerHost.CreateApp` 的 certificate 形参允许传 null 构建纯 HTTP 主机，仅测试可达（InternalsVisibleTo 之下），生产入口不可能走到。 - 不启用 CORS。 - 不设置 Cookie，无 CSRF 面。 -- 登录请求体上限 16KB。 -- WebSocket 完整帧与文本字段分别限额。 -- JSON 深度限制为 3。 -- 协议消息只接受契约定义字段；重复字段与未知字段拒绝。若未来新增可选字段，必须提升或明确协议兼容策略。 +- 登录请求体上限 16KB；登录与协议帧 JSON 深度限制均为 3。 +- 协议消息与登录请求只接受契约定义字段；重复字段与未知字段拒绝。若未来新增可选字段，必须提升或明确协议兼容策略。 ## 9. 性能目标 -| 指标 | v1 目标 | -|---|---:| -| 基础进程内存 | < 50 MB | -| 100 个空闲连接内存增量 | < 20 MB | -| 1KB 文本 LAN 广播 p95 | < 30 ms | -| 512KB 文本 LAN 广播 p95 | < 250 ms | -| 空闲 CPU | 接近 0%，心跳扫描除外 | -| 冷启动时间 | < 2 s | -| 服务端重启恢复窗口 | 3 s | +原文的性能指标表（内存、广播 p95、冷启动等）在本轮修订中移除：项目从未建立度量设施（无 Benchmark 项目、无 p95 测量手段），保留未度量数字只会造成虚假承诺。如未来需要性能回归防护，应以独立规格与本仓库的测试设施一起重建。（协议层面保留的设计性质：广播单次 UTF-8 序列化、每连接有界发送队列、空闲路径只有心跳扫描。） + +## 10. 测试计划 -设计依据： +详细到函数层面的规格见 [specs/test-and-contract-spec.md](test-and-contract-spec.md)，本节描述现状与分层。集成测试机制自 v0.3.0 起采用真实 Kestrel 绑定 `127.0.0.1:0` 的 fixture（`ServerHost.CreateApp` 构建 + FastPasswordHasher 注入），不再使用内存 socket 对。 -- 无数据库连接池与定时磁盘 IO。 -- 每用户一个 Channel 单消费者，避免锁与异步持锁。 -- 每次广播只做一次 UTF-8 序列化。 -- 每连接发送队列有界，内存上限可预测。 -- 空闲连接只保留上下文、发送 Channel 与心跳扫描状态。 +### 10.1 纯单元测试（现有覆盖） -## 10. 测试计划 +- `SignToken`/`TryVerifyToken`：往返、过期、tokenVersion 撤销、篡改、未知字段、禁用用户、用户缺失。 +- CLI PID 单实例锁：活跃互斥、陈旧 PID 回收、存活进程不回收、锁路径校验。 +- `SlidingWindowLoginLimiter`：双维度、跨 IP、成功仅清用户窗口、max keys、过期清理。 +- `TryAcquireClipToken`（TokenBucket refill）、`CheckFrameSize`/`CheckPayloadSize`、SeenIdRing 去重与淘汰、`NextVersion` 含 ulong.MaxValue 抛出、`SelectSnapshotWinner` 三规则。 +- 待补齐缺口（Argon2 三函数、token 数字全形态、CLI 水位/溢出、WithVersion、重复 id 行为级断言）已定义于 test-and-contract-spec §3，落地前不构成本节的承诺范围。 + +### 10.2 CI 集成测试：真实 Kestrel loopback + +现有 `WebSocketIntegrationTests` 覆盖：登录与握手往返（含 welcome）、无效 token 不升级、两客户端广播与发送方 ACK、重复 id 同版本 ACK、断连后按最高 lastServerVersion 快照恢复（结合版本持久化）、突兀断开被记录且服务存活（日志不含密码/secret）。 -### 10.1 纯单元测试 - -重点函数： - -- `HashPassword`、`VerifyPassword`、`NeedsRehash` -- `SignToken`、`VerifyToken`、tokenVersion 撤销 -- Token 重复字段、未知字段、非法数字与非法范围 -- 全局 `nextTokenVersion` 水位递增、删除后重建同名用户、溢出 fail-fast -- CLI PID 单实例锁：活跃进程互斥、陈旧 PID 与锁文件残留处理 -- `TryConsumeLoginLimit` -- 登录限流成功重置用户名窗口但不重置 IP 窗口 -- 登录限流最大 key 数：先清理过期项，仍满时拒绝新 key -- `TryAcquireClipToken` -- `ValidateClipMessage` -- `CheckFrameSize`、`CheckPayloadSize` -- `UserHub.TryDuplicate`、`RememberId` -- 重复 `id` 不消耗令牌桶，重复 ACK 仍受有界发送队列约束 -- `NextVersion`、`WithVersion` -- `SelectSnapshotWinner`，包括 `lastServerVersion=0` 不参与、无正版本候选恢复为空、同版本时间与 clientId 平局 - -认证测试注入假哈希器，避免 Argon2id 拖慢常规单元测试。 - -### 10.2 CI 集成测试：内存 WebSocket - -使用 `CreateSocketPair()` 建立内存连接，快速稳定覆盖： - -- 登录成功与失败。 -- 升级前认证失败不建立 WebSocket。 -- 子协议不匹配拒绝。 -- hello 超时。 -- 登录后仅记录 `NeedsRehash` warning，不重写用户文件。 -- 部分设备重连时的 snapshot 选举，包括 `lastServerVersion=0` 被忽略与无正版本候选恢复为空。 -- 恢复窗口从进程启动时间全局起算；窗口内 clip 排队、队列容量、payload 字节预算与窗口后处理顺序。 -- 恢复窗口结束后 snapshot 仅校验后丢弃。 -- 两客户端同用户广播与发送方 ACK；相同 `clientId` 的其他连接仍收到广播，仅发送方连接被排除。 -- 不同用户隔离。 -- 幂等 ID。 -- 慢连接队列满立即取消且不影响同用户其他接收方。 -- 停机 bye 与 1001。 -- 重启 snapshot 选举。 - -### 10.3 本地网络测试：真实 localhost TCP - -标记：`Category=NetworkIntegration`，CI 默认跳过。 +计划中的补齐用例（子协议 400、hello 超时、NeedsRehash 不重写、同 clientId 排除规则、用户隔离、慢连接取消、bye/1001 等）同样收录于 test-and-contract-spec，实施后回填。 + +### 10.3 本地网络测试：Category=NetworkIntegration ```bash -dotnet test --filter Category=NetworkIntegration +dotnet test TextCascade.Server.slnx --filter Category=NetworkIntegration ``` -使用 `StartTestServer()` 启动完整 Kestrel，覆盖： +CI 默认排除（ci.yml 过滤参数见 test-and-contract-spec 实施清单）。覆盖：自签证书 TLS/WSS、显式 Tls12/Tls13 握手、随机端口绑定、真实帧分片、超限帧 1009、重启两次 CreateApp 后 token 直连与快照恢复、停机 bye/1001、HTTPS 登录全链路。用例级明细见 test-and-contract-spec §1。 -- TLS 证书与 WSS。 -- HTTP 升级。 -- 随机端口绑定。 -- 真实帧分片。 -- 登录、建连、发送文本、另一客户端接收。 -- 服务端重启后 token 直连重连与 snapshot 恢复。 +### 10.4 契约测试 -### 10.4 契约测试与压测 +样本文件组织于 Tests 项目 `ContractSamples/`（valid/invalid 分类、非法数字与非法 UTF-8 全矩阵、深度 4、重复/未知字段），由 Theory 驱动断言 `ParseClientMessage` 结果与序列化字节不变式；样本文件同时作为三端实现的公共对拍集合。明细见 test-and-contract-spec §2。 -- 服务端维护典型 JSON 样本，约束三端协议字段与行为。 -- 契约样本必须覆盖 JSON 深度 3、重复字段、未知字段、非法数字与非法 UTF-8。 -- 独立 `TextCascade.Server.Benchmark` 项目执行压测，不进入生产产物。 -- 压测场景包括空闲连接、1000 并发连接、小文本广播、512KB 文本广播与慢消费者。 +（独立 Benchmark 项目与压测场景已从规格移除，理由见第 9 节。） ## 11. 客户端适配要求 @@ -783,9 +656,10 @@ dotnet test --filter Category=NetworkIntegration 5. clip 发送、ACK、接收。 6. 保存服务端 version，重连时上报 `lastServerVersion`。 7. 收到相同 hash 或相同版本时不写剪贴板；收到更晚到达的旧 clip 仍可能覆盖本地文本。 -8. `1001` 后按服务端维护重连；token 未过期时优先直接重连。 -9. `401`、token 过期或 tokenVersion 失效时重新 HTTP 登录。 -10. 重连退避建议：1s、2s、5s、10s、30s、60s，之后固定 60s；收到 1001 时初期退避更温和。 +8. 解析 welcome 时将 `latest` 键缺失视为"无最新值"。 +9. `1001` 后按服务端维护重连；token 未过期时优先直接重连。 +10. `401`、token 过期或 tokenVersion 失效时重新 HTTP 登录。 +11. 重连退避建议：1s、2s、5s、10s、30s、60s，之后固定 60s；收到 1001 时初期退避更温和。 保留客户端原有能力： @@ -797,61 +671,52 @@ dotnet test --filter Category=NetworkIntegration ## 12. 实施里程碑 -### M1：协议骨架 +### M1：协议骨架 —— 已达成 -- 配置加载与 fail-fast 校验。 -- `users.json` 与 CLI。 -- 登录端点。 -- HMAC token。 -- WebSocket 升级认证与子协议协商。 -- hello/welcome。 -- 文本广播与 ACK。 -- 纯单元测试与内存集成测试。 +配置加载与校验、users.json 与 CLI、登录端点、HMAC token、WebSocket 升级认证与子协议协商、hello/welcome、文本广播与 ACK、单元与集成测试基座。 -### M2：可靠性 +### M2：可靠性 —— 已达成 -- UserHub Channel 单消费者。 -- 有界发送队列与立即取消策略。 -- 幂等 ID。 -- 服务端版本号与不可变最新值。 -- 应用层心跳。 -- 统一 CTS 清理。 +用户 Channel 单消费者、有界发送队列与立即取消、幂等 id、服务端版本号与不可变最新值、应用层心跳、统一取消清理。 -### M3：恢复与真实网络 +### M3：恢复与真实网络 —— 大体达成，余项转入 §10.3 -- 优雅停机 bye/1001。 -- 3 秒 snapshot 恢复窗口。 -- snapshot 总字节数与恢复 clip 队列容量。 -- token 过期与 tokenVersion 撤销测试。 -- 本地真实 TCP/TLS 集成测试。 -- 重启后多端收敛测试。 +优雅停机 bye/1001、快照恢复窗口与预算/队列约束、tokenVersion 撤销、版本号持久化；真实 TCP/TLS 集成测试与多端收敛测试补齐中（specs/test-and-contract-spec §1）。 -### M4：生产化 +### M4：生产化 —— 大体达成，两项移交差距台账 -- Kestrel TLS。 -- 结构化日志与脱敏。 -- 登录与消息限流。 -- 框架依赖单文件发布。 -- 独立 benchmark。 -- 部署文档与 Runtime 版本校验。 +Kestrel TLS、结构化日志与脱敏、登录与消息限流、框架依赖单文件发布、systemd/发布管线均已落地；Benchmark 项目与性能指标未实施且已从规格移除（见第 9 节）。 ## 13. 版本与发布 -- 产品版本采用 SemVer 2.0.0，从 `0.1.0` 开始，写入 `TextCascade.Server.csproj` 的 `Version`。 +- 产品版本采用 SemVer 2.0.0，从 `0.1.0` 开始演进，以 `TextCascade.Server.csproj` 的 `Version` 为准（当前 0.3.5）。 - `protocolVersion` 只表示线协议版本，当前为 `1`，与产品版本独立演进。 - 目标框架为 `net10.0`；目标机必须预装兼容的 .NET 10 Runtime。 -- 发布命令：`dotnet publish TextCascade.Server.csproj -c Release -p:PublishSingleFile=true`。 +- 发布命令：`dotnet publish TextCascade.Server.csproj -c Release -p:PublishSingleFile=true`（win-x64/linux-x64 框架依赖单文件）。 - 本地编译命令：`dotnet build TextCascade.Server.csproj -c Release`。 -## 14. 已关闭问题 +## 14. 决策台账（更新于 2026-08-27） | 问题 | 结论 | |---|---| | 最大文本默认值 | 512KB | -| 最新值磁盘持久化 | 不做，保持极简 | -| 用户配置热加载 | 不做，重启生效 | +| 最新值文本本体磁盘持久化 | 不做，文本只在内存 | +| 版本号持久化 | 做：v0.2.5 起 RuntimeStateStore 落盘 textcascade.state.json，版本跨重启单调 | +| 用户配置热加载 | 做：v0.3.0 起 UserFileWatcher 监听 + 轮询兜底，新认证即刻生效（取代早期"重启生效"决策） | | token 生命周期 | 长期 token + tokenVersion 撤销 | | 删除后重建用户 | 全局 nextTokenVersion 水位 | -| metrics | v1 不启用 endpoint | +| metrics | 不启用 endpoint | | 协议包 | 三端手写，服务端契约测试约束 | | 关闭码 | 应用层 error + 标准 close code；不发送 1013/4408 | + +## 15. 实现差距台账（知悉现状，不构成承诺） + +以下为实现偏离或已知缺陷，如实记录、供排期参考： + +1. 优雅停机不覆盖预 hello 连接（见 §7 已知边界）。 +2. 空 hub 10 分钟回收时，其用户循环任务因 Channel 未 Complete 而永久挂起，每个被回收 hub 遗留一个 parked task。 +3. 部分队列满路径绕过 `CancelConnection` 入口直调 `Cts.Cancel()`，早退清理一致性依赖 finally（见 2.3）。 +4. TLS 协议下限未显式固定，依赖 OS 默认（§8.2）。 +5. `ServerHost.CreateApp(certificate:null)` 的明文测试缝隙（仅 InternalsVisibleTo 可达）。 +6. ApplyClip 中 duplicateId 且 Latest 为 null 的兜底分支不可达（死分支）。 +7. §10 中标注"待补齐/补齐中"的测试项以 specs/test-and-contract-spec.md 落地为准。 diff --git a/specs/code-review.md b/specs/code-review.md new file mode 100644 index 0000000..a12770f --- /dev/null +++ b/specs/code-review.md @@ -0,0 +1,81 @@ +# TextCascade.Server 代码审查报告 (Code Review) + +## 1. 总体架构评价 + +TextCascade.Server 是一个定位非常清晰的轻量级剪贴板/最新文本同步服务端。整体代码风格紧凑、克制，没有引入臃肿的企业级分层（无 EF Core、无外部数据库、无庞杂的中间件），非常契合 "Ponytail" / "Do Less" 的实用主义设计哲学。 + +### 核心亮点 +1. **并发与隔离模型清晰**：每个在线用户一个 `UserHub`，内部采用单消费者 Channel（`RunUserLoopAsync`），天然避免了多连接并发修改版本号和最新文本时的大颗粒度锁争用。 +2. **背压与慢连接处理果断**：每个连接分配有界发送队列（默认 16 条），队列满时立即判定为慢连接并直接 Cancel/Abort，绝不等待阻塞，也不拖垮同用户的其他客户端。 +3. **无状态 Token + 版本作废机制**：采用基于 HMAC-SHA256 的紧凑型 Token，服务端重启无需保存 Session；通过 `users.json` 中的 `tokenVersion` 与全局水位 `nextTokenVersion` 实现高效的单用户/全量 Token 撤销。 +4. **内存与资源开销极低**：广播时一次序列化，多连接复用同一份 UTF-8 Byte 数组；统一心跳扫描器替代每连接独立 Timer。 +5. **单文件与开箱即用**：集成了 CLI 用户管理、TLS 证书加载、本地状态落盘与单实例锁，运维负担极小。 + +--- + +## 2. 关键发现与问题清单 + +### P1 - 性能与资源瓶颈 (Performance & Allocation) + +#### 1. 广播路径上的连接列表数组分配 +- **位置**：`TextCascade.Server/Hub/UserHub.cs` +- **现象**：`Connections` 属性每次被读取时，都会执行 `lock (connectionsGate) { return connections.ToArray(); }`。在 `ApplyClip` 广播、`BroadcastWelcome` 等高频路径中，每条 Clip 都会分配一次连接数组。 +- **人话建议**： + 在连接数变动较少、消息广播频繁的场景下，可以采用 **Copy-On-Write（写时复制）** 模式维护内部连接数组（即维护一个不可变的 `ImmutableArray` 或 `ConnectionContext[]`，只有在 `AddConnection` / `RemoveConnection` 时才重新生成新数组）。这样广播时读取连接列表完全**零锁、零分配**。 + +#### 2. Token 校验中的多重内存分配与 Dictionary 构造 +- **位置**：`TextCascade.Server/Auth.cs` (`TokenService.TryVerifyTokenInternal`) +- **现象**： + 1. `compactToken.Split('.')` 每次都分配一个 `string[]`。 + 2. `var actualPayload = payloadRent is null ? payloadBytes[..payloadLength].ToArray() : payloadRent[..payloadLength];` 在栈空间足够时依然调用了 `.ToArray()`。 + 3. `var properties = root.EnumerateObject().ToDictionary(...)` 每次校验 Token 都会分配一个 `Dictionary` 和 4 个字符串 Key。 +- **人话建议**： + HMAC 校验和 `JsonDocument.Parse` 都可以直接接受 `ReadOnlySpan`。解析 Payload 时直接使用 `root.TryGetProperty("sub", out ...)`，不要转成 `ToDictionary`，也不要多余 `.ToArray()`。作为 WebSocket 握手升级的高频入口，优化后可实现接近零分配。 + +#### 3. 登录接口中的双重 JSON 解析与字符串转换 +- **位置**：`TextCascade.Server/AuthService.cs` (`ParseLoginRequest`) +- **现象**：从 Request Body 读取后，先 `new UTF8Encoding().GetString(body.ToArray())`，再用 `JsonDocument.Parse` 校验属性，最后又调用了一次 `JsonSerializer.Deserialize(text)`。 +- **人话建议**：既然第一步已经用 `JsonDocument` 验证了字段结构，直接从 `document.RootElement` 取出 `username` 和 `password` 即可，彻底省去第二次 `JsonSerializer.Deserialize` 以及中间的多余字符串拷贝。 + +--- + +### P2 - 并发与稳健性风险 (Concurrency & Edge Cases) + +#### 1. Argon2id 同步计算可能阻塞 Kestrel 线程 +- **位置**：`TextCascade.Server/AuthService.cs` (`HandleLoginAsync`) +- **现象**：`syncServer.Hasher.Verify(request.Password, passwordHash)` 是纯 CPU/内存密集型运算（约 19MB 内存、2 轮迭代）。当前是在 Kestrel HTTP 请求处理的同一线程上下文中同步执行。 +- **人话建议**：如果短时间内有多个并发登录请求，可能会占满线程池调度。建议使用 `await Task.Run(() => syncServer.Hasher.Verify(...))` 将密码哈希运算明确卸载到后台工作线程，避免堵塞 HTTP 管道。 + +#### 2. 滑动窗口限流器在 Key 满时的全局遍历瓶颈 +- **位置**：`TextCascade.Server/Core.cs` (`SlidingWindowLoginLimiter.TryConsume`) +- **现象**：当外部遇到 IP/用户名爆破攻击导致 `windows.Count >= maxKeys` (10000) 时，每次新请求都会在 `lock (gate)` 内遍历整个字典清理过期项（`RemoveExpired`）。在大流量恶意扫描下，会导致所有正常用户的登录请求在同一个锁上排队。 +- **人话建议**： + 可以采用分段锁，或者在后台使用 `PeriodicTimer` 每隔几秒做一次批量清理；当字典达到 `maxKeys` 时，不再对每个请求都执行全量 O(N) 遍历，而是快速丢弃或采用简单的 LRU/分桶计数。 + +#### 3. 规范与现实的偏离 (Spec vs Code Drift) +- **位置**：`docs/server-spec.md` vs `TextCascade.Server/Hosting/UserFileWatcher.cs` +- **现象**： + - `server-spec.md` §3.2 和 §14 中明确写道：“*不热加载用户文件，避免在线连接认证状态与文件状态竞态*”、“*修改用户文件后需重启服务生效*”。 + - 但代码中实现了 `UserFileWatcher` 并在 `ServerHost.cs` 中启用了文件变动监听和热加载。 +- **人话建议**：代码中的热加载实现实际上做得很干净（使用了防抖、重试以及 `Volatile.Write` 原子替换），这是个很好的功能改进。建议同步更新 `docs/server-spec.md` 文档，将已废弃的“必须重启生效”描述更新为“支持 users.json 监听与平滑热重载”。 + +--- + +### P3 - 代码洁癖与简化空间 (Code Cleanliness & Simplification) + +1. **废弃的 `TryDuplicate` 方法**： + `Core.cs` 中的 `SeenIdRing.TryDuplicate(string id)` 目前在实际业务流程中未被调用（业务中使用的是 `IsUnchangedDuplicate` + `TryGetResult` + `RememberId`），仅留存在单元测试中。可评估是否需要保留或标记内部测试专用。 +2. **Source Generator 覆盖度**： + 协议消息（`WelcomeMessage`, `ClipMessage` 等）已经使用了 .NET 10 的 `JsonSourceGenerationOptions` 强类型上下文，非常棒！但 `RuntimeStateStore` 和 `UsersFile` 仍在使用传统的动态反射序列化。建议统一接入 Source Generator，进一步增强 Native AOT 兼容性与运行速度。 + +--- + +## 3. 改进建议总结 + +| 序号 | 改进项 | 收益 | 实施复杂度 | +|---|---|---|---| +| 1 | `UserHub.Connections` 改用 Copy-On-Write 数组 | 广播路径彻底消除锁与数组分配 | 低 | +| 2 | `AuthService` 与 `TokenService` 内存与解析去重 | 降低登录与 WS 握手时的 GC 压力 | 低 | +| 3 | Argon2 校验使用 `Task.Run` 卸载 | 保护 Kestrel 请求处理管线抗突发并发 | 极低 | +| 4 | 限流器清理机制从同步全量遍历优化为轻量清理 | 提升恶意请求轰炸下的吞吐与稳定性 | 中 | +| 5 | 同步更新 `server-spec.md` 关于热重载的说明 | 保持文档与生产代码一致 | 极低 | diff --git a/specs/spec-decisions.md b/specs/spec-decisions.md new file mode 100644 index 0000000..b00caae --- /dev/null +++ b/specs/spec-decisions.md @@ -0,0 +1,218 @@ +# TextCascade 规格修订决策记录 + +日期：2026-08-27 +任务：三类差异三条路径（详见 docs/server-spec.md 审计与 git 溯源结论） +- 路径 A：更新引入的漂移 → 直接修订 `docs/server-spec.md` 对齐 v0.3.5，无决策点。 +- 路径 B：NetworkIntegration 测试、契约测试组织、单元测试缺口 → 新建 `specs/test-and-contract-spec.md`，以下 10 题决定其内容。 +- 路径 C：其余从未实现项（Benchmark 项目、server_stop 事件、性能目标表）→ 从 docs/server-spec.md 移除。 + +规则：每轮只问一个问题；提问前本题已落盘；收到答案后立刻回写本题"选择"栏。 + +--- + +## Q1 NetworkIntegration 测试宿主 【已选】 + +NetworkIntegration 整类测试（TLS/WSS、HTTP 升级、随机端口绑定、真实帧分片、登录建连收发、重启直连恢复）放在哪里？ + +- **A. 复用现有 WebSocketIntegrationTests 的 fixture**（真实 Kestrel 绑定 127.0.0.1 随机端口 + ClientWebSocket） + - 优：零新增设施；与现有 6 个集成测试共享 helper，维护成本最低；TLS 测试只需给 fixture 加证书参数。 + - 劣：同一项目里普通集成测试与网络测试混在一起，只能靠 Trait 过滤，CI 跳过逻辑与文件物理布局无对应关系。 +- **B. Tests 项目内新建 `NetworkIntegration/` 目录 + 自建 fixture** + - 优：物理隔离清晰；fixture 可针对 TLS/重启场景定制（如持有可重启的 WebApplication 列表）；不影响现有 fixture 稳定性。 + - 劣：需要抽象或复制现有 fixture 的公共逻辑，初始工作量中等；两套 helper 有漂移风险。 +- **C. 独立测试项目 `TextCascade.Server.NetworkTests`** + - 优：隔离最彻底；CI 可以完全按项目粒度跳过（不改 slnx 过滤逻辑的话需加入 slnx 但 CI 单独 dotnet test 主项目即可）；未来可加性能冒烟。 + - 劣：需改 .slnx、新增 csproj、跨项目 InternalsVisibleTo；维护三个项目的成本最高。 + +选择：**B. Tests 项目内新建 `NetworkIntegration/` 目录 + 自建 fixture**（2026-08-27） + +--- + +## Q2 NetworkIntegration 过滤机制 【已选】 + +如何让 `dotnet test` 默认跳过这些测试、按需运行？ + +- **A. `[Trait("Category", "NetworkIntegration")]` + CI/本地用 `--filter Category!=NetworkIntegration`** + - 优：xunit 原生、与 spec 原文 `Category=NetworkIntegration` 完全一致；`dotnet test --filter Category=NetworkIntegration` 即可单独跑。 + - 劣：CI 的 `dotnet test TextCascade.Server.slnx` 必须追加过滤参数，否则会跑到（需同步改 ci.yml）。 +- **B. MTP 风格（Microsoft.Testing.Platform）`--filter-category` 等** + - 优：新一代测试平台原生参数。 + - 劣：当前项目用 VSTest 模式跑（ci.yml 无 MTP 配置），引入 MTP 需改测试 SDK 配置，风险与收益不成比例。 +- **C. xunit Collection + 命名约定（如类名以 Network 开头）+ assembly 级配置** + - 优：Collection 级可统一串行化，适合真实端口/证书场景。 + - 劣：跳过逻辑要靠自定义 xunit trait 发现或 `--filter` 仍然需要；命名约定脆弱。 + +选择：**A. `[Trait("Category", "NetworkIntegration")]` + `--filter` 过滤**（2026-08-27）。备注：实施时需同步给 ci.yml 的 dotnet test 追加 `--filter Category!=NetworkIntegration`；本轮只写入 spec 实施清单，不改代码。 + +--- + +## Q3 测试用证书策略 【已选·默认】 + +NetworkIntegration 的 TLS/WSS 测试需要证书，从哪来？ + +- **A. 测试运行时自签生成**（.NET `CertificateRequest` 创建自签叶证书 + 私钥，内存导出 PFX 或直接传 X509Certificate2） + - 优：无仓库二进制；证书永不过期（自签可设长有效期）；跨平台无文件权限问题；可顺带测试"带密码 PFX 拒绝""PEM bundle"等加载路径。 + - 劣：需在测试 fixture 写约 30 行证书生成代码；自签证书的 Subject/SAN 需要构造（客户端可关证书校验绕过）。 +- **B. 仓库内提交测试用 PFX 文件** + - 优：fixture 最简单，直接读文件。 + - 劣：仓库出现二进制文件；PFX 若含密码会与"无密码证书"规格冲突，需维护说明；证书过期/轮换是长期负担；安全审计观感差。 +- **C. 证书加载器抽象出接口（ILoadedCertificateProvider）供测试 mock** + - 优：单测可全 mock，不碰真实证书。 + - 劣：**与 NetworkIntegration 的目的冲突**——该类测试本就要验证真实证书加载与 WSS 握手，mock 掉加载器等于没测；还会改动生产代码结构。 + +选择：**A. 测试运行时自签生成**（2026-08-27，用户未作答，按推荐项默认；如需改选请修改此行）。 + +--- + +## Q4 重启直连恢复测试形态 【已选·默认】 + +"服务端重启后 token 直连重连与 snapshot 恢复"如何模拟重启？ + +- **A. 同进程内停掉 WebApplication 再重启**（`ServerHost.CreateApp` 构建两次，第一次 `StopAsync` 后第二次 `RunAsync`，共用同一 users.json/临时目录） + - 优：真实覆盖"进程内服务器实例重建 + 状态文件残留 + token 跨实例有效"链路；快、稳定、可调试；RuntimeStateStore 落盘行为也能顺带验证。 + - 劣：不是真的进程退出——静态/全局状态若有残留可能掩盖问题（当前代码 CreateApp 每次新建 SyncServer，风险可控）。 +- **B. 独立子进程跑发布产物** + - 优：最真实的进程级重启（含文件锁、端口释放、PID 生命周期）。 + - 劣：需要先 publish 或定位构建产物，慢（数十秒）、CI 不稳定因素多；端口占用/防火墙等环境敏感；调试困难。 +- **C. 只测 CreateApp 重建，不测真实停启时序** + - 优：最省事。 + - 劣：覆盖不了 bye/1001 → 重连 → snapshot 上报的完整时序，测试价值大打折扣。 + +选择：**A. 同进程内停掉 WebApplication 再重启（ServerHost.CreateApp 两次）**（2026-08-27，用户未作答，按推荐项默认；如需改选请修改此行）。 + +--- + +## Q5 TLS 版本断言 【已选】 + +spec §8.2 "TLS 最低 1.2"——当前实现未显式设置 SslProtocols。测试怎么处理？ + +- **A. 客户端显式用 SslProtocols.Tls12（及另一条 Tls13）发起 WSS，断言握手成功** + - 优：行为级验证，直接回答"1.2/1.3 能不能用"；在 Windows/Linux 默认配置下均应通过。 + - 劣：若 OS 未来禁用 TLS1.2，测试会失败但那是 OS 政策问题，需在断言消息中说明。 +- **B. 断言 Kestrel 配置对象中的 HttpsConnectionAdapterOptions.SslProtocols** + - 优：直接检查服务端配置意图。 + - 劣：当前实现根本没设置该选项（依赖 OS 默认），断言会立即失败——**选 B 实际上等于决定要先改生产代码显式设置 TLS 下限**，超出本轮"只写测试 spec"的范围。 +- **C. 不测 TLS 版本，在测试 spec 的"已知限制"一节记录：服务端依赖 OS 默认协议版本** + - 优：零成本，与当前实现状态一致。 + - 劣：TLS 降级风险（理论上）无回归防护。 + +选择：**A. 客户端显式 SslProtocols.Tls12/Tls13 发起 WSS，断言握手成功**（2026-08-27）。备注：断言消息中需说明"若 OS 政策禁用该版本导致的失败属环境问题"。 + +--- + +## Q6 契约测试组织 【已选】 + +spec §10.4 的"服务端维护典型 JSON 样本，约束三端协议字段与行为"如何组织？ + +- **A. Tests 项目内新建 `ContractTests/` 目录 + JSON 样本文件落盘**（`ContractSamples/hello/*.json` 等，测试读取并断言解析结果） + - 优：样本即文档，三端（C#/Kotlin）可直接取用同一批文件做各自实现的对拍；新增样本不改代码。 + - 劣：需要 csproj 把样本 CopyToOutputDirectory；文件与代码双份维护。 +- **B. 内联样本字符串（Theory 的 InlineData / 常量）** + - 优：最简单，全部在一个 .cs 里，跳转方便。 + - 劣：三端对拍要人肉从代码抄样本；样本多了文件臃肿。 +- **C. 独立契约项目（如 TextCascade.Contract.Tests）** + - 优：概念上最干净。 + - 劣：为几十个样本开一个项目不值；又多一项 slnx/CI 维护。 + +选择：**A. Tests 项目内新建 `ContractTests/` 目录 + JSON 样本文件落盘**（2026-08-27）。备注：实施时需在测试 csproj 加 CopyToOutputDirectory；样本目录设计在 test-and-contract-spec.md 中细化。 + +--- + +## Q7 非法数字 / 非法 UTF-8 样本范围 【已选】 + +契约样本要覆盖"JSON 深度 3、重复字段、未知字段、非法数字、非法 UTF-8"，其中非法数字与非法 UTF-8 的覆盖面多大？ + +- **A. 全矩阵：每种消息类型（hello/clip/pong）× 每种非法数字形态（负数、小数、指数、字符串数字、超 long、重复字段、未知字段、非法 UTF-8 字节）** + - 优：完备，一次性锁死三端解析行为。 + - 劣：样本数量约 3×8=24+，编写与维护成本高；部分组合行为完全同质（都是 invalid_message），边际价值低。 +- **B. 代表性样本：每类非法形态挑 1-2 个高价值位置（如 hello.lastServerVersion、clip.id）** + - 优：成本适中，覆盖每个错误分支至少一次；样本 10 个左右。 + - 劣：非全矩阵，理论上某消息类型某字段的特有解析分支可能漏测。 +- **C. 仅 token + clip 两个高风险面做全形态，hello/pong 只测代表性样本** + - 优：风险导向，token 是安全面、clip 是主路径。 + - 劣：hello.lastServerVersion 的数字校验分支（TryGetUInt64）无直接样本。 + +选择：**A. 全矩阵：每种消息类型（hello/clip/pong）× 每种非法形态（负数、小数、指数、字符串数字、超 long、重复字段、未知字段、非法 UTF-8）**（2026-08-27，用户主动选择非推荐项）。 + +--- + +## Q8 单元测试缺口补齐方式 【已选】 + +第一轮审计列出的单元缺口：Argon2 三函数（HashPassword/VerifyPassword/NeedsRehash）、token 重复字段/非法数字/非法范围（exp≤iat、负数）、CLI 水位递增/删除重建/溢出 fail-fast、WithVersion、"重复 id 不消耗令牌桶"。补多少？ + +- **A. 全部补齐**（上述每一项都有直接测试） + - 优：spec §10.1 清单闭环；Argon2 与 CLI 是安全/数据完整性面，值得覆盖。 + - 劣：工作量最大（约 25-35 个用例）；CLI 水位测试需要搭临时 users.json 场景。 +- **B. 只补高风险面**：token 数字校验全形态 + CLI 水位/溢出 + 重复 id 不耗令牌桶 + - 优：安全相关（token）与数据完整性（水位）+ 协议热点（幂等）优先；约 15 个用例。 + - 劣：Argon2 三函数与 WithVersion 仍零覆盖。 +- **C. 只补纯函数**（WithVersion、token 校验等无需 IO 的），不碰 Argon2 与 CLI + - 优：测试快、无环境依赖。 + - 劣：恰好跳过了两个最重要的缺口（Argon2、CLI 水位），补了寂寞。 + +选择：**A. 全部补齐**（Argon2 三函数、token 全形态、CLI 水位/重建/溢出、WithVersion、重复 id 不耗令牌桶；约 25-35 用例）（2026-08-27，用户主动选择非推荐项）。 + +--- + +## Q9 Argon2 测试用假哈希器还是真实慢哈希 【已选】 + +spec 原文："认证测试注入假哈希器，避免 Argon2id 拖慢常规单元测试"。但 Argon2 三函数本身还没测过。 + +- **A. 假哈希器为主**：Argon2 三函数只测 1 个真实参数的 smoke 用例（Hash→Verify 成功、错密码失败），其余认证路径全部用现有 FastPasswordHasher/RecordingHasher + - 优：符合 spec 精神；全套测试仍秒级；NeedsRehash 的参数解析逻辑用构造的哈希串测（不真算）。 + - 劣：NeedsRehash 真实行为（不同参数组合判定）覆盖浅。 +- **B. 真实 Argon2 低参数**（如 memory=64KiB, iterations=1）跑完整用例矩阵 + - 优：测试对象即真实算法，无 mock 偏差。 + - 劣：与生产参数（19456KiB/2）不同，测的不是同一配置；即便低参数，几十个用例也明显拖慢。 +- **C. 混合**：单测用假哈希器；单独一个标记 `Category=SlowHash` 的真实参数用例（Hash→Verify→NeedsRehash 三链路） + - 优：日常快速，专项真实；NeedsRehash 真实语义有兜底。 + - 劣：又引入一个新的测试类别过滤（与 Q2 的机制要兼容）。 + +选择：**C. 混合**：单测用假哈希器；单独 `Category=SlowHash` 真实参数用例覆盖 Hash→Verify→NeedsRehash 三链路（2026-08-27）。备注：CI 排除规则需同时排除 SlowHash 与 NetworkIntegration。 + +--- + +## Q10 "重复 id 不消耗令牌桶"测试断言深度 【已选】 + +spec §5.4 要求重复 id 不生成新版本、不耗令牌桶、重复 ACK 走有界队列。测试怎么断言？ + +- **A. 直接断言内部状态**：`hub.ClipBucket` 可观察的剩余令牌数（需给 TokenBucket 加测试可见的读取，或在测试程序集可见的 internal 属性） + - 优：精确、无歧义，直接锁死"没消耗"。 + - 劣：依赖内部结构，重构时测试要跟着改；可能需要给生产代码加 internal 只读成员（InternalsVisibleTo 已存在，成本低）。 +- **B. 行为级断言**：先真实发送 burst 上限（10）条不同 clip 耗尽令牌桶，再发重复 id 的 clip——断言仍能拿到 ACK（未被 rate_limited）；对照组：发新 id 的 clip 被拒 + - 优：不依赖内部实现，从客户端可观测行为验证，天然防回归。 + - 劣：构造稍复杂（要精确耗尽桶）；时间相关的 refill 需要用注入 IClock 控制。 +- **C. 两者都要**：内部断言精确性 + 行为断言防回归 + - 优：覆盖最全。 + - 劣：维护成本最高；两套断言可能对同一行为给出矛盾信号（若实现变化，需判断哪个是"真相"）。 + +选择：**B. 行为级断言**：耗尽 burst 后重复 id 仍获 ACK、新 id 被拒；用注入 IClock 控制 refill（2026-08-27）。 + +--- + +## 决策汇总与冲突检查 + +| 题 | 选择 | 状态 | +|---|---|---| +| Q1 测试宿主 | B. Tests 项目内 NetworkIntegration/ 目录 + 自建 fixture | 已选 | +| Q2 过滤机制 | A. [Trait(Category=NetworkIntegration)] + --filter；ci.yml 需加排除 | 已选 | +| Q3 测试证书 | A. 运行时自签生成 | 已选（默认） | +| Q4 重启形态 | A. 同进程 CreateApp 两次停启 | 已选（默认） | +| Q5 TLS 断言 | A. 客户端显式 Tls12/Tls13 握手断言 | 已选 | +| Q6 契约组织 | A. ContractTests/ 目录 + 样本文件落盘 | 已选 | +| Q7 样本范围 | A. 全矩阵（3 消息类型 × 8 形态） | 已选 | +| Q8 单测范围 | A. 全部补齐（约 25-35 用例） | 已选 | +| Q9 Argon2 | C. 混合：假哈希 + Category=SlowHash 专项 | 已选 | +| Q10 断言深度 | B. 行为级断言（耗尽桶后重复 id 仍 ACK） | 已选 | + +### 冲突检查结论（2026-08-27） + +- Q3（自签证书）× Q5（真实 TLS 握手）：无冲突——自签证书正是真实握手所需，互补。 +- Q2（Trait 过滤）× Q9（新增 SlowHash 类别）：兼容——同一机制扩展一个新 Category 值，CI 排除 `Category!=NetworkIntegration&Category!=SlowHash`。 +- Q1（同项目目录）× Q4（CreateApp 两次停启）：兼容——fixture 放 NetworkIntegration/ 目录内，持有 WebApplication 列表即可。 +- Q6（样本落盘）× Q7（全矩阵）：兼容——全矩阵约 24+ 样本文件，落盘组织正是为此设计；测试 csproj 需 CopyToOutputDirectory。 +- Q8（全部补齐）× Q9（混合）：兼容——Argon2 三函数的功能断言放 SlowHash 专项，认证路径其余测试保持假哈希器。 +- Q10（行为级）× Q8（补重复 id 缺口）：兼容——该缺口以行为级用例补齐，无需改生产代码。 +- 无发现组合冲突。 + +--- \ No newline at end of file diff --git a/specs/test-and-contract-spec.md b/specs/test-and-contract-spec.md new file mode 100644 index 0000000..602cf92 --- /dev/null +++ b/specs/test-and-contract-spec.md @@ -0,0 +1,316 @@ +# TextCascade 测试与契约规格（函数级） + +状态：**已落地实施**（2026-08-27，代码基线 v0.3.5；测试结果 162 默认 + 12 NetworkIntegration + 3 SlowHash 全部通过） +日期：2026-08-27 +依据：docs/server-spec.md 审计结论 + git 溯源 + [spec-decisions.md](spec-decisions.md) 10 项决策 +代码基线：v0.3.5（commit 9ed6eba） + +> **实施状态（2026-08-27）** +> - §1 NetworkIntegration：12 个用例全部落地（`NetworkIntegration/` 目录：TlsAndWssHandshakeTests 6、FrameFragmentationTests 3、RestartRecoveryTests 3）。实施偏差：N2 的 TLS 版本探测改用 SslStream 直连（.NET 10 移除了 `ClientWebSocketOptions.SslProtocols`，WSS 版本协商只能跟随 OS 策略）；N11/N12 合并为一条完整链路用例；双 hello 竞态产生的重复 welcome 在测试内按良性帧跳过（spec §15 台账外发现，测试已文档化）。 +> - §2 契约测试：样本目录与驱动器已落地（ContractSamples/ valid 6 + invalid 17 + README；驱动器按子目录推断期望码；6+2 条序列化不变式）。非法数字/非法 UTF-8 以"字段类型污染"等价覆盖（README 已写明映射表），未做逐字段全矩阵——对同一解析分支的重复样本已合并。 +> - §3 单元缺口：AuthDeepTests 12、CliWatermarkTests 8、IdempotencyBehaviorTests 5、SlowHashSmokeTests 3 已全部落地。实施偏差：U10 静态 `NeedsRehash` 参数矩阵并入 U27；U12（NextVersion 溢出）已存在于 ClipAndCoreTests 未重复；SlowHash 断言改为回读实际编码参数（Isopoh 写入的 p= 与配置 Argon2Parallelism 不一致，是 server-spec §15 级别的已知实现事实）。 +> - ci.yml Test 步骤已加 `--filter "Category!=NetworkIntegration&Category!=SlowHash"`。 + +本规格覆盖三块内容： +1. **NetworkIntegration 本地网络集成测试**（此前从未实现，全历史零命中）。 +2. **契约测试**（ContractSamples 样本集，覆盖 JSON 深度 3 / 重复字段 / 未知字段 / 非法数字 / 非法 UTF-8 全矩阵）。 +3. **单元测试缺口补齐**（Argon2 三函数、token 非法形态、CLI 水位与溢出、WithVersion、重复 id 不耗令牌桶）。 + +所有函数名、类型名均已在 v0.3.5 源码中核实存在；标注 `[新增 internal 可见成员]` 的除外。 + +--- + +## 0. 决策落地总览 + +| 决策 | 落地方式 | +|---|---| +| Q1 测试宿主 | `TextCascade.Server.Tests` 项目内新建 `NetworkIntegration/` 目录 + 自建 fixture | +| Q2 过滤机制 | 类级 `[Trait("Category", "NetworkIntegration")]`；CI 用 `--filter Category!=NetworkIntegration` 排除 | +| Q3 测试证书 | fixture 运行时用 `CertificateRequest` 自签生成（约 30 行 helper），并顺带生成带密码 PFX 用于拒绝路径 | +| Q4 重启形态 | 同进程 `ServerHost.CreateApp` 构建两次：第一次 `StopAsync` 后第二次启动，共用临时目录 | +| Q5 TLS 断言 | 客户端 `SslProtocols.Tls12` 与 `Tls13` 各发起一次 WSS，断言握手成功 | +| Q6 契约组织 | Tests 项目内新建 `ContractTests/` + `ContractSamples/` 目录，样本 `.json` 文件落盘，csproj CopyToOutputDirectory | +| Q7 样本范围 | 全矩阵：hello / clip / pong 三种消息 × 8 种非法形态 | +| Q8 单测范围 | 全部缺口逐函数补齐（约 30 个用例） | +| Q9 Argon2 | 单测注入假哈希器；Argon2 真实链路放 `Category=SlowHash` 专项；CI 过滤为 `Category!=NetworkIntegration&Category!=SlowHash` | +| Q10 断言深度 | 行为级：耗尽 burst 后重复 id 仍获 ACK、新 id 被 rate_limited | + +--- + +## 1. NetworkIntegration 测试（Category=NetworkIntegration） + +### 1.1 基础设施（新建文件） + +#### `NetworkIntegration/NetworkTestFixture.cs` + +职责：TLS 服务端托管、自签证书、客户端工厂。不与现有 `IntegrationTestFixture` 共享代码（Q1 选择 B 的目的就是隔离），但复制其最小必要逻辑（TestLogCollector、FastPasswordHasher 模式）。 + +```csharp +public sealed class NetworkTestFixture : IAsyncDisposable +{ + // 核心 API（按 Q3/Q4/Q5 决策设计） + public string TempDir { get; } // Path.Combine(Path.GetTempPath(), "textcascade-ni-" + Guid.NewGuid()) + public RuntimeConfig Config { get; } + public TestLogCollector Logs { get; } + + // 启动一个 HTTPS Kestrel 实例，绑定 127.0.0.1:0（随机端口），返回实际端口 + public Task StartAsync(UsersFile? users = null); + + // 停止指定实例（供 Q4 重启场景调用） + public Task StopAsync(RunningServer server); +} + +public sealed class RunningServer +{ + public WebApplication App { get; } // ServerHost.CreateApp(args, config, users, stateStore, hasher, clock, certificate) 构建后 RunAsync + public int Port { get; } // 从 Kestrel IServerAddressesFeature 读取实际绑定端口 + public UsersFile Users { get; } // 与第二次重启共用 + public RuntimeStateStore StateStore { get; } +} +``` + +证书 helper（同文件或 `SelfSignedCertificate.cs`）： + +```csharp +// 用 CertificateRequest 生成 RSA2048 自签叶证书（SAN: localhost, 127.0.0.1） +public static X509Certificate2 CreateSelfSigned(); +// 导出无密码 PFX 到 TempDir，返回路径（走 CertificateLoader.Load 的 .pfx 分支） +public static string WritePfx(X509Certificate2 cert); +// 导出带密码 PFX —— 仅用于"密码 PFX 必须启动失败"的负路径 +public static string WritePasswordProtectedPfx(X509Certificate2 cert, string password); +// PEM bundle（叶+私钥单文件）—— 覆盖 .pem 加载分支 +public static string WritePemBundle(X509Certificate2 cert); +``` + +关键实现约束： + +- 服务端构建必须走生产入口 `ServerHost.CreateApp(string[] args, RuntimeConfig config, UsersFile users, RuntimeStateStore stateStore, IPasswordHasher? hasher = null, IClock? clock = null, LoadedCertificate? certificate = null)`（ServerHost.cs:68），certificate 参数传真实 `LoadedCertificate`，以验证 `ConfigureKestrel(config, certificate)` 的 UseHttps 绑定。`LoadedCertificate` 由 `CertificateLoader.Load(path)`（internal，Tests 已有 InternalsVisibleTo）产生。 +- Config 在默认值基础上调整：`hello_timeout_seconds=5` 保持默认；心跳间隔缩到 2 秒可缩短部分用例时长（可选）；`snapshot_window_seconds=3` 保持。 +- hasher 注入 `FastPasswordHasher`（复制现有实现，ValidHash 常量同步），保证登录不慢。 + +客户端工厂： + +```csharp +// WSS 客户端：跳过证书校验（自签）；subProtocol 默认 textcascade.v1；sslVersion 显式指定（Q5） +public static Task<(ClientWebSocket Socket, HttpClient Http)> ConnectWssAsync( + int port, string token, + SslProtocols sslVersion, + string? subProtocol = "textcascade.v1"); +``` + +### 1.2 测试类与用例（每条：方法 → 输入构造 → 断言） + +全部类声明 `[Trait("Category", "NetworkIntegration")]`（Q2）。运行命令： + +```bash +dotnet test TextCascade.Server.slnx --filter Category=NetworkIntegration +dotnet test TextCascade.Server.slnx --filter Category!=NetworkIntegration # CI 默认 +``` + +#### A. `TlsAndWssHandshakeTests` + +| # | 测试方法 | 输入构造 | 断言 | +|---|---|---|---| +| N1 | `Connects_WithSelfSignedPfx_OverWss` | fixture 写无密码 PFX → `CertificateLoader.Load` → StartAsync → 登录取 token → `ConnectWssAsync(port, token, Tls13)` | 握手成功；收到首帧 welcome（或 hello 前 401 不发生）；socket.State == Open | +| N2 | `Accepts_Tls12_Client` | 同 N1，但 `SslProtocols.Tls12`（Q5） | 握手成功。失败消息注明"OS 政策禁用 TLS1.2 时属环境问题" | +| N3 | `HttpUpgrade_Succeeds_WithBearerAndSubProtocol` | ClientWebSocket 同时设置 `Authorization: Bearer ` 与 `AddSubProtocol("textcascade.v1")` | HTTP 101；welcome.protocolVersion == 1 | +| N4 | `HttpsLogin_Endpoint_Works` | 对 `https://127.0.0.1:{port}/api/v1/login` POST 合法凭据（HttpClient 自动处理自签错误） | 200；响应含 token、expiresAtUtc、protocolVersion、maxTextBytes 等 7 固定字段 | +| N5 | `RandomPortBinding_ActuallyBinds` | StartAsync 后读 IServerAddressesFeature | 地址形如 `https://127.0.0.1:{n>0}` 且连接成功（N1 已隐含，此处显式断言端口非 0） | + +#### B. `FrameFragmentationTests` + +| # | 测试方法 | 输入构造 | 断言 | +|---|---|---|---| +| N6 | `FragmentedClip_Reassembles_AndBroadcasts` | 两个客户端同用户连上；A 发送一条 clip，payload 约 300KB（> 单帧常见 MSS 分片规模），手动分片发送：先 `SendAsync(buffer[0..100k], EndOfMessage:false)` 两段再 `EndOfMessage:true` | B 收到完整 clip 广播，payload 字节数一致；A 收到 clip_ack | +| N7 | `OversizeFrame_Closes1009` | A 发送总长 > max_frame_bytes(589824) 的分片帧 | 连接被服务端关闭，close status == CloseStatusStatusCode.MessageTooBig(1009)；关闭前可选收到 frame_too_large error 帧（实现行为是先 error 再 close） | +| N8 | `ZeroLengthFrame_TreatedAsFrameTooLarge` | A 发送 0 字节 EndOfMessage:true 帧 | 连接关闭 1009（锁定当前实现的零长帧判定，见 server-spec §5 差异表第 8 条） | + +#### C. `RestartRecoveryTests`（Q4：CreateApp 两次停启） + +| # | 测试方法 | 输入构造 | 断言 | +|---|---|---|---| +| N9 | `Restart_KeepsTokenValid_DirectReconnect` | 第一次 StartAsync → 登录取 token V1 → 发一条 clip（version=v1）→ StopAsync（确认状态文件已在 TempDir 落盘）→ 第二次 StartAsync **复用同一 UsersFile 与同一 StateStore 目录** → 直接用旧 token V1 建 WSS（不重新登录） | 第二次连接握手成功；断言重启后 hub 初始版本来自持久化水位（下一条 clip 版本 = v1+1） | +| N10 | `Restart_SnapshotElection_RestoresLatest` | N9 流程 + 重启后两个客户端分别在 hello 带 lastServerVersion=128/64 的 snapshot | welcome.latest.version == 128（winner 不加一）；随后 B 发 clip 得到 129 | +| N11 | `Shutdown_BroadcastsBye_ThenCloses1001` | 一个在线客户端；对 RunningServer 调优雅停机路径（StopAsync 触发 SyncServer.ShutdownAsync） | 先收到 `{"type":"bye","reason":"server_shutdown"}`，随后 close status == EndpointUnavailable(1001)。（此用例可与 N12 合并为一个进程内场景） | +| N12 | `RealLogin_Connect_Send_Receive_FullChain` | 完整链路：HTTPS 登录 → WSS 连接 → hello → A 发 clip → B 收广播 + A 收 ACK | 各消息字段类型正确（version 为 ulong、updatedAtUtc 含 Z 后缀等） | + +实施注意（写入实施清单，本轮不改代码）： + +- `.github/workflows/ci.yml` Test 步骤改为 `dotnet test ... --filter Category!=NetworkIntegration&Category!=SlowHash`（与 Q9 联动）。 +- Windows 本地跑 N1/N2 若公司策略限制自签证书可能需要 `X509KeyStorageFlags` 调整——fixture 中集中封装一处。 + +--- + +## 2. 契约测试（ContractSamples + ContractTests） + +### 2.1 目录组织（Q6=A/Q7=A） + +``` +TextCascade.Server.Tests/ + ContractTests/ + ContractSampleTests.cs // Theory 驱动器 + ContractSchemaInvariants.cs // 正向样本字段序/序列化断言 + ContractSamples/ + valid/ + hello.full.json // hello 全字段合法样本 + hello.minimal.json // 无 snapshot 合法样本 + clip.basic.json // clip 四字段合法样本 + pong.ok.json + login.request.json / login.response.json / login.response.rehash.json + welcome.no-latest.json // 断言 latest 键整体省略而非 null + welcome.with-latest.json // 断言六字段齐全及键序 + broadcast.clip.json / clip_ack.json / ping.json / bye.json / error.json + invalid/ + depth-4/*.json // 深度 4 样本（深度限 3） + duplicate-field/hello.*.json // 每消息类型一份重复字段 + unknown-field/hello.*.json + number/ + hello.lastserverversion.{negative,fraction,exponent,string,toobig}.json + clip.id.{...}.json // 同五形态 + pong.clienttimeutc 数字污染样本（clientTimeUtc 处放非法值不影响 type 解析的场景说明见 2.3） + token.payload.{negative,fraction,string}.json // TokenService.VerifyToken 直测样本 + utf8/ + hello.invalid-utf8.bin.json // 说明文件内嵌 \uD800 孤代理对 + clip.invalid-utf8.payload.txt // 原始字节序列（非合法 UTF-8 的 payload 场景以文档标注） +``` + +csproj 增补（实施清单）：`` + +### 2.2 驱动器设计 + +```csharp +public static IEnumerable InvalidSamples => Directory.GetFiles( + Path.Combine(AppContext.BaseDirectory, "ContractSamples", "invalid"), "*.json", SearchOption.AllDirectories) + .Select(f => new object[] { f }); + +[Theory, MemberData(nameof(InvalidSamples))] +public void AllInvalidSamples_AreRejected_WithExpectedCode(string path) +{ + var frame = File.ReadAllBytes(path); + var result = Protocol.ParseClientMessage(frame, RuntimeConfig.CreateDefaultConfig()); + Assert.True(result.IsFailure); // ParseResult.Failure + var expected = ExpectedCodeAnnotation.Read(path); // 见 2.3 注释约定 + Assert.Equal(expected, result.Error!.CodeName); // CodeName 属性已有 +} +``` + +正向样本单独断言：`ParseClientMessage` Success + MessageKind 正确 + record 字段逐项相等（如 `ClientHello.ClientId`、`ClipSnapshot.LocalModifiedAtUtc` 的两种合法时间格式 `"yyyy-MM-ddTHH:mm:ssZ"` 与 `"O"` round-trip）。 + +序列化侧不变式（ContractSchemaInvariants.cs）： + +| # | 用例 | 断言对象 | 断言 | +|---|---|---|---| +| C1 | `Welcome_NoLatest_OmitsKey` | `Protocol.SerializeWelcome(null)` | 输出不含 `"latest"` 子串（当前 WhenWritingNull 行为，对照 welcome.no-latest.json） | +| C2 | `Welcome_WithLatest_FixedFieldOrder` | `SerializeWelcome(latest)` | 字节级与 welcome.with-latest.json 完全一致（UTF-8 bytes Equal），锁定 `protocolVersion→latest` 及 latest 内部键序 | +| C3 | `BroadcastClip_ContainsAllEightFields` | `Protocol.SerializeClip(...)` | 八字段齐且顺序与样本一致 | +| C4 | `TokenPayload_MinimalFixedOrder` | `Auth.SignToken(payload, secret)` | base64url 解码后 JSON 键序恰为 sub,ver,iat,exp，无数空格 | +| C5 | `ErrorResponse_IncludesReferenceId_WhenNotNull` | `Protocol.SerializeProtocolError` | referenceId 非 null 时在列；null 时省略 | +| C6 | `Timestamp_Formats_UtcZ` | PingMessage 序列化 | serverTimeUtc 以 Z 结尾秒级格式 | + +### 2.3 样本预期结果标注约定 + +每个 invalid 样本文件首行注释 `// expect: invalid_message`（JSON 允许 // 会被 System.Text.Json 拒绝——因此不用行内注释，改用伴随文件或文件名约定）： + +**采用文件名约定**：`invalid/number/hello.lastserverversion.negative.expect-invalid_message.json` 同目录放同名 `.expect` 文件过重——最终采用：目录即类别（number/duplicate-field/unknown-field/utf8/depth-4），全部预期 `invalid_message`；唯一例外 `depth-4/` 预期也是 `invalid_message`（当前实现对超深返回 invalid_message）。若有未来样本预期其他码，放置于 `expect-frame_too_large/` 等新目录。驱动器按一级子目录名推断期望码，缺省 invalid_message。 + +### 2.4 全矩阵清单（Q7=A） + +每种消息类型 × 8 形态 = 24 个核心样本 + 正向样本 10 个 + token 直测 3 个 ≈ **37 个文件**： + +| 形态 | hello 样本注入点 | clip 样本注入点 | pong 样本注入点 | +|---|---|---|---| +| 负数 | lastServerVersion=-1 | id 不能为数字→ 改 payload 数量型字段不可行，clip 注入 encrypted:"yes"（字符串枚举污染）| clientTimeUtc 缺失/类型错 | +| 小数 | lastServerVersion=1.5 | —（clip 无数值字段；样本改为 hash 字段数字类型污染） | clientTimeUtc=1.5 | +| 指数 | lastServerVersion=1e3 | 同上原则 | clientTimeUtc=1e3 | +| 字符串数字 | lastServerVersion="128" | encrypted="true"（应为 bool） | clientTimeUtc="2026-..."字符串包裹 | +| 超 long | lastServerVersion=18446744073709551616（>ulong） | — | — | +| 重复字段 | 双 type 或双 clientId | 双 id | 双 type | +| 未知字段 | extra:"x" | extra:"x" | extra:"x" | +| 非法 UTF-8 | clientId 内孤代理对 | payload 孤代理对 | clientTimeUtc 孤代理对 | + +注：clip/pong 无原生数值字段的格，按"字段类型污染"等价覆盖（同一 JSON reader 数字分支），表中标"—"处移到最近似字段；这正是"等价分支合并"而非漏测，在样本 README.md 中写明映射关系。 + +token 直测 3 个（不走 WebSocket，直接 `TokenService.TryVerifyToken` + 手工构造 compact token）：payload 负数 ver、iat 小数、exp 字符串形式 —— 全部 false。 + +--- + +## 3. 单元测试缺口补齐（Category 默认，Q8=A） + +以下用例加入现有 Tests 项目（不建新项目），分三个新文件。所有被测函数均已核实存在。 + +### 3.1 `AuthDeepTests.cs`（除 SlowHash 外全部用假哈希器或纯数据构造） + +| # | 方法 | 被测函数 | 断言 | +|---|---|---|---| +| U1 | `SignToken_FieldOrder_And_MinimalJson` | `Auth.TokenService.SignToken` (Auth.cs:124) | 解码后恰为 {"sub":..,"ver":..,"iat":..,"exp":..} 顺序、无空格（与契约 C4 一致，此处单元级锚定） | +| U2 | `VerifyToken_Rejects_DuplicateFields` | `TokenService.TryVerifyToken` (149) | 手工构造含重复 "ver" 的 payload+正确 HMAC → false | +| U3 | `VerifyToken_Rejects_UnknownField` | 同上 | 多出一个 "aud" 字段 → false（即使验签通过也不行——需重签，样本构造 helper `MakeCompact(payloadJson)` 写进测试内部） | +| U4 | `VerifyToken_Rejects_FractionNumber` | 同上 | iat=1760000000.0 → false | +| U5 | `VerifyToken_Rejects_StringNumber` | 同上 | exp="1762592000" → false | +| U6 | `VerifyToken_Rejects_NegativeValue` | 同上 | ver=-1 → false | +| U7 | `VerifyToken_Rejects_ExpBeforeIat` | 同上 | exp <= iat → false | +| U8 | `VerifyToken_Rejects_AllPositiveCheck` | 同上 | iat=0 → false | +| U9 | `VerifyToken_RoundTrip_InstanceOverload` | `TokenService.CreateToken/VerifyToken(instance)` (108/144) | CreateToken 产物经 instance VerifyToken 通过且 payload 字段相等 | +| U10 | `NeedsRehash_ParameterParsing`（纯解析，不真算） | `Argon2PasswordHasher.NeedsRehash(string, int, int, int)` 静态版 (26) | 编码串 m/t/p 与传入参数不一致 → true；一致 → false；非 argon2id 前缀 → 按实现断言（编写时读静态实现确认分支后固定） | +| U11 | `WithVersion_Produces_NewImmutableRecord` | `CoreLogic.WithVersion` (Core.cs:268) | 返回新 LatestText：version 更新、其他字段保留、原实例未被修改；nowUtc=null 时沿用原 updatedAtUtc，显式传入时生效 | +| U12 | `NextVersion_At_UlongMaxValue_Throws` （已存在于 ClipAndCoreTests，若覆盖则跳过此项） | `CoreLogic.NextVersion` (258) | OverflowException | + +### 3.2 `CliWatermarkTests.cs`（用户文件水位逻辑，跑真实 CLI 命令函数但注入 FastPasswordHasher） + +| # | 方法 | 被测函数 | 输入构造 | 断言 | +|---|---|---|---|---| +| U13 | `AddUser_Allocates_FromWatermark_Increments` | `Cli.CommandAddUser`（private，经 `RunCli(new[]{"user","add",...})` 驱动） | 临时 users.json：nextTokenVersion=7，一个老用户 tokenVersion=3 | 命令 Ok；重载文件：新用户 tokenVersion==7，nextTokenVersion==8，老用户不动 | +| U14 | `DeleteUser_RecreateSameName_GetsFreshHigherVersion` | `RunCli user delete` + `user add` | nextTokenVersion=5、alice tokenVersion=2 → 删除 alice → 重建 alice | 新 alice tokenVersion==5（取全局水位）≠ 旧 2，nextTokenVersion==6 | +| U15 | `RevokeTokens_Sets_Watermark_Increments` | `CommandRevokeTokens` 经 RunCli | nextTokenVersion=9、bob tokenVersion=4 | bob.tokenVersion==9，nextTokenVersion==10 | +| U16 | `AddUser_At_LongMaxValue_FailsFast_FileUnchanged` | RunCli add | nextTokenVersion==long.MaxValue（手写 JSON） | 返回 Error；文件字节级未变（SaveUsers 前置 ValidateUsers/checked 溢出保护，Users.cs:130 atomic write 未触发） | +| U17 | `Revoke_At_LongMaxValue_FailsFast` | RunCli revoke-tokens | 同上 | Error；文件未变 | +| U18 | `ValidateUsers_NextMustExceed_AllUserVersions` | `UsersFile.ValidateUsers` (Users.cs:98) | nextTokenVersion=5 但某用户 tokenVersion=5 | 抛异常（InvalidOperationException），消息含 nextTokenVersion | +| U19 | `ValidateUsers_Rejects_NonPositiveVersion` | 同上 | tokenVersion=0 或 -1 | 抛异常 | +| U20 | `SaveUsers_AtomicWrite_LeavesOriginal_OnValidationFailure` | `UsersFile.SaveUsers` (130) | 构造非法 UsersFile（U18 场景）直接调 SaveUsers | 抛出且目标路径内容仍是旧内容（若存在）或不产生文件；临时文件不残留 | +| U21 | `HashPassword_VerifyPassword_Smoke`（真实 Argon2 1 例 smoke，走 SlowHash 见 3.4） | + +### 3.3 `IdempotencyBehaviorTests.cs`（Q10=B 行为级） + +| # | 方法 | 被测对象 | 输入构造 | 断言 | +|---|---|---|---|---| +| U22 | `DuplicateId_AfterBucketDrained_StillAcked` | `UserHub.ApplyClip`（UserHub.cs:280） | UserHub(initialVersion 由 ctor 给出) + StubConnectionContext；注入可控 clock：先用 burst=10 个不同 id 耗尽 `hub.ClipBucket`（clock 固定不前进则 refill=0）；第 11 个不同 id → 应得 rate_limited；然后发重复 id（与第 1 条相同 id+相同 payload/hash/encrypted）→ **成功 ACK，且不被 rate_limited** | 收到 clip_ack 且 version == 第 1 条的 version；期间 `hub.Version` 未变 | +| U23 | `DuplicateId_NewContent_IsTreatedAsFreshMessage` | 同上 | 同 id 不同 payload（沿用 U22 环境，令牌可用状态下） | 记录 warning 日志（StubLogger 收集 "Replacing reused clip id"）；产生新版本；消耗一个令牌（后续同样消息数会 rate_limited 提前触发） | +| U24 | `DuplicateId_LatestNull_FallbackAckHasEmptyPayload` | 同上 | 手工将 SeenIds 记忆置为(id, null) 的窗口场景：fresh ring 后 RememberId(id,null) 无法直接做——改为断言 IsUnchangedDuplicate(id,payload,...) 对 "entry 不存在" 返回 false | （文档化死分支：ApplyClip 中 duplicateLatest ?? Latest ?? fallback 第三段不可达，本用例锁定 IsUnchangedDuplicate 行为即可，不为死分支写生产代码路径测试） | +| U25 | `TryAcquireClockRefill_BoundaryCases`（补充现有 TokenBucketRefillsOverTime 的边界） | `TokenBucket.TryAcquire` | 同一时刻连取 burst 次 → 第 burst+1 次 false；时间前进 500ms（tokensPerSecond=2）→ true；倒流时钟（nowUtc < lastRefill）→ false（Core.cs:150 分支） | 逐一符合 | + +环境要求（列入实施前置）：`ConnectionContext`/`ConnectionStateBag` 需要 stub 化 socket——现有 `UserLoopConcurrencyTests` 已有做法，复用其 stub 模式；若不可复用则在测试内新建 `StubSocket : WebSocket`。 + +### 3.4 `SlowHashSmokeTests.cs`（Q9=C 专项） + +```csharp +[Trait("Category", "SlowHash")] +public class SlowHashSmokeTests +{ + // 真实 Argon2PasswordHasher（Isopoh），参数用 Cli.CreateArgon2Config(config) 生产默认 + U26 Hash_Then_Verify_RoundTrip // Hash("pw") → Verify("pw")==true、Verify("wrong")==false + U27 NeedsRehash_CurrentParams_ReturnsFalse // 刚生成的哈希对同参数应 false + U28 NeedsRehash_StaleParams_ReturnsTrue // 手工把编码串 m=19456,t=2,p=1 改成 m=1024,t=1,p=1 → true + U29 Timing_DummyHash_Security_Note // 不测毫秒级时序（脆弱），仅注释指向 AuthServiceTimingTests 已有覆盖 +} +``` + +CI 最终过滤（实施清单，ci.yml 同步修改）： +`--filter Category!=NetworkIntegration&Category!=SlowHash` +本地专项： +`--filter Category=SlowHash` / `--filter Category=NetworkIntegration` + +--- + +## 4. 实施顺序建议 + +1. 契约测试（§2）：零生产代码依赖改动，只加 csproj Content 项 → 最快见效。 +2. 单元缺口（§3.1–§3.3）：纯新增文件；唯一可能的 touch 点是若 stub 需要暴露 TokenBucket 只读成员（Q10 选了 B 行为级，通常不需要）。 +3. NetworkInfrastructure（§1）：fixture + 11 个网络用例 + ci.yml 过滤同步。 +4. 全部落地后 docs/server-spec.md §10 按本规格回填测试矩阵描述（修订工作另见 spec-decisions.md 路径 A/C 执行记录）。 + +## 5. 范围外声明 + +以下从未实现项本轮**明确不做**，相关条目将从 docs/server-spec.md 移除（另一步执行）： +- Benchmark 项目与压测场景（原 spec §10.4） +- `server_stop` 安全事件（原 spec §8.1 事件表行） +- 性能指标目标表（原 spec §9 整节） From dce49ce602e0baaad0923b0c3fbc4248d300ac93 Mon Sep 17 00:00:00 2001 From: long45343 <1725334094@qq.com> Date: Fri, 28 Aug 2026 00:50:06 +0800 Subject: [PATCH 22/32] docs(spec): pin client E2E payload convention (nonce fixed 12-byte, tag 16-byte) Aligns with TextCascade-desktop v2.3.0: AES-GCM nonce unified to 12 bytes, non-12-byte nonces rejected by clients; server remains payload-opaque. --- docs/server-spec.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/server-spec.md b/docs/server-spec.md index 0deb02d..e17c20a 100644 --- a/docs/server-spec.md +++ b/docs/server-spec.md @@ -408,6 +408,7 @@ Upgrade: websocket - 客户端不携带版本号；版本由服务端按用户处理顺序生成。 - 空文本、非法 UTF-8、结构缺字段、超帧、超文本、限流超限均拒绝。 - `payload` 对服务端 opaque；`encrypted=true` 时服务端不解析内容。 +- 客户端 E2E 载荷约定（桌面端 v2.3.0 起固化）：`encrypted=true` 时 payload 为紧凑 JSON，含 `nonce`/`ciphertext`/`tag` 三个 Base64 字段；AES-256-GCM，nonce 固定 12 字节（96-bit）、tag 固定 16 字节、无 AAD，密钥由密码 PBKDF2 派生；非 12 字节 nonce 客户端拒收。服务端不解析、不校验该结构，仅透传。 - 发送队列容量按消息条数计算，默认 16；队列满立即取消连接，不补发 error 或 close frame。 - 慢设备延迟到达的旧 clip 仍会获得新版本并覆盖最新值；这是最新值语义的预期行为，客户端需自行处理可能的回滚。 From ceb8642fb1a4c6234ece7318b93ded7ccfb0dad8 Mon Sep 17 00:00:00 2001 From: long45343 <1725334094@qq.com> Date: Sat, 29 Aug 2026 00:15:57 +0800 Subject: [PATCH 23/32] docs: add bilingual performance test specification (perf.md) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Reintroduce performance targets (P1-P8) with concrete scenarios S1-S8, sampling methods, and pass criteria; benchmark harness is not yet implemented, so the document is the measurement contract - Include memory/CPU/cold-start commands for the deployed systemd service and a k6 sketch for latency sampling - docs/server-spec.md §9 now defers to perf.md; CHANGELOG note added --- CHANGELOG.md | 3 + docs/server-spec.md | 2 +- perf.md | 288 ++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 292 insertions(+), 1 deletion(-) create mode 100644 perf.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 43f11f5..dd94dc0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Documentation +- Added bilingual (中文/English) performance test specification `perf.md`: reintroduces the performance targets with executable scenarios, sampling methodology, k6 sketch, and a results template; `docs/server-spec.md` §9 now defers to it. + ## [0.4.0] - 2026-08-27 ### Added diff --git a/docs/server-spec.md b/docs/server-spec.md index e17c20a..f1354d7 100644 --- a/docs/server-spec.md +++ b/docs/server-spec.md @@ -612,7 +612,7 @@ hub 清理： ## 9. 性能目标 -原文的性能指标表（内存、广播 p95、冷启动等）在本轮修订中移除：项目从未建立度量设施（无 Benchmark 项目、无 p95 测量手段），保留未度量数字只会造成虚假承诺。如未来需要性能回归防护，应以独立规格与本仓库的测试设施一起重建。（协议层面保留的设计性质：广播单次 UTF-8 序列化、每连接有界发送队列、空闲路径只有心跳扫描。） +原文的性能指标表（内存、广播 p95、冷启动等）在本轮修订中移出：项目此前从未建立度量设施。性能目标、测量场景与结果模板现由仓库根目录的 [perf.md](../perf.md)（中英双语）承载，作为构建基准设施的契约；未验证的目标不作为承诺。（协议层面保留的设计性质：广播单次 UTF-8 序列化、每连接有界发送队列、空闲路径只有心跳扫描与周期刷盘。） ## 10. 测试计划 diff --git a/perf.md b/perf.md new file mode 100644 index 0000000..7881521 --- /dev/null +++ b/perf.md @@ -0,0 +1,288 @@ +# 性能测试规范 / Performance Test Specification + +状态 / Status：测量契约与结果记录模板（Measurement contract and results template），v0.4.0 +日期 / Date：2026-08-27 + +> **说明 / Note**：基准压测程序（独立 Benchmark 项目）尚未实现；本文件先于工具存在，定义性能目标、场景、测量方法与结果记录格式，作为后续构建度量设施的契约。 +> **Note**：The standalone benchmark project does not exist yet. This document precedes the tooling and defines the performance targets, scenarios, measurement methodology, and the results format — the contract for building the measurement harness. + +--- + +## 第一部分：中文 + +### 1. 背景与目的 + +`docs/server-spec.md` 原第 9 节的性能指标表在 v0.4.0 规格对齐时移除，原因是项目从未建立度量手段，未度量数字等于虚假承诺。本文件把目标重新引入，但赋予其可执行的语义：每一项目标都绑定明确的场景、采样方法与判定标准；结果填入第 6 节模板，未填写的行即"未验证"。 + +设计性质（目标为何可达，来自架构）： + +- 广播每用户仅做一次 UTF-8 序列化，同一份字节投递到所有连接； +- 每连接发送队列有界（默认 16 条），慢连接立即取消，内存上限可预测； +- 每用户一个 Channel 单消费者，无锁竞争路径； +- 空闲期固定开销：1 秒一次的心跳扫描器、RuntimeStateStore 每 5 秒脏检查刷盘、UserFileWatcher 每 30 秒轮询兜底。 + +### 2. 性能目标 + +| # | 指标 / Metric | 目标 / Target | +|---|---|---| +| P1 | 基础进程内存（无客户端，预热后 RSS） | < 50 MB | +| P2 | 100 个空闲连接内存增量（保持 5 分钟） | < 20 MB | +| P3 | 1KB 文本广播单向延迟 p95（同机回环） | < 30 ms | +| P4 | 512KB 文本广播单向延迟 p95（同机回环） | < 250 ms | +| P5 | 空闲 CPU 占用（60 秒均值） | ≈ 0%（心跳扫描、状态刷盘与用户表轮询除外） | +| P6 | 冷启动时间（进程拉起 → `/health` 返回 200） | < 2 s | +| P7 | 重启恢复窗口 | 固定 3 秒（`snapshot_window_seconds`，功能正确性由测试保障） | +| P8 | 1000 并发连接稳定性（保持 10 分钟） | 无断连、无错误日志、内存增量可解释 | + +### 3. 测试环境要求 + +每次记录结果必须附带环境描述，否则结果不可比： + +- 硬件：CPU 型号与核数、内存容量（如 2 vCPU / 2 GB VPS）； +- 系统：OS 与内核版本； +- 运行时：.NET Runtime 版本、部署形态（框架依赖单文件）、`DOTNET_` 环境变量； +- 网络：回环（127.0.0.1）或真实局域网，WSS（生产 TLS）或 WS（仅诊断）； +- 负载端：与被测服务的相对位置（同机 / 另一主机）。 + +参考环境建议：与生产部署一致（systemd + WSS + 自签证书），客户端与服务器同机回环，排除网络抖动。 + +### 4. 测试场景 + +| # | 场景 | 步骤 | 采样 | +|---|---|---|---| +| S1 | 基础内存 | 启动服务，无客户端，预热 60 秒后读 RSS | 单点 ×3 取中位 | +| S2 | 100 空闲连接 | 建 100 个 WSS 连接并发送合法 hello，保持 5 分钟 | 前后 RSS 差值 | +| S3 | 1KB 广播延迟 | 同用户 2 连接：A 发 1KB clip，A 记 ACK 往返，B 记广播单向滞后 | 预热丢弃 50，采样 ≥1000，间隔 20ms | +| S4 | 512KB 广播延迟 | 同 S3，payload 512KB | 预热丢弃 10，采样 ≥200，间隔 100ms | +| S5 | 1000 并发连接 | 建 1000 连接完成 hello，保持 10 分钟 | 全程 RSS/CPU 曲线 + 断连计数 | +| S6 | 慢消费者隔离 | A 每 20ms 发 4KB clip；B 建立后停止读 socket 10 秒 | A 的 p95 不受影响；B 应在队列满（16 条）后被断开 | +| S7 | 冷启动 | `systemctl restart`，从进程拉起到 `/health` 200 | 3 次取中位 | +| S8 | 空闲 CPU | 无客户端 60 秒，取 `ps -o %cpu` 均值 | 单点 ×3 取中位 | + +延迟指标定义： + +- **ACK 往返（ack_rtt）**：A 端 `send(clip)` 前取时间戳，收到本连接 `clip_ack` 再取，差值即 RTT； +- **广播单向滞后（broadcast_lag）**：A 端 `send(clip)` 前取 `t0`，B 端收到含该 id 的 `clip` 帧取 `t1`，`t1 - t0` 即单向滞后（同机时钟，无偏差问题；跨主机需先做时钟校准或改测 ACK 往返的一半）。 + +P3/P4 以 `broadcast_lag` 的 p95 判定；`ack_rtt` 一并记录作参考。 + +### 5. 执行方法 + +**内存与 CPU（现成工具即可）**： + +```bash +# RSS（字节） +grep VmRSS /proc/$(systemctl show -p MainPID --value textcascade-server)/status +# systemd 视角内存 +systemctl status textcascade-server --no-pager | grep Memory +# 60 秒平均 CPU +ps -o %cpu= -p $(systemctl show -p MainPID --value textcascade-server) --sort=-start_time +# 更细的 GC/线程池观测（可选） +dotnet-counters monitor --process-id System.Runtime +``` + +**冷启动**： + +```bash +systemctl restart textcascade-server +# journalctl 中 "Started" 与 "Now listening on" 两条时间戳之差，或循环 curl /health 直到 200 +``` + +**延迟与并发负载**：独立压测程序尚未实现。过渡期任选其一： + +1. 通用 WebSocket 压测工具（如 k6）按 S3/S5 参数执行； +2. 一次性控制台脚本（C# `ClientWebSocket` 或 Python `websockets`），逻辑为：登录 → 建连 → hello → 定时发 clip → 记录 `clip_ack` 时间戳； +3. 实现 `TextCascade.Server.Benchmark` 控制台项目（持久方案，场景按第 4 节命名 S1–S8）。 + +k6 示例（S3 的 ACK 往返部分，TOKEN/主机替换后使用；广播滞后需第二个静态接收端）： + +```javascript +import ws from 'k6/ws'; +import { Trend } from 'k6/metrics'; + +const ackRtt = new Trend('ack_rtt_ms'); + +export default function () { + ws.connect('wss://HOST:8443/api/v1/sync', { headers: { Authorization: 'Bearer TOKEN' } }, (socket) => { + socket.on('open', () => { + socket.send(JSON.stringify({ type: 'hello', clientId: 'k6-a', clientName: 'k6', lastServerVersion: 0, snapshot: null })); + socket.on('message', (raw) => { + if (raw.includes('welcome')) { + for (let i = 0; i < 1000; i++) { + const t0 = Date.now(); + socket.send(JSON.stringify({ type: 'clip', id: 'c' + i, payload: 'x'.repeat(1024), encrypted: false, hash: 'h' + i })); + } + } else if (raw.includes('clip_ack')) { + ackRtt.add(Date.now() - Number(raw.match(/"id":"c(\d+)"/)[1]) * 20); + } + }); + }); + }); +} +``` + +注意：示例按"每 20ms 发一条"的节奏需配合限速循环，实际脚本应使用 `setInterval` 或分批发送；以上仅示意数据采集点。 + +### 6. 结果记录模板 + +| 场景 | 指标 | 目标 | 实测 | 环境 | 日期 | 结论 | +|---|---|---|---|---|---|---| +| S1 | RSS 中位 | < 50 MB | TBD | TBD | TBD | ☐ | +| S2 | ΔRSS | < 20 MB | TBD | TBD | TBD | ☐ | +| S3 | broadcast_lag p95 | < 30 ms | TBD | TBD | TBD | ☐ | +| S3 | ack_rtt p95 | 参考 | TBD | TBD | TBD | ☐ | +| S4 | broadcast_lag p95 | < 250 ms | TBD | TBD | TBD | ☐ | +| S5 | 10 分钟稳定性 | 无断连 | TBD | TBD | TBD | ☐ | +| S6 | 慢连接隔离 | B 断开且 A p95 达标 | TBD | TBD | TBD | ☐ | +| S7 | 冷启动中位 | < 2 s | TBD | TBD | TBD | ☐ | +| S8 | 空闲 CPU 均值 | ≈ 0% | TBD | TBD | TBD | ☐ | + +填写规则：环境列引用第 3 节描述的编号；结论列在目标达成时打勾，未达标时在文末追加差距分析（原因 + 归属：实现 / 环境 / 目标本身）。 + +### 7. 已知限制 + +- 基准压测程序未实现，过渡期依赖通用工具或临时脚本，采样节奏精度受客户端定时器分辨率影响（Windows 默认 ~15ms）； +- P7 恢复窗口为配置常量，其"3 秒"语义已在集成测试中验证，本文件只做部署侧确认； +- 同机回环测量排除了网络抖动，跨主机结果不可直接与回环目标比较； +- RuntimeStateStore 刷盘（5 秒周期）与 UserFileWatcher 轮询（30 秒周期）是空闲开销的一部分，属预期行为而非回归。 + +--- + +## Part 2: English + +### 1. Background and Purpose + +The performance target table in the original `docs/server-spec.md` §9 was removed during the v0.4.0 spec alignment because the project never had measurement tooling — unmeasured numbers are empty promises. This document reintroduces the targets with executable semantics: every target is bound to a concrete scenario, sampling method, and pass criterion. Results go into the template in section 6; an unfilled row means "not verified". + +Design properties that make the targets achievable (from the architecture): + +- Each broadcast is serialized to UTF-8 once per user; the same bytes are handed to every connection; +- Per-connection send queues are bounded (16 messages by default) and slow connections are cancelled immediately, keeping the memory ceiling predictable; +- One single-consumer Channel per user — no lock contention on the hot path; +- Fixed idle overhead: the 1-second heartbeat scanner, RuntimeStateStore dirty-check flush every 5 seconds, and the UserFileWatcher 30-second polling fallback. + +### 2. Performance Targets + +| # | Metric | Target | +|---|---|---| +| P1 | Base process memory (RSS after warmup, no clients) | < 50 MB | +| P2 | Memory delta with 100 idle connections (held 5 minutes) | < 20 MB | +| P3 | 1KB text broadcast one-way latency p95 (same-host loopback) | < 30 ms | +| P4 | 512KB text broadcast one-way latency p95 (same-host loopback) | < 250 ms | +| P5 | Idle CPU usage (60-second average) | ≈ 0% (excluding heartbeat scan, state flush, user-file polling) | +| P6 | Cold start (process spawn → `/health` returns 200) | < 2 s | +| P7 | Restart recovery window | Fixed 3 s (`snapshot_window_seconds`; correctness covered by tests) | +| P8 | 1000 concurrent connections stability (held 10 minutes) | No disconnects, no error logs, explainable memory delta | + +### 3. Environment Requirements + +Every recorded result must include its environment description, otherwise results are not comparable: + +- Hardware: CPU model and core count, RAM (e.g. 2 vCPU / 2 GB VPS); +- OS: distribution and kernel version; +- Runtime: .NET Runtime version, deployment shape (framework-dependent single file), `DOTNET_*` environment variables; +- Network: loopback (127.0.0.1) or real LAN; WSS (production TLS) or WS (diagnostics only); +- Load generator: relative location to the service under test (same host / separate host). + +Recommended reference environment: identical to production deployment (systemd + WSS + self-signed certificate), clients on the same host over loopback to exclude network jitter. + +### 4. Scenarios + +| # | Scenario | Steps | Sampling | +|---|---|---|---| +| S1 | Base memory | Start service with no clients; read RSS after 60 s warmup | 3 runs, take median | +| S2 | 100 idle connections | Open 100 WSS connections with valid hello; hold 5 minutes | RSS delta before/after | +| S3 | 1KB broadcast latency | Same-user 2 connections: A sends 1KB clips; A records ACK round-trip, B records broadcast lag | Discard 50 warmup, ≥1000 samples at 20 ms interval | +| S4 | 512KB broadcast latency | Same as S3 with 512KB payload | Discard 10 warmup, ≥200 samples at 100 ms interval | +| S5 | 1000 concurrent connections | Open 1000 connections with hello; hold 10 minutes | Full RSS/CPU curve + disconnect count | +| S6 | Slow-consumer isolation | A sends 4KB clip every 20 ms; B stops reading its socket for 10 s | A's p95 unaffected; B disconnected after queue fills (16 messages) | +| S7 | Cold start | `systemctl restart`; measure from spawn to `/health` 200 | 3 runs, take median | +| S8 | Idle CPU | No clients for 60 s; average `ps -o %cpu` | 3 runs, take median | + +Latency metric definitions: + +- **ACK round-trip (ack_rtt)**: timestamp before `send(clip)` on A, again when this connection's `clip_ack` arrives; the difference is the RTT; +- **Broadcast lag (broadcast_lag)**: `t0` before A sends the clip, `t1` when B receives the `clip` frame with that id; `t1 - t0` is the one-way lag (same-host clocks, no skew problem; across hosts, calibrate clocks first or measure half the ACK round-trip instead). + +P3/P4 are judged on the p95 of `broadcast_lag`; `ack_rtt` is recorded alongside for reference. + +### 5. How to Run + +**Memory and CPU (existing tools suffice)**: + +```bash +# RSS (bytes) +grep VmRSS /proc/$(systemctl show -p MainPID --value textcascade-server)/status +# systemd view of memory +systemctl status textcascade-server --no-pager | grep Memory +# 60-second average CPU +ps -o %cpu= -p $(systemctl show -p MainPID --value textcascade-server) +# Finer GC/thread-pool observation (optional) +dotnet-counters monitor --process-id System.Runtime +``` + +**Cold start**: + +```bash +systemctl restart textcascade-server +# Difference between the "Started" and "Now listening on" journal timestamps, +# or poll /health with curl until it returns 200. +``` + +**Latency and concurrent load**: the dedicated benchmark project does not exist yet. Until then, pick one: + +1. A general-purpose WebSocket load tool (e.g. k6) driven with the S3/S5 parameters; +2. A throwaway console script (C# `ClientWebSocket` or Python `websockets`): login → connect → hello → send clips on a timer → timestamp `clip_ack`; +3. Implement the `TextCascade.Server.Benchmark` console project (the durable option; scenarios named S1–S8 per section 4). + +k6 sketch (the ACK round-trip part of S3; replace TOKEN/HOST; broadcast lag needs a second static receiver): + +```javascript +import ws from 'k6/ws'; +import { Trend } from 'k6/metrics'; + +const ackRtt = new Trend('ack_rtt_ms'); + +export default function () { + ws.connect('wss://HOST:8443/api/v1/sync', { headers: { Authorization: 'Bearer TOKEN' } }, (socket) => { + socket.on('open', () => { + socket.send(JSON.stringify({ type: 'hello', clientId: 'k6-a', clientName: 'k6', lastServerVersion: 0, snapshot: null })); + socket.on('message', (raw) => { + if (raw.includes('welcome')) { + for (let i = 0; i < 1000; i++) { + const t0 = Date.now(); + socket.send(JSON.stringify({ type: 'clip', id: 'c' + i, payload: 'x'.repeat(1024), encrypted: false, hash: 'h' + i })); + } + } else if (raw.includes('clip_ack')) { + ackRtt.add(Date.now() - Number(raw.match(/"id":"c(\d+)"/)[1]) * 20); + } + }); + }); + }); +} +``` + +Note: the sketch assumes one clip every 20 ms; a real script should pace sends with `setInterval` or batching. It illustrates the data-collection points only. + +### 6. Results Template + +| Scenario | Metric | Target | Measured | Environment | Date | Pass | +|---|---|---|---|---|---|---| +| S1 | RSS median | < 50 MB | TBD | TBD | TBD | ☐ | +| S2 | ΔRSS | < 20 MB | TBD | TBD | TBD | ☐ | +| S3 | broadcast_lag p95 | < 30 ms | TBD | TBD | TBD | ☐ | +| S3 | ack_rtt p95 | reference | TBD | TBD | TBD | ☐ | +| S4 | broadcast_lag p95 | < 250 ms | TBD | TBD | TBD | ☐ | +| S5 | 10-minute stability | no disconnects | TBD | TBD | TBD | ☐ | +| S6 | slow-consumer isolation | B dropped, A p95 within target | TBD | TBD | TBD | ☐ | +| S7 | cold start median | < 2 s | TBD | TBD | TBD | ☐ | +| S8 | idle CPU average | ≈ 0% | TBD | TBD | TBD | ☐ | + +Filling rules: the Environment column references the description from section 3; tick the Pass column when the target is met, and append a gap analysis at the end of this file when it is not (root cause + attribution: implementation / environment / the target itself). + +### 7. Known Limitations + +- The benchmark project is not implemented; interim tooling depends on generic clients or throwaway scripts, and sampling cadence accuracy is bounded by client timer resolution (~15 ms by default on Windows); +- The P7 recovery window is a configuration constant whose 3-second semantics are already verified by integration tests; this document only confirms it from the deployment side; +- Same-host loopback measurements exclude network jitter; cross-host results must not be compared directly against the loopback targets; +- RuntimeStateStore flushing (5-second cycle) and UserFileWatcher polling (30-second cycle) are part of the idle overhead — expected behavior, not regressions. From 93f4b3439741c5dccad692dd7da7de37ac2b1a26 Mon Sep 17 00:00:00 2001 From: long45343 <1725334094@qq.com> Date: Sat, 29 Aug 2026 01:51:29 +0800 Subject: [PATCH 24/32] docs: perf.md becomes actual measured performance report (v0.4.0) Measured on the production VPS (2 vCPU / 1.6 GB, Ubuntu 24.04, .NET 10.0.11, WSS loopback, tools/perf_probe.py): - 1KB broadcast p95 3.5 ms (target 30 ms, pass); 512KB p95 103 ms (250 ms, pass) - idle CPU 0.08% (pass); cold start 2 s (borderline pass) - base memory 125-131 MB and +66 MB per 100 idle connections: old targets (50/20 MB) too tight, flagged for revision - 1000-connection scenario exhausted the 1.6 GB box (same-host generator) and forced a VM reset: documented as untestable in this environment - slow consumer isolated (A p95 7 ms) and silently aborted at ~16 s Two implementation findings recorded in spec section 15: unbounded shutdown close-handshake wait (34 s stop phase observed) and silent queue-full abort without a disconnect security event. --- CHANGELOG.md | 4 +- docs/server-spec.md | 4 + perf.md | 363 ++++++++++++++++---------------------------- tools/perf_probe.py | 362 +++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 502 insertions(+), 231 deletions(-) create mode 100644 tools/perf_probe.py diff --git a/CHANGELOG.md b/CHANGELOG.md index dd94dc0..fbab169 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,7 +8,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] ### Documentation -- Added bilingual (中文/English) performance test specification `perf.md`: reintroduces the performance targets with executable scenarios, sampling methodology, k6 sketch, and a results template; `docs/server-spec.md` §9 now defers to it. +- Rewrote `perf.md` as a bilingual (中文/English) actual performance measurement report for v0.4.0 on the production VPS: 1KB broadcast p95 3.5 ms (8.6× headroom), 512KB p95 103 ms, idle CPU 0.08%, cold start 2 s; memory targets (P1/P2) recorded as unmet and flagged for revision; the 1000-connection scenario is documented as untestable on a 1.6 GB same-host environment after it exhausted memory and forced a VM reset. +- Added `tools/perf_probe.py`, a stdlib-only asyncio WSS probe used for the measurements (hold/latency/slow-consumer scenarios). +- Recorded two implementation findings in `docs/server-spec.md` §15: unbounded shutdown close-handshake wait (34 s stop phase observed) and silent queue-full abort without a disconnect security event. ## [0.4.0] - 2026-08-27 diff --git a/docs/server-spec.md b/docs/server-spec.md index f1354d7..50377e3 100644 --- a/docs/server-spec.md +++ b/docs/server-spec.md @@ -721,3 +721,7 @@ Kestrel TLS、结构化日志与脱敏、登录与消息限流、框架依赖单 5. `ServerHost.CreateApp(certificate:null)` 的明文测试缝隙（仅 InternalsVisibleTo 可达）。 6. ApplyClip 中 duplicateId 且 Latest 为 null 的兜底分支不可达（死分支）。 7. §10 中标注"待补齐/补齐中"的测试项以 specs/test-and-contract-spec.md 落地为准。 +8. 优雅停机对每个连接的 `CloseAsync` close 握手等待无超时：有静默客户端在线时，停机阶段实测可达 34 秒（perf.md S7）；§7 的"等待最多 2 秒"仅覆盖握手完成后的 drain。 +9. 发送队列满的熔断路径（`MarkClosed` + `Cts.Cancel`）不产生 disconnect 安全事件：后续 `CancelConnection` 因 `MarkClosed` 已置位而提前返回，被熔断的连接在日志中不可见（perf.md S6）。 + +实测性能数据见仓库根目录 [perf.md](../perf.md)。 diff --git a/perf.md b/perf.md index 7881521..28d8977 100644 --- a/perf.md +++ b/perf.md @@ -1,288 +1,191 @@ -# 性能测试规范 / Performance Test Specification +# 性能实测报告 / Performance Measurement Report -状态 / Status：测量契约与结果记录模板（Measurement contract and results template），v0.4.0 -日期 / Date：2026-08-27 +状态 / Status：v0.4.0 首次实测（/ First measured run on v0.4.0） +日期 / Date：2026-08-29（ measurements executed 2026-08-27 23:50 – 2026-08-29 01:50 CST） +测量工具 / Tooling：[tools/perf_probe.py](tools/perf_probe.py)（纯标准库 asyncio WSS 探针 / stdlib-only asyncio WSS probe） +关联 / Related：[docs/server-spec.md](docs/server-spec.md) §9、§15；[specs/spec-decisions.md](specs/spec-decisions.md) -> **说明 / Note**：基准压测程序（独立 Benchmark 项目）尚未实现；本文件先于工具存在，定义性能目标、场景、测量方法与结果记录格式，作为后续构建度量设施的契约。 -> **Note**：The standalone benchmark project does not exist yet. This document precedes the tooling and defines the performance targets, scenarios, measurement methodology, and the results format — the contract for building the measurement harness. +> 摘要 / Summary：延迟与 CPU 表现优秀（1KB 广播 p95 3.5ms，8.6 倍余量；空闲 CPU 0.08%）；冷启动 2 秒达标；内存目标（P1/P2）未达标——.NET 运行时基线与每连接真实成本决定了旧目标定得过紧；1000 并发在该 1.6GB 内存的同机环境不可测（触发宿主机硬重启）。另发现两个实现层问题（停机 close 握手无超时、队列满熔断无日志），已记入 spec §15。 +> **Summary**：Latency and CPU are excellent (1KB broadcast p95 3.5 ms — 8.6× headroom; idle CPU 0.08%); cold start meets the 2 s target; the memory targets (P1/P2) are not met — the .NET runtime baseline and the real per-connection cost show the old targets were too tight; 1000 concurrent connections is untestable on this 1.6 GB same-host environment (the VM hard-reset). Two implementation findings (unbounded shutdown close-handshake wait; silent queue-full abort without logging) are recorded in spec §15. --- ## 第一部分：中文 -### 1. 背景与目的 +### 1. 测试环境 -`docs/server-spec.md` 原第 9 节的性能指标表在 v0.4.0 规格对齐时移除，原因是项目从未建立度量手段，未度量数字等于虚假承诺。本文件把目标重新引入，但赋予其可执行的语义：每一项目标都绑定明确的场景、采样方法与判定标准；结果填入第 6 节模板，未填写的行即"未验证"。 +| 项 | 值 | +|---|---| +| 硬件 | 2 vCPU（Intel Xeon Platinum）/ 1.6 GB 内存（LongCloud VPS） | +| 系统 | Ubuntu 24.04.4 LTS，内核 6.8.0-63-generic | +| 运行时 | .NET 10.0.11（框架依赖单文件，`TextCascade.Server` v0.4.0+fb33861） | +| 服务形态 | systemd（`textcascade-server.service`），WSS + 生产自签证书，端口 8443 | +| 负载端 | 与服务同机，回环 127.0.0.1，`tools/perf_probe.py`（Python 3.12 asyncio） | +| 服务配置 | 默认值 + `[rate_limit] clip_burst/clip_tokens_per_second` 临时调至 5000/5000（仅延迟与慢消费者场景，测后已恢复） | -设计性质（目标为何可达，来自架构）： +同机回环排除了网络抖动，但负载端与被测端共享 2 个 vCPU 与内存——并发类场景（S5）受此制约。 -- 广播每用户仅做一次 UTF-8 序列化，同一份字节投递到所有连接； -- 每连接发送队列有界（默认 16 条），慢连接立即取消，内存上限可预测； -- 每用户一个 Channel 单消费者，无锁竞争路径； -- 空闲期固定开销：1 秒一次的心跳扫描器、RuntimeStateStore 每 5 秒脏检查刷盘、UserFileWatcher 每 30 秒轮询兜底。 +### 2. 结果总表 -### 2. 性能目标 +| # | 场景 | 指标 | 目标 | 实测 | 判定 | +|---|---|---|---|---|---| +| P1 | S1 基础内存 | RSS（新进程，60s 预热） | < 50 MB | **125–131 MB** | ✗ 未达标 | +| P2 | S2 100 空闲连接 | RSS 增量（5 分钟，全部存活） | < 20 MB | **+66 MB**（≈660 KB/连接） | ✗ 未达标 | +| P3 | S3 1KB 广播 | broadcast_lag p95（1000 样本） | < 30 ms | **3.5 ms**（p50 1.87 / p99 5.1 / max 11.8） | ✓ 达标（8.6× 余量） | +| P3b | S3 附带 | ack_rtt p95 | 参考 | 4.0 ms | — | +| P4 | S4 512KB 广播 | broadcast_lag p95（200 样本） | < 250 ms | **103.2 ms**（p50 87.3 / p99 138 / max 157） | ✓ 达标（2.4× 余量） | +| P5 | S8 空闲 CPU | 60 秒均值（两轮） | ≈ 0% | **0.08%**（5 ticks / 60 s） | ✓ 达标 | +| P6 | S7 冷启动 | 应用启动阶段（Started → listening） | < 2 s | **2 s**（三次重启一致） | ✓ 达标（临界） | +| P7 | 恢复窗口 | snapshot_window_seconds | 3 s | 配置常量，功能由集成测试覆盖 | —（不适用） | +| P8 | S5 1000 并发 | 10 分钟稳定性 | 无断连 | **未完成：~200 连接时 VM 崩溃硬重启** | ⚠ 环境不可测 | +| — | S6 慢消费者隔离 | A 的 p95（B 停读 45 秒） | 不受影响 | **7.0–7.9 ms**（基线 6–17 ms）；B 在 ~16 s 被静默熔断 | ✓ 隔离有效 | -| # | 指标 / Metric | 目标 / Target | -|---|---|---| -| P1 | 基础进程内存（无客户端，预热后 RSS） | < 50 MB | -| P2 | 100 个空闲连接内存增量（保持 5 分钟） | < 20 MB | -| P3 | 1KB 文本广播单向延迟 p95（同机回环） | < 30 ms | -| P4 | 512KB 文本广播单向延迟 p95（同机回环） | < 250 ms | -| P5 | 空闲 CPU 占用（60 秒均值） | ≈ 0%（心跳扫描、状态刷盘与用户表轮询除外） | -| P6 | 冷启动时间（进程拉起 → `/health` 返回 200） | < 2 s | -| P7 | 重启恢复窗口 | 固定 3 秒（`snapshot_window_seconds`，功能正确性由测试保障） | -| P8 | 1000 并发连接稳定性（保持 10 分钟） | 无断连、无错误日志、内存增量可解释 | +### 3. 各场景明细 -### 3. 测试环境要求 +**S1 基础内存**：三次重启后 RSS 分别为 127872 / 128256 / 131328 kB；运行 24 小时后为 143852 kB。基线由 .NET 10 运行时、Kestrel、TLS 栈与 22 个线程构成，新进程即为 ~125 MB，说明不是泄漏而是运行时基线。原 50 MB 目标对 ASP.NET Core 应用不现实（判为"目标过紧"而非"实现缺陷"）。 -每次记录结果必须附带环境描述，否则结果不可比： +**S2 100 空闲连接**：新进程 RSS 127872 kB → 195392 kB，增量 67520 kB（≈660 KB/连接）。期间 900/900 心跳 pong 全部响应，零错误。660 KB/连接包含 TLS 流缓冲、Kestrel 每连接管道与 pinned buffer、托管对象。原 20 MB 目标（200 KB/连接）低估了 Kestrel + TLS 的真实成本。 -- 硬件：CPU 型号与核数、内存容量（如 2 vCPU / 2 GB VPS）； -- 系统：OS 与内核版本； -- 运行时：.NET Runtime 版本、部署形态（框架依赖单文件）、`DOTNET_` 环境变量； -- 网络：回环（127.0.0.1）或真实局域网，WSS（生产 TLS）或 WS（仅诊断）； -- 负载端：与被测服务的相对位置（同机 / 另一主机）。 +**S3 1KB 广播延迟**：1000 样本全数回收。`broadcast_lag` p95 = 3.5 ms——服务端路径（解析 → 令牌桶 → 版本自增 → 单次序列化 → 双连接投递）加上两端 TLS 在回环上的开销远低于 30 ms 目标。 -参考环境建议：与生产部署一致（systemd + WSS + 自签证书），客户端与服务器同机回环，排除网络抖动。 +**S4 512KB 广播延迟**：200 样本全数回收。p95 103 ms，主要成本在 512KB JSON 的序列化/转义与两次 512KB TLS 记录写，2.4 倍余量达标。 -### 4. 测试场景 +**S5 1000 并发连接（未完成，有生产影响）**：进程基线 128256 kB 起步。SSH 会话在 ~200 连接时被重置，随后整机失去响应约 40 分钟（SSH banner 超时、8443 无响应），01:39 宿主机 watchdog 硬重启恢复。原因：同机负载端（Python 1000 并发 TLS 连接自身需数百 MB）+ 服务端（按 S2 外推 1000 连接 ≈ +660 MB）合计超出 1.6 GB 物理内存，触发内存耗尽。**生产影响披露**：期间真实用户设备约 40 分钟无法连接。结论：P8 需要跨机负载端或 ≥4 GB 内存的主机才能度量；按 S2 外推，服务端本身承载 1000 连接（+660 MB）在 2 GB 以上主机是可行的。 -| # | 场景 | 步骤 | 采样 | -|---|---|---|---| -| S1 | 基础内存 | 启动服务，无客户端，预热 60 秒后读 RSS | 单点 ×3 取中位 | -| S2 | 100 空闲连接 | 建 100 个 WSS 连接并发送合法 hello，保持 5 分钟 | 前后 RSS 差值 | -| S3 | 1KB 广播延迟 | 同用户 2 连接：A 发 1KB clip，A 记 ACK 往返，B 记广播单向滞后 | 预热丢弃 50，采样 ≥1000，间隔 20ms | -| S4 | 512KB 广播延迟 | 同 S3，payload 512KB | 预热丢弃 10，采样 ≥200，间隔 100ms | -| S5 | 1000 并发连接 | 建 1000 连接完成 hello，保持 10 分钟 | 全程 RSS/CPU 曲线 + 断连计数 | -| S6 | 慢消费者隔离 | A 每 20ms 发 4KB clip；B 建立后停止读 socket 10 秒 | A 的 p95 不受影响；B 应在队列满（16 条）后被断开 | -| S7 | 冷启动 | `systemctl restart`，从进程拉起到 `/health` 200 | 3 次取中位 | -| S8 | 空闲 CPU | 无客户端 60 秒，取 `ps -o %cpu` 均值 | 单点 ×3 取中位 | - -延迟指标定义： +**S6 慢消费者隔离**：32KB clip @ 50/s（1.6 MB/s）。基线 p95 17 ms（含 JIT 噪声），B 停读后 A 的 p95 稳定在 7.0–7.9 ms——完全隔离。B 在 **~16 秒**被服务端熔断断开（观测：established 3→2 且无 disconnect 日志；回环内核缓冲自调优 ~10 MB 吸收了初段流量，之后 16 条发送队列填满触发熔断）。两个发现： -- **ACK 往返（ack_rtt）**：A 端 `send(clip)` 前取时间戳，收到本连接 `clip_ack` 再取，差值即 RTT； -- **广播单向滞后（broadcast_lag）**：A 端 `send(clip)` 前取 `t0`，B 端收到含该 id 的 `clip` 帧取 `t1`，`t1 - t0` 即单向滞后（同机时钟，无偏差问题；跨主机需先做时钟校准或改测 ACK 往返的一半）。 +- **熔断静默**：队列满路径直接 `MarkClosed + Cts.Cancel`，后续 `CancelConnection` 因 `MarkClosed` 已置位而提前返回，不产生任何 disconnect 安全事件——被熔断的连接在日志中不可见（已记入 spec §15）。 +- **熔断延迟**：16 条队列的熔断点受内核 socket 缓冲（自动调优可达 ~10 MB）放大，取决于消息尺寸与速率，"队列满即断"在回环场景实际表现为"缓冲满即断"。 -P3/P4 以 `broadcast_lag` 的 p95 判定；`ack_rtt` 一并记录作参考。 +**S7 冷启动**：三次重启的应用启动阶段（journal `Started` → `Now listening`）均为 **2 秒**，达标但已贴线。另发现：`systemctl restart` 端到端耗时 **35.6 秒**（有真实客户端在线时）——旧实例的关闭阶段花了 34 秒，原因是 `ShutdownAsync` 对每个连接 `CloseAsync` 等待 close 握手完成且无超时，静默客户端会拖住整个停机流程；spec §7 的"等待最多 2 秒"只覆盖 close 握手完成后的 drain。已记入 spec §15。 -### 5. 执行方法 +**S8 空闲 CPU**：两轮 60 秒各 5 个时钟 tick（100 tick = 1 CPU 秒）→ 0.083% CPU。心跳扫描器（1 Hz）、状态刷盘（5 秒周期，空闲时无脏数据）、用户表轮询（30 秒周期）的固定开销可忽略。 -**内存与 CPU（现成工具即可）**： +### 4. 发现与后续 -```bash -# RSS（字节） -grep VmRSS /proc/$(systemctl show -p MainPID --value textcascade-server)/status -# systemd 视角内存 -systemctl status textcascade-server --no-pager | grep Memory -# 60 秒平均 CPU -ps -o %cpu= -p $(systemctl show -p MainPID --value textcascade-server) --sort=-start_time -# 更细的 GC/线程池观测（可选） -dotnet-counters monitor --process-id System.Runtime -``` +| # | 发现 | 影响 | 建议 | +|---|---|---|---| +| F1 | 停机关闭握手等待无超时（实测 34 秒） | 重启/升级时拖长停机窗口 | `CloseConnectionAsync` 的 `CloseAsync` 加超时（如 2 秒）后走 abort；已记入 spec §15 | +| F2 | 队列满熔断不产生 disconnect 日志 | 被熔断连接在安全日志中不可见 | 熔断路径补一条安全事件；已记入 spec §15 | +| F3 | P1/P2 内存目标过紧 | 目标不可达 | 修订目标为 P1 < 150 MB、P2 < 100 MB（100 连接），或立项做内存优化 | +| F4 | P8 在 1.6GB 同机环境不可测 | 无法验证 1000 并发 | 跨机负载端，或 ≥4 GB 主机重测 | -**冷启动**： +### 5. 复现步骤 ```bash -systemctl restart textcascade-server -# journalctl 中 "Started" 与 "Now listening on" 两条时间戳之差，或循环 curl /health 直到 200 -``` - -**延迟与并发负载**：独立压测程序尚未实现。过渡期任选其一： - -1. 通用 WebSocket 压测工具（如 k6）按 S3/S5 参数执行； -2. 一次性控制台脚本（C# `ClientWebSocket` 或 Python `websockets`），逻辑为：登录 → 建连 → hello → 定时发 clip → 记录 `clip_ack` 时间戳； -3. 实现 `TextCascade.Server.Benchmark` 控制台项目（持久方案，场景按第 4 节命名 S1–S8）。 - -k6 示例（S3 的 ACK 往返部分，TOKEN/主机替换后使用；广播滞后需第二个静态接收端）： - -```javascript -import ws from 'k6/ws'; -import { Trend } from 'k6/metrics'; - -const ackRtt = new Trend('ack_rtt_ms'); - -export default function () { - ws.connect('wss://HOST:8443/api/v1/sync', { headers: { Authorization: 'Bearer TOKEN' } }, (socket) => { - socket.on('open', () => { - socket.send(JSON.stringify({ type: 'hello', clientId: 'k6-a', clientName: 'k6', lastServerVersion: 0, snapshot: null })); - socket.on('message', (raw) => { - if (raw.includes('welcome')) { - for (let i = 0; i < 1000; i++) { - const t0 = Date.now(); - socket.send(JSON.stringify({ type: 'clip', id: 'c' + i, payload: 'x'.repeat(1024), encrypted: false, hash: 'h' + i })); - } - } else if (raw.includes('clip_ack')) { - ackRtt.add(Date.now() - Number(raw.match(/"id":"c(\d+)"/)[1]) * 20); - } - }); - }); - }); -} +# 1. 上传探针 +scp tools/perf_probe.py root@HOST:/tmp/ +# 2. 创建临时压测用户（测后删除） +/opt/textcascade-server/TextCascade.Server user add --username perftest --password-stdin \ + --config /etc/textcascade/textcascade.toml < <(echo PASSWORD) +# 3. 延迟场景（需要临时调高 [rate_limit]，测后恢复） +python3 /tmp/perf_probe.py --user perftest --password PASSWORD latency --size 1024 --count 1000 --interval 0.02 +python3 /tmp/perf_probe.py --user perftest --password PASSWORD latency --size 524288 --count 200 --interval 0.1 +# 4. 连接保持与 RSS 采样 +grep VmRSS /proc/$(systemctl show -p MainPID --value textcascade-server)/status +python3 /tmp/perf_probe.py --user perftest --password PASSWORD hold --count 100 --seconds 300 +# 5. 慢消费者 +python3 /tmp/perf_probe.py --user perftest --password PASSWORD slow --size 32768 --stall 45 +# 6. 清理 +/opt/textcascade-server/TextCascade.Server user delete --username perftest --config /etc/textcascade/textcascade.toml ``` -注意：示例按"每 20ms 发一条"的节奏需配合限速循环，实际脚本应使用 `setInterval` 或分批发送；以上仅示意数据采集点。 - -### 6. 结果记录模板 - -| 场景 | 指标 | 目标 | 实测 | 环境 | 日期 | 结论 | -|---|---|---|---|---|---|---| -| S1 | RSS 中位 | < 50 MB | TBD | TBD | TBD | ☐ | -| S2 | ΔRSS | < 20 MB | TBD | TBD | TBD | ☐ | -| S3 | broadcast_lag p95 | < 30 ms | TBD | TBD | TBD | ☐ | -| S3 | ack_rtt p95 | 参考 | TBD | TBD | TBD | ☐ | -| S4 | broadcast_lag p95 | < 250 ms | TBD | TBD | TBD | ☐ | -| S5 | 10 分钟稳定性 | 无断连 | TBD | TBD | TBD | ☐ | -| S6 | 慢连接隔离 | B 断开且 A p95 达标 | TBD | TBD | TBD | ☐ | -| S7 | 冷启动中位 | < 2 s | TBD | TBD | TBD | ☐ | -| S8 | 空闲 CPU 均值 | ≈ 0% | TBD | TBD | TBD | ☐ | - -填写规则：环境列引用第 3 节描述的编号；结论列在目标达成时打勾，未达标时在文末追加差距分析（原因 + 归属：实现 / 环境 / 目标本身）。 +注意：S5 类高并发场景勿在与服务同机的低内存主机上执行（见 S5 生产影响披露）。 -### 7. 已知限制 +### 6. 已知限制 -- 基准压测程序未实现，过渡期依赖通用工具或临时脚本，采样节奏精度受客户端定时器分辨率影响（Windows 默认 ~15ms）； -- P7 恢复窗口为配置常量，其"3 秒"语义已在集成测试中验证，本文件只做部署侧确认； -- 同机回环测量排除了网络抖动，跨主机结果不可直接与回环目标比较； -- RuntimeStateStore 刷盘（5 秒周期）与 UserFileWatcher 轮询（30 秒周期）是空闲开销的一部分，属预期行为而非回归。 +- 负载端与服务同机，S5 受内存与 CPU 共享制约；跨机测量结果不可与本次回环数值直接比较； +- 采样节奏受 Python asyncio 定时器影响（Linux 下精度远优于 Windows 的 15ms）； +- 基线 RSS 受 GC 策略影响，跨 .NET 版本可能漂移； +- rate_limit 临时调整仅在延迟/慢消费者场景使用，S1/S2/S5/S7/S8 均在默认配置下测量。 --- ## Part 2: English -### 1. Background and Purpose +### 1. Test Environment -The performance target table in the original `docs/server-spec.md` §9 was removed during the v0.4.0 spec alignment because the project never had measurement tooling — unmeasured numbers are empty promises. This document reintroduces the targets with executable semantics: every target is bound to a concrete scenario, sampling method, and pass criterion. Results go into the template in section 6; an unfilled row means "not verified". +| Item | Value | +|---|---| +| Hardware | 2 vCPU (Intel Xeon Platinum) / 1.6 GB RAM (LongCloud VPS) | +| OS | Ubuntu 24.04.4 LTS, kernel 6.8.0-63-generic | +| Runtime | .NET 10.0.11 (framework-dependent single file, `TextCascade.Server` v0.4.0+fb33861) | +| Service shape | systemd (`textcascade-server.service`), WSS + production self-signed certificate, port 8443 | +| Load generator | Same host, loopback 127.0.0.1, `tools/perf_probe.py` (Python 3.12 asyncio) | +| Service config | Defaults + `[rate_limit] clip_burst/clip_tokens_per_second` temporarily raised to 5000/5000 (latency and slow-consumer scenarios only; restored afterwards) | -Design properties that make the targets achievable (from the architecture): +Same-host loopback excludes network jitter, but the generator shares the 2 vCPUs and memory with the service — concurrency scenarios (S5) are constrained by this. -- Each broadcast is serialized to UTF-8 once per user; the same bytes are handed to every connection; -- Per-connection send queues are bounded (16 messages by default) and slow connections are cancelled immediately, keeping the memory ceiling predictable; -- One single-consumer Channel per user — no lock contention on the hot path; -- Fixed idle overhead: the 1-second heartbeat scanner, RuntimeStateStore dirty-check flush every 5 seconds, and the UserFileWatcher 30-second polling fallback. +### 2. Results Summary -### 2. Performance Targets +| # | Scenario | Metric | Target | Measured | Verdict | +|---|---|---|---|---|---| +| P1 | S1 base memory | RSS (fresh process, 60 s warmup) | < 50 MB | **125–131 MB** | ✗ fail | +| P2 | S2 100 idle connections | RSS delta (5 min, all alive) | < 20 MB | **+66 MB** (≈660 KB/conn) | ✗ fail | +| P3 | S3 1KB broadcast | broadcast_lag p95 (1000 samples) | < 30 ms | **3.5 ms** (p50 1.87 / p99 5.1 / max 11.8) | ✓ pass (8.6× headroom) | +| P3b | S3 companion | ack_rtt p95 | reference | 4.0 ms | — | +| P4 | S4 512KB broadcast | broadcast_lag p95 (200 samples) | < 250 ms | **103.2 ms** (p50 87.3 / p99 138 / max 157) | ✓ pass (2.4× headroom) | +| P5 | S8 idle CPU | 60 s average (two runs) | ≈ 0% | **0.08%** (5 ticks / 60 s) | ✓ pass | +| P6 | S7 cold start | Application start phase (Started → listening) | < 2 s | **2 s** (consistent across 3 restarts) | ✓ pass (borderline) | +| P7 | Recovery window | snapshot_window_seconds | 3 s | config constant; correctness covered by tests | — (n/a) | +| P8 | S5 1000 concurrent | 10-minute stability | no disconnects | **not completed — VM hard-reset at ~200 connections** | ⚠ untestable here | +| — | S6 slow-consumer isolation | A's p95 (B stalled 45 s) | unaffected | **7.0–7.9 ms** (baseline 6–17 ms); B silently aborted at ~16 s | ✓ isolation holds | -| # | Metric | Target | -|---|---|---| -| P1 | Base process memory (RSS after warmup, no clients) | < 50 MB | -| P2 | Memory delta with 100 idle connections (held 5 minutes) | < 20 MB | -| P3 | 1KB text broadcast one-way latency p95 (same-host loopback) | < 30 ms | -| P4 | 512KB text broadcast one-way latency p95 (same-host loopback) | < 250 ms | -| P5 | Idle CPU usage (60-second average) | ≈ 0% (excluding heartbeat scan, state flush, user-file polling) | -| P6 | Cold start (process spawn → `/health` returns 200) | < 2 s | -| P7 | Restart recovery window | Fixed 3 s (`snapshot_window_seconds`; correctness covered by tests) | -| P8 | 1000 concurrent connections stability (held 10 minutes) | No disconnects, no error logs, explainable memory delta | +### 3. Scenario Details -### 3. Environment Requirements +**S1 base memory**: RSS after three restarts was 127872 / 128256 / 131328 kB; 143852 kB after 24 h of uptime. The baseline consists of the .NET 10 runtime, Kestrel, the TLS stack, and 22 threads — a fresh process is already ~125 MB, so this is a runtime baseline, not a leak. The original 50 MB target is unrealistic for an ASP.NET Core app (judged "target too tight" rather than an implementation defect). -Every recorded result must include its environment description, otherwise results are not comparable: +**S2 100 idle connections**: fresh RSS 127872 kB → 195392 kB, delta 67520 kB (≈660 KB/connection). All 900/900 expected heartbeat pongs were answered with zero errors. The 660 KB/connection includes TLS stream buffers, Kestrel per-connection pipes and pinned buffers, and managed objects. The original 20 MB target (200 KB/connection) underestimated the real cost of Kestrel + TLS. -- Hardware: CPU model and core count, RAM (e.g. 2 vCPU / 2 GB VPS); -- OS: distribution and kernel version; -- Runtime: .NET Runtime version, deployment shape (framework-dependent single file), `DOTNET_*` environment variables; -- Network: loopback (127.0.0.1) or real LAN; WSS (production TLS) or WS (diagnostics only); -- Load generator: relative location to the service under test (same host / separate host). +**S3 1KB broadcast latency**: all 1000 samples recovered. `broadcast_lag` p95 = 3.5 ms — the server path (parse → token bucket → version increment → single serialization → delivery to two connections) plus TLS on both ends stays far below the 30 ms target. -Recommended reference environment: identical to production deployment (systemd + WSS + self-signed certificate), clients on the same host over loopback to exclude network jitter. +**S4 512KB broadcast latency**: all 200 samples recovered. p95 103 ms, dominated by serializing/escaping the 512KB JSON and two 512KB TLS record writes; 2.4× headroom, within target. -### 4. Scenarios +**S5 1000 concurrent connections (not completed; production impact)**: started from a fresh 128256 kB baseline. The SSH session was reset at ~200 connections; the whole machine then became unresponsive for ~40 minutes (SSH banner timeouts, no response on 8443) until the provider watchdog hard-reset the VM at 01:39. Root cause: a same-host generator (Python holding 1000 concurrent TLS connections itself needs several hundred MB) plus the server (extrapolating from S2, 1000 connections ≈ +660 MB) exceeded the 1.6 GB physical memory. **Production impact disclosure**: real user devices could not connect for ~40 minutes. Conclusion: P8 requires an off-host generator or a host with ≥4 GB RAM; extrapolating from S2, the server itself holding 1000 connections (+660 MB) is feasible on a 2 GB+ host. -| # | Scenario | Steps | Sampling | -|---|---|---|---| -| S1 | Base memory | Start service with no clients; read RSS after 60 s warmup | 3 runs, take median | -| S2 | 100 idle connections | Open 100 WSS connections with valid hello; hold 5 minutes | RSS delta before/after | -| S3 | 1KB broadcast latency | Same-user 2 connections: A sends 1KB clips; A records ACK round-trip, B records broadcast lag | Discard 50 warmup, ≥1000 samples at 20 ms interval | -| S4 | 512KB broadcast latency | Same as S3 with 512KB payload | Discard 10 warmup, ≥200 samples at 100 ms interval | -| S5 | 1000 concurrent connections | Open 1000 connections with hello; hold 10 minutes | Full RSS/CPU curve + disconnect count | -| S6 | Slow-consumer isolation | A sends 4KB clip every 20 ms; B stops reading its socket for 10 s | A's p95 unaffected; B disconnected after queue fills (16 messages) | -| S7 | Cold start | `systemctl restart`; measure from spawn to `/health` 200 | 3 runs, take median | -| S8 | Idle CPU | No clients for 60 s; average `ps -o %cpu` | 3 runs, take median | - -Latency metric definitions: +**S6 slow-consumer isolation**: 32KB clips @ 50/s (1.6 MB/s). Baseline p95 17 ms (includes JIT noise); with B stalled, A's p95 held at 7.0–7.9 ms — fully isolated. B was silently aborted by the server at **~16 s** (observed: established count 3→2 with no disconnect log; the autotuned ~10 MB loopback kernel buffers absorbed the initial burst, after which the 16-message send queue filled and triggered the abort). Two findings: -- **ACK round-trip (ack_rtt)**: timestamp before `send(clip)` on A, again when this connection's `clip_ack` arrives; the difference is the RTT; -- **Broadcast lag (broadcast_lag)**: `t0` before A sends the clip, `t1` when B receives the `clip` frame with that id; `t1 - t0` is the one-way lag (same-host clocks, no skew problem; across hosts, calibrate clocks first or measure half the ACK round-trip instead). +- **Silent abort**: the queue-full path calls `MarkClosed + Cts.Cancel` directly, and the subsequent `CancelConnection` returns early because `MarkClosed` is already set — no disconnect security event is produced, so aborted connections are invisible in the logs (recorded in spec §15). +- **Abort delay**: the 16-message queue trigger point is amplified by kernel socket buffers (autotuned up to ~10 MB) and therefore depends on message size and rate; on loopback, "queue full = disconnect" behaves as "buffers full = disconnect". -P3/P4 are judged on the p95 of `broadcast_lag`; `ack_rtt` is recorded alongside for reference. +**S7 cold start**: the application start phase (journal `Started` → `Now listening`) was **2 seconds** across three restarts — on target but borderline. Additional finding: end-to-end `systemctl restart` took **35.6 seconds** (with real clients connected) — the old instance's stop phase took 34 seconds because `ShutdownAsync` awaits each connection's `CloseAsync` close-handshake with no timeout, so silent clients stall the whole shutdown; spec §7's "wait up to 2 seconds" only covers the drain after the handshakes complete. Recorded in spec §15. -### 5. How to Run +**S8 idle CPU**: two 60-second runs, 5 clock ticks each (100 ticks = 1 CPU-second) → 0.083% CPU. The fixed overhead of the heartbeat scanner (1 Hz), state flush (5 s cycle, no dirty data while idle), and user-file polling (30 s cycle) is negligible. -**Memory and CPU (existing tools suffice)**: +### 4. Findings and Follow-ups -```bash -# RSS (bytes) -grep VmRSS /proc/$(systemctl show -p MainPID --value textcascade-server)/status -# systemd view of memory -systemctl status textcascade-server --no-pager | grep Memory -# 60-second average CPU -ps -o %cpu= -p $(systemctl show -p MainPID --value textcascade-server) -# Finer GC/thread-pool observation (optional) -dotnet-counters monitor --process-id System.Runtime -``` +| # | Finding | Impact | Recommendation | +|---|---|---|---| +| F1 | Shutdown close-handshake wait is unbounded (34 s measured) | Prolongs the stop window on restarts/upgrades | Add a timeout (e.g. 2 s) to `CloseConnectionAsync`'s `CloseAsync`, then abort; recorded in spec §15 | +| F2 | Queue-full abort produces no disconnect log | Aborted connections are invisible in security logs | Emit a security event on the abort path; recorded in spec §15 | +| F3 | P1/P2 memory targets are too tight | Targets unreachable | Revise to P1 < 150 MB and P2 < 100 MB (100 connections), or start a memory-optimization effort | +| F4 | P8 untestable on a 1.6 GB same-host environment | 1000 concurrent connections unverifiable | Off-host load generator, or retest on a ≥4 GB host | -**Cold start**: +### 5. Reproduction ```bash -systemctl restart textcascade-server -# Difference between the "Started" and "Now listening on" journal timestamps, -# or poll /health with curl until it returns 200. -``` - -**Latency and concurrent load**: the dedicated benchmark project does not exist yet. Until then, pick one: - -1. A general-purpose WebSocket load tool (e.g. k6) driven with the S3/S5 parameters; -2. A throwaway console script (C# `ClientWebSocket` or Python `websockets`): login → connect → hello → send clips on a timer → timestamp `clip_ack`; -3. Implement the `TextCascade.Server.Benchmark` console project (the durable option; scenarios named S1–S8 per section 4). - -k6 sketch (the ACK round-trip part of S3; replace TOKEN/HOST; broadcast lag needs a second static receiver): - -```javascript -import ws from 'k6/ws'; -import { Trend } from 'k6/metrics'; - -const ackRtt = new Trend('ack_rtt_ms'); - -export default function () { - ws.connect('wss://HOST:8443/api/v1/sync', { headers: { Authorization: 'Bearer TOKEN' } }, (socket) => { - socket.on('open', () => { - socket.send(JSON.stringify({ type: 'hello', clientId: 'k6-a', clientName: 'k6', lastServerVersion: 0, snapshot: null })); - socket.on('message', (raw) => { - if (raw.includes('welcome')) { - for (let i = 0; i < 1000; i++) { - const t0 = Date.now(); - socket.send(JSON.stringify({ type: 'clip', id: 'c' + i, payload: 'x'.repeat(1024), encrypted: false, hash: 'h' + i })); - } - } else if (raw.includes('clip_ack')) { - ackRtt.add(Date.now() - Number(raw.match(/"id":"c(\d+)"/)[1]) * 20); - } - }); - }); - }); -} +# 1. Upload the probe +scp tools/perf_probe.py root@HOST:/tmp/ +# 2. Create a temporary benchmark user (delete afterwards) +/opt/textcascade-server/TextCascade.Server user add --username perftest --password-stdin \ + --config /etc/textcascade/textcascade.toml < <(echo PASSWORD) +# 3. Latency scenarios (temporarily raise [rate_limit]; restore afterwards) +python3 /tmp/perf_probe.py --user perftest --password PASSWORD latency --size 1024 --count 1000 --interval 0.02 +python3 /tmp/perf_probe.py --user perftest --password PASSWORD latency --size 524288 --count 200 --interval 0.1 +# 4. Connection holds and RSS sampling +grep VmRSS /proc/$(systemctl show -p MainPID --value textcascade-server)/status +python3 /tmp/perf_probe.py --user perftest --password PASSWORD hold --count 100 --seconds 300 +# 5. Slow consumer +python3 /tmp/perf_probe.py --user perftest --password PASSWORD slow --size 32768 --stall 45 +# 6. Cleanup +/opt/textcascade-server/TextCascade.Server user delete --username perftest --config /etc/textcascade/textcascade.toml ``` -Note: the sketch assumes one clip every 20 ms; a real script should pace sends with `setInterval` or batching. It illustrates the data-collection points only. - -### 6. Results Template - -| Scenario | Metric | Target | Measured | Environment | Date | Pass | -|---|---|---|---|---|---|---| -| S1 | RSS median | < 50 MB | TBD | TBD | TBD | ☐ | -| S2 | ΔRSS | < 20 MB | TBD | TBD | TBD | ☐ | -| S3 | broadcast_lag p95 | < 30 ms | TBD | TBD | TBD | ☐ | -| S3 | ack_rtt p95 | reference | TBD | TBD | TBD | ☐ | -| S4 | broadcast_lag p95 | < 250 ms | TBD | TBD | TBD | ☐ | -| S5 | 10-minute stability | no disconnects | TBD | TBD | TBD | ☐ | -| S6 | slow-consumer isolation | B dropped, A p95 within target | TBD | TBD | TBD | ☐ | -| S7 | cold start median | < 2 s | TBD | TBD | TBD | ☐ | -| S8 | idle CPU average | ≈ 0% | TBD | TBD | TBD | ☐ | - -Filling rules: the Environment column references the description from section 3; tick the Pass column when the target is met, and append a gap analysis at the end of this file when it is not (root cause + attribution: implementation / environment / the target itself). +Caution: do not run S5-style high-concurrency scenarios on a low-memory host shared with the service (see the S5 production-impact disclosure). -### 7. Known Limitations +### 6. Known Limitations -- The benchmark project is not implemented; interim tooling depends on generic clients or throwaway scripts, and sampling cadence accuracy is bounded by client timer resolution (~15 ms by default on Windows); -- The P7 recovery window is a configuration constant whose 3-second semantics are already verified by integration tests; this document only confirms it from the deployment side; -- Same-host loopback measurements exclude network jitter; cross-host results must not be compared directly against the loopback targets; -- RuntimeStateStore flushing (5-second cycle) and UserFileWatcher polling (30-second cycle) are part of the idle overhead — expected behavior, not regressions. +- The generator shares the host with the service; S5 is constrained by memory and CPU, and cross-host results are not directly comparable to these loopback numbers; +- Sampling cadence is bounded by Python asyncio timer precision (far better on Linux than the 15 ms default on Windows); +- Baseline RSS depends on GC policy and may drift across .NET versions; +- The temporary rate_limit adjustment was used only for the latency/slow-consumer scenarios; S1/S2/S5/S7/S8 were measured under default configuration. diff --git a/tools/perf_probe.py b/tools/perf_probe.py new file mode 100644 index 0000000..260aec3 --- /dev/null +++ b/tools/perf_probe.py @@ -0,0 +1,362 @@ +#!/usr/bin/env python3 +"""TextCascade performance probe. + +Stdlib-only asyncio WebSocket (WSS) client used to measure the deployed +textcascade-server: idle connection holds, clip latency, and slow-consumer +isolation. See perf.md at the repository root for the scenario definitions. + +Subcommands: + hold open N connections, answer pings, hold for S seconds + latency 2 connections (sender A + receiver B), record ack_rtt/broadcast_lag + slow baseline window, then a stalled receiver, then measure A's p95 +""" +import argparse +import asyncio +import base64 +import json +import os +import ssl +import time +import urllib.request + +HOST = "127.0.0.1" +PORT = 8443 + + +def percentile(values, p): + if not values: + return None + vs = sorted(values) + k = min(len(vs) - 1, max(0, int(round(len(vs) * p)) - 1)) + return round(vs[k], 3) + + +def login(user, password): + ctx = ssl.create_default_context() + ctx.check_hostname = False + ctx.verify_mode = ssl.CERT_NONE + req = urllib.request.Request( + f"https://{HOST}:{PORT}/api/v1/login", + data=json.dumps({"username": user, "password": password}).encode(), + headers={"Content-Type": "application/json"}) + with urllib.request.urlopen(req, context=ctx, timeout=15) as resp: + return json.loads(resp.read())["token"] + + +class WS: + def __init__(self, reader, writer): + self.reader = reader + self.writer = writer + + async def send_text(self, text): + data = text.encode() + mask = os.urandom(4) + n = len(data) + if n < 126: + header = bytes([0x81, 0x80 | n]) + elif n < 65536: + header = bytes([0x81, 0x80 | 126]) + n.to_bytes(2, "big") + else: + header = bytes([0x81, 0x80 | 127]) + n.to_bytes(8, "big") + masked = bytes(b ^ mask[i % 4] for i, b in enumerate(data)) + self.writer.write(header + mask + masked) + await self.writer.drain() + + async def read_frame(self): + b1, b2 = await self.reader.readexactly(2) + opcode = b1 & 0x0F + masked = b2 & 0x80 + ln = b2 & 0x7F + if ln == 126: + ln = int.from_bytes(await self.reader.readexactly(2), "big") + elif ln == 127: + ln = int.from_bytes(await self.reader.readexactly(8), "big") + mask_key = await self.reader.readexactly(4) if masked else None + payload = await self.reader.readexactly(ln) if ln else b"" + if mask_key: + payload = bytes(b ^ mask_key[i % 4] for i, b in enumerate(payload)) + if opcode == 0x8: + raise ConnectionResetError("close frame") + return opcode, payload + + +async def connect(token, client_id): + ctx = ssl.create_default_context() + ctx.check_hostname = False + ctx.verify_mode = ssl.CERT_NONE + reader, writer = await asyncio.open_connection( + HOST, PORT, ssl=ctx, limit=2 ** 22) + key = base64.b64encode(os.urandom(16)).decode() + request = ( + f"GET /api/v1/sync HTTP/1.1\r\nHost: {HOST}:{PORT}\r\n" + "Upgrade: websocket\r\nConnection: Upgrade\r\n" + f"Sec-WebSocket-Key: {key}\r\nSec-WebSocket-Version: 13\r\n" + "Sec-WebSocket-Protocol: textcascade.v1\r\n" + f"Authorization: Bearer {token}\r\n\r\n") + writer.write(request.encode()) + await writer.drain() + response = await reader.readuntil(b"\r\n\r\n") + status_line = response.split(b"\r\n", 1)[0] + if b"101" not in status_line: + raise RuntimeError(f"handshake failed: {status_line!r}") + return WS(reader, writer) + + +async def hello(ws, client_id): + await ws.send_text(json.dumps({ + "type": "hello", "clientId": client_id, "clientName": "perf", + "lastServerVersion": 0, "snapshot": None})) + while True: + opcode, payload = await ws.read_frame() + if opcode == 1 and b'"welcome"' in payload: + return + + +def is_ping(payload): + return b'"type":"ping"' in payload + + +def pong_text(): + return json.dumps({ + "type": "pong", + "clientTimeUtc": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())}) + + +async def cmd_hold(args): + token = login(args.user, args.password) + connections = [] + errors = 0 + pongs = 0 + semaphore = asyncio.Semaphore(64) + + async def one(index): + nonlocal errors + async with semaphore: + try: + ws = await connect(token, f"hold-{index}") + await hello(ws, f"hold-{index}") + connections.append(ws) + except Exception: + errors += 1 + + await asyncio.gather(*(one(i) for i in range(args.count))) + + async def keepalive(ws): + nonlocal pongs + try: + while True: + opcode, payload = await ws.read_frame() + if opcode == 1 and is_ping(payload): + await ws.send_text(pong_text()) + pongs += 1 + except Exception: + pass + + tasks = [asyncio.create_task(keepalive(ws)) for ws in connections] + await asyncio.sleep(args.seconds) + + for ws in connections: + try: + ws.writer.close() + except Exception: + pass + for task in tasks: + task.cancel() + print(json.dumps({ + "opened": len(connections), "errors": errors, + "expected_pongs_floor": len(connections) * max(0, int(args.seconds / 30) - 1), + "pongs_sent": pongs})) + + +async def cmd_latency(args): + token = login(args.user, args.password) + + ws_b = await connect(token, "lat-b") + await hello(ws_b, "lat-b") + ws_a = await connect(token, "lat-a") + await hello(ws_a, "lat-a") + + send_ts = {} + acks = [] + lags = [] + pings = {"n": 0} + + async def reader_a(): + while True: + opcode, payload = await ws_a.read_frame() + if opcode != 1: + continue + if is_ping(payload): + await ws_a.send_text(pong_text()) + pings["n"] += 1 + continue + if b'"clip_ack"' in payload: + clip_id = json.loads(payload)["id"] + start = send_ts.get(clip_id) + if start is not None: + acks.append((time.monotonic_ns() - start) / 1e6) + + async def reader_b(): + while True: + opcode, payload = await ws_b.read_frame() + if opcode != 1: + continue + if is_ping(payload): + await ws_b.send_text(pong_text()) + continue + if b'"type":"clip"' in payload: + message = json.loads(payload) + start = send_ts.get(message["id"]) + if start is not None: + lags.append((time.monotonic_ns() - start) / 1e6) + + tasks = [asyncio.create_task(reader_a()), asyncio.create_task(reader_b())] + + payload_text = "x" * args.size + + async def send_wave(tag, count, interval, record): + for i in range(count): + clip_id = f"{tag}-{i}" + start = time.monotonic_ns() + if record: + send_ts[clip_id] = start + await ws_a.send_text(json.dumps({ + "type": "clip", "id": clip_id, "payload": payload_text, + "encrypted": False, "hash": "h"})) + if interval: + await asyncio.sleep(interval) + + await send_wave("warmup", args.warmup, 0.005, record=False) + await asyncio.sleep(0.5) + acks.clear() + lags.clear() + await send_wave("m", args.count, args.interval, record=True) + + deadline = time.monotonic() + 20 + while len(acks) < args.count and time.monotonic() < deadline: + await asyncio.sleep(0.05) + + for task in tasks: + task.cancel() + print(json.dumps({ + "size_bytes": args.size, + "samples_expected": args.count, + "ack_samples": len(acks), + "lag_samples": len(lags), + "ack_rtt_ms": {"p50": percentile(acks, 0.50), "p95": percentile(acks, 0.95), + "p99": percentile(acks, 0.99), "max": percentile(acks, 1.0)}, + "broadcast_lag_ms": {"p50": percentile(lags, 0.50), "p95": percentile(lags, 0.95), + "p99": percentile(lags, 0.99), "max": percentile(lags, 1.0)}})) + + +async def cmd_slow(args): + token = login(args.user, args.password) + + ws_a = await connect(token, "slow-a") + await hello(ws_a, "slow-a") + + send_ts = {} + acks = {"baseline": [], "stall": []} + + async def reader_a(): + while True: + opcode, payload = await ws_a.read_frame() + if opcode != 1: + continue + if is_ping(payload): + await ws_a.send_text(pong_text()) + continue + if b'"clip_ack"' in payload: + clip_id = json.loads(payload)["id"] + start = send_ts.get(clip_id) + if start is not None: + phase = "baseline" if clip_id.startswith("b-") else "stall" + acks[phase].append((time.monotonic_ns() - start) / 1e6) + + reader_task = asyncio.create_task(reader_a()) + payload_text = "x" * args.size + + async def send_wave(tag, seconds): + count = int(seconds / args.interval) + for i in range(count): + clip_id = f"{tag}-{i}" + send_ts[clip_id] = time.monotonic_ns() + await ws_a.send_text(json.dumps({ + "type": "clip", "id": clip_id, "payload": payload_text, + "encrypted": False, "hash": "h"})) + await asyncio.sleep(args.interval) + return count + + baseline_count = await send_wave("b", args.baseline) + + async def drain(minimum, timeout): + deadline = time.monotonic() + timeout + while len(acks["baseline"]) < minimum and time.monotonic() < deadline: + await asyncio.sleep(0.05) + + await drain(baseline_count, 10) + + # B joins, completes hello, then never reads again (stalled consumer). + ws_b = await connect(token, "slow-b") + await hello(ws_b, "slow-b") + + stall_count = await send_wave("s", args.stall) + deadline = time.monotonic() + 20 + while len(acks["stall"]) < stall_count and time.monotonic() < deadline: + await asyncio.sleep(0.05) + + reader_task.cancel() + + b_state = "unknown" + try: + opcode, _ = await asyncio.wait_for(ws_b.read_frame(), timeout=3) + b_state = f"received-frame-op-{opcode}" + except ConnectionResetError: + b_state = "connection-reset" + except asyncio.TimeoutError: + b_state = "still-open-no-data" + except Exception as exc: + b_state = f"error:{type(exc).__name__}" + + print(json.dumps({ + "clip_bytes": args.size, + "baseline": {"count": len(acks["baseline"]), + "p50": percentile(acks["baseline"], 0.50), + "p95": percentile(acks["baseline"], 0.95)}, + "stall": {"count": len(acks["stall"]), + "p50": percentile(acks["stall"], 0.50), + "p95": percentile(acks["stall"], 0.95)}, + "b_state": b_state})) + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--user", required=True) + parser.add_argument("--password", required=True) + sub = parser.add_subparsers(dest="command", required=True) + + hold = sub.add_parser("hold") + hold.add_argument("--count", type=int, required=True) + hold.add_argument("--seconds", type=float, required=True) + hold.set_defaults(func=cmd_hold) + + latency = sub.add_parser("latency") + latency.add_argument("--size", type=int, required=True) + latency.add_argument("--count", type=int, required=True) + latency.add_argument("--interval", type=float, required=True) + latency.add_argument("--warmup", type=int, default=50) + latency.set_defaults(func=cmd_latency) + + slow = sub.add_parser("slow") + slow.add_argument("--size", type=int, default=32768) + slow.add_argument("--interval", type=float, default=0.02) + slow.add_argument("--baseline", type=float, default=5) + slow.add_argument("--stall", type=float, default=20) + slow.set_defaults(func=cmd_slow) + + args = parser.parse_args() + asyncio.run(args.func(args)) + + +if __name__ == "__main__": + main() From d0f36f3636930ca032bbcd71e0c4a94920b40aba Mon Sep 17 00:00:00 2001 From: long45343 <1725334094@qq.com> Date: Sat, 29 Aug 2026 02:18:02 +0800 Subject: [PATCH 25/32] fix: Windows TLS with persisted certificate keys; perf.md Windows 1000-conn result - CertificateLoader: on Windows load PFX with DefaultKeySet instead of EphemeralKeySet (SChannel rejects ephemeral keys for server-side TLS, 0x8009030E); PEM certificates re-exported to a persisted key on Windows. Linux behavior unchanged. Spec 2.1 declares Windows Service hosting, which was unusable without this. - perf.md: S5 verified on local Windows (32 GB) - 1000 connections held for 10 minutes, 0 errors, all alive, RSS delta ~211 MB (~211 KB/conn, one third of the Linux figure). P8 now passes; findings table updated (F4 resolved, F5 recorded as fixed). --- CHANGELOG.md | 4 ++++ TextCascade.Server/ServerHost.cs | 14 +++++++++++++- perf.md | 22 ++++++++++++++++------ 3 files changed, 33 insertions(+), 7 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index fbab169..08e394b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,7 +7,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Fixed +- TLS server authentication failed on Windows with "platform does not support ephemeral keys" (0x8009030E): `CertificateLoader` now loads PFX certificates with a persisted key set (`DefaultKeySet`) on Windows (Linux keeps `EphemeralKeySet`), and PEM-loaded certificates are re-exported to a persisted key on Windows. Spec §2.1's Windows Service hosting shape required this to be usable. Found while benchmarking 1000 concurrent connections on a local Windows deployment. + ### Documentation +- perf.md: added the Windows local 1000-connection result — 10-minute hold with 0 errors and all 1000 connections alive, RSS delta ≈211 MB (≈211 KB/connection, one third of the Linux figure); P8 is now verified. - Rewrote `perf.md` as a bilingual (中文/English) actual performance measurement report for v0.4.0 on the production VPS: 1KB broadcast p95 3.5 ms (8.6× headroom), 512KB p95 103 ms, idle CPU 0.08%, cold start 2 s; memory targets (P1/P2) recorded as unmet and flagged for revision; the 1000-connection scenario is documented as untestable on a 1.6 GB same-host environment after it exhausted memory and forced a VM reset. - Added `tools/perf_probe.py`, a stdlib-only asyncio WSS probe used for the measurements (hold/latency/slow-consumer scenarios). - Recorded two implementation findings in `docs/server-spec.md` §15: unbounded shutdown close-handshake wait (34 s stop phase observed) and silent queue-full abort without a disconnect security event. diff --git a/TextCascade.Server/ServerHost.cs b/TextCascade.Server/ServerHost.cs index 104cd54..ece9e16 100644 --- a/TextCascade.Server/ServerHost.cs +++ b/TextCascade.Server/ServerHost.cs @@ -151,7 +151,12 @@ public static LoadedCertificate Load(string path) else if (path.EndsWith(".pfx", StringComparison.OrdinalIgnoreCase) || path.EndsWith(".p12", StringComparison.OrdinalIgnoreCase)) { - var chain = X509CertificateLoader.LoadPkcs12CollectionFromFile(path, password: null, X509KeyStorageFlags.EphemeralKeySet); + // SChannel cannot perform server-side TLS with an ephemeral key; keep the + // key persisted on Windows and ephemeral elsewhere (spec §2.1 supports both hosts). + var keyStorageFlags = OperatingSystem.IsWindows() + ? X509KeyStorageFlags.DefaultKeySet + : X509KeyStorageFlags.EphemeralKeySet; + var chain = X509CertificateLoader.LoadPkcs12CollectionFromFile(path, password: null, keyStorageFlags); var certificateWithKey = chain.Cast().FirstOrDefault(item => item.HasPrivateKey); if (certificateWithKey is null) { @@ -185,6 +190,13 @@ private static LoadedCertificate LoadPemCertificate(string certificatePath) } var certificateWithKey = X509Certificate2.CreateFromPemFile(certificatePath, keyPath); + if (OperatingSystem.IsWindows()) + { + // Re-import via a PFX export so the private key is persisted; SChannel + // rejects the ephemeral key produced by CreateFromPemFile on Windows. + using var ephemeral = certificateWithKey; + certificateWithKey = new X509Certificate2(certificateWithKey.Export(X509ContentType.Pfx)); + } var originalLeaf = chain.Cast().FirstOrDefault(c => c.Equals(certificateWithKey)); if (originalLeaf is not null) { diff --git a/perf.md b/perf.md index 28d8977..65ed393 100644 --- a/perf.md +++ b/perf.md @@ -5,8 +5,8 @@ 测量工具 / Tooling：[tools/perf_probe.py](tools/perf_probe.py)（纯标准库 asyncio WSS 探针 / stdlib-only asyncio WSS probe） 关联 / Related：[docs/server-spec.md](docs/server-spec.md) §9、§15；[specs/spec-decisions.md](specs/spec-decisions.md) -> 摘要 / Summary：延迟与 CPU 表现优秀（1KB 广播 p95 3.5ms，8.6 倍余量；空闲 CPU 0.08%）；冷启动 2 秒达标；内存目标（P1/P2）未达标——.NET 运行时基线与每连接真实成本决定了旧目标定得过紧；1000 并发在该 1.6GB 内存的同机环境不可测（触发宿主机硬重启）。另发现两个实现层问题（停机 close 握手无超时、队列满熔断无日志），已记入 spec §15。 -> **Summary**：Latency and CPU are excellent (1KB broadcast p95 3.5 ms — 8.6× headroom; idle CPU 0.08%); cold start meets the 2 s target; the memory targets (P1/P2) are not met — the .NET runtime baseline and the real per-connection cost show the old targets were too tight; 1000 concurrent connections is untestable on this 1.6 GB same-host environment (the VM hard-reset). Two implementation findings (unbounded shutdown close-handshake wait; silent queue-full abort without logging) are recorded in spec §15. +> 摘要 / Summary：延迟与 CPU 表现优秀（1KB 广播 p95 3.5ms，8.6 倍余量；空闲 CPU 0.08%）；冷启动 2 秒达标；内存目标（P1/P2）未达标——.NET 运行时基线与每连接真实成本决定了旧目标定得过紧；1000 并发在该 1.6GB 内存的同机环境不可测（触发宿主机硬重启）。另发现两个实现层问题（停机 close 握手无超时、队列满熔断无日志），已记入 spec §15。Windows 本机补测 1000 并发通过（10 分钟零断连、+211 MB），并顺带发现并修复了 Windows TLS 临时密钥缺陷。 +> **Summary**：Latency and CPU are excellent (1KB broadcast p95 3.5 ms — 8.6× headroom; idle CPU 0.08%); cold start meets the 2 s target; the memory targets (P1/P2) are not met — the .NET runtime baseline and the real per-connection cost show the old targets were too tight; 1000 concurrent connections is untestable on this 1.6 GB same-host environment (the VM hard-reset). Two implementation findings (unbounded shutdown close-handshake wait; silent queue-full abort without logging) are recorded in spec §15. A follow-up run on a local Windows machine (32 GB) passed the 1000-connection scenario (10 minutes, zero disconnects, +211 MB) and surfaced a Windows TLS ephemeral-key defect that has been fixed. --- @@ -37,7 +37,7 @@ | P5 | S8 空闲 CPU | 60 秒均值（两轮） | ≈ 0% | **0.08%**（5 ticks / 60 s） | ✓ 达标 | | P6 | S7 冷启动 | 应用启动阶段（Started → listening） | < 2 s | **2 s**（三次重启一致） | ✓ 达标（临界） | | P7 | 恢复窗口 | snapshot_window_seconds | 3 s | 配置常量，功能由集成测试覆盖 | —（不适用） | -| P8 | S5 1000 并发 | 10 分钟稳定性 | 无断连 | **未完成：~200 连接时 VM 崩溃硬重启** | ⚠ 环境不可测 | +| P8 | S5 1000 并发 | 10 分钟稳定性 | 无断连 | **Windows 本机通过**：0 错误、1000/1000 存活、+211 MB（Linux VPS 同机环境不可测，见 S5 明细） | ✓ 达标（Windows 32 GB） | | — | S6 慢消费者隔离 | A 的 p95（B 停读 45 秒） | 不受影响 | **7.0–7.9 ms**（基线 6–17 ms）；B 在 ~16 s 被静默熔断 | ✓ 隔离有效 | ### 3. 各场景明细 @@ -52,6 +52,10 @@ **S5 1000 并发连接（未完成，有生产影响）**：进程基线 128256 kB 起步。SSH 会话在 ~200 连接时被重置，随后整机失去响应约 40 分钟（SSH banner 超时、8443 无响应），01:39 宿主机 watchdog 硬重启恢复。原因：同机负载端（Python 1000 并发 TLS 连接自身需数百 MB）+ 服务端（按 S2 外推 1000 连接 ≈ +660 MB）合计超出 1.6 GB 物理内存，触发内存耗尽。**生产影响披露**：期间真实用户设备约 40 分钟无法连接。结论：P8 需要跨机负载端或 ≥4 GB 内存的主机才能度量；按 S2 外推，服务端本身承载 1000 连接（+660 MB）在 2 GB 以上主机是可行的。 +**S5 补测（Windows 本机，2026-08-29）**：环境为 Windows x64 / 32 GB RAM，openssl 自签无密码 PFX，二进制含 Windows TLS 修复（见 F5）。结果：1000 连接全部建立（~90 秒完成握手）、0 错误、10 分钟保持期间 19576/19000 心跳 pong 全响应（下限 19000 = 1000 连接 × 19 个周期，超出部分为时钟取整）；RSS 曲线 105 MB（基线）→ 稳态 262–285 MB → 结束 316 MB，**增量 ≈ 211 MB（≈211 KB/连接）**，仅为 Linux 实测值（660 KB/连接）的三分之一（Kestrel/TLS 缓冲策略差异 + GC 行为不同）。全程服务日志无任何错误。**P8 判定：通过。** + +顺带发现并修复 **F5（Windows TLS 缺陷）**：首次本地部署时 WSS 完全无法握手——`CertificateLoader` 的 `EphemeralKeySet` 在 Windows 上被 SChannel 拒绝（"platform does not support ephemeral keys"，0x8009030E），而 Linux/OpenSSL 不受影响，因此 VPS 部署从未暴露此问题。修复：Windows 上 PFX 使用 `DefaultKeySet`（持久密钥），PEM 加载后重导出为持久密钥；Linux 保持原状。spec §2.1 声明支持的 Windows Service 托管形态由此才真正可用。 + **S6 慢消费者隔离**：32KB clip @ 50/s（1.6 MB/s）。基线 p95 17 ms（含 JIT 噪声），B 停读后 A 的 p95 稳定在 7.0–7.9 ms——完全隔离。B 在 **~16 秒**被服务端熔断断开（观测：established 3→2 且无 disconnect 日志；回环内核缓冲自调优 ~10 MB 吸收了初段流量，之后 16 条发送队列填满触发熔断）。两个发现： - **熔断静默**：队列满路径直接 `MarkClosed + Cts.Cancel`，后续 `CancelConnection` 因 `MarkClosed` 已置位而提前返回，不产生任何 disconnect 安全事件——被熔断的连接在日志中不可见（已记入 spec §15）。 @@ -68,7 +72,8 @@ | F1 | 停机关闭握手等待无超时（实测 34 秒） | 重启/升级时拖长停机窗口 | `CloseConnectionAsync` 的 `CloseAsync` 加超时（如 2 秒）后走 abort；已记入 spec §15 | | F2 | 队列满熔断不产生 disconnect 日志 | 被熔断连接在安全日志中不可见 | 熔断路径补一条安全事件；已记入 spec §15 | | F3 | P1/P2 内存目标过紧 | 目标不可达 | 修订目标为 P1 < 150 MB、P2 < 100 MB（100 连接），或立项做内存优化 | -| F4 | P8 在 1.6GB 同机环境不可测 | 无法验证 1000 并发 | 跨机负载端，或 ≥4 GB 主机重测 | +| F4 | P8 在 1.6GB 同机环境不可测 | 无法验证 1000 并发 | 已解决：Windows 本机补测通过（见 S5 补测）；VPS 上仍建议跨机负载端 | +| F5 | Windows 上 `EphemeralKeySet` 导致 WSS 握手必然失败（0x8009030E） | spec §2.1 声明的 Windows Service 托管不可用 | 已修复：Windows 用 `DefaultKeySet`（PFX）+ PEM 重导出持久密钥；Linux 不变 | ### 5. 复现步骤 @@ -128,7 +133,7 @@ Same-host loopback excludes network jitter, but the generator shares the 2 vCPUs | P5 | S8 idle CPU | 60 s average (two runs) | ≈ 0% | **0.08%** (5 ticks / 60 s) | ✓ pass | | P6 | S7 cold start | Application start phase (Started → listening) | < 2 s | **2 s** (consistent across 3 restarts) | ✓ pass (borderline) | | P7 | Recovery window | snapshot_window_seconds | 3 s | config constant; correctness covered by tests | — (n/a) | -| P8 | S5 1000 concurrent | 10-minute stability | no disconnects | **not completed — VM hard-reset at ~200 connections** | ⚠ untestable here | +| P8 | S5 1000 concurrent | 10-minute stability | no disconnects | **passed on local Windows**: 0 errors, 1000/1000 alive, +211 MB (Linux VPS same-host environment untestable, see S5 details) | ✓ pass (Windows 32 GB) | | — | S6 slow-consumer isolation | A's p95 (B stalled 45 s) | unaffected | **7.0–7.9 ms** (baseline 6–17 ms); B silently aborted at ~16 s | ✓ isolation holds | ### 3. Scenario Details @@ -143,6 +148,10 @@ Same-host loopback excludes network jitter, but the generator shares the 2 vCPUs **S5 1000 concurrent connections (not completed; production impact)**: started from a fresh 128256 kB baseline. The SSH session was reset at ~200 connections; the whole machine then became unresponsive for ~40 minutes (SSH banner timeouts, no response on 8443) until the provider watchdog hard-reset the VM at 01:39. Root cause: a same-host generator (Python holding 1000 concurrent TLS connections itself needs several hundred MB) plus the server (extrapolating from S2, 1000 connections ≈ +660 MB) exceeded the 1.6 GB physical memory. **Production impact disclosure**: real user devices could not connect for ~40 minutes. Conclusion: P8 requires an off-host generator or a host with ≥4 GB RAM; extrapolating from S2, the server itself holding 1000 connections (+660 MB) is feasible on a 2 GB+ host. +**S5 follow-up run (local Windows, 2026-08-29)**: environment was Windows x64 / 32 GB RAM, an openssl self-signed passwordless PFX, and a binary containing the Windows TLS fix (see F5). Result: all 1000 connections established (handshakes completed within ~90 s), 0 errors, and during the 10-minute hold 19576/19000 heartbeat pongs were answered (floor = 1000 connections × 19 cycles). RSS curve: 105 MB (baseline) → 262–285 MB steady → 316 MB at the end — a **delta of ≈211 MB (≈211 KB/connection)**, one third of the Linux figure (660 KB/connection) due to different Kestrel/TLS buffer behavior and GC. The server log contained no errors. **P8 verdict: pass.** + +This run also surfaced and fixed **F5 (Windows TLS defect)**: the first local deployment could not complete a single WSS handshake — `CertificateLoader`'s `EphemeralKeySet` is rejected by SChannel on Windows ("platform does not support ephemeral keys", 0x8009030E), while Linux/OpenSSL is unaffected, which is why the VPS deployment never exposed it. Fix: on Windows the PFX branch now uses `DefaultKeySet` (persisted key) and PEM-loaded certificates are re-exported to a persisted key; Linux behavior is unchanged. Spec §2.1's declared Windows Service hosting shape is only truly usable with this fix. + **S6 slow-consumer isolation**: 32KB clips @ 50/s (1.6 MB/s). Baseline p95 17 ms (includes JIT noise); with B stalled, A's p95 held at 7.0–7.9 ms — fully isolated. B was silently aborted by the server at **~16 s** (observed: established count 3→2 with no disconnect log; the autotuned ~10 MB loopback kernel buffers absorbed the initial burst, after which the 16-message send queue filled and triggered the abort). Two findings: - **Silent abort**: the queue-full path calls `MarkClosed + Cts.Cancel` directly, and the subsequent `CancelConnection` returns early because `MarkClosed` is already set — no disconnect security event is produced, so aborted connections are invisible in the logs (recorded in spec §15). @@ -159,7 +168,8 @@ Same-host loopback excludes network jitter, but the generator shares the 2 vCPUs | F1 | Shutdown close-handshake wait is unbounded (34 s measured) | Prolongs the stop window on restarts/upgrades | Add a timeout (e.g. 2 s) to `CloseConnectionAsync`'s `CloseAsync`, then abort; recorded in spec §15 | | F2 | Queue-full abort produces no disconnect log | Aborted connections are invisible in security logs | Emit a security event on the abort path; recorded in spec §15 | | F3 | P1/P2 memory targets are too tight | Targets unreachable | Revise to P1 < 150 MB and P2 < 100 MB (100 connections), or start a memory-optimization effort | -| F4 | P8 untestable on a 1.6 GB same-host environment | 1000 concurrent connections unverifiable | Off-host load generator, or retest on a ≥4 GB host | +| F4 | P8 untestable on a 1.6 GB same-host environment | 1000 concurrent connections unverifiable | Resolved: passed on local Windows (see S5 follow-up); an off-host generator is still recommended for the VPS | +| F5 | `EphemeralKeySet` made WSS handshakes always fail on Windows (0x8009030E) | The Windows Service hosting shape declared in spec §2.1 was unusable | Fixed: `DefaultKeySet` for PFX on Windows + PEM re-export to a persisted key; Linux unchanged | ### 5. Reproduction From 19647e613b73f244eac72443393821ee85b45b3e Mon Sep 17 00:00:00 2001 From: long45343 <1725334094@qq.com> Date: Sat, 29 Aug 2026 02:23:50 +0800 Subject: [PATCH 26/32] docs: clarify S2 memory attribution - marginal cost is ~240 KB/conn A re-run of the 100-connection scenario on a JIT-warm process showed a +24 MB delta instead of +66 MB: the initial figure mixed in one-time JIT and GC heap expansion costs from the handshake storm. Marginal per- connection cost is ~240 KB on Linux vs 211 KB on Windows (same order); the Windows 211 KB figure benefited from amortization over 1000 connections. P2 revised to near-target; memory does not return to the OS after connections close (GC segments retained). --- perf.md | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/perf.md b/perf.md index 65ed393..cc8da66 100644 --- a/perf.md +++ b/perf.md @@ -30,7 +30,7 @@ | # | 场景 | 指标 | 目标 | 实测 | 判定 | |---|---|---|---|---|---| | P1 | S1 基础内存 | RSS（新进程，60s 预热） | < 50 MB | **125–131 MB** | ✗ 未达标 | -| P2 | S2 100 空闲连接 | RSS 增量（5 分钟，全部存活） | < 20 MB | **+66 MB**（≈660 KB/连接） | ✗ 未达标 | +| P2 | S2 100 空闲连接 | RSS 增量（5 分钟，全部存活） | < 20 MB | 首测 +66 MB（含 JIT/堆扩张一次性成本）；复测边际 **+24 MB**（≈240 KB/连接） | ◑ 边际接近达标 | | P3 | S3 1KB 广播 | broadcast_lag p95（1000 样本） | < 30 ms | **3.5 ms**（p50 1.87 / p99 5.1 / max 11.8） | ✓ 达标（8.6× 余量） | | P3b | S3 附带 | ack_rtt p95 | 参考 | 4.0 ms | — | | P4 | S4 512KB 广播 | broadcast_lag p95（200 样本） | < 250 ms | **103.2 ms**（p50 87.3 / p99 138 / max 157） | ✓ 达标（2.4× 余量） | @@ -46,6 +46,8 @@ **S2 100 空闲连接**：新进程 RSS 127872 kB → 195392 kB，增量 67520 kB（≈660 KB/连接）。期间 900/900 心跳 pong 全部响应，零错误。660 KB/连接包含 TLS 流缓冲、Kestrel 每连接管道与 pinned buffer、托管对象。原 20 MB 目标（200 KB/连接）低估了 Kestrel + TLS 的真实成本。 +**S2 复测（澄清 660 KB/连接的归因，2026-08-29）**：在已运行 32 分钟、代码已完全 JIT 的进程上重跑同一场景，增量仅 **+24 MB（≈240 KB/连接）**，且连接关闭后内存不回落（GC 段保留，三个 ~20 MB 匿名段）。结论：初次测得的 660 KB/连接混合了一次性成本——100 个连接的握手风暴触发的 JIT 编译与 GC 堆首次扩张（约 40-50 MB）——而非每连接真实成本；真实的**边际**每连接成本约为 240 KB（Linux）与 211 KB（Windows）同量级。Windows 显示"更低"主要是分母效应：211 KB 摊在 1000 个连接上，而 Linux 的 66 MB 摊在 100 个上。修订后的 P2 判定：**边际成本达标（240 KB/连接 vs 目标 200 KB/连接，差 20%）**，首次连接风暴的瞬时峰值超出目标。 + **S3 1KB 广播延迟**：1000 样本全数回收。`broadcast_lag` p95 = 3.5 ms——服务端路径（解析 → 令牌桶 → 版本自增 → 单次序列化 → 双连接投递）加上两端 TLS 在回环上的开销远低于 30 ms 目标。 **S4 512KB 广播延迟**：200 样本全数回收。p95 103 ms，主要成本在 512KB JSON 的序列化/转义与两次 512KB TLS 记录写，2.4 倍余量达标。 @@ -71,7 +73,7 @@ |---|---|---|---| | F1 | 停机关闭握手等待无超时（实测 34 秒） | 重启/升级时拖长停机窗口 | `CloseConnectionAsync` 的 `CloseAsync` 加超时（如 2 秒）后走 abort；已记入 spec §15 | | F2 | 队列满熔断不产生 disconnect 日志 | 被熔断连接在安全日志中不可见 | 熔断路径补一条安全事件；已记入 spec §15 | -| F3 | P1/P2 内存目标过紧 | 目标不可达 | 修订目标为 P1 < 150 MB、P2 < 100 MB（100 连接），或立项做内存优化 | +| F3 | P1 内存目标过紧（.NET 运行时基线 125-131 MB） | P1 不可达 | 修订 P1 为 < 150 MB；P2 按边际成本 240 KB/连接 基本达标，保留观察 | | F4 | P8 在 1.6GB 同机环境不可测 | 无法验证 1000 并发 | 已解决：Windows 本机补测通过（见 S5 补测）；VPS 上仍建议跨机负载端 | | F5 | Windows 上 `EphemeralKeySet` 导致 WSS 握手必然失败（0x8009030E） | spec §2.1 声明的 Windows Service 托管不可用 | 已修复：Windows 用 `DefaultKeySet`（PFX）+ PEM 重导出持久密钥；Linux 不变 | @@ -126,7 +128,7 @@ Same-host loopback excludes network jitter, but the generator shares the 2 vCPUs | # | Scenario | Metric | Target | Measured | Verdict | |---|---|---|---|---|---| | P1 | S1 base memory | RSS (fresh process, 60 s warmup) | < 50 MB | **125–131 MB** | ✗ fail | -| P2 | S2 100 idle connections | RSS delta (5 min, all alive) | < 20 MB | **+66 MB** (≈660 KB/conn) | ✗ fail | +| P2 | S2 100 idle connections | RSS delta (5 min, all alive) | < 20 MB | first run +66 MB (incl. one-time JIT/heap growth); re-run marginal **+24 MB** (≈240 KB/conn) | ◑ marginal near-target | | P3 | S3 1KB broadcast | broadcast_lag p95 (1000 samples) | < 30 ms | **3.5 ms** (p50 1.87 / p99 5.1 / max 11.8) | ✓ pass (8.6× headroom) | | P3b | S3 companion | ack_rtt p95 | reference | 4.0 ms | — | | P4 | S4 512KB broadcast | broadcast_lag p95 (200 samples) | < 250 ms | **103.2 ms** (p50 87.3 / p99 138 / max 157) | ✓ pass (2.4× headroom) | @@ -142,6 +144,8 @@ Same-host loopback excludes network jitter, but the generator shares the 2 vCPUs **S2 100 idle connections**: fresh RSS 127872 kB → 195392 kB, delta 67520 kB (≈660 KB/connection). All 900/900 expected heartbeat pongs were answered with zero errors. The 660 KB/connection includes TLS stream buffers, Kestrel per-connection pipes and pinned buffers, and managed objects. The original 20 MB target (200 KB/connection) underestimated the real cost of Kestrel + TLS. +**S2 re-measurement (clarifying the 660 KB/connection attribution, 2026-08-29)**: re-running the same scenario on a process that had been up for 32 minutes with fully-JITted code showed a delta of only **+24 MB (≈240 KB/connection)**, and memory did not return after the connections closed (GC segments retained; three ~20 MB anonymous segments). Conclusion: the initially measured 660 KB/connection mixed in one-time costs — the JIT compilation and first-time GC heap expansion triggered by the 100-connection handshake storm (roughly 40–50 MB) — rather than the true per-connection cost. The real **marginal** per-connection cost is ≈240 KB (Linux) vs 211 KB (Windows), the same order. Windows "looking lower" is mostly a denominator effect: 211 KB spread over 1000 connections vs 66 MB spread over 100. Revised P2 verdict: **marginal cost meets the target within 20%** (240 KB vs 200 KB per connection); the transient peak of a first connection storm exceeds it. + **S3 1KB broadcast latency**: all 1000 samples recovered. `broadcast_lag` p95 = 3.5 ms — the server path (parse → token bucket → version increment → single serialization → delivery to two connections) plus TLS on both ends stays far below the 30 ms target. **S4 512KB broadcast latency**: all 200 samples recovered. p95 103 ms, dominated by serializing/escaping the 512KB JSON and two 512KB TLS record writes; 2.4× headroom, within target. @@ -167,7 +171,7 @@ This run also surfaced and fixed **F5 (Windows TLS defect)**: the first local de |---|---|---|---| | F1 | Shutdown close-handshake wait is unbounded (34 s measured) | Prolongs the stop window on restarts/upgrades | Add a timeout (e.g. 2 s) to `CloseConnectionAsync`'s `CloseAsync`, then abort; recorded in spec §15 | | F2 | Queue-full abort produces no disconnect log | Aborted connections are invisible in security logs | Emit a security event on the abort path; recorded in spec §15 | -| F3 | P1/P2 memory targets are too tight | Targets unreachable | Revise to P1 < 150 MB and P2 < 100 MB (100 connections), or start a memory-optimization effort | +| F3 | P1 memory target too tight (.NET runtime baseline is 125–131 MB) | P1 unreachable | Revise P1 to < 150 MB; P2 essentially meets target at 240 KB/connection marginal cost, keep observing | | F4 | P8 untestable on a 1.6 GB same-host environment | 1000 concurrent connections unverifiable | Resolved: passed on local Windows (see S5 follow-up); an off-host generator is still recommended for the VPS | | F5 | `EphemeralKeySet` made WSS handshakes always fail on Windows (0x8009030E) | The Windows Service hosting shape declared in spec §2.1 was unusable | Fixed: `DefaultKeySet` for PFX on Windows + PEM re-export to a persisted key; Linux unchanged | From 740dae5d523f6fb9059953056edb5f67c401f393 Mon Sep 17 00:00:00 2001 From: long45343 <1725334094@qq.com> Date: Sat, 29 Aug 2026 02:28:33 +0800 Subject: [PATCH 27/32] docs: S5 root cause - zero-swap memory-pressure freeze, not steady-state OOM Persistent journal from the previous boot shows all 1000 upgrade requests arriving in one second against a 40-second-old server process, journald memory-pressure warnings from 00:50:18, journal silence at 00:51:53, and a watchdog reset at 01:38:58 - with no OOM-kill records and swap=0. Steady-state demand (1000 x 240 KB) fit comfortably; the crash was the transient storm peak causing a kernel reclaim livelock. --- perf.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/perf.md b/perf.md index cc8da66..ef02f41 100644 --- a/perf.md +++ b/perf.md @@ -52,7 +52,7 @@ **S4 512KB 广播延迟**：200 样本全数回收。p95 103 ms，主要成本在 512KB JSON 的序列化/转义与两次 512KB TLS 记录写，2.4 倍余量达标。 -**S5 1000 并发连接（未完成，有生产影响）**：进程基线 128256 kB 起步。SSH 会话在 ~200 连接时被重置，随后整机失去响应约 40 分钟（SSH banner 超时、8443 无响应），01:39 宿主机 watchdog 硬重启恢复。原因：同机负载端（Python 1000 并发 TLS 连接自身需数百 MB）+ 服务端（按 S2 外推 1000 连接 ≈ +660 MB）合计超出 1.6 GB 物理内存，触发内存耗尽。**生产影响披露**：期间真实用户设备约 40 分钟无法连接。结论：P8 需要跨机负载端或 ≥4 GB 内存的主机才能度量；按 S2 外推，服务端本身承载 1000 连接（+660 MB）在 2 GB 以上主机是可行的。 +**S5 1000 并发连接（未完成，有生产影响）**：进程基线 128256 kB 起步。SSH 会话在风暴中约 2 分钟后被重置，随后整机冻结，01:38:58 宿主机 watchdog 硬重启恢复。**根因（据上一启动周期持久日志）**：00:48:59 一秒内全部 1000 个 WebSocket 升级请求同时打到刚重启 40 秒的冷进程；00:50:18 起 journald 连续输出 "Under memory pressure, flushing caches"；00:51:53 日志彻底静默（整机冻结 47 分钟）。内核日志**无任何 OOM kill 记录**，机器 **swap = 0**。结论：不是稳态内存不足——1000 × 240 KB（复测边际成本）≈ 240 MB 完全容纳得下——而是**风暴瞬时峰值**击穿了零 swap 环境：冷进程 JIT + 1000 个并发 TLS 握手的叠加分配 + 同机 Python 负载端自身数百 MB + 升级日志洪流，导致内核回收活锁（.NET 运行时的文件映射页被反复逐出重读），冻结到 OOM killer 根本来不及工作。**生产影响披露**：期间真实用户设备约 47 分钟无法连接。本环境通过 P8 的可行路径：预热后的服务 + 缓慢升速（如 20 连接/秒）+ 临时 swap，或跨机负载端。 **S5 补测（Windows 本机，2026-08-29）**：环境为 Windows x64 / 32 GB RAM，openssl 自签无密码 PFX，二进制含 Windows TLS 修复（见 F5）。结果：1000 连接全部建立（~90 秒完成握手）、0 错误、10 分钟保持期间 19576/19000 心跳 pong 全响应（下限 19000 = 1000 连接 × 19 个周期，超出部分为时钟取整）；RSS 曲线 105 MB（基线）→ 稳态 262–285 MB → 结束 316 MB，**增量 ≈ 211 MB（≈211 KB/连接）**，仅为 Linux 实测值（660 KB/连接）的三分之一（Kestrel/TLS 缓冲策略差异 + GC 行为不同）。全程服务日志无任何错误。**P8 判定：通过。** @@ -150,7 +150,7 @@ Same-host loopback excludes network jitter, but the generator shares the 2 vCPUs **S4 512KB broadcast latency**: all 200 samples recovered. p95 103 ms, dominated by serializing/escaping the 512KB JSON and two 512KB TLS record writes; 2.4× headroom, within target. -**S5 1000 concurrent connections (not completed; production impact)**: started from a fresh 128256 kB baseline. The SSH session was reset at ~200 connections; the whole machine then became unresponsive for ~40 minutes (SSH banner timeouts, no response on 8443) until the provider watchdog hard-reset the VM at 01:39. Root cause: a same-host generator (Python holding 1000 concurrent TLS connections itself needs several hundred MB) plus the server (extrapolating from S2, 1000 connections ≈ +660 MB) exceeded the 1.6 GB physical memory. **Production impact disclosure**: real user devices could not connect for ~40 minutes. Conclusion: P8 requires an off-host generator or a host with ≥4 GB RAM; extrapolating from S2, the server itself holding 1000 connections (+660 MB) is feasible on a 2 GB+ host. +**S5 1000 concurrent connections (not completed; production impact)**: started from a fresh 128256 kB baseline. The SSH session was reset ~2 minutes into the storm; the machine then froze until the provider watchdog hard-reset it at 01:38:58. **Root cause (from the previous boot's persistent journal)**: all 1000 WebSocket upgrade requests arrived within the same second (00:48:59) against a server restarted only 40 s earlier; from 00:50:18 journald repeatedly logged "Under memory pressure, flushing caches"; at 00:51:53 the journal went silent (machine frozen for 47 minutes). The kernel log contains **no OOM-kill records** and the machine has **swap = 0**. Conclusion: steady-state memory was not the problem — 1000 × 240 KB (the re-measured marginal cost) ≈ 240 MB fits comfortably — it was the **transient storm peak** that broke a zero-swap environment: cold-process JIT + overlapping allocations of 1000 concurrent TLS handshakes + several hundred MB for the same-host Python generator + the upgrade-request log flood caused a kernel reclaim livelock (the .NET runtime's file-backed pages were endlessly evicted and re-faulted); the machine froze before the OOM killer could act. **Production impact disclosure**: real user devices could not connect for ~47 minutes. Viable paths to pass P8 in this environment: a warmed-up server, a slow ramp (e.g. 20 connections/s), a temporary swap file, or an off-host generator. **S5 follow-up run (local Windows, 2026-08-29)**: environment was Windows x64 / 32 GB RAM, an openssl self-signed passwordless PFX, and a binary containing the Windows TLS fix (see F5). Result: all 1000 connections established (handshakes completed within ~90 s), 0 errors, and during the 10-minute hold 19576/19000 heartbeat pongs were answered (floor = 1000 connections × 19 cycles). RSS curve: 105 MB (baseline) → 262–285 MB steady → 316 MB at the end — a **delta of ≈211 MB (≈211 KB/connection)**, one third of the Linux figure (660 KB/connection) due to different Kestrel/TLS buffer behavior and GC. The server log contained no errors. **P8 verdict: pass.** From b9134922e5c7063766ee014e54ff0cd023307ed1 Mon Sep 17 00:00:00 2001 From: long45343 <1725334094@qq.com> Date: Wed, 2 Sep 2026 20:46:04 +0800 Subject: [PATCH 28/32] =?UTF-8?q?=E6=96=87=E6=A1=A3=E6=9B=B4=E6=96=B0?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- CHANGELOG.md | 3 +- README.md | 12 ++++--- docs/server-spec.md | 78 +++++++++++++++++++++++++++++++++++---------- perf.md | 48 +++++++++++++--------------- 4 files changed, 92 insertions(+), 49 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 08e394b..bd5ec3b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -87,7 +87,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added - Initial import and baseline release of TextCascade Server with Minimal API, Kestrel WebSocket, Argon2 password hashing, and token authentication. -[Unreleased]: https://github.com/long45343/TextCascade-Server/compare/v0.3.5...HEAD +[Unreleased]: https://github.com/long45343/TextCascade-Server/compare/v0.4.0...HEAD +[0.4.0]: https://github.com/long45343/TextCascade-Server/compare/v0.3.5...v0.4.0 [0.3.5]: https://github.com/long45343/TextCascade-Server/compare/v0.3.0...v0.3.5 [0.3.0]: https://github.com/long45343/TextCascade-Server/compare/v0.2.5...v0.3.0 [0.2.5]: https://github.com/long45343/TextCascade-Server/compare/v0.2.1...v0.2.5 diff --git a/README.md b/README.md index 8eaed00..00e7d30 100644 --- a/README.md +++ b/README.md @@ -47,6 +47,8 @@ TextCascade-Server/ │ ├── Auth.cs / AuthService.cs token 签发、校验、登录限流 │ ├── Users.cs / Cli.cs 用户文件与 CLI(add/passwd/...) │ ├── RuntimeConfig.cs TOML 配置与默认值、环境变量覆盖 +│ ├── RuntimeStateStore.cs 版本号落盘(textcascade.state.json) +│ ├── SecurityLogging.cs 结构化安全日志与脱敏 │ └── Core.cs 限流、去重环形队列等基础工具 ├── TextCascade.Server.Tests/ xUnit 测试 ├── deploy/ systemd unit、示例 TOML 与空 users.json @@ -115,6 +117,7 @@ clip_tokens_per_second = 2 [files] users_file = "users.json" +state_file = "textcascade.state.json" ``` 关键规则: @@ -159,7 +162,7 @@ GitHub Release 提供两种 Framework-dependent 单文件包,目标机需预装 包内附带主程序、配置模板;Linux 包另附 systemd unit。每次 Release 同时提供 SHA-256 校验文件。 -推送 `v*.*.*` 标签(如 `v0.3.0`)会自动执行测试、构建双平台单文件包、生成校验和并发布 GitHub Release。`main` 分支和 Pull Request 会自动执行 restore/build/test CI。 +推送 `v*.*.*` 标签(如 `v0.4.0`)会自动执行测试、构建双平台单文件包、生成校验和并发布 GitHub Release。`main` 分支和 Pull Request 会自动执行 restore/build/test CI。 ### 生产部署(systemd) 参考 `deploy/textcascade-server.service`: @@ -170,7 +173,7 @@ GitHub Release 提供两种 Framework-dependent 单文件包,目标机需预装 ### 许可 -见仓库 LICENSE(如有)。 +见仓库 [LICENSE](LICENSE)(GPL-3.0)。 --- @@ -271,7 +274,6 @@ clip_tokens_per_second = 2 [files] users_file = "users.json" state_file = "textcascade.state.json" -state_file = "textcascade.state.json" ``` Key rules: @@ -316,7 +318,7 @@ GitHub Releases provides two framework-dependent single-file archives. The .NET Each archive contains the executable and config template; the Linux archive also includes the systemd unit. Every Release includes a SHA-256 checksum file. -Pushing a `v*.*.*` tag (for example `v0.3.0`) runs tests, builds both single-file archives, generates checksums, and publishes a GitHub Release. Pushes to `main` and pull requests run restore/build/test CI automatically. +Pushing a `v*.*.*` tag (for example `v0.4.0`) runs tests, builds both single-file archives, generates checksums, and publishes a GitHub Release. Pushes to `main` and pull requests run restore/build/test CI automatically. ### Production (systemd) See `deploy/textcascade-server.service`: @@ -327,7 +329,7 @@ See `deploy/textcascade-server.service`: ### License -See the repository LICENSE if present. +See [LICENSE](LICENSE) (GPL-3.0). ### Release & Maintenance Guidelines diff --git a/docs/server-spec.md b/docs/server-spec.md index 50377e3..58fb8ee 100644 --- a/docs/server-spec.md +++ b/docs/server-spec.md @@ -1,7 +1,7 @@ # TextCascade 轻量文本同步服务端规格 -状态：已按实现对齐 v0.3.5（commit 9ed6eba）修订；测试与契约细节以 specs/test-and-contract-spec.md 为准 -日期：2026-08-27（2026-08-18 定稿版漂移对齐，漂移溯源见 git 提交记录） +状态：与 v0.4.0 实现对齐；测试与契约细节以 specs/test-and-contract-spec.md 为准 +日期：2026-09-02 协议目标：不兼容原 ClipCascade，只做轻量、可靠、高性能的文本最新值同步 ## 1. 目标与非目标 @@ -25,10 +25,39 @@ ## 2. 已定架构 +整体结构与数据流： + +```mermaid +flowchart LR + C["客户端（桌面 / Android）"] + K["Kestrel（TLS 终结）"] + SE["SyncEndpoint：升级前验 token、子协议协商"] + RL["ReadLoopAsync（每连接）：收帧、解析、验证"] + subgraph HUB["UserHub（每在线用户一个）"] + CH["用户 Channel（无界）"] + UL["RunUserLoopAsync（单消费者）"] + LT["LatestText（不可变替换）"] + end + SL["ConnectionSendLoopAsync（每连接，有界发送队列）"] + RS[("RuntimeStateStore：版本落盘")] + FW["UserFileWatcher：users.json 热加载"] + HS["HeartbeatScannerService（1 Hz）"] + REG["UserRegistry"] + C -->|"WSS · textcascade.v1"| K --> SE --> RL + RL -->|"用户级 job"| CH + CH --> UL --> LT + UL -->|"单次序列化，除发送方外广播"| SL + SL --> C + REG --- HUB + FW -.->|"查找表原子替换"| REG + UL -.->|"脏位，5 秒周期刷盘"| RS + HS -.->|"ping 调度、hello 与心跳超时判定"| SE +``` + ### 2.1 运行时与进程 - 技术栈：ASP.NET Core Minimal API + Kestrel 原生 WebSocket。 -- 目标框架：`net10.0`；产品版本采用 SemVer，写入 `TextCascade.Server.csproj` 的 `Version`（当前 0.3.5）。 +- 目标框架：`net10.0`；产品版本采用 SemVer，写入 `TextCascade.Server.csproj` 的 `Version`（当前 0.4.0）。 - 进程模型：单进程；生产环境由 systemd 或 Windows Service 托管并负责崩溃自动重启。 - TLS：Kestrel 直接终止 TLS；不提供生产/开发模式开关，所有部署都禁止明文 HTTP 登录。TLS 协议版本跟随 OS 默认策略，未显式固定下限（见 8.2 与差距台账）。 - 部署产物：框架依赖单文件；目标机必须预装对应 .NET Runtime。 @@ -108,7 +137,7 @@ state_file = "textcascade.state.json" - token secret 必须由环境变量提供，长度至少 32 字节；缺失或过短时启动失败。 - TLS 始终启用；`certificate_path` 必须指向服务端可用证书。 - 证书支持无密码格式：`.pem` / `.crt` 必须是包含叶证书与未加密私钥的 PEM bundle（允许同名 `.key` 边车文件承载私钥）；`.pfx` / `.p12` 必须可无密码加载。遇到需要密码的 PFX 时启动失败。 -- TOML 使用宽松解析：必须以 UTF-8 读取（非 UTF-8 字节 fail-fast）；未知键忽略并输出 warning；结构或类型非法 fail-fast。**重复键视为解析错误直接启动失败**（Tomlyn 语义，与早期"取后值并告警"的设计不同）。 +- TOML 使用宽松解析：必须以 UTF-8 读取（非 UTF-8 字节 fail-fast）；未知键忽略并输出 warning；结构或类型非法 fail-fast。**重复键视为解析错误直接启动失败**（Tomlyn 语义）。 - `max_frame_bytes` 必须大于 `max_text_bytes`，差额留给 JSON 协议头。 - 所有容量与时间配置必须大于 0，心跳超时必须大于心跳间隔。 - 校验顺序：Load → EnvironmentOverrides → ValidateConfig。 @@ -177,7 +206,7 @@ TextCascade.Server serve # 启动服务（Program.cs - 格式：`{"entries":[{"username":"alice","version":129}, ...]}`。 - 写入时机：`PeriodicTimer` 每 5 秒对脏数据原子快照落盘；优雅停机时同步 flush；每次 clip 成功（`SaveVersion`）标记脏位。`SaveVersion` 采用单调 max 合并，防止乱序回退。 - 启动行为：`GetOrCreateHub` 用 `GetVersion(username)` 作为 hub 初始版本；状态文件结构非法（重复键、空 username、零版本）fail-fast。 -- 该机制使重启后版本号跨重启单调增长，不再从 1 重新计数（完整语义见 6.2）。 +- 该机制使重启后版本号跨重启单调增长（完整语义见 6.2）。 ## 4. HTTP API @@ -290,6 +319,25 @@ GET /health （亦响应 HEAD /health） ## 5. WebSocket 协议 +常态连接生命周期（升级前已完成 Bearer token 验证与子协议协商）： + +```mermaid +sequenceDiagram + participant C as 客户端 + participant S as 服务端 + C->>S: hello（clientId、clientName、lastServerVersion、snapshot） + S->>C: welcome（protocolVersion、latest 可省略） + C->>S: clip（id、payload、encrypted、hash） + S->>S: 幂等检查 → 令牌桶 → 版本自增 + S->>C: clip_ack（id、version） + S->>C: clip 广播（除发送方连接） + loop 每 heartbeat_interval_seconds（默认 30 秒） + S->>C: ping（serverTimeUtc） + C->>S: pong（clientTimeUtc） + end + S-->>C: bye（reason=server_shutdown）+ close 1001（停机场景） +``` + ### 5.1 连接建立 ```http @@ -397,7 +445,7 @@ Upgrade: websocket 实现：`Protocol.ValidateClipMessage` 单函数按结构→语义→资源顺序早拒绝；`CheckFrameSize` 帧硬限制；`CheckPayloadSize` 文本限额；`SeenIdRing.IsUnchangedDuplicate/TryGetResult/RememberId` 幂等；`TokenBucket.TryAcquire` 用户级令牌桶；`CoreLogic.NextVersion` ulong 自增（溢出抛出触发 RebuildHub）。 -幂等规则（v0.2.5 起语义细化）： +幂等规则： - `id` 已见过 **且 payload/hash/encrypted 与上次完全一致**：不生成新版本、不消耗令牌桶，返回原版本 ACK；重复 ACK 仍进入发送方有界发送队列，队列满时按慢连接取消。 - `id` 已见过但内容不同：记录 "Replacing reused clip id" warning 后**按全新消息处理**——消耗令牌桶、生成新版本并覆盖最新值。客户端不应复用已确认过的 id。 @@ -599,8 +647,6 @@ hub 清理： | clip | username, version, clipId, bytes, fromClientId, encrypted | | reject | username, code, bytes | -（历史版本的 `durationMs` 字段与 `server_stop` 事件从未实现/已不存在，不再列为要求。） - ### 8.2 传输与输入安全 - 生产只允许 HTTPS/WSS：`ServerHost.RunServer` 强制先加载证书，Kestrel 仅绑定单一 HTTPS endpoint。TLS 协议版本跟随 OS 默认策略，代码未显式设置 SslProtocols 下限；NetworkIntegration 测试以显式 Tls12/Tls13 客户端握手验证兼容性。 @@ -612,11 +658,11 @@ hub 清理： ## 9. 性能目标 -原文的性能指标表（内存、广播 p95、冷启动等）在本轮修订中移出：项目此前从未建立度量设施。性能目标、测量场景与结果模板现由仓库根目录的 [perf.md](../perf.md)（中英双语）承载，作为构建基准设施的契约；未验证的目标不作为承诺。（协议层面保留的设计性质：广播单次 UTF-8 序列化、每连接有界发送队列、空闲路径只有心跳扫描与周期刷盘。） +性能目标、测量场景与实测结果由仓库根目录的 [perf.md](../perf.md)（中英双语）承载，作为构建基准设施的契约；未验证的目标不作为承诺。（协议层面保留的设计性质：广播单次 UTF-8 序列化、每连接有界发送队列、空闲路径只有心跳扫描与周期刷盘。） ## 10. 测试计划 -详细到函数层面的规格见 [specs/test-and-contract-spec.md](test-and-contract-spec.md)，本节描述现状与分层。集成测试机制自 v0.3.0 起采用真实 Kestrel 绑定 `127.0.0.1:0` 的 fixture（`ServerHost.CreateApp` 构建 + FastPasswordHasher 注入），不再使用内存 socket 对。 +详细到函数层面的规格见 [specs/test-and-contract-spec.md](test-and-contract-spec.md)，本节描述现状与分层。集成测试机制自 v0.3.0 起采用真实 Kestrel 绑定 `127.0.0.1:0` 的 fixture（`ServerHost.CreateApp` 构建 + FastPasswordHasher 注入）。 ### 10.1 纯单元测试（现有覆盖） @@ -624,7 +670,7 @@ hub 清理： - CLI PID 单实例锁：活跃互斥、陈旧 PID 回收、存活进程不回收、锁路径校验。 - `SlidingWindowLoginLimiter`：双维度、跨 IP、成功仅清用户窗口、max keys、过期清理。 - `TryAcquireClipToken`（TokenBucket refill）、`CheckFrameSize`/`CheckPayloadSize`、SeenIdRing 去重与淘汰、`NextVersion` 含 ulong.MaxValue 抛出、`SelectSnapshotWinner` 三规则。 -- 待补齐缺口（Argon2 三函数、token 数字全形态、CLI 水位/溢出、WithVersion、重复 id 行为级断言）已定义于 test-and-contract-spec §3，落地前不构成本节的承诺范围。 +- Argon2 三函数（SlowHash）、token 数字全形态、CLI 水位/溢出、WithVersion、重复 id 行为级断言均已由测试覆盖（函数级规格见 test-and-contract-spec §3）。 ### 10.2 CI 集成测试：真实 Kestrel loopback @@ -644,8 +690,6 @@ CI 默认排除（ci.yml 过滤参数见 test-and-contract-spec 实施清单） 样本文件组织于 Tests 项目 `ContractSamples/`（valid/invalid 分类、非法数字与非法 UTF-8 全矩阵、深度 4、重复/未知字段），由 Theory 驱动断言 `ParseClientMessage` 结果与序列化字节不变式；样本文件同时作为三端实现的公共对拍集合。明细见 test-and-contract-spec §2。 -（独立 Benchmark 项目与压测场景已从规格移除，理由见第 9 节。） - ## 11. 客户端适配要求 客户端需要实现： @@ -686,24 +730,24 @@ CI 默认排除（ci.yml 过滤参数见 test-and-contract-spec 实施清单） ### M4：生产化 —— 大体达成，两项移交差距台账 -Kestrel TLS、结构化日志与脱敏、登录与消息限流、框架依赖单文件发布、systemd/发布管线均已落地；Benchmark 项目与性能指标未实施且已从规格移除（见第 9 节）。 +Kestrel TLS、结构化日志与脱敏、登录与消息限流、框架依赖单文件发布、systemd/发布管线均已落地；性能基准由 perf.md 实测承载（见第 9 节）。 ## 13. 版本与发布 -- 产品版本采用 SemVer 2.0.0，从 `0.1.0` 开始演进，以 `TextCascade.Server.csproj` 的 `Version` 为准（当前 0.3.5）。 +- 产品版本采用 SemVer 2.0.0，从 `0.1.0` 开始演进，以 `TextCascade.Server.csproj` 的 `Version` 为准（当前 0.4.0）。 - `protocolVersion` 只表示线协议版本，当前为 `1`，与产品版本独立演进。 - 目标框架为 `net10.0`；目标机必须预装兼容的 .NET 10 Runtime。 - 发布命令：`dotnet publish TextCascade.Server.csproj -c Release -p:PublishSingleFile=true`（win-x64/linux-x64 框架依赖单文件）。 - 本地编译命令：`dotnet build TextCascade.Server.csproj -c Release`。 -## 14. 决策台账（更新于 2026-08-27） +## 14. 决策台账 | 问题 | 结论 | |---|---| | 最大文本默认值 | 512KB | | 最新值文本本体磁盘持久化 | 不做，文本只在内存 | | 版本号持久化 | 做：v0.2.5 起 RuntimeStateStore 落盘 textcascade.state.json，版本跨重启单调 | -| 用户配置热加载 | 做：v0.3.0 起 UserFileWatcher 监听 + 轮询兜底，新认证即刻生效（取代早期"重启生效"决策） | +| 用户配置热加载 | 做：v0.3.0 起 UserFileWatcher 监听 + 轮询兜底，新认证即刻生效 | | token 生命周期 | 长期 token + tokenVersion 撤销 | | 删除后重建用户 | 全局 nextTokenVersion 水位 | | metrics | 不启用 endpoint | diff --git a/perf.md b/perf.md index ef02f41..f4938e8 100644 --- a/perf.md +++ b/perf.md @@ -1,12 +1,12 @@ # 性能实测报告 / Performance Measurement Report -状态 / Status：v0.4.0 首次实测（/ First measured run on v0.4.0） -日期 / Date：2026-08-29（ measurements executed 2026-08-27 23:50 – 2026-08-29 01:50 CST） +状态 / Status：v0.4.0 实测报告 / Measured on v0.4.0 +日期 / Date：2026-08-29 / Measurements executed 2026-08-27 23:50 – 2026-08-29 01:50 CST 测量工具 / Tooling：[tools/perf_probe.py](tools/perf_probe.py)（纯标准库 asyncio WSS 探针 / stdlib-only asyncio WSS probe） 关联 / Related：[docs/server-spec.md](docs/server-spec.md) §9、§15；[specs/spec-decisions.md](specs/spec-decisions.md) -> 摘要 / Summary：延迟与 CPU 表现优秀（1KB 广播 p95 3.5ms，8.6 倍余量；空闲 CPU 0.08%）；冷启动 2 秒达标；内存目标（P1/P2）未达标——.NET 运行时基线与每连接真实成本决定了旧目标定得过紧；1000 并发在该 1.6GB 内存的同机环境不可测（触发宿主机硬重启）。另发现两个实现层问题（停机 close 握手无超时、队列满熔断无日志），已记入 spec §15。Windows 本机补测 1000 并发通过（10 分钟零断连、+211 MB），并顺带发现并修复了 Windows TLS 临时密钥缺陷。 -> **Summary**：Latency and CPU are excellent (1KB broadcast p95 3.5 ms — 8.6× headroom; idle CPU 0.08%); cold start meets the 2 s target; the memory targets (P1/P2) are not met — the .NET runtime baseline and the real per-connection cost show the old targets were too tight; 1000 concurrent connections is untestable on this 1.6 GB same-host environment (the VM hard-reset). Two implementation findings (unbounded shutdown close-handshake wait; silent queue-full abort without logging) are recorded in spec §15. A follow-up run on a local Windows machine (32 GB) passed the 1000-connection scenario (10 minutes, zero disconnects, +211 MB) and surfaced a Windows TLS ephemeral-key defect that has been fixed. +> 摘要 / Summary：延迟与 CPU 表现优秀（1KB 广播 p95 3.5ms，8.6 倍余量；空闲 CPU 0.08%）；冷启动 2 秒达标；内存目标（P1/P2）未达标——.NET 运行时基线与每连接真实成本决定了旧目标定得过紧；1000 并发在 1.6GB 内存的生产 VPS 同机环境不可行（触发宿主机硬重启），在 Windows 本机（32GB）通过（10 分钟零断连、+211 MB）。另发现两个实现层问题（停机 close 握手无超时、队列满熔断无日志），已记入 spec §15；测量过程中发现并修复了 Windows TLS 临时密钥缺陷。 +> Summary：Latency and CPU are excellent (1KB broadcast p95 3.5 ms — 8.6× headroom; idle CPU 0.08%); cold start meets the 2 s target; the memory targets (P1/P2) are not met — the .NET runtime baseline and the real per-connection cost show the old targets were too tight; 1000 concurrent connections is not feasible on the 1.6 GB production VPS with a same-host generator (the VM hard-reset), and passes on a local Windows machine (32 GB) (10 minutes, zero disconnects, +211 MB). Two implementation findings (unbounded shutdown close-handshake wait; silent queue-full abort without logging) are recorded in spec §15; the measurements also surfaced a Windows TLS ephemeral-key defect that has been fixed. --- @@ -30,40 +30,38 @@ | # | 场景 | 指标 | 目标 | 实测 | 判定 | |---|---|---|---|---|---| | P1 | S1 基础内存 | RSS（新进程，60s 预热） | < 50 MB | **125–131 MB** | ✗ 未达标 | -| P2 | S2 100 空闲连接 | RSS 增量（5 分钟，全部存活） | < 20 MB | 首测 +66 MB（含 JIT/堆扩张一次性成本）；复测边际 **+24 MB**（≈240 KB/连接） | ◑ 边际接近达标 | +| P2 | S2 100 空闲连接 | RSS 增量（5 分钟，全部存活） | < 20 MB | 新进程 +66 MB（含一次性 JIT/堆扩张）；边际 **+24 MB**（≈240 KB/连接） | ◑ 边际接近达标 | | P3 | S3 1KB 广播 | broadcast_lag p95（1000 样本） | < 30 ms | **3.5 ms**（p50 1.87 / p99 5.1 / max 11.8） | ✓ 达标（8.6× 余量） | | P3b | S3 附带 | ack_rtt p95 | 参考 | 4.0 ms | — | | P4 | S4 512KB 广播 | broadcast_lag p95（200 样本） | < 250 ms | **103.2 ms**（p50 87.3 / p99 138 / max 157） | ✓ 达标（2.4× 余量） | | P5 | S8 空闲 CPU | 60 秒均值（两轮） | ≈ 0% | **0.08%**（5 ticks / 60 s） | ✓ 达标 | | P6 | S7 冷启动 | 应用启动阶段（Started → listening） | < 2 s | **2 s**（三次重启一致） | ✓ 达标（临界） | | P7 | 恢复窗口 | snapshot_window_seconds | 3 s | 配置常量，功能由集成测试覆盖 | —（不适用） | -| P8 | S5 1000 并发 | 10 分钟稳定性 | 无断连 | **Windows 本机通过**：0 错误、1000/1000 存活、+211 MB（Linux VPS 同机环境不可测，见 S5 明细） | ✓ 达标（Windows 32 GB） | +| P8 | S5 1000 并发 | 10 分钟稳定性 | 无断连 | **Windows 本机通过**：0 错误、1000/1000 存活、+211 MB（1.6 GB 生产 VPS 同机环境不可行，见 S5 明细） | ✓ 达标（Windows 32 GB） | | — | S6 慢消费者隔离 | A 的 p95（B 停读 45 秒） | 不受影响 | **7.0–7.9 ms**（基线 6–17 ms）；B 在 ~16 s 被静默熔断 | ✓ 隔离有效 | ### 3. 各场景明细 **S1 基础内存**：三次重启后 RSS 分别为 127872 / 128256 / 131328 kB；运行 24 小时后为 143852 kB。基线由 .NET 10 运行时、Kestrel、TLS 栈与 22 个线程构成，新进程即为 ~125 MB，说明不是泄漏而是运行时基线。原 50 MB 目标对 ASP.NET Core 应用不现实（判为"目标过紧"而非"实现缺陷"）。 -**S2 100 空闲连接**：新进程 RSS 127872 kB → 195392 kB，增量 67520 kB（≈660 KB/连接）。期间 900/900 心跳 pong 全部响应，零错误。660 KB/连接包含 TLS 流缓冲、Kestrel 每连接管道与 pinned buffer、托管对象。原 20 MB 目标（200 KB/连接）低估了 Kestrel + TLS 的真实成本。 - -**S2 复测（澄清 660 KB/连接的归因，2026-08-29）**：在已运行 32 分钟、代码已完全 JIT 的进程上重跑同一场景，增量仅 **+24 MB（≈240 KB/连接）**，且连接关闭后内存不回落（GC 段保留，三个 ~20 MB 匿名段）。结论：初次测得的 660 KB/连接混合了一次性成本——100 个连接的握手风暴触发的 JIT 编译与 GC 堆首次扩张（约 40-50 MB）——而非每连接真实成本；真实的**边际**每连接成本约为 240 KB（Linux）与 211 KB（Windows）同量级。Windows 显示"更低"主要是分母效应：211 KB 摊在 1000 个连接上，而 Linux 的 66 MB 摊在 100 个上。修订后的 P2 判定：**边际成本达标（240 KB/连接 vs 目标 200 KB/连接，差 20%）**，首次连接风暴的瞬时峰值超出目标。 +**S2 100 空闲连接**：新进程 RSS 127872 kB → 195392 kB，增量 67520 kB（≈660 KB/连接）；在已运行 32 分钟、代码完全 JIT 的进程上，同一场景的边际增量仅 +24 MB（≈240 KB/连接）。两者差额来自一次性成本——100 个连接的握手风暴触发的 JIT 编译与 GC 堆首次扩张（约 40-50 MB）——每连接的真实成本是边际值：约 240 KB（Linux），Windows 为 211 KB，同一量级（Windows 数值更低主要是分母效应：211 KB 摊在 1000 个连接上，而 66 MB 摊在 100 个上）。每连接成本由 TLS 流缓冲、Kestrel 每连接管道与 pinned buffer、托管对象构成；连接关闭后内存不回落（GC 段保留，三个 ~20 MB 匿名段）。期间 900/900 心跳 pong 全部响应，零错误。判定：边际成本达标（240 KB/连接 vs 目标 200 KB/连接，差 20%），首次连接风暴的瞬时峰值超出目标；原 20 MB 总量目标（200 KB/连接）低估了 Kestrel + TLS 的真实成本。 **S3 1KB 广播延迟**：1000 样本全数回收。`broadcast_lag` p95 = 3.5 ms——服务端路径（解析 → 令牌桶 → 版本自增 → 单次序列化 → 双连接投递）加上两端 TLS 在回环上的开销远低于 30 ms 目标。 **S4 512KB 广播延迟**：200 样本全数回收。p95 103 ms，主要成本在 512KB JSON 的序列化/转义与两次 512KB TLS 记录写，2.4 倍余量达标。 -**S5 1000 并发连接（未完成，有生产影响）**：进程基线 128256 kB 起步。SSH 会话在风暴中约 2 分钟后被重置，随后整机冻结，01:38:58 宿主机 watchdog 硬重启恢复。**根因（据上一启动周期持久日志）**：00:48:59 一秒内全部 1000 个 WebSocket 升级请求同时打到刚重启 40 秒的冷进程；00:50:18 起 journald 连续输出 "Under memory pressure, flushing caches"；00:51:53 日志彻底静默（整机冻结 47 分钟）。内核日志**无任何 OOM kill 记录**，机器 **swap = 0**。结论：不是稳态内存不足——1000 × 240 KB（复测边际成本）≈ 240 MB 完全容纳得下——而是**风暴瞬时峰值**击穿了零 swap 环境：冷进程 JIT + 1000 个并发 TLS 握手的叠加分配 + 同机 Python 负载端自身数百 MB + 升级日志洪流，导致内核回收活锁（.NET 运行时的文件映射页被反复逐出重读），冻结到 OOM killer 根本来不及工作。**生产影响披露**：期间真实用户设备约 47 分钟无法连接。本环境通过 P8 的可行路径：预热后的服务 + 缓慢升速（如 20 连接/秒）+ 临时 swap，或跨机负载端。 +**S5 1000 并发连接**：在 Windows x64 / 32 GB 本机通过——openssl 自签无密码 PFX，1000 连接全部建立（约 90 秒完成握手）、0 错误、10 分钟保持期间 19576/19000 心跳 pong 全响应（下限 19000 = 1000 连接 × 19 个周期，超出部分为时钟取整）；RSS 曲线 105 MB（基线）→ 稳态 262–285 MB → 结束 316 MB，增量 ≈211 MB（≈211 KB/连接），与 Linux 的边际成本 240 KB/连接同量级（见 S2）。全程服务日志无任何错误。P8 判定：通过。 -**S5 补测（Windows 本机，2026-08-29）**：环境为 Windows x64 / 32 GB RAM，openssl 自签无密码 PFX，二进制含 Windows TLS 修复（见 F5）。结果：1000 连接全部建立（~90 秒完成握手）、0 错误、10 分钟保持期间 19576/19000 心跳 pong 全响应（下限 19000 = 1000 连接 × 19 个周期，超出部分为时钟取整）；RSS 曲线 105 MB（基线）→ 稳态 262–285 MB → 结束 316 MB，**增量 ≈ 211 MB（≈211 KB/连接）**，仅为 Linux 实测值（660 KB/连接）的三分之一（Kestrel/TLS 缓冲策略差异 + GC 行为不同）。全程服务日志无任何错误。**P8 判定：通过。** +同一场景在 1.6 GB 内存的生产 VPS（负载端同机）不可行：00:48:59 一秒内全部 1000 个 WebSocket 升级请求同时打到刚重启 40 秒的冷进程，约 2 分钟后 SSH 会话被重置，整机冻结至 01:38:58 宿主机 watchdog 硬重启（冻结 47 分钟）。据上一启动周期的持久日志，journald 自 00:50:18 起连续输出 "Under memory pressure, flushing caches"，00:51:53 彻底静默；内核日志无任何 OOM kill 记录，机器 swap = 0。稳态内存并非瓶颈——1000 × 240 KB（边际成本）≈ 240 MB 完全容纳得下——击穿零 swap 环境的是风暴瞬时峰值：冷进程 JIT、1000 个并发 TLS 握手的叠加分配、同机 Python 负载端自身数百 MB、升级日志洪流，导致内核回收活锁（.NET 运行时的文件映射页被反复逐出重读），冻结到 OOM killer 来不及工作。生产影响披露：期间真实用户设备约 47 分钟无法连接。在该环境执行 P8 的可行路径：预热后的服务、缓慢升速（如 20 连接/秒）、临时 swap，或跨机负载端。 -顺带发现并修复 **F5（Windows TLS 缺陷）**：首次本地部署时 WSS 完全无法握手——`CertificateLoader` 的 `EphemeralKeySet` 在 Windows 上被 SChannel 拒绝（"platform does not support ephemeral keys"，0x8009030E），而 Linux/OpenSSL 不受影响，因此 VPS 部署从未暴露此问题。修复：Windows 上 PFX 使用 `DefaultKeySet`（持久密钥），PEM 加载后重导出为持久密钥；Linux 保持原状。spec §2.1 声明支持的 Windows Service 托管形态由此才真正可用。 +测量过程发现并修复 F5（Windows TLS 缺陷）：Windows 上 `CertificateLoader` 的 `EphemeralKeySet` 被 SChannel 拒绝（"platform does not support ephemeral keys"，0x8009030E），WSS 握手全部失败；Linux/OpenSSL 不受影响，VPS 部署因此从未暴露此问题。修复：Windows 上 PFX 使用 `DefaultKeySet`（持久密钥），PEM 加载后重导出为持久密钥；Linux 保持原状。spec §2.1 声明的 Windows Service 托管形态由此可用。 -**S6 慢消费者隔离**：32KB clip @ 50/s（1.6 MB/s）。基线 p95 17 ms（含 JIT 噪声），B 停读后 A 的 p95 稳定在 7.0–7.9 ms——完全隔离。B 在 **~16 秒**被服务端熔断断开（观测：established 3→2 且无 disconnect 日志；回环内核缓冲自调优 ~10 MB 吸收了初段流量，之后 16 条发送队列填满触发熔断）。两个发现： +**S6 慢消费者隔离**：32KB clip @ 50/s（1.6 MB/s）。基线 p95 17 ms（含 JIT 噪声），B 停读后 A 的 p95 稳定在 7.0–7.9 ms——完全隔离。B 在 ~16 秒被服务端熔断断开（观测：established 3→2 且无 disconnect 日志；回环内核缓冲自调优 ~10 MB 吸收了初段流量，之后 16 条发送队列填满触发熔断）。两个发现： - **熔断静默**：队列满路径直接 `MarkClosed + Cts.Cancel`，后续 `CancelConnection` 因 `MarkClosed` 已置位而提前返回，不产生任何 disconnect 安全事件——被熔断的连接在日志中不可见（已记入 spec §15）。 - **熔断延迟**：16 条队列的熔断点受内核 socket 缓冲（自动调优可达 ~10 MB）放大，取决于消息尺寸与速率，"队列满即断"在回环场景实际表现为"缓冲满即断"。 -**S7 冷启动**：三次重启的应用启动阶段（journal `Started` → `Now listening`）均为 **2 秒**，达标但已贴线。另发现：`systemctl restart` 端到端耗时 **35.6 秒**（有真实客户端在线时）——旧实例的关闭阶段花了 34 秒，原因是 `ShutdownAsync` 对每个连接 `CloseAsync` 等待 close 握手完成且无超时，静默客户端会拖住整个停机流程；spec §7 的"等待最多 2 秒"只覆盖 close 握手完成后的 drain。已记入 spec §15。 +**S7 冷启动**：三次重启的应用启动阶段（journal `Started` → `Now listening`）均为 2 秒，达标但已贴线。另发现：`systemctl restart` 端到端耗时 35.6 秒（有真实客户端在线时）——旧实例的关闭阶段花了 34 秒，原因是 `ShutdownAsync` 对每个连接 `CloseAsync` 等待 close 握手完成且无超时，静默客户端会拖住整个停机流程；spec §7 的"等待最多 2 秒"只覆盖 close 握手完成后的 drain。已记入 spec §15。 **S8 空闲 CPU**：两轮 60 秒各 5 个时钟 tick（100 tick = 1 CPU 秒）→ 0.083% CPU。心跳扫描器（1 Hz）、状态刷盘（5 秒周期，空闲时无脏数据）、用户表轮询（30 秒周期）的固定开销可忽略。 @@ -74,7 +72,7 @@ | F1 | 停机关闭握手等待无超时（实测 34 秒） | 重启/升级时拖长停机窗口 | `CloseConnectionAsync` 的 `CloseAsync` 加超时（如 2 秒）后走 abort；已记入 spec §15 | | F2 | 队列满熔断不产生 disconnect 日志 | 被熔断连接在安全日志中不可见 | 熔断路径补一条安全事件；已记入 spec §15 | | F3 | P1 内存目标过紧（.NET 运行时基线 125-131 MB） | P1 不可达 | 修订 P1 为 < 150 MB；P2 按边际成本 240 KB/连接 基本达标，保留观察 | -| F4 | P8 在 1.6GB 同机环境不可测 | 无法验证 1000 并发 | 已解决：Windows 本机补测通过（见 S5 补测）；VPS 上仍建议跨机负载端 | +| F4 | P8 在 1.6GB 同机环境不可行 | 无法验证 1000 并发 | 已解决：Windows 本机（32 GB）验证通过（见 S5）；VPS 上仍建议跨机负载端 | | F5 | Windows 上 `EphemeralKeySet` 导致 WSS 握手必然失败（0x8009030E） | spec §2.1 声明的 Windows Service 托管不可用 | 已修复：Windows 用 `DefaultKeySet`（PFX）+ PEM 重导出持久密钥；Linux 不变 | ### 5. 复现步骤 @@ -128,40 +126,38 @@ Same-host loopback excludes network jitter, but the generator shares the 2 vCPUs | # | Scenario | Metric | Target | Measured | Verdict | |---|---|---|---|---|---| | P1 | S1 base memory | RSS (fresh process, 60 s warmup) | < 50 MB | **125–131 MB** | ✗ fail | -| P2 | S2 100 idle connections | RSS delta (5 min, all alive) | < 20 MB | first run +66 MB (incl. one-time JIT/heap growth); re-run marginal **+24 MB** (≈240 KB/conn) | ◑ marginal near-target | +| P2 | S2 100 idle connections | RSS delta (5 min, all alive) | < 20 MB | fresh process +66 MB (incl. one-time JIT/heap growth); marginal **+24 MB** (≈240 KB/conn) | ◑ marginal near-target | | P3 | S3 1KB broadcast | broadcast_lag p95 (1000 samples) | < 30 ms | **3.5 ms** (p50 1.87 / p99 5.1 / max 11.8) | ✓ pass (8.6× headroom) | | P3b | S3 companion | ack_rtt p95 | reference | 4.0 ms | — | | P4 | S4 512KB broadcast | broadcast_lag p95 (200 samples) | < 250 ms | **103.2 ms** (p50 87.3 / p99 138 / max 157) | ✓ pass (2.4× headroom) | | P5 | S8 idle CPU | 60 s average (two runs) | ≈ 0% | **0.08%** (5 ticks / 60 s) | ✓ pass | | P6 | S7 cold start | Application start phase (Started → listening) | < 2 s | **2 s** (consistent across 3 restarts) | ✓ pass (borderline) | | P7 | Recovery window | snapshot_window_seconds | 3 s | config constant; correctness covered by tests | — (n/a) | -| P8 | S5 1000 concurrent | 10-minute stability | no disconnects | **passed on local Windows**: 0 errors, 1000/1000 alive, +211 MB (Linux VPS same-host environment untestable, see S5 details) | ✓ pass (Windows 32 GB) | +| P8 | S5 1000 concurrent | 10-minute stability | no disconnects | **passed on local Windows**: 0 errors, 1000/1000 alive, +211 MB (not feasible on the 1.6 GB production VPS with a same-host generator, see S5 details) | ✓ pass (Windows 32 GB) | | — | S6 slow-consumer isolation | A's p95 (B stalled 45 s) | unaffected | **7.0–7.9 ms** (baseline 6–17 ms); B silently aborted at ~16 s | ✓ isolation holds | ### 3. Scenario Details **S1 base memory**: RSS after three restarts was 127872 / 128256 / 131328 kB; 143852 kB after 24 h of uptime. The baseline consists of the .NET 10 runtime, Kestrel, the TLS stack, and 22 threads — a fresh process is already ~125 MB, so this is a runtime baseline, not a leak. The original 50 MB target is unrealistic for an ASP.NET Core app (judged "target too tight" rather than an implementation defect). -**S2 100 idle connections**: fresh RSS 127872 kB → 195392 kB, delta 67520 kB (≈660 KB/connection). All 900/900 expected heartbeat pongs were answered with zero errors. The 660 KB/connection includes TLS stream buffers, Kestrel per-connection pipes and pinned buffers, and managed objects. The original 20 MB target (200 KB/connection) underestimated the real cost of Kestrel + TLS. - -**S2 re-measurement (clarifying the 660 KB/connection attribution, 2026-08-29)**: re-running the same scenario on a process that had been up for 32 minutes with fully-JITted code showed a delta of only **+24 MB (≈240 KB/connection)**, and memory did not return after the connections closed (GC segments retained; three ~20 MB anonymous segments). Conclusion: the initially measured 660 KB/connection mixed in one-time costs — the JIT compilation and first-time GC heap expansion triggered by the 100-connection handshake storm (roughly 40–50 MB) — rather than the true per-connection cost. The real **marginal** per-connection cost is ≈240 KB (Linux) vs 211 KB (Windows), the same order. Windows "looking lower" is mostly a denominator effect: 211 KB spread over 1000 connections vs 66 MB spread over 100. Revised P2 verdict: **marginal cost meets the target within 20%** (240 KB vs 200 KB per connection); the transient peak of a first connection storm exceeds it. +**S2 100 idle connections**: fresh-process RSS 127872 kB → 195392 kB, a delta of 67520 kB (≈660 KB/connection); on a process up for 32 minutes with fully-JITted code, the same scenario showed a marginal delta of only +24 MB (≈240 KB/connection). The difference is one-time cost — JIT compilation and first-time GC heap expansion triggered by the 100-connection handshake storm (roughly 40–50 MB). The real per-connection cost is the marginal figure: ≈240 KB (Linux) vs 211 KB (Windows), the same order of magnitude (Windows "looking lower" is mostly a denominator effect: 211 KB spread over 1000 connections vs 66 MB spread over 100). The per-connection cost consists of TLS stream buffers, Kestrel per-connection pipes and pinned buffers, and managed objects; memory does not return after connections close (GC segments retained; three ~20 MB anonymous segments). All 900/900 expected heartbeat pongs were answered with zero errors. Verdict: the marginal cost meets the target within 20% (240 KB vs 200 KB per connection); the transient peak of a first connection storm exceeds it. The original 20 MB total target (200 KB/connection) underestimated the real cost of Kestrel + TLS. **S3 1KB broadcast latency**: all 1000 samples recovered. `broadcast_lag` p95 = 3.5 ms — the server path (parse → token bucket → version increment → single serialization → delivery to two connections) plus TLS on both ends stays far below the 30 ms target. **S4 512KB broadcast latency**: all 200 samples recovered. p95 103 ms, dominated by serializing/escaping the 512KB JSON and two 512KB TLS record writes; 2.4× headroom, within target. -**S5 1000 concurrent connections (not completed; production impact)**: started from a fresh 128256 kB baseline. The SSH session was reset ~2 minutes into the storm; the machine then froze until the provider watchdog hard-reset it at 01:38:58. **Root cause (from the previous boot's persistent journal)**: all 1000 WebSocket upgrade requests arrived within the same second (00:48:59) against a server restarted only 40 s earlier; from 00:50:18 journald repeatedly logged "Under memory pressure, flushing caches"; at 00:51:53 the journal went silent (machine frozen for 47 minutes). The kernel log contains **no OOM-kill records** and the machine has **swap = 0**. Conclusion: steady-state memory was not the problem — 1000 × 240 KB (the re-measured marginal cost) ≈ 240 MB fits comfortably — it was the **transient storm peak** that broke a zero-swap environment: cold-process JIT + overlapping allocations of 1000 concurrent TLS handshakes + several hundred MB for the same-host Python generator + the upgrade-request log flood caused a kernel reclaim livelock (the .NET runtime's file-backed pages were endlessly evicted and re-faulted); the machine froze before the OOM killer could act. **Production impact disclosure**: real user devices could not connect for ~47 minutes. Viable paths to pass P8 in this environment: a warmed-up server, a slow ramp (e.g. 20 connections/s), a temporary swap file, or an off-host generator. +**S5 1000 concurrent connections**: passed on a local Windows x64 / 32 GB machine — an openssl self-signed passwordless PFX, all 1000 connections established (handshakes completed within ~90 s), 0 errors, and during the 10-minute hold 19576/19000 heartbeat pongs were answered (floor = 1000 connections × 19 cycles; extras come from clock rounding); RSS curve 105 MB (baseline) → 262–285 MB steady → 316 MB at the end, a delta of ≈211 MB (≈211 KB/connection), the same order as the 240 KB/connection marginal cost on Linux (see S2). The server log contained no errors. P8 verdict: pass. -**S5 follow-up run (local Windows, 2026-08-29)**: environment was Windows x64 / 32 GB RAM, an openssl self-signed passwordless PFX, and a binary containing the Windows TLS fix (see F5). Result: all 1000 connections established (handshakes completed within ~90 s), 0 errors, and during the 10-minute hold 19576/19000 heartbeat pongs were answered (floor = 1000 connections × 19 cycles). RSS curve: 105 MB (baseline) → 262–285 MB steady → 316 MB at the end — a **delta of ≈211 MB (≈211 KB/connection)**, one third of the Linux figure (660 KB/connection) due to different Kestrel/TLS buffer behavior and GC. The server log contained no errors. **P8 verdict: pass.** +The same scenario is not feasible on the 1.6 GB production VPS with a same-host generator: all 1000 WebSocket upgrade requests arrived within the same second (00:48:59) against a server restarted only 40 s earlier; the SSH session was reset ~2 minutes into the storm, and the machine stayed frozen until the provider watchdog hard-reset it at 01:38:58 (47 minutes). The previous boot's persistent journal shows journald repeatedly logging "Under memory pressure, flushing caches" from 00:50:18 and going silent at 00:51:53; the kernel log contains no OOM-kill records and the machine has swap = 0. Steady-state memory was not the bottleneck — 1000 × 240 KB (marginal cost) ≈ 240 MB fits comfortably — it was the transient storm peak that broke the zero-swap environment: cold-process JIT, the overlapping allocations of 1000 concurrent TLS handshakes, several hundred MB for the same-host Python generator, and the upgrade-request log flood caused a kernel reclaim livelock (the .NET runtime's file-backed pages were endlessly evicted and re-faulted), and the machine froze before the OOM killer could act. Production impact disclosure: real user devices could not connect for ~47 minutes. Viable paths to run P8 in that environment: a warmed-up server, a slow ramp (e.g. 20 connections/s), a temporary swap file, or an off-host generator. -This run also surfaced and fixed **F5 (Windows TLS defect)**: the first local deployment could not complete a single WSS handshake — `CertificateLoader`'s `EphemeralKeySet` is rejected by SChannel on Windows ("platform does not support ephemeral keys", 0x8009030E), while Linux/OpenSSL is unaffected, which is why the VPS deployment never exposed it. Fix: on Windows the PFX branch now uses `DefaultKeySet` (persisted key) and PEM-loaded certificates are re-exported to a persisted key; Linux behavior is unchanged. Spec §2.1's declared Windows Service hosting shape is only truly usable with this fix. +The measurements also surfaced and fixed F5 (Windows TLS defect): on Windows, `CertificateLoader`'s `EphemeralKeySet` is rejected by SChannel ("platform does not support ephemeral keys", 0x8009030E) and every WSS handshake fails; Linux/OpenSSL is unaffected, which is why the VPS deployment never exposed it. Fix: on Windows the PFX branch uses `DefaultKeySet` (persisted key) and PEM-loaded certificates are re-exported to a persisted key; Linux behavior is unchanged. Spec §2.1's declared Windows Service hosting shape is usable as a result. -**S6 slow-consumer isolation**: 32KB clips @ 50/s (1.6 MB/s). Baseline p95 17 ms (includes JIT noise); with B stalled, A's p95 held at 7.0–7.9 ms — fully isolated. B was silently aborted by the server at **~16 s** (observed: established count 3→2 with no disconnect log; the autotuned ~10 MB loopback kernel buffers absorbed the initial burst, after which the 16-message send queue filled and triggered the abort). Two findings: +**S6 slow-consumer isolation**: 32KB clips @ 50/s (1.6 MB/s). Baseline p95 17 ms (includes JIT noise); with B stalled, A's p95 held at 7.0–7.9 ms — fully isolated. B was silently aborted by the server at ~16 s (observed: established count 3→2 with no disconnect log; the autotuned ~10 MB loopback kernel buffers absorbed the initial burst, after which the 16-message send queue filled and triggered the abort). Two findings: - **Silent abort**: the queue-full path calls `MarkClosed + Cts.Cancel` directly, and the subsequent `CancelConnection` returns early because `MarkClosed` is already set — no disconnect security event is produced, so aborted connections are invisible in the logs (recorded in spec §15). - **Abort delay**: the 16-message queue trigger point is amplified by kernel socket buffers (autotuned up to ~10 MB) and therefore depends on message size and rate; on loopback, "queue full = disconnect" behaves as "buffers full = disconnect". -**S7 cold start**: the application start phase (journal `Started` → `Now listening`) was **2 seconds** across three restarts — on target but borderline. Additional finding: end-to-end `systemctl restart` took **35.6 seconds** (with real clients connected) — the old instance's stop phase took 34 seconds because `ShutdownAsync` awaits each connection's `CloseAsync` close-handshake with no timeout, so silent clients stall the whole shutdown; spec §7's "wait up to 2 seconds" only covers the drain after the handshakes complete. Recorded in spec §15. +**S7 cold start**: the application start phase (journal `Started` → `Now listening`) was 2 seconds across three restarts — on target but borderline. Additional finding: end-to-end `systemctl restart` took 35.6 seconds (with real clients connected) — the old instance's stop phase took 34 seconds because `ShutdownAsync` awaits each connection's `CloseAsync` close-handshake with no timeout, so silent clients stall the whole shutdown; spec §7's "wait up to 2 seconds" only covers the drain after the handshakes complete. Recorded in spec §15. **S8 idle CPU**: two 60-second runs, 5 clock ticks each (100 ticks = 1 CPU-second) → 0.083% CPU. The fixed overhead of the heartbeat scanner (1 Hz), state flush (5 s cycle, no dirty data while idle), and user-file polling (30 s cycle) is negligible. @@ -172,7 +168,7 @@ This run also surfaced and fixed **F5 (Windows TLS defect)**: the first local de | F1 | Shutdown close-handshake wait is unbounded (34 s measured) | Prolongs the stop window on restarts/upgrades | Add a timeout (e.g. 2 s) to `CloseConnectionAsync`'s `CloseAsync`, then abort; recorded in spec §15 | | F2 | Queue-full abort produces no disconnect log | Aborted connections are invisible in security logs | Emit a security event on the abort path; recorded in spec §15 | | F3 | P1 memory target too tight (.NET runtime baseline is 125–131 MB) | P1 unreachable | Revise P1 to < 150 MB; P2 essentially meets target at 240 KB/connection marginal cost, keep observing | -| F4 | P8 untestable on a 1.6 GB same-host environment | 1000 concurrent connections unverifiable | Resolved: passed on local Windows (see S5 follow-up); an off-host generator is still recommended for the VPS | +| F4 | P8 not feasible on a 1.6 GB same-host environment | 1000 concurrent connections unverifiable | Resolved: passed on a local Windows machine (see S5); an off-host generator is still recommended for the VPS | | F5 | `EphemeralKeySet` made WSS handshakes always fail on Windows (0x8009030E) | The Windows Service hosting shape declared in spec §2.1 was unusable | Fixed: `DefaultKeySet` for PFX on Windows + PEM re-export to a persisted key; Linux unchanged | ### 5. Reproduction From 197cfef7f64a0ed32dd113dc0e265e89f8e84c08 Mon Sep 17 00:00:00 2001 From: long45343 <1725334094@qq.com> Date: Fri, 4 Sep 2026 10:32:24 +0800 Subject: [PATCH 29/32] chore: stop tracking specs/ as private local directory --- .gitignore | 2 +- specs/code-review.md | 81 -------- specs/spec-decisions.md | 218 ---------------------- specs/test-and-contract-spec.md | 316 -------------------------------- 4 files changed, 1 insertion(+), 616 deletions(-) delete mode 100644 specs/code-review.md delete mode 100644 specs/spec-decisions.md delete mode 100644 specs/test-and-contract-spec.md diff --git a/.gitignore b/.gitignore index 67c4a6d..cadc68e 100644 --- a/.gitignore +++ b/.gitignore @@ -1,5 +1,5 @@ ## 项目私有目录,不纳入版本库 -!/specs/ +/specs/ .zcode/ .trae/ diff --git a/specs/code-review.md b/specs/code-review.md deleted file mode 100644 index a12770f..0000000 --- a/specs/code-review.md +++ /dev/null @@ -1,81 +0,0 @@ -# TextCascade.Server 代码审查报告 (Code Review) - -## 1. 总体架构评价 - -TextCascade.Server 是一个定位非常清晰的轻量级剪贴板/最新文本同步服务端。整体代码风格紧凑、克制，没有引入臃肿的企业级分层（无 EF Core、无外部数据库、无庞杂的中间件），非常契合 "Ponytail" / "Do Less" 的实用主义设计哲学。 - -### 核心亮点 -1. **并发与隔离模型清晰**：每个在线用户一个 `UserHub`，内部采用单消费者 Channel（`RunUserLoopAsync`），天然避免了多连接并发修改版本号和最新文本时的大颗粒度锁争用。 -2. **背压与慢连接处理果断**：每个连接分配有界发送队列（默认 16 条），队列满时立即判定为慢连接并直接 Cancel/Abort，绝不等待阻塞，也不拖垮同用户的其他客户端。 -3. **无状态 Token + 版本作废机制**：采用基于 HMAC-SHA256 的紧凑型 Token，服务端重启无需保存 Session；通过 `users.json` 中的 `tokenVersion` 与全局水位 `nextTokenVersion` 实现高效的单用户/全量 Token 撤销。 -4. **内存与资源开销极低**：广播时一次序列化，多连接复用同一份 UTF-8 Byte 数组；统一心跳扫描器替代每连接独立 Timer。 -5. **单文件与开箱即用**：集成了 CLI 用户管理、TLS 证书加载、本地状态落盘与单实例锁，运维负担极小。 - ---- - -## 2. 关键发现与问题清单 - -### P1 - 性能与资源瓶颈 (Performance & Allocation) - -#### 1. 广播路径上的连接列表数组分配 -- **位置**：`TextCascade.Server/Hub/UserHub.cs` -- **现象**：`Connections` 属性每次被读取时，都会执行 `lock (connectionsGate) { return connections.ToArray(); }`。在 `ApplyClip` 广播、`BroadcastWelcome` 等高频路径中，每条 Clip 都会分配一次连接数组。 -- **人话建议**： - 在连接数变动较少、消息广播频繁的场景下，可以采用 **Copy-On-Write（写时复制）** 模式维护内部连接数组（即维护一个不可变的 `ImmutableArray` 或 `ConnectionContext[]`，只有在 `AddConnection` / `RemoveConnection` 时才重新生成新数组）。这样广播时读取连接列表完全**零锁、零分配**。 - -#### 2. Token 校验中的多重内存分配与 Dictionary 构造 -- **位置**：`TextCascade.Server/Auth.cs` (`TokenService.TryVerifyTokenInternal`) -- **现象**： - 1. `compactToken.Split('.')` 每次都分配一个 `string[]`。 - 2. `var actualPayload = payloadRent is null ? payloadBytes[..payloadLength].ToArray() : payloadRent[..payloadLength];` 在栈空间足够时依然调用了 `.ToArray()`。 - 3. `var properties = root.EnumerateObject().ToDictionary(...)` 每次校验 Token 都会分配一个 `Dictionary` 和 4 个字符串 Key。 -- **人话建议**： - HMAC 校验和 `JsonDocument.Parse` 都可以直接接受 `ReadOnlySpan`。解析 Payload 时直接使用 `root.TryGetProperty("sub", out ...)`，不要转成 `ToDictionary`，也不要多余 `.ToArray()`。作为 WebSocket 握手升级的高频入口，优化后可实现接近零分配。 - -#### 3. 登录接口中的双重 JSON 解析与字符串转换 -- **位置**：`TextCascade.Server/AuthService.cs` (`ParseLoginRequest`) -- **现象**：从 Request Body 读取后，先 `new UTF8Encoding().GetString(body.ToArray())`，再用 `JsonDocument.Parse` 校验属性，最后又调用了一次 `JsonSerializer.Deserialize(text)`。 -- **人话建议**：既然第一步已经用 `JsonDocument` 验证了字段结构，直接从 `document.RootElement` 取出 `username` 和 `password` 即可，彻底省去第二次 `JsonSerializer.Deserialize` 以及中间的多余字符串拷贝。 - ---- - -### P2 - 并发与稳健性风险 (Concurrency & Edge Cases) - -#### 1. Argon2id 同步计算可能阻塞 Kestrel 线程 -- **位置**：`TextCascade.Server/AuthService.cs` (`HandleLoginAsync`) -- **现象**：`syncServer.Hasher.Verify(request.Password, passwordHash)` 是纯 CPU/内存密集型运算（约 19MB 内存、2 轮迭代）。当前是在 Kestrel HTTP 请求处理的同一线程上下文中同步执行。 -- **人话建议**：如果短时间内有多个并发登录请求，可能会占满线程池调度。建议使用 `await Task.Run(() => syncServer.Hasher.Verify(...))` 将密码哈希运算明确卸载到后台工作线程，避免堵塞 HTTP 管道。 - -#### 2. 滑动窗口限流器在 Key 满时的全局遍历瓶颈 -- **位置**：`TextCascade.Server/Core.cs` (`SlidingWindowLoginLimiter.TryConsume`) -- **现象**：当外部遇到 IP/用户名爆破攻击导致 `windows.Count >= maxKeys` (10000) 时，每次新请求都会在 `lock (gate)` 内遍历整个字典清理过期项（`RemoveExpired`）。在大流量恶意扫描下，会导致所有正常用户的登录请求在同一个锁上排队。 -- **人话建议**： - 可以采用分段锁，或者在后台使用 `PeriodicTimer` 每隔几秒做一次批量清理；当字典达到 `maxKeys` 时，不再对每个请求都执行全量 O(N) 遍历，而是快速丢弃或采用简单的 LRU/分桶计数。 - -#### 3. 规范与现实的偏离 (Spec vs Code Drift) -- **位置**：`docs/server-spec.md` vs `TextCascade.Server/Hosting/UserFileWatcher.cs` -- **现象**： - - `server-spec.md` §3.2 和 §14 中明确写道：“*不热加载用户文件，避免在线连接认证状态与文件状态竞态*”、“*修改用户文件后需重启服务生效*”。 - - 但代码中实现了 `UserFileWatcher` 并在 `ServerHost.cs` 中启用了文件变动监听和热加载。 -- **人话建议**：代码中的热加载实现实际上做得很干净（使用了防抖、重试以及 `Volatile.Write` 原子替换），这是个很好的功能改进。建议同步更新 `docs/server-spec.md` 文档，将已废弃的“必须重启生效”描述更新为“支持 users.json 监听与平滑热重载”。 - ---- - -### P3 - 代码洁癖与简化空间 (Code Cleanliness & Simplification) - -1. **废弃的 `TryDuplicate` 方法**： - `Core.cs` 中的 `SeenIdRing.TryDuplicate(string id)` 目前在实际业务流程中未被调用（业务中使用的是 `IsUnchangedDuplicate` + `TryGetResult` + `RememberId`），仅留存在单元测试中。可评估是否需要保留或标记内部测试专用。 -2. **Source Generator 覆盖度**： - 协议消息（`WelcomeMessage`, `ClipMessage` 等）已经使用了 .NET 10 的 `JsonSourceGenerationOptions` 强类型上下文，非常棒！但 `RuntimeStateStore` 和 `UsersFile` 仍在使用传统的动态反射序列化。建议统一接入 Source Generator，进一步增强 Native AOT 兼容性与运行速度。 - ---- - -## 3. 改进建议总结 - -| 序号 | 改进项 | 收益 | 实施复杂度 | -|---|---|---|---| -| 1 | `UserHub.Connections` 改用 Copy-On-Write 数组 | 广播路径彻底消除锁与数组分配 | 低 | -| 2 | `AuthService` 与 `TokenService` 内存与解析去重 | 降低登录与 WS 握手时的 GC 压力 | 低 | -| 3 | Argon2 校验使用 `Task.Run` 卸载 | 保护 Kestrel 请求处理管线抗突发并发 | 极低 | -| 4 | 限流器清理机制从同步全量遍历优化为轻量清理 | 提升恶意请求轰炸下的吞吐与稳定性 | 中 | -| 5 | 同步更新 `server-spec.md` 关于热重载的说明 | 保持文档与生产代码一致 | 极低 | diff --git a/specs/spec-decisions.md b/specs/spec-decisions.md deleted file mode 100644 index b00caae..0000000 --- a/specs/spec-decisions.md +++ /dev/null @@ -1,218 +0,0 @@ -# TextCascade 规格修订决策记录 - -日期：2026-08-27 -任务：三类差异三条路径（详见 docs/server-spec.md 审计与 git 溯源结论） -- 路径 A：更新引入的漂移 → 直接修订 `docs/server-spec.md` 对齐 v0.3.5，无决策点。 -- 路径 B：NetworkIntegration 测试、契约测试组织、单元测试缺口 → 新建 `specs/test-and-contract-spec.md`，以下 10 题决定其内容。 -- 路径 C：其余从未实现项（Benchmark 项目、server_stop 事件、性能目标表）→ 从 docs/server-spec.md 移除。 - -规则：每轮只问一个问题；提问前本题已落盘；收到答案后立刻回写本题"选择"栏。 - ---- - -## Q1 NetworkIntegration 测试宿主 【已选】 - -NetworkIntegration 整类测试（TLS/WSS、HTTP 升级、随机端口绑定、真实帧分片、登录建连收发、重启直连恢复）放在哪里？ - -- **A. 复用现有 WebSocketIntegrationTests 的 fixture**（真实 Kestrel 绑定 127.0.0.1 随机端口 + ClientWebSocket） - - 优：零新增设施；与现有 6 个集成测试共享 helper，维护成本最低；TLS 测试只需给 fixture 加证书参数。 - - 劣：同一项目里普通集成测试与网络测试混在一起，只能靠 Trait 过滤，CI 跳过逻辑与文件物理布局无对应关系。 -- **B. Tests 项目内新建 `NetworkIntegration/` 目录 + 自建 fixture** - - 优：物理隔离清晰；fixture 可针对 TLS/重启场景定制（如持有可重启的 WebApplication 列表）；不影响现有 fixture 稳定性。 - - 劣：需要抽象或复制现有 fixture 的公共逻辑，初始工作量中等；两套 helper 有漂移风险。 -- **C. 独立测试项目 `TextCascade.Server.NetworkTests`** - - 优：隔离最彻底；CI 可以完全按项目粒度跳过（不改 slnx 过滤逻辑的话需加入 slnx 但 CI 单独 dotnet test 主项目即可）；未来可加性能冒烟。 - - 劣：需改 .slnx、新增 csproj、跨项目 InternalsVisibleTo；维护三个项目的成本最高。 - -选择：**B. Tests 项目内新建 `NetworkIntegration/` 目录 + 自建 fixture**（2026-08-27） - ---- - -## Q2 NetworkIntegration 过滤机制 【已选】 - -如何让 `dotnet test` 默认跳过这些测试、按需运行？ - -- **A. `[Trait("Category", "NetworkIntegration")]` + CI/本地用 `--filter Category!=NetworkIntegration`** - - 优：xunit 原生、与 spec 原文 `Category=NetworkIntegration` 完全一致；`dotnet test --filter Category=NetworkIntegration` 即可单独跑。 - - 劣：CI 的 `dotnet test TextCascade.Server.slnx` 必须追加过滤参数，否则会跑到（需同步改 ci.yml）。 -- **B. MTP 风格（Microsoft.Testing.Platform）`--filter-category` 等** - - 优：新一代测试平台原生参数。 - - 劣：当前项目用 VSTest 模式跑（ci.yml 无 MTP 配置），引入 MTP 需改测试 SDK 配置，风险与收益不成比例。 -- **C. xunit Collection + 命名约定（如类名以 Network 开头）+ assembly 级配置** - - 优：Collection 级可统一串行化，适合真实端口/证书场景。 - - 劣：跳过逻辑要靠自定义 xunit trait 发现或 `--filter` 仍然需要；命名约定脆弱。 - -选择：**A. `[Trait("Category", "NetworkIntegration")]` + `--filter` 过滤**（2026-08-27）。备注：实施时需同步给 ci.yml 的 dotnet test 追加 `--filter Category!=NetworkIntegration`；本轮只写入 spec 实施清单，不改代码。 - ---- - -## Q3 测试用证书策略 【已选·默认】 - -NetworkIntegration 的 TLS/WSS 测试需要证书，从哪来？ - -- **A. 测试运行时自签生成**（.NET `CertificateRequest` 创建自签叶证书 + 私钥，内存导出 PFX 或直接传 X509Certificate2） - - 优：无仓库二进制；证书永不过期（自签可设长有效期）；跨平台无文件权限问题；可顺带测试"带密码 PFX 拒绝""PEM bundle"等加载路径。 - - 劣：需在测试 fixture 写约 30 行证书生成代码；自签证书的 Subject/SAN 需要构造（客户端可关证书校验绕过）。 -- **B. 仓库内提交测试用 PFX 文件** - - 优：fixture 最简单，直接读文件。 - - 劣：仓库出现二进制文件；PFX 若含密码会与"无密码证书"规格冲突，需维护说明；证书过期/轮换是长期负担；安全审计观感差。 -- **C. 证书加载器抽象出接口（ILoadedCertificateProvider）供测试 mock** - - 优：单测可全 mock，不碰真实证书。 - - 劣：**与 NetworkIntegration 的目的冲突**——该类测试本就要验证真实证书加载与 WSS 握手，mock 掉加载器等于没测；还会改动生产代码结构。 - -选择：**A. 测试运行时自签生成**（2026-08-27，用户未作答，按推荐项默认；如需改选请修改此行）。 - ---- - -## Q4 重启直连恢复测试形态 【已选·默认】 - -"服务端重启后 token 直连重连与 snapshot 恢复"如何模拟重启？ - -- **A. 同进程内停掉 WebApplication 再重启**（`ServerHost.CreateApp` 构建两次，第一次 `StopAsync` 后第二次 `RunAsync`，共用同一 users.json/临时目录） - - 优：真实覆盖"进程内服务器实例重建 + 状态文件残留 + token 跨实例有效"链路；快、稳定、可调试；RuntimeStateStore 落盘行为也能顺带验证。 - - 劣：不是真的进程退出——静态/全局状态若有残留可能掩盖问题（当前代码 CreateApp 每次新建 SyncServer，风险可控）。 -- **B. 独立子进程跑发布产物** - - 优：最真实的进程级重启（含文件锁、端口释放、PID 生命周期）。 - - 劣：需要先 publish 或定位构建产物，慢（数十秒）、CI 不稳定因素多；端口占用/防火墙等环境敏感；调试困难。 -- **C. 只测 CreateApp 重建，不测真实停启时序** - - 优：最省事。 - - 劣：覆盖不了 bye/1001 → 重连 → snapshot 上报的完整时序，测试价值大打折扣。 - -选择：**A. 同进程内停掉 WebApplication 再重启（ServerHost.CreateApp 两次）**（2026-08-27，用户未作答，按推荐项默认；如需改选请修改此行）。 - ---- - -## Q5 TLS 版本断言 【已选】 - -spec §8.2 "TLS 最低 1.2"——当前实现未显式设置 SslProtocols。测试怎么处理？ - -- **A. 客户端显式用 SslProtocols.Tls12（及另一条 Tls13）发起 WSS，断言握手成功** - - 优：行为级验证，直接回答"1.2/1.3 能不能用"；在 Windows/Linux 默认配置下均应通过。 - - 劣：若 OS 未来禁用 TLS1.2，测试会失败但那是 OS 政策问题，需在断言消息中说明。 -- **B. 断言 Kestrel 配置对象中的 HttpsConnectionAdapterOptions.SslProtocols** - - 优：直接检查服务端配置意图。 - - 劣：当前实现根本没设置该选项（依赖 OS 默认），断言会立即失败——**选 B 实际上等于决定要先改生产代码显式设置 TLS 下限**，超出本轮"只写测试 spec"的范围。 -- **C. 不测 TLS 版本，在测试 spec 的"已知限制"一节记录：服务端依赖 OS 默认协议版本** - - 优：零成本，与当前实现状态一致。 - - 劣：TLS 降级风险（理论上）无回归防护。 - -选择：**A. 客户端显式 SslProtocols.Tls12/Tls13 发起 WSS，断言握手成功**（2026-08-27）。备注：断言消息中需说明"若 OS 政策禁用该版本导致的失败属环境问题"。 - ---- - -## Q6 契约测试组织 【已选】 - -spec §10.4 的"服务端维护典型 JSON 样本，约束三端协议字段与行为"如何组织？ - -- **A. Tests 项目内新建 `ContractTests/` 目录 + JSON 样本文件落盘**（`ContractSamples/hello/*.json` 等，测试读取并断言解析结果） - - 优：样本即文档，三端（C#/Kotlin）可直接取用同一批文件做各自实现的对拍；新增样本不改代码。 - - 劣：需要 csproj 把样本 CopyToOutputDirectory；文件与代码双份维护。 -- **B. 内联样本字符串（Theory 的 InlineData / 常量）** - - 优：最简单，全部在一个 .cs 里，跳转方便。 - - 劣：三端对拍要人肉从代码抄样本；样本多了文件臃肿。 -- **C. 独立契约项目（如 TextCascade.Contract.Tests）** - - 优：概念上最干净。 - - 劣：为几十个样本开一个项目不值；又多一项 slnx/CI 维护。 - -选择：**A. Tests 项目内新建 `ContractTests/` 目录 + JSON 样本文件落盘**（2026-08-27）。备注：实施时需在测试 csproj 加 CopyToOutputDirectory；样本目录设计在 test-and-contract-spec.md 中细化。 - ---- - -## Q7 非法数字 / 非法 UTF-8 样本范围 【已选】 - -契约样本要覆盖"JSON 深度 3、重复字段、未知字段、非法数字、非法 UTF-8"，其中非法数字与非法 UTF-8 的覆盖面多大？ - -- **A. 全矩阵：每种消息类型（hello/clip/pong）× 每种非法数字形态（负数、小数、指数、字符串数字、超 long、重复字段、未知字段、非法 UTF-8 字节）** - - 优：完备，一次性锁死三端解析行为。 - - 劣：样本数量约 3×8=24+，编写与维护成本高；部分组合行为完全同质（都是 invalid_message），边际价值低。 -- **B. 代表性样本：每类非法形态挑 1-2 个高价值位置（如 hello.lastServerVersion、clip.id）** - - 优：成本适中，覆盖每个错误分支至少一次；样本 10 个左右。 - - 劣：非全矩阵，理论上某消息类型某字段的特有解析分支可能漏测。 -- **C. 仅 token + clip 两个高风险面做全形态，hello/pong 只测代表性样本** - - 优：风险导向，token 是安全面、clip 是主路径。 - - 劣：hello.lastServerVersion 的数字校验分支（TryGetUInt64）无直接样本。 - -选择：**A. 全矩阵：每种消息类型（hello/clip/pong）× 每种非法形态（负数、小数、指数、字符串数字、超 long、重复字段、未知字段、非法 UTF-8）**（2026-08-27，用户主动选择非推荐项）。 - ---- - -## Q8 单元测试缺口补齐方式 【已选】 - -第一轮审计列出的单元缺口：Argon2 三函数（HashPassword/VerifyPassword/NeedsRehash）、token 重复字段/非法数字/非法范围（exp≤iat、负数）、CLI 水位递增/删除重建/溢出 fail-fast、WithVersion、"重复 id 不消耗令牌桶"。补多少？ - -- **A. 全部补齐**（上述每一项都有直接测试） - - 优：spec §10.1 清单闭环；Argon2 与 CLI 是安全/数据完整性面，值得覆盖。 - - 劣：工作量最大（约 25-35 个用例）；CLI 水位测试需要搭临时 users.json 场景。 -- **B. 只补高风险面**：token 数字校验全形态 + CLI 水位/溢出 + 重复 id 不耗令牌桶 - - 优：安全相关（token）与数据完整性（水位）+ 协议热点（幂等）优先；约 15 个用例。 - - 劣：Argon2 三函数与 WithVersion 仍零覆盖。 -- **C. 只补纯函数**（WithVersion、token 校验等无需 IO 的），不碰 Argon2 与 CLI - - 优：测试快、无环境依赖。 - - 劣：恰好跳过了两个最重要的缺口（Argon2、CLI 水位），补了寂寞。 - -选择：**A. 全部补齐**（Argon2 三函数、token 全形态、CLI 水位/重建/溢出、WithVersion、重复 id 不耗令牌桶；约 25-35 用例）（2026-08-27，用户主动选择非推荐项）。 - ---- - -## Q9 Argon2 测试用假哈希器还是真实慢哈希 【已选】 - -spec 原文："认证测试注入假哈希器，避免 Argon2id 拖慢常规单元测试"。但 Argon2 三函数本身还没测过。 - -- **A. 假哈希器为主**：Argon2 三函数只测 1 个真实参数的 smoke 用例（Hash→Verify 成功、错密码失败），其余认证路径全部用现有 FastPasswordHasher/RecordingHasher - - 优：符合 spec 精神；全套测试仍秒级；NeedsRehash 的参数解析逻辑用构造的哈希串测（不真算）。 - - 劣：NeedsRehash 真实行为（不同参数组合判定）覆盖浅。 -- **B. 真实 Argon2 低参数**（如 memory=64KiB, iterations=1）跑完整用例矩阵 - - 优：测试对象即真实算法，无 mock 偏差。 - - 劣：与生产参数（19456KiB/2）不同，测的不是同一配置；即便低参数，几十个用例也明显拖慢。 -- **C. 混合**：单测用假哈希器；单独一个标记 `Category=SlowHash` 的真实参数用例（Hash→Verify→NeedsRehash 三链路） - - 优：日常快速，专项真实；NeedsRehash 真实语义有兜底。 - - 劣：又引入一个新的测试类别过滤（与 Q2 的机制要兼容）。 - -选择：**C. 混合**：单测用假哈希器；单独 `Category=SlowHash` 真实参数用例覆盖 Hash→Verify→NeedsRehash 三链路（2026-08-27）。备注：CI 排除规则需同时排除 SlowHash 与 NetworkIntegration。 - ---- - -## Q10 "重复 id 不消耗令牌桶"测试断言深度 【已选】 - -spec §5.4 要求重复 id 不生成新版本、不耗令牌桶、重复 ACK 走有界队列。测试怎么断言？ - -- **A. 直接断言内部状态**：`hub.ClipBucket` 可观察的剩余令牌数（需给 TokenBucket 加测试可见的读取，或在测试程序集可见的 internal 属性） - - 优：精确、无歧义，直接锁死"没消耗"。 - - 劣：依赖内部结构，重构时测试要跟着改；可能需要给生产代码加 internal 只读成员（InternalsVisibleTo 已存在，成本低）。 -- **B. 行为级断言**：先真实发送 burst 上限（10）条不同 clip 耗尽令牌桶，再发重复 id 的 clip——断言仍能拿到 ACK（未被 rate_limited）；对照组：发新 id 的 clip 被拒 - - 优：不依赖内部实现，从客户端可观测行为验证，天然防回归。 - - 劣：构造稍复杂（要精确耗尽桶）；时间相关的 refill 需要用注入 IClock 控制。 -- **C. 两者都要**：内部断言精确性 + 行为断言防回归 - - 优：覆盖最全。 - - 劣：维护成本最高；两套断言可能对同一行为给出矛盾信号（若实现变化，需判断哪个是"真相"）。 - -选择：**B. 行为级断言**：耗尽 burst 后重复 id 仍获 ACK、新 id 被拒；用注入 IClock 控制 refill（2026-08-27）。 - ---- - -## 决策汇总与冲突检查 - -| 题 | 选择 | 状态 | -|---|---|---| -| Q1 测试宿主 | B. Tests 项目内 NetworkIntegration/ 目录 + 自建 fixture | 已选 | -| Q2 过滤机制 | A. [Trait(Category=NetworkIntegration)] + --filter；ci.yml 需加排除 | 已选 | -| Q3 测试证书 | A. 运行时自签生成 | 已选（默认） | -| Q4 重启形态 | A. 同进程 CreateApp 两次停启 | 已选（默认） | -| Q5 TLS 断言 | A. 客户端显式 Tls12/Tls13 握手断言 | 已选 | -| Q6 契约组织 | A. ContractTests/ 目录 + 样本文件落盘 | 已选 | -| Q7 样本范围 | A. 全矩阵（3 消息类型 × 8 形态） | 已选 | -| Q8 单测范围 | A. 全部补齐（约 25-35 用例） | 已选 | -| Q9 Argon2 | C. 混合：假哈希 + Category=SlowHash 专项 | 已选 | -| Q10 断言深度 | B. 行为级断言（耗尽桶后重复 id 仍 ACK） | 已选 | - -### 冲突检查结论（2026-08-27） - -- Q3（自签证书）× Q5（真实 TLS 握手）：无冲突——自签证书正是真实握手所需，互补。 -- Q2（Trait 过滤）× Q9（新增 SlowHash 类别）：兼容——同一机制扩展一个新 Category 值，CI 排除 `Category!=NetworkIntegration&Category!=SlowHash`。 -- Q1（同项目目录）× Q4（CreateApp 两次停启）：兼容——fixture 放 NetworkIntegration/ 目录内，持有 WebApplication 列表即可。 -- Q6（样本落盘）× Q7（全矩阵）：兼容——全矩阵约 24+ 样本文件，落盘组织正是为此设计；测试 csproj 需 CopyToOutputDirectory。 -- Q8（全部补齐）× Q9（混合）：兼容——Argon2 三函数的功能断言放 SlowHash 专项，认证路径其余测试保持假哈希器。 -- Q10（行为级）× Q8（补重复 id 缺口）：兼容——该缺口以行为级用例补齐，无需改生产代码。 -- 无发现组合冲突。 - ---- \ No newline at end of file diff --git a/specs/test-and-contract-spec.md b/specs/test-and-contract-spec.md deleted file mode 100644 index 602cf92..0000000 --- a/specs/test-and-contract-spec.md +++ /dev/null @@ -1,316 +0,0 @@ -# TextCascade 测试与契约规格（函数级） - -状态：**已落地实施**（2026-08-27，代码基线 v0.3.5；测试结果 162 默认 + 12 NetworkIntegration + 3 SlowHash 全部通过） -日期：2026-08-27 -依据：docs/server-spec.md 审计结论 + git 溯源 + [spec-decisions.md](spec-decisions.md) 10 项决策 -代码基线：v0.3.5（commit 9ed6eba） - -> **实施状态（2026-08-27）** -> - §1 NetworkIntegration：12 个用例全部落地（`NetworkIntegration/` 目录：TlsAndWssHandshakeTests 6、FrameFragmentationTests 3、RestartRecoveryTests 3）。实施偏差：N2 的 TLS 版本探测改用 SslStream 直连（.NET 10 移除了 `ClientWebSocketOptions.SslProtocols`，WSS 版本协商只能跟随 OS 策略）；N11/N12 合并为一条完整链路用例；双 hello 竞态产生的重复 welcome 在测试内按良性帧跳过（spec §15 台账外发现，测试已文档化）。 -> - §2 契约测试：样本目录与驱动器已落地（ContractSamples/ valid 6 + invalid 17 + README；驱动器按子目录推断期望码；6+2 条序列化不变式）。非法数字/非法 UTF-8 以"字段类型污染"等价覆盖（README 已写明映射表），未做逐字段全矩阵——对同一解析分支的重复样本已合并。 -> - §3 单元缺口：AuthDeepTests 12、CliWatermarkTests 8、IdempotencyBehaviorTests 5、SlowHashSmokeTests 3 已全部落地。实施偏差：U10 静态 `NeedsRehash` 参数矩阵并入 U27；U12（NextVersion 溢出）已存在于 ClipAndCoreTests 未重复；SlowHash 断言改为回读实际编码参数（Isopoh 写入的 p= 与配置 Argon2Parallelism 不一致，是 server-spec §15 级别的已知实现事实）。 -> - ci.yml Test 步骤已加 `--filter "Category!=NetworkIntegration&Category!=SlowHash"`。 - -本规格覆盖三块内容： -1. **NetworkIntegration 本地网络集成测试**（此前从未实现，全历史零命中）。 -2. **契约测试**（ContractSamples 样本集，覆盖 JSON 深度 3 / 重复字段 / 未知字段 / 非法数字 / 非法 UTF-8 全矩阵）。 -3. **单元测试缺口补齐**（Argon2 三函数、token 非法形态、CLI 水位与溢出、WithVersion、重复 id 不耗令牌桶）。 - -所有函数名、类型名均已在 v0.3.5 源码中核实存在；标注 `[新增 internal 可见成员]` 的除外。 - ---- - -## 0. 决策落地总览 - -| 决策 | 落地方式 | -|---|---| -| Q1 测试宿主 | `TextCascade.Server.Tests` 项目内新建 `NetworkIntegration/` 目录 + 自建 fixture | -| Q2 过滤机制 | 类级 `[Trait("Category", "NetworkIntegration")]`；CI 用 `--filter Category!=NetworkIntegration` 排除 | -| Q3 测试证书 | fixture 运行时用 `CertificateRequest` 自签生成（约 30 行 helper），并顺带生成带密码 PFX 用于拒绝路径 | -| Q4 重启形态 | 同进程 `ServerHost.CreateApp` 构建两次：第一次 `StopAsync` 后第二次启动，共用临时目录 | -| Q5 TLS 断言 | 客户端 `SslProtocols.Tls12` 与 `Tls13` 各发起一次 WSS，断言握手成功 | -| Q6 契约组织 | Tests 项目内新建 `ContractTests/` + `ContractSamples/` 目录，样本 `.json` 文件落盘，csproj CopyToOutputDirectory | -| Q7 样本范围 | 全矩阵：hello / clip / pong 三种消息 × 8 种非法形态 | -| Q8 单测范围 | 全部缺口逐函数补齐（约 30 个用例） | -| Q9 Argon2 | 单测注入假哈希器；Argon2 真实链路放 `Category=SlowHash` 专项；CI 过滤为 `Category!=NetworkIntegration&Category!=SlowHash` | -| Q10 断言深度 | 行为级：耗尽 burst 后重复 id 仍获 ACK、新 id 被 rate_limited | - ---- - -## 1. NetworkIntegration 测试（Category=NetworkIntegration） - -### 1.1 基础设施（新建文件） - -#### `NetworkIntegration/NetworkTestFixture.cs` - -职责：TLS 服务端托管、自签证书、客户端工厂。不与现有 `IntegrationTestFixture` 共享代码（Q1 选择 B 的目的就是隔离），但复制其最小必要逻辑（TestLogCollector、FastPasswordHasher 模式）。 - -```csharp -public sealed class NetworkTestFixture : IAsyncDisposable -{ - // 核心 API（按 Q3/Q4/Q5 决策设计） - public string TempDir { get; } // Path.Combine(Path.GetTempPath(), "textcascade-ni-" + Guid.NewGuid()) - public RuntimeConfig Config { get; } - public TestLogCollector Logs { get; } - - // 启动一个 HTTPS Kestrel 实例，绑定 127.0.0.1:0（随机端口），返回实际端口 - public Task StartAsync(UsersFile? users = null); - - // 停止指定实例（供 Q4 重启场景调用） - public Task StopAsync(RunningServer server); -} - -public sealed class RunningServer -{ - public WebApplication App { get; } // ServerHost.CreateApp(args, config, users, stateStore, hasher, clock, certificate) 构建后 RunAsync - public int Port { get; } // 从 Kestrel IServerAddressesFeature 读取实际绑定端口 - public UsersFile Users { get; } // 与第二次重启共用 - public RuntimeStateStore StateStore { get; } -} -``` - -证书 helper（同文件或 `SelfSignedCertificate.cs`）： - -```csharp -// 用 CertificateRequest 生成 RSA2048 自签叶证书（SAN: localhost, 127.0.0.1） -public static X509Certificate2 CreateSelfSigned(); -// 导出无密码 PFX 到 TempDir，返回路径（走 CertificateLoader.Load 的 .pfx 分支） -public static string WritePfx(X509Certificate2 cert); -// 导出带密码 PFX —— 仅用于"密码 PFX 必须启动失败"的负路径 -public static string WritePasswordProtectedPfx(X509Certificate2 cert, string password); -// PEM bundle（叶+私钥单文件）—— 覆盖 .pem 加载分支 -public static string WritePemBundle(X509Certificate2 cert); -``` - -关键实现约束： - -- 服务端构建必须走生产入口 `ServerHost.CreateApp(string[] args, RuntimeConfig config, UsersFile users, RuntimeStateStore stateStore, IPasswordHasher? hasher = null, IClock? clock = null, LoadedCertificate? certificate = null)`（ServerHost.cs:68），certificate 参数传真实 `LoadedCertificate`，以验证 `ConfigureKestrel(config, certificate)` 的 UseHttps 绑定。`LoadedCertificate` 由 `CertificateLoader.Load(path)`（internal，Tests 已有 InternalsVisibleTo）产生。 -- Config 在默认值基础上调整：`hello_timeout_seconds=5` 保持默认；心跳间隔缩到 2 秒可缩短部分用例时长（可选）；`snapshot_window_seconds=3` 保持。 -- hasher 注入 `FastPasswordHasher`（复制现有实现，ValidHash 常量同步），保证登录不慢。 - -客户端工厂： - -```csharp -// WSS 客户端：跳过证书校验（自签）；subProtocol 默认 textcascade.v1；sslVersion 显式指定（Q5） -public static Task<(ClientWebSocket Socket, HttpClient Http)> ConnectWssAsync( - int port, string token, - SslProtocols sslVersion, - string? subProtocol = "textcascade.v1"); -``` - -### 1.2 测试类与用例（每条：方法 → 输入构造 → 断言） - -全部类声明 `[Trait("Category", "NetworkIntegration")]`（Q2）。运行命令： - -```bash -dotnet test TextCascade.Server.slnx --filter Category=NetworkIntegration -dotnet test TextCascade.Server.slnx --filter Category!=NetworkIntegration # CI 默认 -``` - -#### A. `TlsAndWssHandshakeTests` - -| # | 测试方法 | 输入构造 | 断言 | -|---|---|---|---| -| N1 | `Connects_WithSelfSignedPfx_OverWss` | fixture 写无密码 PFX → `CertificateLoader.Load` → StartAsync → 登录取 token → `ConnectWssAsync(port, token, Tls13)` | 握手成功；收到首帧 welcome（或 hello 前 401 不发生）；socket.State == Open | -| N2 | `Accepts_Tls12_Client` | 同 N1，但 `SslProtocols.Tls12`（Q5） | 握手成功。失败消息注明"OS 政策禁用 TLS1.2 时属环境问题" | -| N3 | `HttpUpgrade_Succeeds_WithBearerAndSubProtocol` | ClientWebSocket 同时设置 `Authorization: Bearer ` 与 `AddSubProtocol("textcascade.v1")` | HTTP 101；welcome.protocolVersion == 1 | -| N4 | `HttpsLogin_Endpoint_Works` | 对 `https://127.0.0.1:{port}/api/v1/login` POST 合法凭据（HttpClient 自动处理自签错误） | 200；响应含 token、expiresAtUtc、protocolVersion、maxTextBytes 等 7 固定字段 | -| N5 | `RandomPortBinding_ActuallyBinds` | StartAsync 后读 IServerAddressesFeature | 地址形如 `https://127.0.0.1:{n>0}` 且连接成功（N1 已隐含，此处显式断言端口非 0） | - -#### B. `FrameFragmentationTests` - -| # | 测试方法 | 输入构造 | 断言 | -|---|---|---|---| -| N6 | `FragmentedClip_Reassembles_AndBroadcasts` | 两个客户端同用户连上；A 发送一条 clip，payload 约 300KB（> 单帧常见 MSS 分片规模），手动分片发送：先 `SendAsync(buffer[0..100k], EndOfMessage:false)` 两段再 `EndOfMessage:true` | B 收到完整 clip 广播，payload 字节数一致；A 收到 clip_ack | -| N7 | `OversizeFrame_Closes1009` | A 发送总长 > max_frame_bytes(589824) 的分片帧 | 连接被服务端关闭，close status == CloseStatusStatusCode.MessageTooBig(1009)；关闭前可选收到 frame_too_large error 帧（实现行为是先 error 再 close） | -| N8 | `ZeroLengthFrame_TreatedAsFrameTooLarge` | A 发送 0 字节 EndOfMessage:true 帧 | 连接关闭 1009（锁定当前实现的零长帧判定，见 server-spec §5 差异表第 8 条） | - -#### C. `RestartRecoveryTests`（Q4：CreateApp 两次停启） - -| # | 测试方法 | 输入构造 | 断言 | -|---|---|---|---| -| N9 | `Restart_KeepsTokenValid_DirectReconnect` | 第一次 StartAsync → 登录取 token V1 → 发一条 clip（version=v1）→ StopAsync（确认状态文件已在 TempDir 落盘）→ 第二次 StartAsync **复用同一 UsersFile 与同一 StateStore 目录** → 直接用旧 token V1 建 WSS（不重新登录） | 第二次连接握手成功；断言重启后 hub 初始版本来自持久化水位（下一条 clip 版本 = v1+1） | -| N10 | `Restart_SnapshotElection_RestoresLatest` | N9 流程 + 重启后两个客户端分别在 hello 带 lastServerVersion=128/64 的 snapshot | welcome.latest.version == 128（winner 不加一）；随后 B 发 clip 得到 129 | -| N11 | `Shutdown_BroadcastsBye_ThenCloses1001` | 一个在线客户端；对 RunningServer 调优雅停机路径（StopAsync 触发 SyncServer.ShutdownAsync） | 先收到 `{"type":"bye","reason":"server_shutdown"}`，随后 close status == EndpointUnavailable(1001)。（此用例可与 N12 合并为一个进程内场景） | -| N12 | `RealLogin_Connect_Send_Receive_FullChain` | 完整链路：HTTPS 登录 → WSS 连接 → hello → A 发 clip → B 收广播 + A 收 ACK | 各消息字段类型正确（version 为 ulong、updatedAtUtc 含 Z 后缀等） | - -实施注意（写入实施清单，本轮不改代码）： - -- `.github/workflows/ci.yml` Test 步骤改为 `dotnet test ... --filter Category!=NetworkIntegration&Category!=SlowHash`（与 Q9 联动）。 -- Windows 本地跑 N1/N2 若公司策略限制自签证书可能需要 `X509KeyStorageFlags` 调整——fixture 中集中封装一处。 - ---- - -## 2. 契约测试（ContractSamples + ContractTests） - -### 2.1 目录组织（Q6=A/Q7=A） - -``` -TextCascade.Server.Tests/ - ContractTests/ - ContractSampleTests.cs // Theory 驱动器 - ContractSchemaInvariants.cs // 正向样本字段序/序列化断言 - ContractSamples/ - valid/ - hello.full.json // hello 全字段合法样本 - hello.minimal.json // 无 snapshot 合法样本 - clip.basic.json // clip 四字段合法样本 - pong.ok.json - login.request.json / login.response.json / login.response.rehash.json - welcome.no-latest.json // 断言 latest 键整体省略而非 null - welcome.with-latest.json // 断言六字段齐全及键序 - broadcast.clip.json / clip_ack.json / ping.json / bye.json / error.json - invalid/ - depth-4/*.json // 深度 4 样本（深度限 3） - duplicate-field/hello.*.json // 每消息类型一份重复字段 - unknown-field/hello.*.json - number/ - hello.lastserverversion.{negative,fraction,exponent,string,toobig}.json - clip.id.{...}.json // 同五形态 - pong.clienttimeutc 数字污染样本（clientTimeUtc 处放非法值不影响 type 解析的场景说明见 2.3） - token.payload.{negative,fraction,string}.json // TokenService.VerifyToken 直测样本 - utf8/ - hello.invalid-utf8.bin.json // 说明文件内嵌 \uD800 孤代理对 - clip.invalid-utf8.payload.txt // 原始字节序列（非合法 UTF-8 的 payload 场景以文档标注） -``` - -csproj 增补（实施清单）：`` - -### 2.2 驱动器设计 - -```csharp -public static IEnumerable InvalidSamples => Directory.GetFiles( - Path.Combine(AppContext.BaseDirectory, "ContractSamples", "invalid"), "*.json", SearchOption.AllDirectories) - .Select(f => new object[] { f }); - -[Theory, MemberData(nameof(InvalidSamples))] -public void AllInvalidSamples_AreRejected_WithExpectedCode(string path) -{ - var frame = File.ReadAllBytes(path); - var result = Protocol.ParseClientMessage(frame, RuntimeConfig.CreateDefaultConfig()); - Assert.True(result.IsFailure); // ParseResult.Failure - var expected = ExpectedCodeAnnotation.Read(path); // 见 2.3 注释约定 - Assert.Equal(expected, result.Error!.CodeName); // CodeName 属性已有 -} -``` - -正向样本单独断言：`ParseClientMessage` Success + MessageKind 正确 + record 字段逐项相等（如 `ClientHello.ClientId`、`ClipSnapshot.LocalModifiedAtUtc` 的两种合法时间格式 `"yyyy-MM-ddTHH:mm:ssZ"` 与 `"O"` round-trip）。 - -序列化侧不变式（ContractSchemaInvariants.cs）： - -| # | 用例 | 断言对象 | 断言 | -|---|---|---|---| -| C1 | `Welcome_NoLatest_OmitsKey` | `Protocol.SerializeWelcome(null)` | 输出不含 `"latest"` 子串（当前 WhenWritingNull 行为，对照 welcome.no-latest.json） | -| C2 | `Welcome_WithLatest_FixedFieldOrder` | `SerializeWelcome(latest)` | 字节级与 welcome.with-latest.json 完全一致（UTF-8 bytes Equal），锁定 `protocolVersion→latest` 及 latest 内部键序 | -| C3 | `BroadcastClip_ContainsAllEightFields` | `Protocol.SerializeClip(...)` | 八字段齐且顺序与样本一致 | -| C4 | `TokenPayload_MinimalFixedOrder` | `Auth.SignToken(payload, secret)` | base64url 解码后 JSON 键序恰为 sub,ver,iat,exp，无数空格 | -| C5 | `ErrorResponse_IncludesReferenceId_WhenNotNull` | `Protocol.SerializeProtocolError` | referenceId 非 null 时在列；null 时省略 | -| C6 | `Timestamp_Formats_UtcZ` | PingMessage 序列化 | serverTimeUtc 以 Z 结尾秒级格式 | - -### 2.3 样本预期结果标注约定 - -每个 invalid 样本文件首行注释 `// expect: invalid_message`（JSON 允许 // 会被 System.Text.Json 拒绝——因此不用行内注释，改用伴随文件或文件名约定）： - -**采用文件名约定**：`invalid/number/hello.lastserverversion.negative.expect-invalid_message.json` 同目录放同名 `.expect` 文件过重——最终采用：目录即类别（number/duplicate-field/unknown-field/utf8/depth-4），全部预期 `invalid_message`；唯一例外 `depth-4/` 预期也是 `invalid_message`（当前实现对超深返回 invalid_message）。若有未来样本预期其他码，放置于 `expect-frame_too_large/` 等新目录。驱动器按一级子目录名推断期望码，缺省 invalid_message。 - -### 2.4 全矩阵清单（Q7=A） - -每种消息类型 × 8 形态 = 24 个核心样本 + 正向样本 10 个 + token 直测 3 个 ≈ **37 个文件**： - -| 形态 | hello 样本注入点 | clip 样本注入点 | pong 样本注入点 | -|---|---|---|---| -| 负数 | lastServerVersion=-1 | id 不能为数字→ 改 payload 数量型字段不可行，clip 注入 encrypted:"yes"（字符串枚举污染）| clientTimeUtc 缺失/类型错 | -| 小数 | lastServerVersion=1.5 | —（clip 无数值字段；样本改为 hash 字段数字类型污染） | clientTimeUtc=1.5 | -| 指数 | lastServerVersion=1e3 | 同上原则 | clientTimeUtc=1e3 | -| 字符串数字 | lastServerVersion="128" | encrypted="true"（应为 bool） | clientTimeUtc="2026-..."字符串包裹 | -| 超 long | lastServerVersion=18446744073709551616（>ulong） | — | — | -| 重复字段 | 双 type 或双 clientId | 双 id | 双 type | -| 未知字段 | extra:"x" | extra:"x" | extra:"x" | -| 非法 UTF-8 | clientId 内孤代理对 | payload 孤代理对 | clientTimeUtc 孤代理对 | - -注：clip/pong 无原生数值字段的格，按"字段类型污染"等价覆盖（同一 JSON reader 数字分支），表中标"—"处移到最近似字段；这正是"等价分支合并"而非漏测，在样本 README.md 中写明映射关系。 - -token 直测 3 个（不走 WebSocket，直接 `TokenService.TryVerifyToken` + 手工构造 compact token）：payload 负数 ver、iat 小数、exp 字符串形式 —— 全部 false。 - ---- - -## 3. 单元测试缺口补齐（Category 默认，Q8=A） - -以下用例加入现有 Tests 项目（不建新项目），分三个新文件。所有被测函数均已核实存在。 - -### 3.1 `AuthDeepTests.cs`（除 SlowHash 外全部用假哈希器或纯数据构造） - -| # | 方法 | 被测函数 | 断言 | -|---|---|---|---| -| U1 | `SignToken_FieldOrder_And_MinimalJson` | `Auth.TokenService.SignToken` (Auth.cs:124) | 解码后恰为 {"sub":..,"ver":..,"iat":..,"exp":..} 顺序、无空格（与契约 C4 一致，此处单元级锚定） | -| U2 | `VerifyToken_Rejects_DuplicateFields` | `TokenService.TryVerifyToken` (149) | 手工构造含重复 "ver" 的 payload+正确 HMAC → false | -| U3 | `VerifyToken_Rejects_UnknownField` | 同上 | 多出一个 "aud" 字段 → false（即使验签通过也不行——需重签，样本构造 helper `MakeCompact(payloadJson)` 写进测试内部） | -| U4 | `VerifyToken_Rejects_FractionNumber` | 同上 | iat=1760000000.0 → false | -| U5 | `VerifyToken_Rejects_StringNumber` | 同上 | exp="1762592000" → false | -| U6 | `VerifyToken_Rejects_NegativeValue` | 同上 | ver=-1 → false | -| U7 | `VerifyToken_Rejects_ExpBeforeIat` | 同上 | exp <= iat → false | -| U8 | `VerifyToken_Rejects_AllPositiveCheck` | 同上 | iat=0 → false | -| U9 | `VerifyToken_RoundTrip_InstanceOverload` | `TokenService.CreateToken/VerifyToken(instance)` (108/144) | CreateToken 产物经 instance VerifyToken 通过且 payload 字段相等 | -| U10 | `NeedsRehash_ParameterParsing`（纯解析，不真算） | `Argon2PasswordHasher.NeedsRehash(string, int, int, int)` 静态版 (26) | 编码串 m/t/p 与传入参数不一致 → true；一致 → false；非 argon2id 前缀 → 按实现断言（编写时读静态实现确认分支后固定） | -| U11 | `WithVersion_Produces_NewImmutableRecord` | `CoreLogic.WithVersion` (Core.cs:268) | 返回新 LatestText：version 更新、其他字段保留、原实例未被修改；nowUtc=null 时沿用原 updatedAtUtc，显式传入时生效 | -| U12 | `NextVersion_At_UlongMaxValue_Throws` （已存在于 ClipAndCoreTests，若覆盖则跳过此项） | `CoreLogic.NextVersion` (258) | OverflowException | - -### 3.2 `CliWatermarkTests.cs`（用户文件水位逻辑，跑真实 CLI 命令函数但注入 FastPasswordHasher） - -| # | 方法 | 被测函数 | 输入构造 | 断言 | -|---|---|---|---|---| -| U13 | `AddUser_Allocates_FromWatermark_Increments` | `Cli.CommandAddUser`（private，经 `RunCli(new[]{"user","add",...})` 驱动） | 临时 users.json：nextTokenVersion=7，一个老用户 tokenVersion=3 | 命令 Ok；重载文件：新用户 tokenVersion==7，nextTokenVersion==8，老用户不动 | -| U14 | `DeleteUser_RecreateSameName_GetsFreshHigherVersion` | `RunCli user delete` + `user add` | nextTokenVersion=5、alice tokenVersion=2 → 删除 alice → 重建 alice | 新 alice tokenVersion==5（取全局水位）≠ 旧 2，nextTokenVersion==6 | -| U15 | `RevokeTokens_Sets_Watermark_Increments` | `CommandRevokeTokens` 经 RunCli | nextTokenVersion=9、bob tokenVersion=4 | bob.tokenVersion==9，nextTokenVersion==10 | -| U16 | `AddUser_At_LongMaxValue_FailsFast_FileUnchanged` | RunCli add | nextTokenVersion==long.MaxValue（手写 JSON） | 返回 Error；文件字节级未变（SaveUsers 前置 ValidateUsers/checked 溢出保护，Users.cs:130 atomic write 未触发） | -| U17 | `Revoke_At_LongMaxValue_FailsFast` | RunCli revoke-tokens | 同上 | Error；文件未变 | -| U18 | `ValidateUsers_NextMustExceed_AllUserVersions` | `UsersFile.ValidateUsers` (Users.cs:98) | nextTokenVersion=5 但某用户 tokenVersion=5 | 抛异常（InvalidOperationException），消息含 nextTokenVersion | -| U19 | `ValidateUsers_Rejects_NonPositiveVersion` | 同上 | tokenVersion=0 或 -1 | 抛异常 | -| U20 | `SaveUsers_AtomicWrite_LeavesOriginal_OnValidationFailure` | `UsersFile.SaveUsers` (130) | 构造非法 UsersFile（U18 场景）直接调 SaveUsers | 抛出且目标路径内容仍是旧内容（若存在）或不产生文件；临时文件不残留 | -| U21 | `HashPassword_VerifyPassword_Smoke`（真实 Argon2 1 例 smoke，走 SlowHash 见 3.4） | - -### 3.3 `IdempotencyBehaviorTests.cs`（Q10=B 行为级） - -| # | 方法 | 被测对象 | 输入构造 | 断言 | -|---|---|---|---|---| -| U22 | `DuplicateId_AfterBucketDrained_StillAcked` | `UserHub.ApplyClip`（UserHub.cs:280） | UserHub(initialVersion 由 ctor 给出) + StubConnectionContext；注入可控 clock：先用 burst=10 个不同 id 耗尽 `hub.ClipBucket`（clock 固定不前进则 refill=0）；第 11 个不同 id → 应得 rate_limited；然后发重复 id（与第 1 条相同 id+相同 payload/hash/encrypted）→ **成功 ACK，且不被 rate_limited** | 收到 clip_ack 且 version == 第 1 条的 version；期间 `hub.Version` 未变 | -| U23 | `DuplicateId_NewContent_IsTreatedAsFreshMessage` | 同上 | 同 id 不同 payload（沿用 U22 环境，令牌可用状态下） | 记录 warning 日志（StubLogger 收集 "Replacing reused clip id"）；产生新版本；消耗一个令牌（后续同样消息数会 rate_limited 提前触发） | -| U24 | `DuplicateId_LatestNull_FallbackAckHasEmptyPayload` | 同上 | 手工将 SeenIds 记忆置为(id, null) 的窗口场景：fresh ring 后 RememberId(id,null) 无法直接做——改为断言 IsUnchangedDuplicate(id,payload,...) 对 "entry 不存在" 返回 false | （文档化死分支：ApplyClip 中 duplicateLatest ?? Latest ?? fallback 第三段不可达，本用例锁定 IsUnchangedDuplicate 行为即可，不为死分支写生产代码路径测试） | -| U25 | `TryAcquireClockRefill_BoundaryCases`（补充现有 TokenBucketRefillsOverTime 的边界） | `TokenBucket.TryAcquire` | 同一时刻连取 burst 次 → 第 burst+1 次 false；时间前进 500ms（tokensPerSecond=2）→ true；倒流时钟（nowUtc < lastRefill）→ false（Core.cs:150 分支） | 逐一符合 | - -环境要求（列入实施前置）：`ConnectionContext`/`ConnectionStateBag` 需要 stub 化 socket——现有 `UserLoopConcurrencyTests` 已有做法，复用其 stub 模式；若不可复用则在测试内新建 `StubSocket : WebSocket`。 - -### 3.4 `SlowHashSmokeTests.cs`（Q9=C 专项） - -```csharp -[Trait("Category", "SlowHash")] -public class SlowHashSmokeTests -{ - // 真实 Argon2PasswordHasher（Isopoh），参数用 Cli.CreateArgon2Config(config) 生产默认 - U26 Hash_Then_Verify_RoundTrip // Hash("pw") → Verify("pw")==true、Verify("wrong")==false - U27 NeedsRehash_CurrentParams_ReturnsFalse // 刚生成的哈希对同参数应 false - U28 NeedsRehash_StaleParams_ReturnsTrue // 手工把编码串 m=19456,t=2,p=1 改成 m=1024,t=1,p=1 → true - U29 Timing_DummyHash_Security_Note // 不测毫秒级时序（脆弱），仅注释指向 AuthServiceTimingTests 已有覆盖 -} -``` - -CI 最终过滤（实施清单，ci.yml 同步修改）： -`--filter Category!=NetworkIntegration&Category!=SlowHash` -本地专项： -`--filter Category=SlowHash` / `--filter Category=NetworkIntegration` - ---- - -## 4. 实施顺序建议 - -1. 契约测试（§2）：零生产代码依赖改动，只加 csproj Content 项 → 最快见效。 -2. 单元缺口（§3.1–§3.3）：纯新增文件；唯一可能的 touch 点是若 stub 需要暴露 TokenBucket 只读成员（Q10 选了 B 行为级，通常不需要）。 -3. NetworkInfrastructure（§1）：fixture + 11 个网络用例 + ci.yml 过滤同步。 -4. 全部落地后 docs/server-spec.md §10 按本规格回填测试矩阵描述（修订工作另见 spec-decisions.md 路径 A/C 执行记录）。 - -## 5. 范围外声明 - -以下从未实现项本轮**明确不做**，相关条目将从 docs/server-spec.md 移除（另一步执行）： -- Benchmark 项目与压测场景（原 spec §10.4） -- `server_stop` 安全事件（原 spec §8.1 事件表行） -- 性能指标目标表（原 spec §9 整节） From f4b7fe467baab6b5ad7b4892b3e23354460a7d54 Mon Sep 17 00:00:00 2001 From: long45343 <1725334094@qq.com> Date: Fri, 4 Sep 2026 10:32:24 +0800 Subject: [PATCH 30/32] refactor: replace hand-rolled helpers with BCL equivalents (v0.5.0) A-tier stdlib audit replacements, zero behavior change: - Auth.cs: Base64Url codec -> System.Buffers.Text.Base64Url - IClock/SystemClock -> System.TimeProvider (DI singleton, tests pass TimeProvider.System) - HeartbeatScannerService: System.Threading.Timer -> PeriodicTimer in BackgroundService (scan exceptions logged, stop order preserved) - SingleInstanceLock: drop PID probing/stale recovery; OpenOrCreate + FileShare.None (OS releases on holder death, leftover files reused) - ParseLoginRequest: single strict JsonSerializer.DeserializeAsync pass (net10 AllowDuplicateProperties=false, UnmappedMemberHandling.Disallow); 16KB cap via IHttpMaxRequestBodySizeFeature; "Invalid login request." merges into "Invalid JSON." (400/invalid_request unchanged) Tests: 176/176 passing (Release x2 + Debug x1). Deployed to piyansvm.top and verified. --- CHANGELOG.md | 7 +- .../AuthServiceTimingTests.cs | 6 +- .../NetworkIntegration/NetworkTestFixture.cs | 2 +- .../RuntimeStateAndProtocolTests.cs | 4 +- .../SingleInstanceLockTests.cs | 22 ----- .../UserFileWatcherTests.cs | 8 +- .../WebSocketIntegrationTests.cs | 2 +- TextCascade.Server/Auth.cs | 41 +++------- TextCascade.Server/AuthService.cs | 81 +++++++------------ TextCascade.Server/Cli.cs | 73 ++--------------- .../Hosting/HeartbeatScannerService.cs | 53 +++++++----- TextCascade.Server/ServerHost.cs | 6 +- TextCascade.Server/SyncServer.cs | 18 +---- TextCascade.Server/TextCascade.Server.csproj | 2 +- docs/server-spec.md | 4 +- 15 files changed, 106 insertions(+), 223 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index bd5ec3b..6771122 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,7 +5,12 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). -## [Unreleased] +## [0.5.0] - 2026-09-04 + +### Changed +- Replaced hand-rolled helpers with standard library equivalents, with no behavior change: `TokenService` base64url codec now uses `System.Buffers.Text.Base64Url` (.NET 9 BCL) instead of a custom encode/decode pair; the `IClock`/`SystemClock` seam was replaced by `System.TimeProvider` (registered in DI, tests pass `TimeProvider.System`); `HeartbeatScannerService` was rewritten from a `System.Threading.Timer` callback to `PeriodicTimer` inside `BackgroundService` (scan exceptions are now logged instead of being unobserved). +- Login request parsing (`AuthService.ParseLoginRequest`) now performs a single strict `JsonSerializer.DeserializeAsync` pass with .NET 10 `AllowDuplicateProperties = false` and `JsonUnmappedMemberHandling.Disallow`, replacing the manual 4 KB chunked read loop plus a double `JsonDocument`/`JsonSerializer` parse; the 16 KB cap is enforced via `IHttpMaxRequestBodySizeFeature` (chunked bodies included). Structural violations previously reported as "Invalid login request." now return "Invalid JSON." — status 400 and the `invalid_request` code are unchanged. +- `SingleInstanceLock` no longer probes or recovers PIDs: the lock file (`users.json.lock`) is opened with `FileMode.OpenOrCreate` + `FileShare.None`, so the OS releases it when the holder process dies and a leftover file from a crash is simply reopened instead of blocking. The PID written into the file is diagnostic only. Contention behavior (3 retries, then graceful failure) is unchanged. ### Fixed - TLS server authentication failed on Windows with "platform does not support ephemeral keys" (0x8009030E): `CertificateLoader` now loads PFX certificates with a persisted key set (`DefaultKeySet`) on Windows (Linux keeps `EphemeralKeySet`), and PEM-loaded certificates are re-exported to a persisted key on Windows. Spec §2.1's Windows Service hosting shape required this to be usable. Found while benchmarking 1000 concurrent connections on a local Windows deployment. diff --git a/TextCascade.Server.Tests/AuthServiceTimingTests.cs b/TextCascade.Server.Tests/AuthServiceTimingTests.cs index 4fb6092..43071f1 100644 --- a/TextCascade.Server.Tests/AuthServiceTimingTests.cs +++ b/TextCascade.Server.Tests/AuthServiceTimingTests.cs @@ -80,7 +80,7 @@ public async Task LoginVerificationRunsForMissingUser() try { var stateStore = new RuntimeStateStore(tempState); - var server = new SyncServer(config, users, stateStore, hasher, new SystemClock(), Microsoft.Extensions.Logging.Abstractions.NullLogger.Instance); + var server = new SyncServer(config, users, stateStore, hasher, TimeProvider.System, Microsoft.Extensions.Logging.Abstractions.NullLogger.Instance); Assert.Equal(server.LoginDummyHash, hasher.DummyHashReturn); @@ -121,7 +121,7 @@ public void LoginDummyHashUsesConfiguredArgon2Parameters() try { var stateStore = new RuntimeStateStore(tempState); - var server = new SyncServer(config, users, stateStore, hasher, new SystemClock(), Microsoft.Extensions.Logging.Abstractions.NullLogger.Instance); + var server = new SyncServer(config, users, stateStore, hasher, TimeProvider.System, Microsoft.Extensions.Logging.Abstractions.NullLogger.Instance); Assert.Single(hasher.HashCalls); var call = hasher.HashCalls[0]; @@ -152,7 +152,7 @@ public async Task DisabledUserStillReturnsUnifiedInvalidCredentials() try { var stateStore = new RuntimeStateStore(tempState); - var server = new SyncServer(config, users, stateStore, hasher, new SystemClock(), Microsoft.Extensions.Logging.Abstractions.NullLogger.Instance); + var server = new SyncServer(config, users, stateStore, hasher, TimeProvider.System, Microsoft.Extensions.Logging.Abstractions.NullLogger.Instance); var (context, responseBody) = CreateHttpContext("dave", "correct-password"); await AuthService.HandleLoginAsync(context, config, server, logger); diff --git a/TextCascade.Server.Tests/NetworkIntegration/NetworkTestFixture.cs b/TextCascade.Server.Tests/NetworkIntegration/NetworkTestFixture.cs index 6a68769..239f071 100644 --- a/TextCascade.Server.Tests/NetworkIntegration/NetworkTestFixture.cs +++ b/TextCascade.Server.Tests/NetworkIntegration/NetworkTestFixture.cs @@ -85,7 +85,7 @@ public async Task StartAsync() Users, new RuntimeStateStore(StatePath), hasher: new FastPasswordHasher(), - clock: new SystemClock(), + clock: TimeProvider.System, certificate: new LoadedCertificate( new X509Certificate2(PfxPath), new X509Certificate2Collection(new X509Certificate2(PfxPath)))); diff --git a/TextCascade.Server.Tests/RuntimeStateAndProtocolTests.cs b/TextCascade.Server.Tests/RuntimeStateAndProtocolTests.cs index 36a98dc..f5e5180 100644 --- a/TextCascade.Server.Tests/RuntimeStateAndProtocolTests.cs +++ b/TextCascade.Server.Tests/RuntimeStateAndProtocolTests.cs @@ -152,7 +152,7 @@ public void RecoveryWindowRestoresSnapshotAtPersistedVersion() new UsersFile(), stateStore, new Argon2PasswordHasher(), - new SystemClock(), + TimeProvider.System, NullLogger.Instance); var hub = new UserHub("alice", config, TestStartTime, server, server.RuntimeStateStore, 7UL); var modified = DateTimeOffset.FromUnixTimeSeconds(1759999990); @@ -193,7 +193,7 @@ public void RecoveryWindowIgnoresStaleSnapshot() new UsersFile(), stateStore, new Argon2PasswordHasher(), - new SystemClock(), + TimeProvider.System, NullLogger.Instance); var hub = new UserHub("alice", config, TestStartTime, server, server.RuntimeStateStore, 7UL); hub.AcceptSnapshot(new ClientHello( diff --git a/TextCascade.Server.Tests/SingleInstanceLockTests.cs b/TextCascade.Server.Tests/SingleInstanceLockTests.cs index 48672f4..15b2a3e 100644 --- a/TextCascade.Server.Tests/SingleInstanceLockTests.cs +++ b/TextCascade.Server.Tests/SingleInstanceLockTests.cs @@ -72,28 +72,6 @@ public void StaleLockIsRecovered() } } - [Fact] - public void LiveProcessLockIsNotRecovered() - { - var tempDir = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString("N")); - Directory.CreateDirectory(tempDir); - try - { - var lockPath = Path.Combine(tempDir, "users.json.lock"); - var currentPid = Environment.ProcessId; - File.WriteAllText(lockPath, currentPid.ToString(CultureInfo.InvariantCulture), Encoding.UTF8); - - using var handle = SingleInstanceLock.Acquire(lockPath, TimeSpan.FromMilliseconds(10)); - Assert.Null(handle); - Assert.True(File.Exists(lockPath)); - Assert.Equal(currentPid.ToString(CultureInfo.InvariantCulture), File.ReadAllText(lockPath).Trim()); - } - finally - { - if (Directory.Exists(tempDir)) Directory.Delete(tempDir, true); - } - } - [Fact] public void AcquireRejectsPathWithoutDirectory() { diff --git a/TextCascade.Server.Tests/UserFileWatcherTests.cs b/TextCascade.Server.Tests/UserFileWatcherTests.cs index 52f4bb6..7efc906 100644 --- a/TextCascade.Server.Tests/UserFileWatcherTests.cs +++ b/TextCascade.Server.Tests/UserFileWatcherTests.cs @@ -44,7 +44,7 @@ public async Task ReloadReplacesUserLookupAfterSave() initialUsers, new RuntimeStateStore(tempState), new Argon2PasswordHasher(), - new SystemClock(), + TimeProvider.System, NullLogger.Instance); using var watcher = new UserFileWatcher( @@ -106,7 +106,7 @@ public async Task InvalidReloadRetainsPreviousLookup() initialUsers, new RuntimeStateStore(tempState), new Argon2PasswordHasher(), - new SystemClock(), + TimeProvider.System, NullLogger.Instance); using var watcher = new UserFileWatcher( @@ -169,7 +169,7 @@ public async Task ConcurrentReloadObserversAlwaysSeeCompleteDictionary() usersA, new RuntimeStateStore(tempState), new Argon2PasswordHasher(), - new SystemClock(), + TimeProvider.System, NullLogger.Instance); using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(1)); @@ -224,7 +224,7 @@ public void WatcherDisposeIsIdempotent() initialUsers, new RuntimeStateStore(tempState), new Argon2PasswordHasher(), - new SystemClock(), + TimeProvider.System, NullLogger.Instance); var watcher = new UserFileWatcher(tempUsers, server, NullLogger.Instance); diff --git a/TextCascade.Server.Tests/WebSocketIntegrationTests.cs b/TextCascade.Server.Tests/WebSocketIntegrationTests.cs index 5602431..c0c29ca 100644 --- a/TextCascade.Server.Tests/WebSocketIntegrationTests.cs +++ b/TextCascade.Server.Tests/WebSocketIntegrationTests.cs @@ -102,7 +102,7 @@ public static async Task CreateAsync( users, stateStore, hasher: new FastPasswordHasher(), - clock: new SystemClock(), + clock: TimeProvider.System, certificate: null); var logs = new TestLogCollector(); diff --git a/TextCascade.Server/Auth.cs b/TextCascade.Server/Auth.cs index f402f4a..f0fa6f7 100644 --- a/TextCascade.Server/Auth.cs +++ b/TextCascade.Server/Auth.cs @@ -1,3 +1,4 @@ +using System.Buffers.Text; using System.Globalization; using System.Security.Cryptography; using System.Text; @@ -136,9 +137,9 @@ public static string SignToken(TokenPayload payload, byte[] secret) } var payloadBytes = stream.ToArray(); - var payloadSegment = Base64UrlEncode(payloadBytes); + var payloadSegment = Base64Url.EncodeToString(payloadBytes); var signature = HMACSHA256.HashData(secret, payloadBytes); - return $"{payloadSegment}.{Base64UrlEncode(signature)}"; + return $"{payloadSegment}.{Base64Url.EncodeToString(signature)}"; } public bool VerifyToken(string compactToken, DateTimeOffset nowUtc, IReadOnlyDictionary userLookup) @@ -265,46 +266,22 @@ private static bool TryParsePositiveInteger(string raw, out long value) return long.TryParse(span, NumberStyles.None, CultureInfo.InvariantCulture, out value) && value > 0; } - private static string Base64UrlEncode(byte[] bytes) - { - return Convert.ToBase64String(bytes).TrimEnd('=').Replace('+', '-').Replace('/', '_'); - } - - private static bool TryBase64UrlDecode(string text, ref byte[]? rented, Span buffer, out int length) + private static bool TryBase64UrlDecode(ReadOnlySpan text, ref byte[]? rented, Span buffer, out int length) { rented = null; - if (string.IsNullOrEmpty(text) || text.Length % 4 == 1) + if (text.IsEmpty || text.Length % 4 == 1) { length = 0; return false; } - foreach (var character in text) - { - if (!char.IsAsciiLetterOrDigit(character) && character is not ('-' or '_')) - { - length = 0; - return false; - } - } - - var paddedLength = text.Length + (4 - text.Length % 4) % 4; - if (paddedLength > buffer.Length) + var maxDecodedLength = Base64Url.GetMaxDecodedLength(text.Length); + if (maxDecodedLength > buffer.Length) { - rented = new byte[paddedLength]; + rented = new byte[maxDecodedLength]; buffer = rented; } - var chars = new char[paddedLength]; - for (var i = 0; i < text.Length; i++) - { - chars[i] = text[i] switch { '-' => '+', '_' => '/', _ => text[i] }; - } - - for (var i = text.Length; i < paddedLength; i++) - { - chars[i] = '='; - } - return Convert.TryFromBase64Chars(chars, buffer, out length) && length > 0; + return Base64Url.TryDecodeFromChars(text, buffer, out length) && length > 0; } } diff --git a/TextCascade.Server/AuthService.cs b/TextCascade.Server/AuthService.cs index da9304e..586cab0 100644 --- a/TextCascade.Server/AuthService.cs +++ b/TextCascade.Server/AuthService.cs @@ -1,7 +1,7 @@ -using System.IO; -using System.Text; using System.Text.Json; +using System.Text.Json.Serialization; using Microsoft.AspNetCore.Http; +using Microsoft.AspNetCore.Http.Features; namespace TextCascade.Server; @@ -12,14 +12,14 @@ public static async Task HandleLoginAsync(HttpContext context, RuntimeConfig con var limiter = syncServer.LoginLimiter; var clock = syncServer.Clock; var ip = context.Connection.RemoteIpAddress?.ToString() ?? "unknown"; - var now = clock.UtcNow; + var now = clock.GetUtcNow(); LoginRequest? request; try { request = await ParseLoginRequest(context); } - catch (Exception exception) when (exception is LoginParseException or JsonException or DecoderFallbackException) + catch (Exception exception) when (exception is LoginParseException or JsonException) { await WriteError(context, 400, "invalid_request", exception.Message); return; @@ -65,51 +65,39 @@ public static async Task HandleLoginAsync(HttpContext context, RuntimeConfig con await context.Response.BodyWriter.WriteAsync(bytes); } + private const int MaxLoginBodyBytes = 16384; + + // Login contract (spec §4.1): unknown fields, duplicate fields and nesting beyond + // depth 3 are rejected; names are exact lowercase via JsonPropertyName on LoginRequest. + private static readonly JsonSerializerOptions StrictLoginOptions = new() + { + MaxDepth = 3, + AllowDuplicateProperties = false, + UnmappedMemberHandling = JsonUnmappedMemberHandling.Disallow, + }; + private static async Task ParseLoginRequest(HttpContext context) { - if (context.Request.ContentLength is > 16384) + if (context.Request.ContentLength is > MaxLoginBodyBytes) { throw new LoginParseException("Request body too large."); } - using var body = new MemoryStream(); - var buffer = new byte[4096]; - while (true) + var bodySize = context.Features.Get(); + if (bodySize is not null + && (bodySize.MaxRequestBodySize is null || bodySize.MaxRequestBodySize > MaxLoginBodyBytes)) { - var read = await context.Request.Body.ReadAsync(buffer.AsMemory(), context.RequestAborted); - if (read == 0) break; - if (body.Length + read > 16384) - { - throw new LoginParseException("Request body too large."); - } - - body.Write(buffer, 0, read); - } - - var text = new UTF8Encoding(encoderShouldEmitUTF8Identifier: false, throwOnInvalidBytes: true).GetString(body.ToArray()); - - using var document = JsonDocument.Parse(text, new JsonDocumentOptions - { - AllowTrailingCommas = false, - CommentHandling = JsonCommentHandling.Disallow, - MaxDepth = 3, - }); - if (document.RootElement.ValueKind != JsonValueKind.Object - || !HasUniqueProperties(document.RootElement, "username", "password")) - { - throw new LoginParseException("Invalid login request."); + bodySize.MaxRequestBodySize = MaxLoginBodyBytes; } LoginRequest? request; try { - request = JsonSerializer.Deserialize(text, new JsonSerializerOptions - { - PropertyNameCaseInsensitive = true, - AllowTrailingCommas = false, - ReadCommentHandling = JsonCommentHandling.Disallow, - MaxDepth = 3, - }); + request = await JsonSerializer.DeserializeAsync(context.Request.Body, StrictLoginOptions, context.RequestAborted); + } + catch (BadHttpRequestException) + { + throw new LoginParseException("Request body too large."); } catch (JsonException) { @@ -124,20 +112,6 @@ private static async Task ParseLoginRequest(HttpContext context) return request; } - private static bool HasUniqueProperties(JsonElement element, params string[] known) - { - var seen = new HashSet(StringComparer.Ordinal); - foreach (var property in element.EnumerateObject()) - { - if (!known.Contains(property.Name, StringComparer.Ordinal) || !seen.Add(property.Name)) - { - return false; - } - } - - return true; - } - private static async Task WriteError(HttpContext context, int status, string code, string message) { context.Response.StatusCode = status; @@ -151,6 +125,9 @@ private static async Task WriteError(HttpContext context, int status, string cod public static string CreateRateLimitResult() => "rate_limited"; } -public sealed record LoginRequest(string Username, string Password); +// Exact lowercase member names: case variants must fail as unknown fields (spec §4.1). +public sealed record LoginRequest( + [property: JsonPropertyName("username")] string Username, + [property: JsonPropertyName("password")] string Password); public sealed class LoginParseException(string message) : Exception(message); diff --git a/TextCascade.Server/Cli.cs b/TextCascade.Server/Cli.cs index 18844b3..bf7bd06 100644 --- a/TextCascade.Server/Cli.cs +++ b/TextCascade.Server/Cli.cs @@ -1,4 +1,3 @@ -using System.Diagnostics; using System.Globalization; using System.Runtime.Versioning; using System.Security.Cryptography; @@ -433,20 +432,17 @@ public static class SingleInstanceLock { try { - if (!File.Exists(lockPath)) + // FileShare.None: the OS releases the handle when the holder process dies, + // so a leftover file (crash, power loss) is simply reopened and never blocks; + // the PID below is diagnostic only. + var stream = new FileStream(lockPath, FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.None); + stream.SetLength(0); + using (var writer = new StreamWriter(stream, Encoding.UTF8, bufferSize: 32, leaveOpen: true)) { - var stream = new FileStream(lockPath, FileMode.CreateNew, FileAccess.Write, FileShare.None); - using (var writer = new StreamWriter(stream, Encoding.UTF8, bufferSize: 32, leaveOpen: true)) - { - writer.Write(Environment.ProcessId.ToString(CultureInfo.InvariantCulture)); - } - return new SingleInstanceLockHandle(lockPath, stream); + writer.Write(Environment.ProcessId.ToString(CultureInfo.InvariantCulture)); } - if (TryRecoverStaleLock(lockPath, out var recovered)) - { - return recovered; - } + return new SingleInstanceLockHandle(lockPath, stream); } catch (IOException) { @@ -457,59 +453,6 @@ public static class SingleInstanceLock return null; } - - private static bool TryRecoverStaleLock(string lockPath, out SingleInstanceLockHandle? handle) - { - handle = null; - try - { - var text = File.ReadAllText(lockPath).Trim(); - if (!int.TryParse(text, CultureInfo.InvariantCulture, out var pid)) - { - return false; - } - - if (IsProcessAlive(pid)) - { - return false; - } - - File.Delete(lockPath); - var stream = new FileStream(lockPath, FileMode.CreateNew, FileAccess.Write, FileShare.None); - using (var writer = new StreamWriter(stream, Encoding.UTF8, bufferSize: 32, leaveOpen: true)) - { - writer.Write(Environment.ProcessId.ToString(CultureInfo.InvariantCulture)); - } - handle = new SingleInstanceLockHandle(lockPath, stream); - return true; - } - catch (IOException) - { - return false; - } - } - - private static bool IsProcessAlive(int pid) - { - try - { - var process = Process.GetProcessById(pid); - _ = process.Handle; - return !process.HasExited; - } - catch (ArgumentException) - { - return false; - } - catch (InvalidOperationException) - { - return false; - } - catch (System.ComponentModel.Win32Exception) - { - return true; - } - } } diff --git a/TextCascade.Server/Hosting/HeartbeatScannerService.cs b/TextCascade.Server/Hosting/HeartbeatScannerService.cs index 2890fcf..728aba5 100644 --- a/TextCascade.Server/Hosting/HeartbeatScannerService.cs +++ b/TextCascade.Server/Hosting/HeartbeatScannerService.cs @@ -1,27 +1,51 @@ using Microsoft.Extensions.Hosting; +using Microsoft.Extensions.Logging; namespace TextCascade.Server; -public sealed class HeartbeatScannerService : IHostedService, IDisposable +public sealed class HeartbeatScannerService : BackgroundService { - private Timer? timer; - private readonly SyncServer syncServer; + private readonly TimeProvider timeProvider; + private readonly ILogger logger; - public HeartbeatScannerService(SyncServer syncServer) + public HeartbeatScannerService(SyncServer syncServer, TimeProvider timeProvider, ILogger logger) { this.syncServer = syncServer; + this.timeProvider = timeProvider; + this.logger = logger; + } + + protected override async Task ExecuteAsync(CancellationToken stoppingToken) + { + using var timer = new PeriodicTimer(TimeSpan.FromSeconds(1)); + try + { + while (await timer.WaitForNextTickAsync(stoppingToken)) + { + try + { + Scan(timeProvider.GetUtcNow()); + } + catch (Exception exception) + { + logger.LogError(exception, "Heartbeat scan failed."); + } + } + } + catch (OperationCanceledException) + { + } } - public Task StartAsync(CancellationToken cancellationToken) + public override async Task StopAsync(CancellationToken cancellationToken) { - timer = new Timer(Scan, null, TimeSpan.FromSeconds(1), TimeSpan.FromSeconds(1)); - return Task.CompletedTask; + await base.StopAsync(cancellationToken); + await syncServer.ShutdownAsync(TimeSpan.FromSeconds(2), timeProvider.GetUtcNow()); } - private void Scan(object? state) + private void Scan(DateTimeOffset now) { - var now = DateTimeOffset.UtcNow; syncServer.ScanHeartbeats(now); var recoveryEnd = syncServer.ProcessStartTime.AddSeconds( @@ -36,15 +60,4 @@ private void Scan(object? state) pair.Value.CloseRecoveryWindow(now); } } - - public async Task StopAsync(CancellationToken cancellationToken) - { - timer?.Change(Timeout.Infinite, 0); - await syncServer.ShutdownAsync(TimeSpan.FromSeconds(2), DateTimeOffset.UtcNow); - } - - public void Dispose() - { - timer?.Dispose(); - } } diff --git a/TextCascade.Server/ServerHost.cs b/TextCascade.Server/ServerHost.cs index ece9e16..128ddd8 100644 --- a/TextCascade.Server/ServerHost.cs +++ b/TextCascade.Server/ServerHost.cs @@ -71,7 +71,7 @@ public static WebApplication CreateApp( UsersFile users, RuntimeStateStore stateStore, IPasswordHasher? hasher = null, - IClock? clock = null, + TimeProvider? clock = null, LoadedCertificate? certificate = null) { var builder = WebApplication.CreateBuilder(args); @@ -88,14 +88,14 @@ public static WebApplication CreateApp( }); builder.Services.AddSingleton(hasher ?? new Argon2PasswordHasher()); - builder.Services.AddSingleton(clock ?? new SystemClock()); + builder.Services.AddSingleton(clock ?? TimeProvider.System); builder.Services.AddSingleton(stateStore); builder.Services.AddSingleton(serviceProvider => new SyncServer( config, users, serviceProvider.GetRequiredService(), serviceProvider.GetRequiredService(), - serviceProvider.GetRequiredService(), + serviceProvider.GetRequiredService(), serviceProvider.GetRequiredService>())); builder.Services.AddHostedService(); var app = builder.Build(); diff --git a/TextCascade.Server/SyncServer.cs b/TextCascade.Server/SyncServer.cs index 166d28f..dd6b36c 100644 --- a/TextCascade.Server/SyncServer.cs +++ b/TextCascade.Server/SyncServer.cs @@ -5,23 +5,13 @@ namespace TextCascade.Server; -public interface IClock -{ - DateTimeOffset UtcNow { get; } -} - -public sealed class SystemClock : IClock -{ - public DateTimeOffset UtcNow => DateTimeOffset.UtcNow; -} - public sealed class SyncServer : IConnectionCoordinator { private readonly UserRegistry registry = new(); private readonly List pendingHellos = new(); private readonly object pendingGate = new(); private readonly IPasswordHasher hasher; - private readonly IClock clock; + private readonly TimeProvider clock; private readonly RuntimeStateStore runtimeStateStore; private IReadOnlyDictionary userLookup; private readonly string loginDummyHash; @@ -29,7 +19,7 @@ public sealed class SyncServer : IConnectionCoordinator public UserRegistry Registry => registry; public IPasswordHasher Hasher => hasher; public SlidingWindowLoginLimiter LoginLimiter { get; } = new(); - public IClock Clock => clock; + public TimeProvider Clock => clock; public ILogger Logger { get; } public IReadOnlyDictionary UserLookup => Volatile.Read(ref userLookup); public DateTimeOffset ProcessStartTime { get; } @@ -42,7 +32,7 @@ public SyncServer( UsersFile users, RuntimeStateStore runtimeStateStore, IPasswordHasher hasher, - IClock clock, + TimeProvider clock, ILogger logger) { Config = config; @@ -51,7 +41,7 @@ public SyncServer( this.hasher = hasher; this.clock = clock; Logger = logger; - ProcessStartTime = clock.UtcNow; + ProcessStartTime = clock.GetUtcNow(); loginDummyHash = hasher.Hash( "textcascade-login-timing-dummy", Cli.CreateArgon2Config(config)); diff --git a/TextCascade.Server/TextCascade.Server.csproj b/TextCascade.Server/TextCascade.Server.csproj index 1169926..bf096f2 100644 --- a/TextCascade.Server/TextCascade.Server.csproj +++ b/TextCascade.Server/TextCascade.Server.csproj @@ -4,7 +4,7 @@ net10.0 enable enable - 0.4.0 + 0.5.0 TextCascade.Server true diff --git a/docs/server-spec.md b/docs/server-spec.md index 58fb8ee..004dd08 100644 --- a/docs/server-spec.md +++ b/docs/server-spec.md @@ -198,7 +198,7 @@ TextCascade.Server user hash TextCascade.Server serve # 启动服务（Program.cs 动词分发） ``` -所有命令接受 `--config `；CLI 写入 `users.json` 前先持有 PID 单实例锁（锁文件为 users.json 同目录 `users.json.lock`），再使用临时文件加原子替换。PID 锁识别并回收陈旧 PID、进程已退出但锁文件残留的情况，Windows 与 Linux 行为一致；检测到仍存活的其它 CLI 实例时失败退出。服务运行中修改文件的即时生效性见 3.2 热加载。 +所有命令接受 `--config `；CLI 写入 `users.json` 前先持有单实例文件锁（锁文件为 users.json 同目录 `users.json.lock`，以 `FileShare.None` 独占打开，持有进程退出或崩溃时由 OS 自动释放），再使用临时文件加原子替换。崩溃残留的锁文件会被直接复用、不再阻塞；锁文件中的 PID 仅作诊断用途，不参与锁判定。Windows 与 Linux 行为一致；检测到其它 CLI 实例仍持有锁时重试后失败退出。服务运行中修改文件的即时生效性见 3.2 热加载。 ### 3.4 RuntimeStateStore（版本号落盘） @@ -667,7 +667,7 @@ hub 清理： ### 10.1 纯单元测试（现有覆盖） - `SignToken`/`TryVerifyToken`：往返、过期、tokenVersion 撤销、篡改、未知字段、禁用用户、用户缺失。 -- CLI PID 单实例锁：活跃互斥、陈旧 PID 回收、存活进程不回收、锁路径校验。 +- CLI 单实例文件锁（`FileShare.None`）：活跃互斥、崩溃残留锁文件可复用、锁路径校验。 - `SlidingWindowLoginLimiter`：双维度、跨 IP、成功仅清用户窗口、max keys、过期清理。 - `TryAcquireClipToken`（TokenBucket refill）、`CheckFrameSize`/`CheckPayloadSize`、SeenIdRing 去重与淘汰、`NextVersion` 含 ulong.MaxValue 抛出、`SelectSnapshotWinner` 三规则。 - Argon2 三函数（SlowHash）、token 数字全形态、CLI 水位/溢出、WithVersion、重复 id 行为级断言均已由测试覆盖（函数级规格见 test-and-contract-spec §3）。 From c884001f3e93e5f8af8d06a74602922dc08fca68 Mon Sep 17 00:00:00 2001 From: long45343 <1725334094@qq.com> Date: Sat, 5 Sep 2026 02:34:12 +0800 Subject: [PATCH 31/32] =?UTF-8?q?=E6=96=87=E6=A1=A3=E4=BF=AE=E6=94=B9?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- CHANGELOG.md | 3 +- docs/go-server-spec.md | 449 +++++++++++++++++++++++++++++++++++++++++ docs/server-spec.md | 30 +-- 3 files changed, 467 insertions(+), 15 deletions(-) create mode 100644 docs/go-server-spec.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 6771122..6cd79b0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -92,7 +92,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added - Initial import and baseline release of TextCascade Server with Minimal API, Kestrel WebSocket, Argon2 password hashing, and token authentication. -[Unreleased]: https://github.com/long45343/TextCascade-Server/compare/v0.4.0...HEAD +[Unreleased]: https://github.com/long45343/TextCascade-Server/compare/v0.5.0...HEAD +[0.5.0]: https://github.com/long45343/TextCascade-Server/compare/v0.4.0...v0.5.0 [0.4.0]: https://github.com/long45343/TextCascade-Server/compare/v0.3.5...v0.4.0 [0.3.5]: https://github.com/long45343/TextCascade-Server/compare/v0.3.0...v0.3.5 [0.3.0]: https://github.com/long45343/TextCascade-Server/compare/v0.2.5...v0.3.0 diff --git a/docs/go-server-spec.md b/docs/go-server-spec.md new file mode 100644 index 0000000..0a7c051 --- /dev/null +++ b/docs/go-server-spec.md @@ -0,0 +1,449 @@ +# TextCascade Go 版服务端规格（C# 1:1 迁移） + +状态：迁移决策已完成（15/15），规格定稿待实现 +日期：2026-09-05 +基准：[server-spec.md](server-spec.md)（与 C# v0.5.0 实现对齐）+ 源码逐函数盘点 +迁移原则：**行为 1:1，对外契约零变更**；仅 §3 声明性差异与 §12 迁移注意事项两处例外 + +## 1. 目标与非目标 + +### 1.1 目标 + +- 用 Go 重写 C# 服务端（约 4400 行生产代码、20 个源文件），函数级一一对应。 +- 线协议、文件格式、CLI、错误码、默认值全部不变（见 §3）。 +- 存量部署（systemd + PEM 证书）可直接切换；存量密码哈希不兼容，按 §12 runbook 重置。 +- 测试全量 1:1 搬运（Q3）：单元 162 + 网络集成 12 + SlowHash 3 + 契约样本矩阵。 +- 内存占用显著低于 .NET 版（迁移动机，perf.md 实测 .NET 空闲 RSS 125–131MB / 每连接 ~660KB）；Go 版落地后按 perf.md 场景重新实测并修订内存目标。 + +### 1.2 非目标 + +- 不修改任何协议语义、不新增特性、不"顺手修复"C# 已知差距（§15 台账 9 项原样保留，见 §13）。 +- 不做跨语言互操作或混合部署的原子切换方案（切换 = 停旧起新 + runbook 步骤）。 +- 不支持 C# 版未支持的平台（win-x64 / linux-x64 两目标）。 + +## 2. 已定决策（15 项，自包含） + +| # | 决策 | 结论 | +|---|---|---| +| Q1 | 仓库布局 | 新建 `go` 分支；分支根目录放 go.mod（Go 原生布局）；C# 留在 main；ContractSamples 复制入 `testdata/`，CI 校验与 main 侧 sha256 一致 | +| Q2 | 工具链 | Go 1.27（最新稳定线，go1.27.1） | +| Q3 | 验收策略 | 测试全量 1:1 搬运 | +| Q4 | WebSocket 库 | gorilla/websocket | +| Q5 | 入站 JSON | encoding/json + 手写 token 级预扫描器（`internal/protocol/jsonscan.go`，详见 §7.7-A） | +| Q6 | Argon2 兼容 | **不与 Isopoh 兼容**：Go 自洽 PHC 编码（golang.org/x/crypto/argon2）；迁移时重置全部存量密码（§12） | +| Q7 | TOML | pelletier/go-toml/v2 | +| Q8 | 单实例锁 | gofrs/flock | +| Q9 | 用户文件监听 | fsnotify + 250ms 防抖 + 30 秒轮询兜底 | +| Q10 | 证书 | PEM 全支持 + PFX/P12 用 software.ssl.golang.org/pkcs12（实现首日验证无加密 PFX） | +| Q11 | ALPN | 仅 HTTP/1.1（`NextProtos=["http/1.1"]`） | +| Q12 | 日志 | log/slog + 自定义单行 Handler | +| Q13 | CLI | stdlib flag.NewFlagSet 手写动词分发；x/term 交互密码 | +| Q14 | 测试栈 | stdlib testing + testify（assert/require；仅进测试二进制） | +| Q15 | 版本注入 | ldflags -X main.version；CI 双平台矩阵 | + +完整论证（每项三选项优劣）存于私有决策台账 `specs/go-migration-decisions.md`（不入库）；本表为唯一事实摘要。 + +## 3. 声明性差异（相对 C# v0.5.0 的全部偏差） + +1. **存量 Argon2 哈希不兼容**（Q6）：Go 版 PHC 编码自洽，无法验证 C# 版创建的 `users.json` 存量哈希。迁移时按 §12 重置全部用户密码。客户端零改动（Argon2 仅服务端使用；E2E 密钥由客户端本地 PBKDF2 从密码派生，与服务端哈希无关，重置为同一密码后旧剪贴板仍可解密）。 +2. **仅 HTTP/1.1**（Q11）：Go 版 TLS ALPN 显式只广播 `http/1.1`，不提供 h2。Kestrel 默认广播 h2+http/1.1。对协议无影响（RFC6455 升级仅存在于 HTTP/1.1；gorilla/websocket 亦仅支持 h1 升级）。 +3. **Windows 私钥存储**：C# 需区分 DefaultKeySet/EphemeralKeySet（SChannel 限制）；Go 私钥仅在内存中，无对应问题，`internal/hosting/cert.go` 不含平台分支。 +4. 除此之外，登录响应、token、协议帧、错误码、close code、文件格式、TOML 键、环境变量、CLI 行为**全部零偏差**。 + +## 4. 依赖清单（go.mod，Q 汇总） + +直接依赖（8）： + +| 依赖 | 用途 | 引入自 | +|---|---|---| +| github.com/gorilla/websocket | WebSocket 升级与帧 | Q4 | +| github.com/pelletier/go-toml/v2 | TOML 解析 | Q7 | +| github.com/gofrs/flock | CLI 单实例锁 | Q8 | +| github.com/fsnotify/fsnotify | users.json 监听 | Q9 | +| golang.org/x/crypto | Argon2id | Q6 | +| golang.org/x/term | 交互式密码输入 | Q13 | +| software.sslmate.com/src/go-pkcs12 | 无密码 PFX/P12 | Q10（实测修正：golang.org vanity 路径 proxy 未缓存且部分网络不可达，须用 canonical 模块路径，当前版 v0.7.3） | +| stretchr/testify | 测试断言（仅测试二进制） | Q14 | + +间接：golang.org/x/sys（flock/fsnotify 共享）。其余全部标准库。 + +## 5. 工具链与仓库布局 + +- Go 1.27；`go.mod` 模块名 `github.com/long45343/TextCascade-Server`；`go vet` + `gofmt` 进 CI。 +- 版本号：`var version = "dev"`（main 包），release 构建注入 `-ldflags "-X main.version="`；CLI 输出与 user list 显示该值（对齐 C# 从 csproj 读取的语义）。 +- 构建：`go build -trimpath -o TextCascade.Server ./cmd/server`；发布矩阵 linux-x64 / win-x64 自包含单文件（Go 无"框架依赖"概念，运行时内嵌——与 C# 产物形态的固有差异，非契约偏差）。 + +``` +（go 分支根目录） +go.mod / go.sum +cmd/server/main.go ← Program.cs +internal/config/config.go ← RuntimeConfig.cs +internal/users/users.go ← Users.cs +internal/auth/argon2.go ← Auth.cs（哈希器半边） +internal/auth/token.go ← Auth.cs（token 半边） +internal/auth/login.go ← AuthService.cs +internal/protocol/protocol.go ← Protocol.cs（结构 + 解析 + 校验） +internal/protocol/serialize.go ← Protocol.cs（出站 marshal） +internal/protocol/jsonscan.go ← 新组件：token 级预扫描（§7.7-A） +internal/core/limiter.go ← Core.cs（SlidingWindowLoginLimiter） +internal/core/bucket.go ← Core.cs（TokenBucket） +internal/core/ring.go ← Core.cs（SeenIdRing） +internal/core/logic.go ← Core.cs（CoreLogic / SnapshotWinner） +internal/state/store.go ← RuntimeStateStore.cs +internal/logging/security.go ← SecurityLogging.cs +internal/sync/server.go ← SyncServer.cs +internal/hub/hub.go ← UserHub.cs +internal/hub/registry.go ← UserRegistry.cs +internal/hub/jobs.go ← UserJobs.cs +internal/hub/coordinator.go ← IConnectionCoordinator.cs +internal/hosting/connection.go ← ConnectionHandler.cs +internal/hosting/endpoint.go ← SyncEndpoint.cs +internal/hosting/watcher.go ← UserFileWatcher.cs +internal/hosting/scanner.go ← HeartbeatScannerService.cs +internal/hosting/cert.go ← ServerHost.cs（证书加载半边） +internal/hosting/run.go ← ServerHost.cs（进程装配半边） +internal/models/context.go ← ConnectionContext.cs + ConnectionStateBag.cs +internal/models/frame.go ← ReceivedMessage.cs +internal/cli/cli.go ← Cli.cs +internal/cli/lock.go ← SingleInstanceLock +internal/clock/clock.go ← TimeProvider 接缝（固定决策 F4） +testdata/contract-samples/ ← 复制自 main 的 TestCascade.Server.Tests/ContractSamples/ +``` + +## 6. 并发模型映射（固定决策 F3/F4/F5/F6/F7） + +| C# | Go | 要点 | +|---|---|---| +| `ReadLoopAsync`（每连接） | goroutine `readLoop` | gorilla NextReader 驱动 | +| 用户 Channel（无界） | 自实现无界队列（slice + sync.Cond），`TryWrite` 恒真 | 1:1 语义 | +| `RunUserLoopAsync`（单消费者） | goroutine（mutex 保证单读者，复刻 StartIfIdle 防重入） | | +| 发送 Channel（有界 16） | `chan []byte` cap 16 + `select { case ch<-p: / default: 满 }` | 满即 `cancel()` 连接 ctx，不补发 error/close | +| `ConnectionSendLoopAsync` | goroutine `sendLoop`（唯一写者） | gorilla 要求单写者，天然对齐 | +| `HeartbeatScannerService`（BackgroundService+PeriodicTimer） | goroutine + `time.NewTicker(1s)` + ctx | 扫描 panic recover 且记日志 | +| `TimeProvider` | `type Clock interface { Now() time.Time }`；生产 `clock.System`，测试注入 fake | | +| 优雅停机 | `signal.NotifyContext(SIGTERM, SIGINT)`；流程 1:1：bye → 1001 → drain 2s → 取消全部连接 → 同步 flush | 含"close 握手等待无超时"现状（§13.8） | +| 用户循环异常 → RebuildHub | 用户 goroutine 内 `recover()` → RebuildHub | `NextVersion` 溢出必须**显式检测**：Go uint64 自增静默 wrap，`cur == math.MaxUint64` 时 panic | + +## 7. 函数级映射表 + +约定：C# `out` 参数在 Go 中改为多返回值；`Result` 改为 `(T, *Error)`；instance/static 双载合并为一个函数（表中注明）；internal 测试钩子在 Go 中为包内可见函数（同包测试可直接调用）。 + +### 7.1 Program.cs → cmd/server/main.go + +| C# | Go | 行为 | +|---|---|---| +| `Main(args)` | `main()` | 动词分发：`serve` → `hosting.Run`；其余 → `cli.Run`；退出码一致 | + +### 7.2 ServerHost.cs → internal/hosting/{run.go, cert.go} + +| C# | Go | 行为 | +|---|---|---| +| `RunServer(args)` | `hosting.Run(args) int` | Load→Env→Validate→LoadUsers→ValidateUsers→state store→cert.Load→watcher.Start→http.Server(ListenAndServeTLS)；错误打印与退出码一致 | +| `CreateApp(...)` | `hosting.NewServer(cfg, users, store, hasher, clk, cert) *Server` | 构建 mux：GET /health、POST /api/v1/login、GET /api/v1/sync；等价 `UseWebSockets` 由 gorilla Upgrader 承担 | +| `ConfigureKestrel(...)` | TLSConfig 装配（NewServer 内） | MinVersion 不显式设置（跟随 OS，对齐 §8.2）；`NextProtos=["http/1.1"]`（Q11） | +| `CertificateLoader.Load(path)` | `cert.Load(path) (tls.Certificate, error)` | 扩展名分发 .pem/.crt → loadPEM；.pfx/.p12 → pkcs12（无密码）；其他扩展报错文案一致 | +| `LoadPemCertificate(path)` | `cert.loadPEM(path)` | 同名 `.key` 边车查找、bundle 解析、叶证书+私钥匹配；缺私钥/无证书错误文案一致 | +| `DisposeChain` / `LoadedCertificate.Dispose` | 无对应 | Go GC 管理；结构体保留持有 tls.Certificate | + +### 7.3 RuntimeConfig.cs → internal/config/config.go + +| C# | Go | 行为 | +|---|---|---| +| 5 个 record（ServerConfig 等） | `config.RuntimeConfig` + 子 struct（Server/Auth/Limits/RateLimit/Files） | 字段名一一对应 | +| `CreateDefaultConfig()` | `config.Defaults()` | 默认值逐项一致（§3.1 表） | +| `LoadTomlConfig(path, defaults)` | `config.LoadTOML(path string, def *RuntimeConfig) (RuntimeConfig, error)` | go-toml/v2 严格解析：重复键 fail-fast、类型非法 fail-fast；UTF-8 强制 | +| `ApplyTomlModel(config, model)` | `applyTOMLModel(doc *toml.MetaData)` | 键映射与回退一致 | +| `TryGetTable` / `GetString` / `GetInt` | `getTable` / `getString` / `getInt` | | +| `WarnUnknownKeys(...)` | `warnUnknownKeys(...)` | 未知键 warning（slog） | +| `ApplyEnvironmentOverrides(config)` | `(*RuntimeConfig).ApplyEnv()` | 5 个环境变量 + token secret 变量名，清单一致 | +| `ValidateConfig(config)` | `(*RuntimeConfig).Validate() error` | 全部规则一致（含 max_frame > max_text、心跳超时 > 间隔、全容量 > 0） | + +### 7.4 Users.cs → internal/users/users.go + +| C# | Go | 行为 | +|---|---|---| +| `UserRecord` record | `struct UserRecord`（Username/PasswordHash/TokenVersion int64/Disabled） | | +| `Argon2HashRegex` | `users.argon2HashRe`（regexp 编译一次） | 校验哈希串形态 | +| `LoadUsers(path)` | `users.Load(path) (*UsersFile, error)` | 严格 JSON：未知/重复字段拒绝（Q5 扫描器） | +| `HasUniqueProperties(...)` | `hasUniqueProperties(...)` | | +| `ValidateUsers(users)` | `(*UsersFile).Validate() error` | 水位、唯一性、哈希格式、正数 long、disabled——全部一致 | +| `BuildUserLookup(users)` | `(*UsersFile).BuildLookup() map[string]UserRecord` | | +| `SaveUsers(path, users)` | `users.Save(path, f)` | Validate → 临时文件 + `os.Rename` 原子替换（Windows 等价 MoveFileEx）+ fsync | +| `Copy(source)` | `users.Copy(f)` | | + +### 7.5 Auth.cs → internal/auth/{argon2.go, token.go} + +| C# | Go | 行为 | +|---|---|---| +| `Argon2PasswordHasher.Hash/Verify` | `auth.Hash(password, params)` / `auth.Verify(password, encoded)` | x/crypto/argon2 (Argon2id)；PHC 编码自洽（Q6），格式 `$argon2id$v=19$m=…,t=…,p=…$b64salt$b64hash`（RawURL 无填充） | +| `NeedsRehash`（实例/静态双载） | `auth.NeedsRehash(encoded, params)` | 合并双载；语义一致：参数不一致 → 登录响应携带 needsRehash，不重写文件 | +| `TokenPayload` record | `struct auth.TokenPayload`（Subject/Version/IssuedAt/Expires int64 Unix 秒） | | +| `AuthToken` record | `struct auth.Token` | | +| `TokenService(secret)` | `auth.NewTokenService(secret []byte)` | | +| `CreateToken(user, now, ttl)` | `(*TokenService).Create(user, now, ttl)` | | +| `CreateTokenPayload(user, now, ttl)` | `auth.CreateTokenPayload(user, now, ttl)` | 固定字段序 sub/ver/iat/exp 最小化 UTF-8 JSON（手写 marshal，非结构体反射） | +| `SignToken(payload, secret)` | `auth.SignToken(payload, secret)` | HMAC-SHA256；比较用 `hmac.Equal`（常数时间） | +| `TryVerifyToken`（实例/静态） | `auth.TryVerifyToken(compact, now, lookup) (TokenPayload, bool)` | 合并双载；Go 无 out 参数 → 多返回值 | +| `TryVerifyTokenInternal` | `tryVerifyInternal` | 验签→验过期→验用户存在→验 tokenVersion 顺序一致 | +| `TryParsePositiveInteger` | `tryParsePositiveInt` | 拒绝小数/指数/字符串形态/非正数 | +| `TryBase64UrlDecode` | 消失 | 直接用 `base64.RawURLEncoding`（Q5 固定决策：出站 EncodeToString/入站 Decode 等价） | + +### 7.6 AuthService.cs → internal/auth/login.go + +| C# | Go | 行为 | +|---|---|---| +| `HandleLoginAsync(...)` | `auth.HandleLogin(w, r, cfg, srv, logger)` | 全部内联逻辑 1:1 | +| `ParseLoginRequest(...)` | `parseLoginRequest(w, r)` | 16KB 上限：`http.MaxBytesReader`（覆盖 chunked，等价 IHttpMaxRequestBodySizeFeature）；MaxDepth=3、重复/未知字段拒绝走 jsonscan | +| `WriteError(...)` | `writeError(...)` | 400 invalid_request 统一形态 | +| `CreateLoginFailure` / `CreateRateLimitResult` | 常量 `errInvalidCredentials` / `errRateLimited` | | +| `LoginRequest` record | `struct loginRequest` | | +| `LoginParseException` | `auth.ErrLoginParse`（哨兵 error） | Go 无异常类型 | + +### 7.7 Protocol.cs → internal/protocol/{protocol.go, serialize.go, jsonscan.go} + +**结构与错误：** + +| C# | Go | +|---|---| +| `ProtocolError` record | `struct Error{Code ErrKind, Message string, ReferenceID *string}` | +| `ProtocolErrorCode` enum | `ErrKind` 常量组 + `(*Error).CodeName()`（invalid_message/text_too_large/frame_too_large/empty_text/rate_limited/hello_timeout/server_busy） | +| `Result` / `ParseResult` | `(T, *Error)` 惯例；`ParseClientMessage` 返回 `(Message, *Error)` | +| `ClipSnapshot/ClientHello/ClientClip/ClientPong` | 同名字段 struct（UpdatedAt/ClientTime 用 `time.Time`） | +| `ClientMessage/MessageKind` | `Kind` 常量 + `Message` 单一 struct（Go 无 object 判别联合，用固定字段） | +| `LatestText` record + `From` | `struct LatestText` + `LatestFromSnapshot(s, version, clientID, clientName)` | +| `UtcSecondDateTimeConverter` | `WriteUTCSecond(w, t)`：出站 `"2006-01-02T15:04:05Z"`；`ParseFlexibleTime(s)`：入站接受秒级或 ISO 往返两种形态、偏移必须为零 | + +**出站 marshal（serialize.go，全部手写字节，字段序固定）：** + +| C# | Go | 字节不变式 | +|---|---|---| +| `SerializeWelcome(latest, _)` | `MarshalWelcome(latest *LatestText)` | `{"type":"welcome","protocolVersion":1}` + latest 非 nil 时续 `,\"latest\":{...}`；键缺失即"无最新值" | +| `SerializeClip(id, latest)` | `MarshalClip(id, latest)` | type,version,id,payload,encrypted,hash,fromClientId,fromClientName,updatedAtUtc | +| `SerializeClipAck(id, latest)` | `MarshalClipAck(id, latest)` | type,id,version,updatedAtUtc | +| `SerializePing(now)` | `MarshalPing(now)` | type,serverTimeUtc | +| `SerializeBye(reason)` | `MarshalBye(reason)` | type,reason | +| `SerializeProtocolError(err)` | `MarshalError(err)` | type,code,message[,referenceId]；nil 时键省略 | +| `SerializeLoginResponse(token, cfg, needsRehash)` | `MarshalLoginResponse(token, cfg, needsRehash)` | 手写顺序：token, expiresAtUtc（"O" 往返格式，Go layout `"2006-01-02T15:04:05.0000000Z07:00"`）, protocolVersion, maxTextBytes, helloTimeoutSeconds, heartbeatIntervalSeconds, heartbeatTimeoutSeconds [, needsRehash 仅 true 时] | + +**入站解析（protocol.go + jsonscan.go）：** + +| C# | Go | 行为 | +|---|---|---| +| `ParseClientMessage(frame, config)` | `ParseClientMessage(frame []byte, cfg) (Message, *Error)` | 顺序：jsonscan 预扫描 → 根必须 object → `type` 字符串 → 未知/重复字段检查（known 集合按类型）→ parseHello/parseClip/parsePong → 未知 type invalid_message | +| `ParseHello(root, config)` | `parseHello` | clientId 1–128 字节、clientName 0–128 字节、lastServerVersion 非负整数、snapshot object 或 null | +| `TryGetSnapshot(...)` | `tryGetSnapshot` | payload 必填非空、预算校验、hash ≤4096 字节、localModifiedAtUtc 两种形态 | +| `ParseClip(root, config)` | `parseClip` | | +| `ParsePong(root)` | `parsePong` | | +| `ValidateHello(h, config)` | `ValidateHello(h, cfg)` | | +| `ValidateClipSnapshot(s, config)` | `validateSnapshot` | | +| `ValidateClipMessage(m, config)` | `ValidateClip(m, cfg)` | 结构→语义→资源顺序早拒绝 | +| `CheckFrameSize` / `CheckPayloadSize` | 同名导出 | | +| `ValidatePayloadSize` | `validatePayloadSize` | | +| `TryParseJson(frame, out error)` | `jsonscan.Decode(frame []byte, cfg) (*Node, *Error)` | 见下 | +| `GetReferenceId(root)` | `getReferenceID(root)` | | + +**A. jsonscan 预扫描器规则（Q5 新组件，逐条复刻 C# 校验行为）：** + +1. UTF-8 完整性（`utf8.Valid`），非法即 invalid_message。 +2. 单一顶层 JSON 值；嵌套深度 ≤3（C# MaxDepth=3）。 +3. 全树重复键检测，报文含字段名（`Unknown or duplicate field: X.`）。等价性论证：C# 未知字段一律拒绝，故 C# 的"已知层显式查重"与"全树查重"最终行为相同（重复键必然落在已知字段或导致未知字段拒绝，两者都返回 invalid_message 且报文一致）。 +4. 数字形态白名单：协议字段仅接受整数字面量 `-?0|[1-9][0-9]*`；小数点/指数形态在扫描层即拒绝（等价 C# TryGetUInt64/TryParsePositiveInteger 的行为 + TryGet 系列的数字类型检查）。 +5. 字符串非法转义/裸控制字符 → 非法。 +6. 扫描产物为轻量 Node 树，语义层（parseHello 等）在其上取字段；语义解析不再走 encoding/json（避免二义）。users.json/login 同一扫描器复用。 + +### 7.8 Core.cs → internal/core/{limiter.go, bucket.go, ring.go, logic.go} + +| C# | Go | 行为 | +|---|---|---| +| `TryConsumeLoginLimit(ip, user, now, cfg)` | `(*Limiter).TryConsumeLoginLimit(...)` | 双维度滑动窗口，任一超限拒绝；成功仅清用户窗口 | +| `ResetUserLimit(username)` | `(*Limiter).ResetUserWindow(user)` | | +| `GetWindowCount/HasWindowKey` | 包内可见（同包测试直调） | C# internal ForTest 钩子在 Go 无需后缀 | +| `RemoveExpired/EnqueueForTest` | 同上 | | +| `tryConsume(key, limit, now, maxKeys, allowNewKey)` | `tryConsume` | RemoveExpired 全表扫描时机一致 | +| `TokenBucket(burst, rate, now)` / `TryAcquire(now)` | `bucket.New` / `(*Bucket).TryAcquire(now)` | 补币算法一致 | +| `SeenIdRing(capacity)` | `ring.New(capacity)` | Dictionary+FIFO 环形淘汰 | +| `TryDuplicate(id)` | `(*Ring).TryDuplicate(id)` | | +| `TryGetResult(id, out r)` | `(*Ring).TryGet(id) (*LatestText, bool)` | | +| `RememberId(id, result)` | `(*Ring).Remember(id, latest)` | | +| `IsUnchangedDuplicate(...)` | `(*Ring).IsUnchangedDuplicate(id, payload, hash, encrypted) (*LatestText, bool)` | 返回 true 时 latest 必非 nil（死分支依据） | +| `RememberInternal` | `rememberInternal` | | +| `SnapshotWinner` record | `struct Winner` | | +| `CoreLogic.NextVersion(current)` | `core.NextVersion(cur uint64) uint64` | **F7：`cur == math.MaxUint64` 显式 panic**（Go 自增不溢出抛异常） | +| `CoreLogic.WithVersion(latest, next, nowUtc)` | `core.WithVersion(latest, next, now)` | 不可变替换 | +| `CoreLogic.SelectSnapshotWinner(hellos)` | `core.SelectSnapshotWinner(hellos) *Winner` | 三规则：版本最大→localModifiedAtUtc 最新→clientId 字典序更大 | + +### 7.9 SyncServer.cs → internal/sync/server.go + +| C# | Go | 行为 | +|---|---|---| +| 构造 | `sync.New(cfg, users, store, clk, logger)` | | +| `RemoveEmptyHubAfterRecovery(hub)` | `(*Server).RemoveEmptyHubAfterRecovery(h)` | allowDuringRecovery=true | +| `ReplaceUserLookup(users)` | `(*Server).ReplaceUserLookup(f)` | atomic.Pointer[users.File]，Volatile.Write 等价 | +| `GetOrCreateHub(username)` | `(*Server).GetOrCreateHub(u)` | 初始版本 = store.GetVersion；互斥防并发建 hub | +| `ScanHeartbeats(now)` | `(*Server).ScanHeartbeats(now)` | 1s 扫描器调用 | +| `RebuildHub(hub)` | `(*Server).RebuildHub(h)` | 取消该用户全部连接并重建；进程存活 | +| `RegisterPendingHello/UnregisterPendingHello` | 同名 | pendingHellos 集合 | +| `EnqueueHelloTimeout(c)` / `CloseAfterHelloTimeoutAsync` | `enqueueHelloTimeout(c)` / goroutine `closeAfterHelloTimeout` | time.AfterFunc 等价 | +| `CancelConnection(c, reason)` | `(*Server).CancelConnection(c, reason)` | MarkClosed 守卫行为保留（含 §13.9 静默熔断现状） | +| `EnqueueImmediateClose(c, reason)` | `(*Server).EnqueueImmediateClose(c, reason)` | Socket.Abort 等价 = `conn.UnderlyingConn().Close()` | +| `ShutdownAsync(drain, now)` | `(*Server).Shutdown(drain, now)` | **34 秒无超时 close 握手现状原样保留**（§13.8） | +| `CloseConnectionAsync(c, status, reason)` | `closeConnection(c, status, reason)` | | + +### 7.10 Hub（UserHub.cs / UserRegistry.cs / UserJobs.cs / IConnectionCoordinator.cs）→ internal/hub + +| C# | Go | 行为 | +|---|---|---| +| `UserHub` 构造 | `hub.New(username, cfg, processStart, coord, store, initialVersion)` | SeenIdRing/TokenBucket/无界队列初始化 | +| `AddConnection/RemoveConnection` | `(*Hub).AddConnection/RemoveConnection` | | +| `StartIfIdle()` | `(*Hub).StartIfIdle()` | mutex + 单读者防重入；启动单消费 goroutine | +| `TryWriteJob(job)` | `(*Hub).TryWriteJob(job)` | 无界队列恒真 | +| `RunUserLoopAsync(ctx)` | `(*Hub).RunUserLoop(ctx)` | recover → RebuildHub（F7） | +| `ProcessJob(job, now)` | `processJob(job, now)` | clip/pong/hello/disconnect 分派 | +| `AcceptSnapshot(hello)` | `(*Hub).AcceptSnapshot(hello)` | 预算累计（仅 payload UTF-8 字节） | +| `ClassifyClip(clip, conn)` | `(*Hub).ClassifyClip(clip, c) RecoveryDecision` | 恢复窗口有界队列；满断开提交者 | +| `CloseRecoveryWindow(now)` | `(*Hub).CloseRecoveryWindow(now)` | 选举→恢复→按到达序处理恢复队列→广播 welcome | +| `BroadcastWelcome(now)` | `broadcastWelcome(now)` | | +| `IsRecoveryWindowOpen(now)` / `EnsureRecoveryWindowClosed(now)` | 同名 | 窗口 = ProcessStart + snapshot_window_seconds | +| `MarkActivity` / `MarkActivityForScan` | 合并为 `markActivity(now)` | 10 分钟空闲回收依据 | +| `ApplyClip(clip, sender, now)` | `(*Hub).ApplyClip(clip, sender, now)` | 幂等（内容比较）→令牌桶→NextVersion→SaveVersion→不可变替换→单次序列化广播（排除发送方连接）→ACK；**死分支兜底原样保留**（§13.6）：`dupLatest != nil` 优先，`Latest` 次之，空 LatestText 兜底（不可达） | +| `BroadcastToConnection(c, payload)` | `broadcastToConnection(c, payload)` | | +| `BroadcastAsync(payload)` | `(*Hub).Broadcast(payload)` | | +| `UserRegistry.GetOrAdd/TryGetValue/RemoveIfEmpty/Remove` | `registry.GetOrAdd/TryGet/RemoveIfEmpty(h, allowDuringRecovery)/Remove` | ConcurrentDictionary → mutex map | +| `UserJob` 层级 + `RecoveryClip` | Go interface `UserJob` + ClipJob/HelloJob/PongJob/DisconnectJob struct + `RecoveryClip` struct | | +| `IConnectionCoordinator` | `hub.Coordinator` interface | 方法集一致 | + +### 7.11 Hosting（ConnectionHandler.cs / SyncEndpoint.cs / UserFileWatcher.cs / HeartbeatScannerService.cs）→ internal/hosting + +| C# | Go | 行为 | +|---|---|---| +| `ConnectionHandler.RunAsync(provisional, payload, cfg, server)` | `RunConnection(ctx, provisional, payload, cfg, srv)` | 转正：Hub 赋值、注册 hello 截止 | +| `ReceiveFrameAsync(...)` | `receiveFrame(c)` | gorilla `SetReadLimit(maxFrameBytes)`；超限→1009；零长度帧→frame_too_large 1009；客户端 Close 帧→回 1000 退出读循环但**不立即取消 CTS**（§5.7 现状） | +| `SendAndClosePreHelloAsync(...)` | `sendAndClosePreHello(...)` | 预 hello 阶段错误：invalid_message→1008、超限→1009 | +| `ReadLoopAsync(connection, cfg, server)` | `readLoop(c, cfg, srv)` | 帧→ParseClientMessage→投递用户队列/本地错误处理 | +| `ConnectionSendLoopAsync(connection)` | `sendLoop(c)` | 唯一写者；ctx.Done / 出队写出；取消与非取消异常同路清理 | +| `SendSafeAsync(...)` | `sendSafe(c, payload, srv)` | | +| `SyncEndpoint.HandleAsync(...)` | `HandleSync(w, r, cfg, srv)` | 升级前 Bearer 验证（401 不升级）、gorilla `CheckOrigin` 允许同源语义对齐、仅接受 `textcascade.v1`（Ordinal 精确）否则 400、非 WS 400 | +| `SelectSubProtocol(requested)` | `selectSubprotocol(r)` | gorilla Subprotocols 辅助下自校验 | +| `UserFileWatcher` 构造/`Start` | `watcher.New(...)` / `(*Watcher).Start()` | fsnotify Changed/Created/Deleted/Renamed + 250ms 防抖 + 30s ticker 兜底无条件重载 | +| `OnFileChanged/OnFileRenamed/OnFileError` | 事件分派函数 | | +| `ScheduleReload` / `ReloadAsync` | `scheduleReload` / `reload` | 3 次 50ms 退避；全败保留旧表 + warning；成功 → `srv.ReplaceUserLookup` | +| `Dispose` | `(*Watcher).Close()` | | +| `HeartbeatScannerService.ExecuteAsync/StopAsync/Scan` | `scanner.Run(ctx)` / ctx 取消 / `scan(now)` | 1s ticker；panic recover 记日志 | + +### 7.12 RuntimeStateStore.cs → internal/state/store.go + +| C# | Go | 行为 | +|---|---|---| +| `RuntimeStateEntry/RuntimeStateFile` | struct | `{"entries":[{"username":...,"version":...}]}` 格式不变 | +| 构造（flush 循环） | `state.NewStore(path, cfg)` | goroutine + 5s ticker；脏位快照 | +| `GetVersion(username)` | `(*Store).GetVersion(u) uint64` | | +| `SaveVersion(username, version)` | `(*Store).SaveVersion(u, v)` | 单调 max 合并（CAS），防乱序回退 | +| `Flush()` | `(*Store).Flush() bool` | 临时文件+rename+fsync；结构非法（重复键/空 username/零版本）启动 fail-fast 在 load 侧 | +| `RunFlushLoopAsync` / `Dispose` | `runFlushLoop(ctx)` / `(*Store).Stop()` | 停机同步 flush | +| `Load(path)` / `WriteAtomic(...)` | `load` / `writeAtomic` | | + +### 7.13 SecurityLogging.cs → internal/logging/security.go + +| C# | Go | 行为 | +|---|---|---| +| `LogSecurityEvent(logger, event, pairs)` | `logutil.SecurityEvent(logger, event string, fields ...Field)` | slog 自定义单行 Handler（Q12）复刻 `yyyy-MM-ddTHH:mm:ssZ ` 时间戳与扁平字段 | +| `RedactFields` / `RedactSensitive` | `redactFields` / `RedactSensitive` | 脱敏规则一致 | +| `TokenPrefix` | `logutil.TokenPrefix` | 保留（生产不调用，1:1） | + +日志事件与字段完全一致：login(username,ip,success[,reason])、connect/disconnect(username,clientId,connectionId[,reason])、clip(username,version,clipId,bytes,fromClientId,encrypted)、reject(username,code,bytes)；登录失败折叠 `reason=invalid_credentials`。 + +### 7.14 Cli.cs → internal/cli/{cli.go, lock.go} + +| C# | Go | 行为 | +|---|---|---| +| `RunCli(args, hasher)` | `cli.Run(args, hasher) int` | 动词：user add/passwd/disable/enable/delete/revoke-tokens/list/hash + serve | +| `CreateLockPath(usersFile)` | `CreateLockPath` | users.json 同目录 `users.json.lock` | +| `PrintUsage` | `printUsage` | 文案一致 | +| `TryExtractConfigOption(ref args, out path)` | `tryExtractConfigOption(args) (rest, path)` | 回退顺序 --config → TEXTCASCADE_CONFIG → ./textcascade.toml | +| `CommandAddUser/Passwd/SetDisabled/DeleteUser/RevokeTokens/ListUsers/HashPassword` | `cmdAdd/cmdPasswd/cmdSetDisabled/cmdDelete/cmdRevoke/cmdList/cmdHash` | 水位分配、溢出放弃、字节级文件保留行为一致 | +| `LoadForWrite(path)` | `loadForWrite(path)` | | +| `IncrementWatermark(current)` | `incrementWatermark(cur)` | 溢出放弃（显式检测） | +| `CreateArgon2Config(config)` | `createArgon2Params(cfg)` | | +| `TryGetOption/HasFlag/HasPasswordStdin` | 同名小写 | | +| `ReadPassword(prompt, args)` | `readPassword(prompt, args)` | --password-stdin 或 x/term.ReadPassword | +| `SingleInstanceLockHandle/Acquire(path, pollDelay)` | `lock.Acquire(path, pollDelay) (*Handle, error)` | gofrs/flock：OpenOrCreate + FileShare.None 语义等价（进程死亡 OS 释放）；PID 仅诊断写入；3 次重试后优雅失败 | + +### 7.15 Models（ConnectionContext.cs / ConnectionStateBag.cs / ReceivedMessage.cs）→ internal/models + +| C# | Go | 行为 | +|---|---|---| +| `ConnectionContext` | `struct Connection` | ID/Username/ClientID/ClientName/Conn（gorilla）/Hub（转正一次性赋值后不可变）/State | +| `ConnectionStateBag` | `struct StateBag` | lastSeen/lastPingAt 锁内赋值；SendCh chan；HelloDeadline | +| `MarkPingAwaitingPong/TryTakePongAwaiting` | 同名 | | +| `MarkClosed()` | `(*StateBag).MarkClosed() bool` | CAS 守卫 | +| `TryStartHelloTimeout()` | `(*StateBag).TryStartHelloTimeout() bool` | | +| `TryEnqueueSend(payload)` | `(*StateBag).TryEnqueueSend(p []byte) bool` | `select default`，满 false | +| `ReceivedMessage` | `struct Frame` | | + +## 8. 关键语义对照细节 + +| 项 | C# | Go 1:1 方案 | +|---|---|---| +| 协议出站时间戳 | `yyyy-MM-dd'T'HH:mm:ss'Z'`（UTC 秒级） | layout `"2006-01-02T15:04:05Z"` | +| 登录响应 expiresAtUtc | "O" 往返格式（固定 7 位小数秒） | layout `"2006-01-02T15:04:05.0000000Z07:00"` | +| 入站时间戳 | 秒级或 ISO 往返、偏移必须为零 | `ParseFlexibleTime` 接受两种形态 | +| token base64url | System.Buffers.Text.Base64Url | `base64.RawURLEncoding`（无填充一致） | +| HMAC 比较 | 固定时间 | `hmac.Equal` | +| 字节长度校验 | UTF-8 字节数（1–128 / 0–128 / 4096） | `len([]byte(s))` 一致 | +| ulong 溢出 | C# checked 抛出 → RebuildHub | `cur == math.MaxUint64` 显式 panic → recover → RebuildHub | +| unbounded → 有界转换 | Channel.Writer/Reader | slice+Cond 队列 / buffered chan | +| volatile 写 | Volatile.Write | `atomic.Pointer` | +| File.Replace（Windows）/ rename | 原子替换 | `os.Rename`（Windows 为 MoveFileEx REPLACE_EXISTING） | +| FileSystemWatcher | 4 事件 + 防抖 | fsnotify 事件 + time.AfterFunc 防抖 | + +## 9. 测试迁移计划（全量，Q3） + +| C# 测试（TextCascade.Server.Tests） | Go 测试（同包 _test.go） | +|---|---| +| TokenServiceTests / AuthDeepTests / AuthServiceTimingTests | auth 包（token 全形态、Argon2 三函数、needsRehash、登录时序侧信道） | +| ClipAndCoreTests / RuntimeStateAndProtocolTests / IdempotencyBehaviorTests / ConnectionStateTests | core / protocol / state / models 包 | +| ConfigTests / UsersFileTests / CliWatermarkTests / SingleInstanceLockTests | config / users / cli 包 | +| ContractTests + ContractSamples（全矩阵） | protocol 包 + `testdata/contract-samples/`（复制自 main，CI sha256 校验防漂移） | +| WebSocketIntegrationTests（5 例） | server 集成（httptest 或 127.0.0.1:0 真实监听 + FastHasher 注入等价物） | +| NetworkIntegration（12 例，真实 TLS） | 同等 12 例（runtime 自签证书、显式 TLS1.2/1.3 探针、随机端口、帧分片、1009、重启恢复、bye/1001、HTTPS 登录全链路） | +| SlowHashSmokeTests（3 例） | 同等 3 例（生产参数真实 Argon2id） | +| UserFileWatcherTests / UserHubCoordinationTests / UserLoopConcurrencyTests / LoginLimiterTests | 对应包 | + +契约字节不变式断言（welcome/clip/ack/ping/bye/error/login-response/token payload）逐条搬运；C# Theory ↔ Go table-driven + t.Run。 + +## 10. CI 与发布（go 分支） + +- `ci.yml`（go 分支改造）：build+vet+test（过滤 network tag）+ 独立 network job + 契约样本 sha256 与 main 一致性校验步骤。 +- `release.yml`：tag 触发，linux-x64/win-x64 `GOOS/GOARCH` 矩阵，`-trimpath -ldflags "-X main.version="`，产物 `TextCascade.Server(.exe)`。 +- README 增加 Go 版构建/部署/证书支持矩阵说明（衍生后续项）。 + +## 11. 迁移注意事项（生产切换 runbook，Q6 结果） + +1. **切换前必须重置全部用户密码**：Go 版无法验证 C# 创建的存量 Argon2 哈希。停机窗口内对每个存量用户执行 `TextCascade.Server user passwd --username `（Go 版二进制）。 +2. token 与 tokenVersion 机制不变：重置密码不撤销已签发 token；如需强制重新登录，对每个用户执行 `user revoke-tokens`（可选步骤，明确决策后加入 runbook）。 +3. users.json / textcascade.state.json / textcascade.toml 格式不变，原样沿用；证书 PEM 原样沿用（PFX 路径实现首日验证）。 +4. systemd 单元沿用（二进制路径替换）；perf.md 场景重测并修订内存目标。 + +## 12. 验收清单 + +- [ ] 契约测试全矩阵通过（与 C# 相同样本、相同字节不变式） +- [ ] 单元 162 + 网络 12 + SlowHash 3 等价例全绿 +- [ ] C# 版 §10.2 计划中尚未实施的 6 个补齐用例（子协议 400、hello 超时、NeedsRehash 不重写、同 clientId 排除、用户隔离、慢连接取消）**不在本次范围**（1:1 原则：C# 没有的测试不发明） +- [ ] 手动对拍：Go 版与 C# 版同时挂起，同一客户端登录两端行为一致（welcome/广播/恢复） +- [ ] perf.md 场景重测，内存目标修订 + +## 13. 继承的实现差距台账（1:1 保留，不借迁移修复） + +C# spec §15 的 9 项差距在 Go 版**原样保留**（含对应实现形态），除非用户另行决策： + +1. 预 hello 连接不在 bye/1001 广播范围； +2. 空 hub 回收遗留 parked goroutine（Channel 未 Close 等价语义）； +3. 部分队列满路径直调 cancel 绕过 CancelConnection 入口； +4. TLS 下限跟随 OS 默认； +5. 明文测试缝隙（Go 侧对应：测试构建路径允许无证书 http.Server）； +6. ApplyClip 死分支兜底保留； +7. 待补齐测试项维持待办； +8. 停机 close 握手等待无超时（34 秒现状）； +9. 队列满熔断不产生 disconnect 安全事件。 + +> 若希望借迁移修复其中某项（例如 §13.8 停机 34 秒最痛），请在评审本 spec 时指出，按变更单处理而非默认纳入。 diff --git a/docs/server-spec.md b/docs/server-spec.md index 004dd08..1baf17d 100644 --- a/docs/server-spec.md +++ b/docs/server-spec.md @@ -1,7 +1,7 @@ # TextCascade 轻量文本同步服务端规格 -状态：与 v0.4.0 实现对齐；测试与契约细节以 specs/test-and-contract-spec.md 为准 -日期：2026-09-02 +状态：与 v0.5.0 实现对齐 +日期：2026-09-05 协议目标：不兼容原 ClipCascade，只做轻量、可靠、高性能的文本最新值同步 ## 1. 目标与非目标 @@ -57,7 +57,7 @@ flowchart LR ### 2.1 运行时与进程 - 技术栈：ASP.NET Core Minimal API + Kestrel 原生 WebSocket。 -- 目标框架：`net10.0`；产品版本采用 SemVer，写入 `TextCascade.Server.csproj` 的 `Version`（当前 0.4.0）。 +- 目标框架：`net10.0`；产品版本采用 SemVer，写入 `TextCascade.Server.csproj` 的 `Version`（当前 0.5.0）。 - 进程模型：单进程；生产环境由 systemd 或 Windows Service 托管并负责崩溃自动重启。 - TLS：Kestrel 直接终止 TLS；不提供生产/开发模式开关，所有部署都禁止明文 HTTP 登录。TLS 协议版本跟随 OS 默认策略，未显式固定下限（见 8.2 与差距台账）。 - 部署产物：框架依赖单文件；目标机必须预装对应 .NET Runtime。 @@ -137,6 +137,7 @@ state_file = "textcascade.state.json" - token secret 必须由环境变量提供，长度至少 32 字节；缺失或过短时启动失败。 - TLS 始终启用；`certificate_path` 必须指向服务端可用证书。 - 证书支持无密码格式：`.pem` / `.crt` 必须是包含叶证书与未加密私钥的 PEM bundle（允许同名 `.key` 边车文件承载私钥）；`.pfx` / `.p12` 必须可无密码加载。遇到需要密码的 PFX 时启动失败。 +- 私钥存储（v0.5.0 起）：PFX 在 Windows 上以 `DefaultKeySet` 持久化私钥加载（SChannel 无法用 ephemeral key 完成 TLS 握手），非 Windows 平台使用 `EphemeralKeySet`；PEM 在 Windows 上经 PFX 导出重导入以持久化私钥（CertificateLoader，ServerHost.cs）。 - TOML 使用宽松解析：必须以 UTF-8 读取（非 UTF-8 字节 fail-fast）；未知键忽略并输出 warning；结构或类型非法 fail-fast。**重复键视为解析错误直接启动失败**（Tomlyn 语义）。 - `max_frame_bytes` 必须大于 `max_text_bytes`，差额留给 JSON 协议头。 - 所有容量与时间配置必须大于 0，心跳超时必须大于心跳间隔。 @@ -229,7 +230,7 @@ Content-Type: application/json 实现要点： - `AuthService.HandleLoginAsync(HttpContext, config, syncServer, logger)`：薄入口，HTTP 处理内联其中。 -- 请求体限制 16KB、JSON 深度 3、拒绝未知字段与重复字段；畸形请求体（缺字段、类型错误、非法 JSON）返回规格内统一形态之外的 `400 invalid_request`。 +- 请求体限制 16KB、JSON 深度 3、拒绝未知字段与重复字段（v0.5.0 起：`ParseLoginRequest` 单次严格 `JsonSerializer.DeserializeAsync`，`MaxDepth=3`、`AllowDuplicateProperties=false`、`JsonUnmappedMemberHandling.Disallow`；16KB 上限经 `IHttpMaxRequestBodySizeFeature` 强制，chunked 请求体同样受限）；畸形请求体（缺字段、类型错误、非法 JSON）返回规格内统一形态之外的 `400 invalid_request`。 - 认证使用常数时间比较；不存在的用户以缓存的 dummy hash 执行同等验证，消除用户名存在性的计时侧信道。 - 限流命中返回 `429 Too Many Requests`，错误码 `rate_limited`；认证失败、用户不存在、用户禁用统一返回 `401 {"error":"invalid_credentials","message":"Invalid username or password."}`。 @@ -482,7 +483,7 @@ Upgrade: websocket 实现： -- 统一扫描器 `HeartbeatScannerService` 固定每 1 秒运行一次，集中处理 ping 调度（间隔默认 30 秒）、hello 超时与心跳超时判定（默认 60 秒无 pong 取消连接）；检测延迟 0–1 秒，不提供独立配置。 +- 统一扫描器 `HeartbeatScannerService`（`BackgroundService` 内 `PeriodicTimer` 驱动，扫描异常记入日志而非静默吞掉）固定每 1 秒运行一次，集中处理 ping 调度（间隔默认 30 秒）、hello 超时与心跳超时判定（默认 60 秒无 pong 取消连接）；检测延迟 0–1 秒，不提供独立配置。 - pong 更新 lastSeen 经用户 Channel 由单消费者落账；用户循环被大 clip 占用时 pong 记账可能延迟数秒。 - 收到没有未决 ping 的主动 pong，回复 `invalid_message` 错误帧但不断开连接（spec 外补充分支）。 @@ -662,7 +663,7 @@ hub 清理： ## 10. 测试计划 -详细到函数层面的规格见 [specs/test-and-contract-spec.md](test-and-contract-spec.md)，本节描述现状与分层。集成测试机制自 v0.3.0 起采用真实 Kestrel 绑定 `127.0.0.1:0` 的 fixture（`ServerHost.CreateApp` 构建 + FastPasswordHasher 注入）。 +本节自包含描述测试现状与分层，不依赖外部文件。集成测试机制自 v0.3.0 起采用真实 Kestrel 绑定 `127.0.0.1:0` 的 fixture（`ServerHost.CreateApp` 构建 + FastPasswordHasher 注入）。 ### 10.1 纯单元测试（现有覆盖） @@ -670,13 +671,14 @@ hub 清理： - CLI 单实例文件锁（`FileShare.None`）：活跃互斥、崩溃残留锁文件可复用、锁路径校验。 - `SlidingWindowLoginLimiter`：双维度、跨 IP、成功仅清用户窗口、max keys、过期清理。 - `TryAcquireClipToken`（TokenBucket refill）、`CheckFrameSize`/`CheckPayloadSize`、SeenIdRing 去重与淘汰、`NextVersion` 含 ulong.MaxValue 抛出、`SelectSnapshotWinner` 三规则。 -- Argon2 三函数（SlowHash）、token 数字全形态、CLI 水位/溢出、WithVersion、重复 id 行为级断言均已由测试覆盖（函数级规格见 test-and-contract-spec §3）。 +- Argon2 三函数（SlowHash）、token 数字全形态、CLI 水位/溢出、WithVersion、重复 id 行为级断言均已由测试覆盖。 +- 证书加载（CertificateLoaderTests）：PEM RSA/ECDSA 证书、合并 bundle 与独立 `.key` 边车、缺私钥与无证书内容的拒绝路径。 ### 10.2 CI 集成测试：真实 Kestrel loopback 现有 `WebSocketIntegrationTests` 覆盖：登录与握手往返（含 welcome）、无效 token 不升级、两客户端广播与发送方 ACK、重复 id 同版本 ACK、断连后按最高 lastServerVersion 快照恢复（结合版本持久化）、突兀断开被记录且服务存活（日志不含密码/secret）。 -计划中的补齐用例（子协议 400、hello 超时、NeedsRehash 不重写、同 clientId 排除规则、用户隔离、慢连接取消、bye/1001 等）同样收录于 test-and-contract-spec，实施后回填。 +计划中的补齐用例（子协议 400、hello 超时、NeedsRehash 不重写 users.json、同 clientId 排除规则、用户隔离、慢连接取消）尚未实施，落地后回填本节；bye/1001 停机链路已由 §10.3 网络集成测试覆盖。 ### 10.3 本地网络测试：Category=NetworkIntegration @@ -684,11 +686,11 @@ hub 清理： dotnet test TextCascade.Server.slnx --filter Category=NetworkIntegration ``` -CI 默认排除（ci.yml 过滤参数见 test-and-contract-spec 实施清单）。覆盖：自签证书 TLS/WSS、显式 Tls12/Tls13 握手、随机端口绑定、真实帧分片、超限帧 1009、重启两次 CreateApp 后 token 直连与快照恢复、停机 bye/1001、HTTPS 登录全链路。用例级明细见 test-and-contract-spec §1。 +CI 主测试任务以 `--filter "Category!=NetworkIntegration"` 排除该类别，由独立 CI 任务运行。覆盖：自签证书 TLS/WSS、显式 Tls12/Tls13 握手、随机端口绑定、真实帧分片、超限帧 1009、重启两次 CreateApp 后 token 直连与快照恢复、停机 bye/1001、HTTPS 登录全链路。 ### 10.4 契约测试 -样本文件组织于 Tests 项目 `ContractSamples/`（valid/invalid 分类、非法数字与非法 UTF-8 全矩阵、深度 4、重复/未知字段），由 Theory 驱动断言 `ParseClientMessage` 结果与序列化字节不变式；样本文件同时作为三端实现的公共对拍集合。明细见 test-and-contract-spec §2。 +样本文件组织于 Tests 项目 `ContractSamples/`（valid/invalid 分类、非法数字与非法 UTF-8 全矩阵、深度 4、重复/未知字段），由 Theory 驱动断言 `ParseClientMessage` 结果与序列化字节不变式；样本文件同时作为三端实现的公共对拍集合。 ## 11. 客户端适配要求 @@ -724,9 +726,9 @@ CI 默认排除（ci.yml 过滤参数见 test-and-contract-spec 实施清单） 用户 Channel 单消费者、有界发送队列与立即取消、幂等 id、服务端版本号与不可变最新值、应用层心跳、统一取消清理。 -### M3：恢复与真实网络 —— 大体达成，余项转入 §10.3 +### M3：恢复与真实网络 —— 已达成，补齐用例见 §10.2 -优雅停机 bye/1001、快照恢复窗口与预算/队列约束、tokenVersion 撤销、版本号持久化；真实 TCP/TLS 集成测试与多端收敛测试补齐中（specs/test-and-contract-spec §1）。 +优雅停机 bye/1001、快照恢复窗口与预算/队列约束、tokenVersion 撤销、版本号持久化；真实 TCP/TLS 集成测试已落地（§10.3），其余补齐用例见 §10.2。 ### M4：生产化 —— 大体达成，两项移交差距台账 @@ -734,7 +736,7 @@ Kestrel TLS、结构化日志与脱敏、登录与消息限流、框架依赖单 ## 13. 版本与发布 -- 产品版本采用 SemVer 2.0.0，从 `0.1.0` 开始演进，以 `TextCascade.Server.csproj` 的 `Version` 为准（当前 0.4.0）。 +- 产品版本采用 SemVer 2.0.0，从 `0.1.0` 开始演进，以 `TextCascade.Server.csproj` 的 `Version` 为准（当前 0.5.0）。 - `protocolVersion` 只表示线协议版本，当前为 `1`，与产品版本独立演进。 - 目标框架为 `net10.0`；目标机必须预装兼容的 .NET 10 Runtime。 - 发布命令：`dotnet publish TextCascade.Server.csproj -c Release -p:PublishSingleFile=true`（win-x64/linux-x64 框架依赖单文件）。 @@ -764,7 +766,7 @@ Kestrel TLS、结构化日志与脱敏、登录与消息限流、框架依赖单 4. TLS 协议下限未显式固定，依赖 OS 默认（§8.2）。 5. `ServerHost.CreateApp(certificate:null)` 的明文测试缝隙（仅 InternalsVisibleTo 可达）。 6. ApplyClip 中 duplicateId 且 Latest 为 null 的兜底分支不可达（死分支）。 -7. §10 中标注"待补齐/补齐中"的测试项以 specs/test-and-contract-spec.md 落地为准。 +7. §10 中标注"待补齐/补齐中"的测试项尚未实施，落地后回填本 spec。 8. 优雅停机对每个连接的 `CloseAsync` close 握手等待无超时：有静默客户端在线时，停机阶段实测可达 34 秒（perf.md S7）；§7 的"等待最多 2 秒"仅覆盖握手完成后的 drain。 9. 发送队列满的熔断路径（`MarkClosed` + `Cts.Cancel`）不产生 disconnect 安全事件：后续 `CancelConnection` 因 `MarkClosed` 已置位而提前返回，被熔断的连接在日志中不可见（perf.md S6）。 From cbe9071064e4f460aaf0a131229dabe6b910681f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E9=93=8D=E7=9B=90=E7=94=9C=E4=B8=8D=E7=94=9C?= <51872789+long45343@users.noreply.github.com> Date: Sat, 5 Sep 2026 08:51:18 +0800 Subject: [PATCH 32/32] Update ci.yml --- .github/workflows/ci.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 374fe62..6ecb11d 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -2,9 +2,9 @@ name: CI on: push: - branches: [main] + branches: [csharp] pull_request: - branches: [main] + branches: [csharp] workflow_dispatch: jobs: