Fix GetRotateCropImage: corner selection broken by OpenCV 4.13 RotatedRect convention change

[Kserol]

Summary

PaddleOcrAll.GetRotateCropImage selects the four crop corners from RotatedRect.Angle and
RotatedRect.Points() using a hard-coded ordering that assumes the OpenCV 4.11 RotatedRect
convention. OpenCV 4.13 changed that convention (angle range / points() order), so the same
code now picks the wrong corners. The perspective transform then maps corners incorrectly,
producing mirrored/rotated text-line crops and garbled recognition.

This is not a build/binding issue — recompiling Sdcb.PaddleOCR against OpenCvSharp 4.13 does not
help, because rect.Angle / rect.Points() are runtime native values and the C# selection logic
stays wrong.

Impact / reproduction

Same code, same model (LocalFullModels.LatinV5), same images (degraded French document scans),
only OpenCvSharp/OpenCV version differs:

OpenCvSharp	char accuracy	mean confidence
4.11.0.20250507	99.8 %	0.98
4.13.0.20260602 (NuGet win & linux)	~21 %	0.34
4.13.0.20260602 + this fix	99.9 %	0.98

The 4.13 result was reproduced with the NuGet OpenCvSharp4.official.runtime.* natives and
with the official source-built native in ghcr.io/shimat/opencvsharp/ubuntu24-dotnet10-opencv4.13.0,
so it is the OpenCV 4.13 RotatedRect behavior, not a packaging artifact.

This matters because the OpenCvSharp 4.11 native does not load on modern glibc (≥ 2.34) base images
(ld.so dl-version assertion), so users on .NET 8/10 Linux images are pushed to 4.13 — where OCR
silently degrades.

Root cause

// PaddleOcrAll.GetRotateCropImage — current
bool wider = rect.Size.Width > rect.Size.Height;
float angle = rect.Angle;
...
Point2f[] srcPoints = (wider, angle) switch
{
    (true, >= 0 and < 45) => new[] { rp[1], rp[2], rp[3], rp[0] },
    _                     => new[] { rp[0], rp[3], rp[2], rp[1] }
};
...
if (!wider)        Cv2.Transpose(dest, dest);
else if (angle > 45) Cv2.Flip(dest, dest, FlipMode.X);

Both the rp[...] index ordering and the angle-based flip/transpose depend on the exact
RotatedRect convention, which changed in OpenCV 4.13.

Proposed fix

Order the four corners geometrically (convention-independent) and derive crop size from the
corner distances, instead of relying on angle / Points() ordering:

public static Mat GetRotateCropImage(Mat src, RotatedRect rect)
{
    Size srcSize = src.Size();
    Rect boundingRect = rect.BoundingRect();

    int expTop    = Math.Max(0, 0 - boundingRect.Top);
    int expBottom = Math.Max(0, boundingRect.Bottom - srcSize.Height);
    int expLeft   = Math.Max(0, 0 - boundingRect.Left);
    int expRight  = Math.Max(0, boundingRect.Right - srcSize.Width);

    Rect rectToExp = boundingRect + new Point(expTop, expLeft);
    Rect roiRect = Rect.FromLTRB(
        boundingRect.Left + expLeft, boundingRect.Top + expTop,
        boundingRect.Right - expRight, boundingRect.Bottom - expBottom);
    using Mat boundingMat = src[roiRect];
    using Mat expanded = boundingMat.CopyMakeBorder(expTop, expBottom, expLeft, expRight, BorderTypes.Replicate);
    Point2f[] rp = rect.Points()
        .Select(v => new Point2f(v.X - rectToExp.X, v.Y - rectToExp.Y))
        .ToArray();

    // Convention-independent corner ordering (robust to OpenCV RotatedRect.Angle/Points() changes).
    Point2f tl = rp.Aggregate((a, b) => (a.X + a.Y) <= (b.X + b.Y) ? a : b);
    Point2f br = rp.Aggregate((a, b) => (a.X + a.Y) >= (b.X + b.Y) ? a : b);
    Point2f tr = rp.Aggregate((a, b) => (a.Y - a.X) <= (b.Y - b.X) ? a : b);
    Point2f bl = rp.Aggregate((a, b) => (a.Y - a.X) >= (b.Y - b.X) ? a : b);

    static float Dist(Point2f p, Point2f q) => (float)Math.Sqrt((p.X - q.X) * (p.X - q.X) + (p.Y - q.Y) * (p.Y - q.Y));
    int cw = Math.Max(1, (int)Math.Round(Math.Max(Dist(tl, tr), Dist(bl, br))));
    int ch = Math.Max(1, (int)Math.Round(Math.Max(Dist(tl, bl), Dist(tr, br))));

    Point2f[] srcPoints = { tl, tr, br, bl };
    Point2f[] dstPoints = { new(0, 0), new(cw, 0), new(cw, ch), new(0, ch) };

    using Mat matrix = Cv2.GetPerspectiveTransform(srcPoints, dstPoints);
    Mat dest = expanded.WarpPerspective(matrix, new Size(cw, ch), InterpolationFlags.Nearest, BorderTypes.Replicate);

    // Vertical text line -> rotate 90° CW so the recognizer sees a horizontal strip.
    if (ch >= cw * 1.5)
    {
        Cv2.Transpose(dest, dest);
        Cv2.Flip(dest, dest, FlipMode.X);
    }
    return dest;
}

This mirrors PaddleOCR's reference get_rotate_crop_image (geometric corner order + size from edge
lengths) and is robust for the near-horizontal text lines detectors emit. Validated at 99.9 % char
accuracy on both OpenCvSharp 4.11 and 4.13.

Notes / open questions

• Geometric min/max(x±y) ordering is robust for skew up to ~45°; for steeply rotated lines a
  centroid-angle sort could be considered, but detector boxes are near-horizontal in practice.
• Once merged, please consider bumping the OpenCvSharp4 dependency floor and noting 4.13 support,
  so downstreams know they can move off 4.11.

[n0099]

OpenCvSharp 4.11 native does not load on modern glibc (≥ 2.34) base images
(ld.so dl-version assertion)

Re-compile its underlaying opencv with older glibc and using https://github.com/NixOS/patchelf to load the standalone glibc in runtime is a way.

[n0099]

OpenCV 4.13 changed that convention (angle range / points() order),

Citation needed, I can't find issue about this in https://github.com/search?q=repo%3Aopencv%2Fopencv%20point%20order&type=issues&s=created&o=desc since the release of 4.13 on 2025-12-31.
And the doc of cv::RotatedRect::points() and angle() is unchanged between 4.11 and 4.13:

returns 4 vertices of the rotated rectangle

Parameters

pts

The points array for storing rectangle vertices. The order is bottomLeft, topLeft, topRight, bottomRight.

Note
Bottom, Top, Left and Right sides refer to the original rectangle (angle is 0), so after 180 degree rotation bottomLeft point will be located at the top right corner of the rectangle.

That note was added by me in opencv/opencv#23342 to clarify opencv/opencv#23335 as opencvsharp refused to re-order points: shimat/opencvsharp#1541 like this PR.

opencvsharp's own implement of RotatedRect.Points() is also stable: https://github.com/shimat/opencvsharp/blame/7e2f06056567be6fce93112ccd09812d7861ccc1/src/OpenCvSharp/Modules/core/Struct/RotatedRect.cs#L112

[n0099]

If we are targeting .NET 6.0+ we may use .MinBy() and .MaxBy(), but sadly we are on .NET Standard 2.0:

PaddleSharp/src/Sdcb.PaddleOCR/Sdcb.PaddleOCR.csproj

Line 3 in 6a004e0

<TargetFrameworks>netstandard2.0</TargetFrameworks>

[n0099]

#183
#178
#177

[sdcb]

Thanks for the detailed analysis and for tracking this down. I agree with the direction: GetRotateCropImage should not depend on RotatedRect.Angle or the native Points() ordering convention.

Before merging, I would prefer to adjust the geometric ordering slightly. I already solved the same issue in OpenVINO.NET by using an OrderPointsClockwise helper:

• sort points by X
• split the two left-most and two right-most points
• order the left-most pair by Y as top-left / bottom-left
• pick bottom-right as the right-most point farthest from top-left
• use the remaining right-most point as top-right

That avoids relying on angle/point-order conventions, but is also safer than min/max(x+y) and min/max(y-x), which can select duplicate points around exact +/-45 degree cases and produce a degenerate perspective transform.

Would you be open to updating this PR to use that ordering approach? The rest of the fix, especially deriving crop width/height from edge distances and rotating vertical crops based on the resulting aspect ratio, looks aligned with what we need.

[n0099]

https://github.com/sdcb/OpenVINO.NET/blob/bc748dc7b42f7f25fe9ae6a3c373932f82b32262/projects/PaddleOCR/Sdcb.OpenVINO.PaddleOCR/PaddleOcrAll.cs#L244

[Kserol]

Thanks, that makes sense — min/max(x±y) can indeed collapse two corners into the same point near ±45° and produce a degenerate transform. I've updated the PR to use the OrderPointsClockwise approach from OpenVINO.NET (sort by X, split the left/right pairs, then disambiguate the left pair by Y and pick bottom-right as the right-most point farthest from top-left).

The rest is unchanged: crop width/height derived from edge distances, and 90° rotation of vertical crops based on the aspect ratio. I kept the existing border/interpolation behaviour to minimize the diff.

Re-validated on my French degraded-document set: still ~99.8% char accuracy / 0.98 confidence with OpenCvSharp 4.13. Let me know if you'd like any further tweaks.