Spec optional conversion to RGB in VideoFrame.copyTo()

Author: Djuffin

index.src.html

@@ -38,6 +38,8 @@
 spec: html; urlPrefix: https://html.spec.whatwg.org/multipage/;
     for: ImageBitmap;
         type: dfn; text: bitmap data; url: imagebitmap-and-animations.html#concept-imagebitmap-bitmap-data
+    for: ImageData
+        type: dfn; text: canvas pixel data; url: canvas.html#imagedata
     for: Canvas;
         type: dfn; text: Check the usability of the image argument; url: canvas.html#check-the-usability-of-the-image-argument
         type: dfn; text: is not origin-clean; url: canvas.html#the-image-argument-is-not-origin-clean
@@ -3745,8 +3747,8 @@
 
 : <dfn method for=VideoFrame>copyTo(|destination|, |options|)</dfn>
 :: Asynchronously copies the planes of this frame into |destination| according
-    to |options|. The format of the data is the same as this {{VideoFrame}}'s
-    {{VideoFrame/format}}.
+    to |options|. The format of the data is specified by {{VideoFrameCopyToOptions/format}},
+    if present, or the same as {{VideoFrame/format}}.
 
     NOTE: Promises that are returned by several calls to
         {{VideoFrame/copyTo()}} are not guaranteed to resolve in the order they
@@ -3764,10 +3766,20 @@
     5. If `destination.byteLength` is less than |combinedLayout|'s [=combined
         buffer layout/allocationSize=], return a promise rejected with a
         {{TypeError}}.
-    6. Let |p| be a new {{Promise}}.
-    7. Let |copyStepsQueue| be the result of starting a new [=parallel queue=].
-    8. Let |planeLayouts| be a new [=list=].
-    9. Enqueue the following steps to |copyStepsQueue|:
+    6. If `options.format` does not equal to {{VideoFrame/[[format]]}} and
+        `options.format` is equal to one of {{RGBA}}, {{RGBX}}, {{BGRA}}, {{BGRX}}
+        then:
+          1. Let |serializedOptions| be [$StructuredSerialize$](|options|)
+          2. Let |newOptions| be [$StructuredDeserialize$](|serializedOptions|, [=the current Realm=]).
+          3. Set |newOptions.format| to `undefined`
+          4. Let |rgbFrame| be the result of running the [=VideoFrame/Convert to RGB frame=]
+               algorithm with `options.format`, `options.rect`, and  `options.colorSpace`.
+          5. Return the result of calling {{VideoFrame/copyTo()}} on |rgbFrame| with
+               |destination| and |newOptions|.
+    7. Let |p| be a new {{Promise}}.
+    8. Let |copyStepsQueue| be the result of starting a new [=parallel queue=].
+    9. Let |planeLayouts| be a new [=list=].
+    10. Enqueue the following steps to |copyStepsQueue|:
         1. Let resource be the [=media resource=] referenced by
             [[resource reference]].
         2. Let |numPlanes| be the number of planes as defined by
@@ -3804,7 +3816,7 @@
             10. Increment |planeIndex| by `1`.
             11. Append |layout| to |planeLayouts|.
         5. [=Queue a task=] to resolve |p| with |planeLayouts|.
-    10. Return |p|.
+    11. Return |p|.
 
 : <dfn method for=VideoFrame>clone()</dfn>
 :: Creates a new {{VideoFrame}} with a reference to the same
@@ -4048,10 +4060,17 @@
     6. Let |optLayout| be `undefined`.
     7. If |options|.{{VideoFrameCopyToOptions/layout}} [=map/exists=], assign
         its value to |optLayout|.
-    8. Let |combinedLayout| be the result of running the [=VideoFrame/Compute
-        Layout and Allocation Size=] algorithm with |parsedRect|,
-        {{VideoFrame/[[format]]}}, and |optLayout|.
-    9. Return |combinedLayout|.
+    8. Let |format| be `undefined`.
+    9. If |options|.{{VideoFrameCopyToOptions/format}} is `undefined`,
+        assign {{VideoFrame/[[format]]}} to |format|.
+    10. Otherwise, if |options|.{{VideoFrameCopyToOptions/format}} is equal to
+        one of {{RGBA}}, {{RGBX}}, {{BGRA}}, {{BGRX}}, then assign
+        |options|.{{VideoFrameCopyToOptions/format}} to |format|,
+        otherwise return {{NotSupportedError}}.
+    11. Let |combinedLayout| be the result of running the [=VideoFrame/Compute
+        Layout and Allocation Size=] algorithm with |parsedRect|, |format|,
+        and |optLayout|.
+    12. Return |combinedLayout|.
 
 : <dfn for=VideoFrame>Verify Rect Offset Alignment</dfn> (with |format| and
     |rect|)
@@ -4183,10 +4202,36 @@
             [=combined buffer layout/allocationSize=].
     9. Return |combinedLayout|.
 
+: <dfn for=VideoFrame>Convert to RGB frame</dfn> (with |format|, |rect| and |colorSpace|)
+ :: 1. Let |canvas| be a new {{OffscreenCanvas}} constructed with
+        |rect|.|width| and |rect|.|height|.
+    2. Let |canvasSettings| be a new {{CanvasRenderingContext2DSettings}} with
+        {{CanvasRenderingContext2DSettings/colorSpace}} set to |colorSpace|.
+    3. Let |context| be the result of calling {{OffscreenCanvas/getContext()}}
+        on |canvas| with `"2d"` and |canvasSettings|
+    3. Call {{CanvasDrawImage/drawImage()}} on |context| with
+        `rect.x`, `rect.y`, `rect.width`, `rect.height`, `0`, `0`, `rect.width`,
+        `rect.height`.
+    4. Let |imageData| be the result of calling {{CanvasImageData/getImageData()}} with
+        `0`, `0`, |rect.width|, |rect.height|.
+    5. If |format| is equal to {{BGRA}} or {{BGRX}}:
+        1. Let |pixelIndex| be `0`
+        2. While |pixelIndex| is less than `rect.width` multiplied by `rect.height`:
+            1. Swap items in |imageData|.{{ImageData/data}} at indexes `4 * pixelIndex`
+                 and `4 * pixelIndex + 2`.
+            2. Increment |pixelIndex| by `1`
+    6. Let |init| be a new {{VideoFrameBufferInit}} constructed with
+        {{VideoFrameBufferInit/format}} = |format|,
+        {{VideoFrameBufferInit/codedWidth}} = `rect.width`,
+        {{VideoFrameBufferInit/codedHeight}} = `rect.height`
+    7. Let |convertedFrame| be a new {{VideoFrame}} constructed with
+        |imageData|.{{ImageData/data}} and |init|
+    8. Return |convertedFrame|
+
 : <dfn for=VideoFrame>Copy VideoFrame metadata</dfn> (with |metadata|)
 :: 1. Let |metadataCopySerialized| be [$StructuredSerialize$](|metadata|).
     2. Let |metadataCopy| be [$StructuredDeserialize$](|metadataCopySerialized|, [=the current Realm=]).
-    3. return |metadataCopy|.
+    3. Return |metadataCopy|.
 
 The goal of this algorithm is to ensure that metadata owned by a {{VideoFrame}} is immutable.
 
@@ -4239,13 +4284,15 @@
 
 VideoFrame CopyTo() Options {#videoframe-copyto-options}
 ------------------------------------------------------------
-Options to specify a rectangle of pixels to copy and the offset and stride of
-planes in the destination buffer.
+Options to specify a rectangle of pixels to copy, their format and the offset
+and stride of planes in the destination buffer.
 
 <xmp class='idl'>
 dictionary VideoFrameCopyToOptions {
   DOMRectInit rect;
   sequence<PlaneLayout> layout;
+  VideoPixelFormat format;
+  PredefinedColorSpace colorSpace;
 };
 </xmp>
 
@@ -4273,6 +4320,16 @@
     to specify an offset and stride for each plane in the destination
     {{BufferSource}}. If unspecified, the planes will be tightly packed. It is
     invalid to specify planes that overlap.
+: <dfn dict-member for=VideoFrameCopyToOptions>format</dfn>
+:: A {{VideoPixelFormat}} for the pixel data in the destination
+    {{BufferSource}}. Explicity values cab be:
+    {{RGBA}}, {{RGBX}}, {{BGRA}}, {{BGRX}} and the same as {{VideoFrame/format}}.
+    If unspecified, the {{VideoFrame/format}} is used.
+: <dfn dict-member for=VideoFrameCopyToOptions>colorSpace</dfn>
+:: A {{PredefinedColorSpace}} that <em class="rfc2119">SHALL</em> be used to as
+    a target color space for for the pixel data in the destination
+    {{BufferSource}}.
+
 
 DOMRects in VideoFrame {#videoframe-domrect}
 --------------------------------------------