frame metadata for ARKitScenes

[Jimntu]

Hi, thanks for the amazing work!

I would like to ask why there is no frame metadata generation script for ARKitScenes dataset? How can you generate appearance order Q&A data without frame metadata?

Thank you so much!

[Jimntu]

            # Instance mask is optional? Check ScanNet++ documentation. Assume optional for now.
            if not os.path.exists(instance_path_abs):
                logger.debug(f"Instance mask file missing for frame {frame_id_base} in {scene_id}: {instance_path_abs}.")
                instance_path_rel = None # Indicate missing mask

 # --- 4d. Extract BBoxes (Optional) ---
            # TODO: Implement bbox extraction from instance mask if needed
            bboxes_2d = [] # Placeholder
            if instance_path_rel: # If mask exists

I notice that the frame_metadata for scannetpp does not include instance id and bbox of objects in each frame. In this case, how can you generate object_first_appearance qa? I would really appreciate if you could answer the question. Thank you so much!

[CQC-gogopro]

您好，感谢您出色的工作！

我想请问为什么ARKitScenes数据集没有帧元数据生成脚本？如何在没有帧元数据的情况下生成外观顺序问答数据？

太感谢了！

I have the same question.

[jzh15]

Sorry for the late reply — I’ve been quite tied up with other matters recently.

1. ARKITScenes:
   For ARKitScenes, the official release only provides bounding box annotations, but does not provide accurate scene-level instance segmentation. As a result, frame-level metadata (e.g., per-frame instance maps) cannot be directly generated, and we therefore did not generate or release frame metadata for this dataset.

2. ScanNet++ frame data:
   For ScanNet++, although we experimented with frame-level data generation internally, it was not ultimately used in our final pipeline, and we did not release a fully polished implementation. The relevant code in the repository was not carefully refactored or cleaned up, so its usability may indeed be limited.

We released the complete ScanNet processing pipeline as a working example. ScanNet++ frame data is more complex to handle, mainly because the official ScanNet++ release does not directly provide frame-level annotations (such as instance maps or semantic maps). Generating such frame data requires an additional rendering step, and the corresponding rendering code can be found in the original ScanNet++ repository.

Hope this clarifies the design choices and code status. Thanks again for your interest, and feel free to follow up if you have further questions!