Issues

• An LLM-judge reply that fails the assert in llm_fuzzy_match/llm_ua_match is excluded from the reported score instead of counting as a failure

  web-arena-x/visualwebarena#87 · Filter by author shaurya416 opened on Sep 30, 2026

• False negatives in evaluation due to tokenization

  web-arena-x/visualwebarena#84 · Filter by author cherry979988 opened on Jan 31, 2026

• AMI not found

  web-arena-x/visualwebarena#83 · Filter by author khanddorjmendbayar-svg opened on Jan 27, 2026
  2

• How to upload results to the leaderboard

  web-arena-x/visualwebarena#79 · Filter by author kevinhikali opened on Aug 27, 2025
  2

• playwright timeout error

  web-arena-x/visualwebarena#78 · Filter by author c4chuan opened on Jun 21, 2025
  1

• pytest -x failed

  web-arena-x/visualwebarena#77 · Filter by author payphone131 opened on Mar 21, 2025

• The instruction of installation is non-reproducible

  web-arena-x/visualwebarena#76 · Filter by author LEON-gittech opened on Mar 1, 2025
  1

• General question on how much need to spend on API call

  web-arena-x/visualwebarena#75 · Filter by author zhengqigao opened on Jan 3, 2025

• Action String Format for SoM Click Operation

  web-arena-x/visualwebarena#74 · Filter by author yangpei-comp opened on Dec 29, 2024

• trajectory need to be deepcopy!

  web-arena-x/visualwebarena#73 · Filter by author indeedzcr opened on Dec 4, 2024

• Wrong evaluation

  web-arena-x/visualwebarena#71 · Filter by author yeonjooooni opened on Nov 3, 2024
  1

• Classifieds is impossible to evaluate for vision-based CC agents

  web-arena-x/visualwebarena#70 · Filter by author liaopeiyuan opened on Nov 3, 2024
  2

• clear action related test failed

  web-arena-x/visualwebarena#66 · Filter by author shuyanzhou opened on Sep 23, 2024

• Access to Dockerfile and compose.yaml files

  web-arena-x/visualwebarena#64 · Filter by author Philmod opened on Sep 19, 2024
  2

• How to set up multiple docker for the same website to conduct the parallel evalution.

  web-arena-x/visualwebarena#61 · Filter by author junleiz opened on Aug 23, 2024
  1

• Need for resetting in run som scripts

  web-arena-x/visualwebarena#60 · Filter by author sanjari-orb opened on Aug 16, 2024
  2

• Pytest code is not working

  web-arena-x/visualwebarena#56 · Filter by author pokerme7777 opened on Jul 19, 2024
  1

• How to open trace files without display server?

  web-arena-x/visualwebarena#53 · Filter by author piperwolters opened on Jun 28, 2024
  1

• May I know any solution for using apptainer instead of docker for starting up website?

  web-arena-x/visualwebarena#51 · Filter by author pokerme7777 opened on Jun 26, 2024

• PyTest fail on test_click_open_new_tab

  web-arena-x/visualwebarena#28 · Filter by author anithselva opened on Apr 30, 2024
  

• Reproducing open-source model results

  web-arena-x/visualwebarena#27 · Filter by author anithselva opened on Apr 29, 2024
  2