Skip to content

Addressing issue #876 to add RRF scores - #886

Open
hyi wants to merge 8 commits into
masterfrom
876-add-RRF
Open

Addressing issue #876 to add RRF scores#886
hyi wants to merge 8 commits into
masterfrom
876-add-RRF

Conversation

@hyi

@hyi hyi commented Sep 10, 2026

Copy link
Copy Markdown

Addressing issue #876 to add RRF scores.

@MarkDWilliams It'd be great if you could test this out since I don't know the ARS workflow enough to test it locally. The following three new environment variables are introduced as shown below:

  • ARS_RRF_ENABLED: setting it to "true" or "false" with "true" enabling RRF ranking score fusion and "false" otherwise. So only setting this env variable to "true" will add RRF ranking scores as part of post-processing. The default is "true"
  • ARS_RRF_ARAGORN_WEIGHT: setting the ARAGORN ranker weight in RRF to a float number between 0 and 1. ARAX ranker weight will be automatically set as 1 minus this value since the two weights must sum to 1. If it is not set, the default value is 0.9, since that was the weight value that resulted in the best-performing overall F1 score from my testing.
  • ARS_RRF_C: setting the RRF dampening constant used in RRF formula for ranker fusion. If it is not set, the default value 40 is used since that was the value that resulted in the best-performing overall F1 score from my testing.

Let me know if you have any questions once you get a chance to do code review. Thanks

@hyi
hyi requested a review from MarkDWilliams as a code owner September 10, 2026 01:48

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well done! You have taken the first step into a larger world

@MarkDWilliams

Copy link
Copy Markdown
Collaborator

Looks good to me on review, but yes, I'll test locally before merging it. Thanks for putting this together @hyi

@hyi

hyi commented Sep 11, 2026

Copy link
Copy Markdown
Author

@MarkDWilliams Thanks for the prompt review. I realized I needed to handle tied RRF scores for ranking, so I just made a new commit to handle tied RRF score ranking, so that if two query results have the same RRF ranker combination score, their ranks will be set as the same. Take a quick look at that commit and let me know if you see any issues or have questions. I'll address any potential issues from your local testing if any. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants