Skip to content

Write launch blog post: 'Why we run 5 AI agents and pick the best' #141

Closed
Closed
Write launch blog post: 'Why we run 5 AI agents and pick the best'#141
Labels
enhancementNew feature or request

Description

@that-github-user
that-github-user
opened on Mar 29, 2026

Summary

For HackerNews / dev.to / blog launch, we need a compelling narrative:

Proposed structure

  1. The problem: AI coding is unreliable. pass@1 is a gamble.
  2. The insight: pass@5 >> pass@1 (academic evidence)
  3. The solution: thinktank runs N agents in parallel
  4. The twist: we use social choice theory (Copeland) to pick the winner
  5. Real data: our scoring evaluation shows Copeland and Borda agree 86%
  6. Dogfooding: thinktank built itself (63 PRs, 237 tests)
  7. The future: A* showcase, arXiv paper

Key metrics to include

  • 63 PRs merged in one session
  • 237 tests, all passing
  • 45 thinktank runs dogfooding itself
  • Copeland-Borda 86% concordance finding
  • Test pass rate improvement from 21% to 72%

Activity

  1. that-github-user
    added
    enhancementNew feature or request
    on Mar 29, 2026

  2. that-github-user commented on Mar 31, 2026

    @that-github-user
    that-github-user
    on Mar 31, 2026
    OwnerAuthor

    Blog post is written and reviewed. Final version at C:\Users\silon\Documents\thinktank-blog-final.md.

    Ran a 5-agent ensemble review of the post against the competitive landscape (Star Chamber, llm-council, Composio, Roundtable, Aider, EnsLLM). All 5 agents converged: thinktank's differentiation is real — no other tool combines same-task ensemble + test verification + Copeland pairwise scoring.

    Key fixes applied during review:

    • Corrected arXiv IDs for EnsLLM and "Wisdom and Delusion" papers
    • Fixed GitHub URLs for Star Chamber, Roundtable
    • Added single-model limitation caveat with named alternatives
    • Reframed convergence as confidence signal (not correctness)
    • Fixed 96% stat to show honest 84-96% range with subset context
    • Removed "no tool exposes this — until now" from README
    • Added landscape section acknowledging related tools

    Ready to publish on dev.to. PR #161 blog assets in thinktank-blog-assets/.

  3. that-github-user
    closed this as completedon Mar 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions