Conversation
fix body read err
Add daydaymap as a new supported search engine with complete integration: - Add daydaymap agent implementation in sources/agent/daydaymap/ - Integrate daydaymap CLI option (-ddm/--daydaymap) in runner options - Add daydaymap API key configuration in provider and keys - Register daydaymap in available engines list - Support DAYDAYMAP_API_KEY environment variable The implementation follows the existing agent pattern and maintains consistency with other search engine integrations (greynoise, driftnet, etc).
feat(agent): add daydaymap search engine support
…ugh-agents fix: thread ctx through sources.Agent (#724)
Contributor
WalkthroughThis PR refactors uncover's query execution model to be context-aware, enabling cancellation propagation across agents, and adds DayDayMap as a new search engine with CLI, provider/keys, wiring, and README updates. ChangesContext-aware execution and agent integration
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes
🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 9
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (3)
sources/agent/criminalip/criminalip.go (1)
83-109:⚠️ Potential issue | 🟠 Major | ⚡ Quick winClose
resp.Bodyinquery.Each paginated request returns without closing the HTTP body. That will leak connections across larger scans and eventually hurt retries/reuse.
Suggested fix
func (agent *Agent) query(ctx context.Context, URL string, session *sources.Session, criminalipRequest *CriminalIPRequest, results chan sources.Result) *CriminalIPResponse { resp, err := agent.queryURL(ctx, session, URL, criminalipRequest) if err != nil { sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err}) return nil } + defer func() { + _ = resp.Body.Close() + }() criminalipResponse := &CriminalIPResponse{} if err := json.NewDecoder(resp.Body).Decode(criminalipResponse); err != nil { sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err}) return nil🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@sources/agent/criminalip/criminalip.go` around lines 83 - 109, In function query, you never close the HTTP response body from agent.queryURL which leaks connections; immediately after receiving resp (and after checking err) add a defer resp.Body.Close() so the body is always closed before any early returns or after json decoding in CriminalIPResponse handling; ensure the defer is placed before json.NewDecoder(resp.Body).Decode(...) and before any returns inside the loop so resp.Body is always released even on decode errors or when SendResult returns false.sources/agent/google/google.go (2)
49-53:⚠️ Potential issue | 🟠 Major | ⚡ Quick winFix pagination increment to use result count, not 1.
The
StartIndexin the Google request maps to thestartparameter of the Google Custom Search API, which is a 1-based result index. According to Google's API documentation, whenCount(thenumparameter) is set to a value, the next request must incrementstartby that same count value. Currently,pageQueryincrements by 1 on line 69, producing overlapping result windows (start=1, 2, 3...) instead of sequential pages (start=1, 11, 21...). This causes duplicate results to be fetched and early termination.Suggested fix
- pageQuery += 1 + pageQuery += size🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@sources/agent/google/google.go` around lines 49 - 53, The pagination advances by 1 currently, causing overlapping windows; update the loop that builds the Google requests to increment the page start by the request Count instead of 1: when constructing the Request (googleRequest with fields SearchTerms, Count, StartIndex) use the existing size variable as Count and advance pageQuery by size (or set StartIndex = 1 + n*size) on each iteration so StartIndex becomes 1, 1+size, 1+2*size, ... ensuring non-overlapping pages.
76-123:⚠️ Potential issue | 🟠 Major | ⚡ Quick winAlways close
resp.Body, even on gzip responses.
gzipReader.Close()does not close the underlying HTTP body. Without closingresp.Body, each request leaks a connection slot during pagination, exhausting the connection pool.Suggested fix
func (agent *Agent) query(ctx context.Context, session *sources.Session, googleRequest *Request, results chan sources.Result) []string { resp, err := agent.queryURL(ctx, session, googleRequest) if err != nil { sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err}) return nil } + defer func() { + _ = resp.Body.Close() + }() var apiResponse Response if resp.Header.Get("Content-Encoding") == "gzip" {🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@sources/agent/google/google.go` around lines 76 - 123, In the Agent.query function the HTTP response body (resp.Body) is never closed which leaks connections during pagination; after successfully getting resp from agent.queryURL(...) add a defer resp.Body.Close() so the underlying connection is always released, and keep closing gzipReader when used (gzipReader.Close()) after decoding; ensure the defer for resp.Body.Close() is placed immediately after checking err from agent.queryURL so both gzip and non-gzip branches return with resp.Body closed.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@sources/agent/daydaymap/daydaymap.go`:
- Around line 100-110: The HTTP response body from agent.queryURL (resp) is
never closed, causing connection leaks; after the resp, err :=
agent.queryURL(...) success path, ensure the response body is closed by calling
resp.Body.Close()—preferably via defer immediately after confirming err == nil
(right before decoding into DaydaymapResponse) or alternatively ensure
agent.queryURL transfers ownership and callers close resp.Body; update the code
around agent.queryURL, resp, and the DaydaymapResponse decoding to guarantee
resp.Body.Close() is always executed.
In `@sources/agent/driftnet/driftnet.go`:
- Around line 78-90: The loops in querySearchTerm and queryIPCIDR are deferring
resp.Body.Close() per iteration which leaks file descriptors; instead, after
each call to agent.queryURL (and after checking resp for nil), do not defer:
immediately use json.Decode (or check resp.StatusCode == http.StatusNoContent)
and then call resp.Body.Close() before continuing/returning so the body is
closed each iteration; specifically update the ENDPOINT_LOOP handling where you
currently continue on resp != nil && resp.StatusCode == http.StatusNoContent to
call resp.Body.Close() before continue, and replace the deferred close after
successful json decoding with an explicit resp.Body.Close() right after decoding
in both querySearchTerm and queryIPCIDR (and any other loop that uses resp).
In `@sources/agent/hunterhow/hunterhow.go`:
- Around line 68-80: In query (method Agent.query) you never close resp.Body
which leaks connections; after the successful call to agent.queryURL (i.e.,
right after the if err != nil check where resp is non-nil) add a defer
resp.Body.Close() so the body is closed in all subsequent paths (including after
json.NewDecoder(resp.Body).Decode(&apiResponse) and any early returns that send
errors via sources.SendResult).
In `@sources/agent/netlas/netlas.go`:
- Around line 61-72: The HTTP response body from agent.queryURL is never closed,
leaking connections; in the query function (and after a successful call to
agent.queryURL where resp is non-nil and err == nil) add a defer
resp.Body.Close() immediately after receiving resp so the body is closed in all
code paths (including when json.NewDecoder(resp.Body).Decode(netlasResponse)
returns an error) before sending results/errors via sources.SendResult;
reference symbols: query, queryURL, resp.Body, Response, json.NewDecoder.
In `@sources/agent/publicwww/publicwww.go`:
- Around line 66-77: In Agent.query (the method that calls queryURL and reads
resp.Body) ensure the HTTP response body is closed to avoid connection leaks:
after a successful call to agent.queryURL (i.e., once err == nil and you have
resp), add a defer resp.Body.Close() so the body is always closed whether
io.ReadAll succeeds or fails; keep the existing error handling that sends
sources.Result via sources.SendResult when reading fails.
In `@sources/agent/quake/quake.go`:
- Around line 70-93: The function Agent.query leaks network connections because
resp.Body is never closed; after the successful call to agent.queryURL (the one
returning resp, err) add a defer resp.Body.Close() immediately so the response
body is closed on all return paths (including the early returns after io.ReadAll
error and JSON decode errors). Update the query function in quake.go (around the
resp, err := agent.queryURL(...) block) to ensure resp.Body.Close() is always
invoked before any return.
In `@sources/agent/shodan/shodan.go`:
- Around line 73-83: In the Agent.query function you never close resp.Body from
agent.queryURL which leaks connections across paginated requests; immediately
after getting resp and verifying err (inside query, before decoding or any early
returns) add a defer resp.Body.Close() so every HTTP response body is closed on
all code paths in query (including error/early returns) to prevent connection
leaks when calling agent.queryURL repeatedly for pagination.
In `@sources/agent/shodanidb/shodan.go`:
- Around line 89-100: The loop reuses a single sources.Result (result) and does
not clear result.Host before sending the port-only result, causing stale host
values to be emitted; inside the ports loop in shodan.go (where
shodanResponse.Ports and shodanResponse.Hostnames are iterated) clear
result.Host (e.g., set to empty string) immediately before calling
sources.SendResult for the port-only emission, then populate result.Host for the
hostname-specific sends so each SendResult uses the correct host state.
- Around line 73-86: The loop that calls agent.queryURL and decodes into
ShodanResponse never closes resp.Body, leaking connections; update the code
around the resp variable returned by agent.queryURL so resp.Body is always
closed after use (do NOT use defer inside the loop without scoping), e.g. call
resp.Body.Close() in both the success path after decoding (or use io.ReadAll
then close) and in any early error/continue paths; ensure this change touches
the block handling resp from agent.queryURL and the decoding into ShodanResponse
so resp.Body is reliably closed on all execution paths.
---
Outside diff comments:
In `@sources/agent/criminalip/criminalip.go`:
- Around line 83-109: In function query, you never close the HTTP response body
from agent.queryURL which leaks connections; immediately after receiving resp
(and after checking err) add a defer resp.Body.Close() so the body is always
closed before any early returns or after json decoding in CriminalIPResponse
handling; ensure the defer is placed before
json.NewDecoder(resp.Body).Decode(...) and before any returns inside the loop so
resp.Body is always released even on decode errors or when SendResult returns
false.
In `@sources/agent/google/google.go`:
- Around line 49-53: The pagination advances by 1 currently, causing overlapping
windows; update the loop that builds the Google requests to increment the page
start by the request Count instead of 1: when constructing the Request
(googleRequest with fields SearchTerms, Count, StartIndex) use the existing size
variable as Count and advance pageQuery by size (or set StartIndex = 1 + n*size)
on each iteration so StartIndex becomes 1, 1+size, 1+2*size, ... ensuring
non-overlapping pages.
- Around line 76-123: In the Agent.query function the HTTP response body
(resp.Body) is never closed which leaks connections during pagination; after
successfully getting resp from agent.queryURL(...) add a defer resp.Body.Close()
so the underlying connection is always released, and keep closing gzipReader
when used (gzipReader.Close()) after decoding; ensure the defer for
resp.Body.Close() is placed immediately after checking err from agent.queryURL
so both gzip and non-gzip branches return with resp.Body closed.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 21f7305f-fdda-45c4-9b42-dd83883d3dd2
📒 Files selected for processing (28)
README.mdrunner/options.gosources/agent.gosources/agent/binaryedge/binaryedge.gosources/agent/censys/censys.gosources/agent/criminalip/criminalip.gosources/agent/daydaymap/daydaymap.gosources/agent/daydaymap/request.gosources/agent/daydaymap/response.gosources/agent/driftnet/driftnet.gosources/agent/fofa/fofa.gosources/agent/google/google.gosources/agent/greynoise/greynoise.gosources/agent/hunter/hunter.gosources/agent/hunterhow/hunterhow.gosources/agent/netlas/netlas.gosources/agent/odin/odin.gosources/agent/onyphe/onyphe.gosources/agent/publicwww/publicwww.gosources/agent/quake/quake.gosources/agent/shodan/shodan.gosources/agent/shodanidb/shodan.gosources/agent/shodanidb/shodan_test.gosources/agent/zoomeye/zoomeye.gosources/keys.gosources/provider.gosources/util.gouncover.go
Comment on lines
+100
to
+110
| resp, err := agent.queryURL(ctx, session, URL, daydaymapRequest) | ||
| if err != nil { | ||
| sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err}) | ||
| return nil | ||
| } | ||
|
|
||
| daydaymapResponse := &DaydaymapResponse{} | ||
| if err := json.NewDecoder(resp.Body).Decode(daydaymapResponse); err != nil { | ||
| sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err}) | ||
| return nil | ||
| } |
Contributor
There was a problem hiding this comment.
Close response bodies in query to avoid connection leaks.
Line 100 obtains an HTTP response, but resp.Body is never closed. In pagination this can exhaust sockets/file descriptors and degrade reliability.
Suggested fix
func (agent *Agent) query(ctx context.Context, URL string, session *sources.Session, daydaymapRequest *DaydaymapRequest, results chan sources.Result) *DaydaymapResponse {
resp, err := agent.queryURL(ctx, session, URL, daydaymapRequest)
if err != nil {
sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err})
return nil
}
+ defer resp.Body.Close()
daydaymapResponse := &DaydaymapResponse{}
if err := json.NewDecoder(resp.Body).Decode(daydaymapResponse); err != nil {
sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err})
return nil
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| resp, err := agent.queryURL(ctx, session, URL, daydaymapRequest) | |
| if err != nil { | |
| sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err}) | |
| return nil | |
| } | |
| daydaymapResponse := &DaydaymapResponse{} | |
| if err := json.NewDecoder(resp.Body).Decode(daydaymapResponse); err != nil { | |
| sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err}) | |
| return nil | |
| } | |
| resp, err := agent.queryURL(ctx, session, URL, daydaymapRequest) | |
| if err != nil { | |
| sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err}) | |
| return nil | |
| } | |
| defer resp.Body.Close() | |
| daydaymapResponse := &DaydaymapResponse{} | |
| if err := json.NewDecoder(resp.Body).Decode(daydaymapResponse); err != nil { | |
| sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err}) | |
| return nil | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@sources/agent/daydaymap/daydaymap.go` around lines 100 - 110, The HTTP
response body from agent.queryURL (resp) is never closed, causing connection
leaks; after the resp, err := agent.queryURL(...) success path, ensure the
response body is closed by calling resp.Body.Close()—preferably via defer
immediately after confirming err == nil (right before decoding into
DaydaymapResponse) or alternatively ensure agent.queryURL transfers ownership
and callers close resp.Body; update the code around agent.queryURL, resp, and
the DaydaymapResponse decoding to guarantee resp.Body.Close() is always
executed.
Comment on lines
+78
to
90
| resp, queryError := agent.queryURL(ctx, session, apiEndpoint, driftnetRequest) | ||
|
|
||
| if queryError != nil { | ||
| // Driftnet will return 204 if no results are found for a query | ||
| if resp != nil && resp.StatusCode == http.StatusNoContent { | ||
| // Try the next endpoint | ||
| continue ENDPOINT_LOOP | ||
| } | ||
|
|
||
| // Some non 204 error | ||
| results <- sources.Result{Source: agent.Name(), Error: queryError} | ||
| sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: queryError}) | ||
| return | ||
| } | ||
| defer func() { | ||
| _ = resp.Body.Close() | ||
| }() |
Contributor
There was a problem hiding this comment.
❓ Verification inconclusive
Script executed:
cat -n sources/agent/driftnet/driftnet.go | head -200Repository: projectdiscovery/uncover
Repository: projectdiscovery/uncover
Exit code: 0
stdout:
1 package driftnet
2
3 import (
4 "context"
5 "encoding/json"
6 "errors"
7 "fmt"
8 "net/http"
9 "net/url"
10 "strconv"
11 "strings"
12 "time"
13
14 "github.com/projectdiscovery/mapcidr"
15 "github.com/projectdiscovery/uncover/sources"
16 iputil "github.com/projectdiscovery/utils/ip"
17 )
18
19 const (
20 OpenPortIPPortsURL = "https://api.driftnet.io/v1/scan/ipports?from=%s&ip=%s"
21 DomainsURL = "https://api.driftnet.io/v1/scan/domains?from=%s&most_recent=true&%s"
22 ProtocolURL = "https://api.driftnet.io/v1/scan/protocols?from=%s&most_recent=true&%s"
23 PageMaxLimit = 100
24 )
25
26 type DriftnetRequest struct {
27 Query string
28 ResultLimit int
29 From string
30 Page int
31 }
32
33 type Agent struct{}
34
35 func (agent *Agent) Name() string {
36 return "driftnet"
37 }
38
39 func (agent *Agent) Query(ctx context.Context, session *sources.Session, query *sources.Query) (chan sources.Result, error) {
40 if session.Keys.DriftnetToken == "" {
41 return nil, errors.New("empty driftnet keys")
42 }
43
44 results := make(chan sources.Result)
45
46 go func() {
47 defer close(results)
48
49 queryTime := time.Now().AddDate(0, 0, -30)
50
51 driftnetRequest := &DriftnetRequest{Query: query.Query, From: queryTime.Format(time.DateOnly), ResultLimit: query.Limit}
52
53 agent.query(ctx, session, driftnetRequest, results)
54 }()
55
56 return results, nil
57 }
58
59 func (agent *Agent) query(ctx context.Context, session *sources.Session, driftnetRequest *DriftnetRequest, results chan sources.Result) {
60 if iputil.IsIP(driftnetRequest.Query) || iputil.IsCIDR(driftnetRequest.Query) {
61 agent.queryIPCIDR(ctx, session, driftnetRequest, results)
62 } else {
63 agent.querySearchTerm(ctx, session, driftnetRequest, results)
64 }
65 }
66
67 func (agent *Agent) querySearchTerm(ctx context.Context, session *sources.Session, driftnetRequest *DriftnetRequest, results chan sources.Result) {
68 totalReportedResults := 0
69
70 ENDPOINT_LOOP:
71 for _, apiEndpoint := range []string{ProtocolURL, DomainsURL} {
72 for currentPage := 0; currentPage < PageMaxLimit; currentPage++ {
73 if ctx.Err() != nil {
74 return
75 }
76 driftnetRequest.Page = currentPage
77
78 resp, queryError := agent.queryURL(ctx, session, apiEndpoint, driftnetRequest)
79
80 if queryError != nil {
81 if resp != nil && resp.StatusCode == http.StatusNoContent {
82 continue ENDPOINT_LOOP
83 }
84
85 sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: queryError})
86 return
87 }
88 defer func() {
89 _ = resp.Body.Close()
90 }()
91
92 driftnetResponse := &DriftnetAPIPaginatedResponse{}
93 if err := json.NewDecoder(resp.Body).Decode(driftnetResponse); err != nil {
94 sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err})
95 return
96 }
97
98 for _, result := range driftnetResponse.Results {
99 var ip string
100 var port string
101 var host string
102 for _, item := range result.Items {
103 if item.Context == "" {
104 if strings.HasPrefix(item.Type, "port-") {
105 port = item.Value
106 }
107
108 if item.Type == "ip" {
109 ip = item.Value
110 }
111
112 if item.Type == "host" {
113 host = item.Value
114 }
115 }
116 }
117
118 if len(port) > 0 && len(ip) > 0 {
119 portAsInt, conversionError := strconv.Atoi(port)
120 if conversionError != nil {
121 if !sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: conversionError}) {
122 return
123 }
124 continue
125 }
126
127 out := sources.Result{Source: agent.Name(), IP: ip, Port: portAsInt}
128
129 if len(host) > 0 {
130 out.Host = host
131 }
132
133 out.Raw, _ = json.Marshal(out)
134
135 if !sources.SendResult(ctx, results, out) {
136 return
137 }
138 totalReportedResults += 1
139 }
140
141 if driftnetRequest.ResultLimit > 0 && totalReportedResults >= driftnetRequest.ResultLimit {
142 return
143 }
144 }
145
146 if len(driftnetResponse.Results) < 100 {
147 continue ENDPOINT_LOOP
148 }
149 }
150 }
151 }
152
153 func (agent *Agent) queryIPCIDR(ctx context.Context, session *sources.Session, driftnetRequest *DriftnetRequest, results chan sources.Result) {
154 var targetCIDR = driftnetRequest.Query
155
156 if iputil.IsIP(targetCIDR) {
157 if iputil.IsIPv4(targetCIDR) {
158 targetCIDR = iputil.AsIPV4CIDR(targetCIDR)
159 } else if iputil.IsIPv6(targetCIDR) {
160 targetCIDR = iputil.AsIPV6CIDR(targetCIDR)
161 }
162 }
163
164 requestCIDRs, splitError := mapcidr.SplitByNumber(targetCIDR, 1024)
165 if splitError != nil {
166 sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: splitError})
167 return
168 }
169
170 totalReportedResults := 0
171 for _, requestCIDR := range requestCIDRs {
172 if ctx.Err() != nil {
173 return
174 }
175 driftnetRequest.Query = requestCIDR.String()
176
177 resp, queryError := agent.queryURL(ctx, session, OpenPortIPPortsURL, driftnetRequest)
178
179 if queryError != nil {
180 if resp != nil && resp.StatusCode == http.StatusNoContent {
181 continue
182 }
183
184 sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: queryError})
185 return
186 }
187 defer func() {
188 _ = resp.Body.Close()
189 }()
190
191 driftnetResponse := &DriftnetAPIOpenIPPortResponse{}
192 if err := json.NewDecoder(resp.Body).Decode(driftnetResponse); err != nil {
193 sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err})
194 return
195 }
196
197 for ip, portsResponse := range driftnetResponse.Values {
198 result := sources.Result{Source: agent.Name(), IP: ip}
199 result.Raw, _ = json.Marshal(driftnetResponse)
200
Close each response body immediately after use instead of deferring inside loops.
Both loops defer resp.Body.Close() inside iteration blocks. In querySearchTerm (lines 88-90), the defer stacks within a loop that runs up to PageMaxLimit (100) iterations per endpoint. In queryIPCIDR (lines 187-189), the defer stacks within a loop over split CIDRs, potentially hundreds of times. This keeps file descriptors open until the function returns, exhausting descriptor limits and blocking connection reuse.
Close the body immediately after json.Decode() completes in each iteration, rather than deferring until function exit.
Also applies to: 177-189
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@sources/agent/driftnet/driftnet.go` around lines 78 - 90, The loops in
querySearchTerm and queryIPCIDR are deferring resp.Body.Close() per iteration
which leaks file descriptors; instead, after each call to agent.queryURL (and
after checking resp for nil), do not defer: immediately use json.Decode (or
check resp.StatusCode == http.StatusNoContent) and then call resp.Body.Close()
before continuing/returning so the body is closed each iteration; specifically
update the ENDPOINT_LOOP handling where you currently continue on resp != nil &&
resp.StatusCode == http.StatusNoContent to call resp.Body.Close() before
continue, and replace the deferred close after successful json decoding with an
explicit resp.Body.Close() right after decoding in both querySearchTerm and
queryIPCIDR (and any other loop that uses resp).
Comment on lines
+68
to
80
| func (agent *Agent) query(ctx context.Context, URL string, session *sources.Session, results chan sources.Result) []string { | ||
| resp, err := agent.queryURL(ctx, session, URL) | ||
| if err != nil { | ||
| results <- sources.Result{Source: agent.Name(), Error: err} | ||
| sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err}) | ||
| return nil | ||
| } | ||
|
|
||
| var apiResponse Response | ||
| err = json.NewDecoder(resp.Body).Decode(&apiResponse) | ||
| if err != nil { | ||
| results <- sources.Result{Source: agent.Name(), Error: err} | ||
| sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err}) | ||
| return nil | ||
| } |
Contributor
There was a problem hiding this comment.
Close the response body in query.
resp.Body is never closed here. Under pagination this leaks connections and can eventually stall further requests from this agent.
Suggested fix
func (agent *Agent) query(ctx context.Context, URL string, session *sources.Session, results chan sources.Result) []string {
resp, err := agent.queryURL(ctx, session, URL)
if err != nil {
sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err})
return nil
}
+ defer func() {
+ _ = resp.Body.Close()
+ }()
var apiResponse Response
err = json.NewDecoder(resp.Body).Decode(&apiResponse)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@sources/agent/hunterhow/hunterhow.go` around lines 68 - 80, In query (method
Agent.query) you never close resp.Body which leaks connections; after the
successful call to agent.queryURL (i.e., right after the if err != nil check
where resp is non-nil) add a defer resp.Body.Close() so the body is closed in
all subsequent paths (including after
json.NewDecoder(resp.Body).Decode(&apiResponse) and any early returns that send
errors via sources.SendResult).
Comment on lines
+61
to
72
| func (agent *Agent) query(ctx context.Context, URL string, session *sources.Session, results chan sources.Result) *Response { | ||
| resp, err := agent.queryURL(ctx, session, URL) | ||
| if err != nil { | ||
| results <- sources.Result{Source: agent.Name(), Error: err} | ||
| sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err}) | ||
| return nil | ||
| } | ||
|
|
||
| netlasResponse := &Response{} | ||
| if err := json.NewDecoder(resp.Body).Decode(netlasResponse); err != nil { | ||
| results <- sources.Result{Source: agent.Name(), Error: err} | ||
| sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err}) | ||
| return nil | ||
| } |
Contributor
There was a problem hiding this comment.
Close the HTTP response body after decoding.
query never closes resp.Body, so each page leaks a connection back to the transport pool.
Suggested fix
func (agent *Agent) query(ctx context.Context, URL string, session *sources.Session, results chan sources.Result) *Response {
resp, err := agent.queryURL(ctx, session, URL)
if err != nil {
sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err})
return nil
}
+ defer func() {
+ _ = resp.Body.Close()
+ }()
netlasResponse := &Response{}
if err := json.NewDecoder(resp.Body).Decode(netlasResponse); err != nil {📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| func (agent *Agent) query(ctx context.Context, URL string, session *sources.Session, results chan sources.Result) *Response { | |
| resp, err := agent.queryURL(ctx, session, URL) | |
| if err != nil { | |
| results <- sources.Result{Source: agent.Name(), Error: err} | |
| sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err}) | |
| return nil | |
| } | |
| netlasResponse := &Response{} | |
| if err := json.NewDecoder(resp.Body).Decode(netlasResponse); err != nil { | |
| results <- sources.Result{Source: agent.Name(), Error: err} | |
| sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err}) | |
| return nil | |
| } | |
| func (agent *Agent) query(ctx context.Context, URL string, session *sources.Session, results chan sources.Result) *Response { | |
| resp, err := agent.queryURL(ctx, session, URL) | |
| if err != nil { | |
| sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err}) | |
| return nil | |
| } | |
| defer func() { | |
| _ = resp.Body.Close() | |
| }() | |
| netlasResponse := &Response{} | |
| if err := json.NewDecoder(resp.Body).Decode(netlasResponse); err != nil { | |
| sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err}) | |
| return nil | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@sources/agent/netlas/netlas.go` around lines 61 - 72, The HTTP response body
from agent.queryURL is never closed, leaking connections; in the query function
(and after a successful call to agent.queryURL where resp is non-nil and err ==
nil) add a defer resp.Body.Close() immediately after receiving resp so the body
is closed in all code paths (including when
json.NewDecoder(resp.Body).Decode(netlasResponse) returns an error) before
sending results/errors via sources.SendResult; reference symbols: query,
queryURL, resp.Body, Response, json.NewDecoder.
Comment on lines
+66
to
77
| func (agent *Agent) query(ctx context.Context, URL string, session *sources.Session, results chan sources.Result) []string { | ||
| resp, err := agent.queryURL(ctx, session, URL) | ||
| if err != nil { | ||
| results <- sources.Result{Source: agent.Name(), Error: err} | ||
| sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err}) | ||
| return nil | ||
| } | ||
|
|
||
| body, err := io.ReadAll(resp.Body) | ||
| if err != nil { | ||
| results <- sources.Result{Source: agent.Name(), Error: err} | ||
| sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err}) | ||
| return nil | ||
| } |
Contributor
There was a problem hiding this comment.
Close resp.Body after queryURL succeeds.
This path reads the entire body but never closes it, so repeated queries will leak HTTP connections.
Suggested fix
func (agent *Agent) query(ctx context.Context, URL string, session *sources.Session, results chan sources.Result) []string {
resp, err := agent.queryURL(ctx, session, URL)
if err != nil {
sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err})
return nil
}
+ defer func() {
+ _ = resp.Body.Close()
+ }()
body, err := io.ReadAll(resp.Body)
if err != nil {🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@sources/agent/publicwww/publicwww.go` around lines 66 - 77, In Agent.query
(the method that calls queryURL and reads resp.Body) ensure the HTTP response
body is closed to avoid connection leaks: after a successful call to
agent.queryURL (i.e., once err == nil and you have resp), add a defer
resp.Body.Close() so the body is always closed whether io.ReadAll succeeds or
fails; keep the existing error handling that sends sources.Result via
sources.SendResult when reading fails.
Comment on lines
+70
to
93
| func (agent *Agent) query(ctx context.Context, URL string, session *sources.Session, quakeRequest *Request, results chan sources.Result) *Response { | ||
| resp, err := agent.queryURL(ctx, session, URL, quakeRequest) | ||
| if err != nil { | ||
| results <- sources.Result{Source: agent.Name(), Error: err} | ||
| sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err}) | ||
| return nil | ||
| } | ||
|
|
||
| quakeResponse := &Response{} | ||
| respdata, err := io.ReadAll(resp.Body) | ||
| if err != nil { | ||
| results <- sources.Result{Source: agent.Name(), Error: fmt.Errorf("%v: %v", err, string(respdata))} | ||
| sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: fmt.Errorf("%v: %v", err, string(respdata))}) | ||
| return nil | ||
| } | ||
| if err := json.NewDecoder(bytes.NewReader(respdata)).Decode(quakeResponse); err != nil { | ||
| errx := errorutil.NewWithErr(err) | ||
| // quake has different json format for error messages try to unmarshal it in map and print map | ||
| var errMap map[string]interface{} | ||
| if err := json.NewDecoder(bytes.NewReader(respdata)).Decode(&errMap); err == nil { | ||
| errx = errx.Msgf("failed to decode quake response: %v", errMap) | ||
| } else { | ||
| errx = errx.Msgf("failed to decode quake response: %s", string(respdata)) | ||
| } | ||
| results <- sources.Result{Source: agent.Name(), Error: errx} | ||
| sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: errx}) | ||
| return nil | ||
| } |
Contributor
There was a problem hiding this comment.
Release the response body before returning from query.
resp.Body is never closed on either the success or decode-error paths here. That leaks connections across pages.
Suggested fix
func (agent *Agent) query(ctx context.Context, URL string, session *sources.Session, quakeRequest *Request, results chan sources.Result) *Response {
resp, err := agent.queryURL(ctx, session, URL, quakeRequest)
if err != nil {
sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err})
return nil
}
+ defer func() {
+ _ = resp.Body.Close()
+ }()
quakeResponse := &Response{}
respdata, err := io.ReadAll(resp.Body)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| func (agent *Agent) query(ctx context.Context, URL string, session *sources.Session, quakeRequest *Request, results chan sources.Result) *Response { | |
| resp, err := agent.queryURL(ctx, session, URL, quakeRequest) | |
| if err != nil { | |
| results <- sources.Result{Source: agent.Name(), Error: err} | |
| sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err}) | |
| return nil | |
| } | |
| quakeResponse := &Response{} | |
| respdata, err := io.ReadAll(resp.Body) | |
| if err != nil { | |
| results <- sources.Result{Source: agent.Name(), Error: fmt.Errorf("%v: %v", err, string(respdata))} | |
| sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: fmt.Errorf("%v: %v", err, string(respdata))}) | |
| return nil | |
| } | |
| if err := json.NewDecoder(bytes.NewReader(respdata)).Decode(quakeResponse); err != nil { | |
| errx := errorutil.NewWithErr(err) | |
| // quake has different json format for error messages try to unmarshal it in map and print map | |
| var errMap map[string]interface{} | |
| if err := json.NewDecoder(bytes.NewReader(respdata)).Decode(&errMap); err == nil { | |
| errx = errx.Msgf("failed to decode quake response: %v", errMap) | |
| } else { | |
| errx = errx.Msgf("failed to decode quake response: %s", string(respdata)) | |
| } | |
| results <- sources.Result{Source: agent.Name(), Error: errx} | |
| sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: errx}) | |
| return nil | |
| } | |
| func (agent *Agent) query(ctx context.Context, URL string, session *sources.Session, quakeRequest *Request, results chan sources.Result) *Response { | |
| resp, err := agent.queryURL(ctx, session, URL, quakeRequest) | |
| if err != nil { | |
| sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err}) | |
| return nil | |
| } | |
| defer func() { | |
| _ = resp.Body.Close() | |
| }() | |
| quakeResponse := &Response{} | |
| respdata, err := io.ReadAll(resp.Body) | |
| if err != nil { | |
| sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: fmt.Errorf("%v: %v", err, string(respdata))}) | |
| return nil | |
| } | |
| if err := json.NewDecoder(bytes.NewReader(respdata)).Decode(quakeResponse); err != nil { | |
| errx := errorutil.NewWithErr(err) | |
| var errMap map[string]interface{} | |
| if err := json.NewDecoder(bytes.NewReader(respdata)).Decode(&errMap); err == nil { | |
| errx = errx.Msgf("failed to decode quake response: %v", errMap) | |
| } else { | |
| errx = errx.Msgf("failed to decode quake response: %s", string(respdata)) | |
| } | |
| sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: errx}) | |
| return nil | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@sources/agent/quake/quake.go` around lines 70 - 93, The function Agent.query
leaks network connections because resp.Body is never closed; after the
successful call to agent.queryURL (the one returning resp, err) add a defer
resp.Body.Close() immediately so the response body is closed on all return paths
(including the early returns after io.ReadAll error and JSON decode errors).
Update the query function in quake.go (around the resp, err :=
agent.queryURL(...) block) to ensure resp.Body.Close() is always invoked before
any return.
Comment on lines
+73
to
83
| func (agent *Agent) query(ctx context.Context, URL string, session *sources.Session, shodanRequest *ShodanRequest, results chan sources.Result) *ShodanResponse { | ||
| resp, err := agent.queryURL(ctx, session, URL, shodanRequest) | ||
| if err != nil { | ||
| results <- sources.Result{Source: agent.Name(), Error: err} | ||
| sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err}) | ||
| return nil | ||
| } | ||
|
|
||
| shodanResponse := &ShodanResponse{} | ||
| if err := json.NewDecoder(resp.Body).Decode(shodanResponse); err != nil { | ||
| results <- sources.Result{Source: agent.Name(), Error: err} | ||
| sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err}) | ||
| return nil |
Contributor
There was a problem hiding this comment.
Close resp.Body in the paginated query path.
Line 74 fetches a new response for every page, but this helper never closes resp.Body. That will leak connections across multi-page queries and can eventually stall later requests.
Suggested fix
func (agent *Agent) query(ctx context.Context, URL string, session *sources.Session, shodanRequest *ShodanRequest, results chan sources.Result) *ShodanResponse {
resp, err := agent.queryURL(ctx, session, URL, shodanRequest)
if err != nil {
sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err})
return nil
}
+ defer func() {
+ _ = resp.Body.Close()
+ }()
shodanResponse := &ShodanResponse{}
if err := json.NewDecoder(resp.Body).Decode(shodanResponse); err != nil {
sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err})
return nil📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| func (agent *Agent) query(ctx context.Context, URL string, session *sources.Session, shodanRequest *ShodanRequest, results chan sources.Result) *ShodanResponse { | |
| resp, err := agent.queryURL(ctx, session, URL, shodanRequest) | |
| if err != nil { | |
| results <- sources.Result{Source: agent.Name(), Error: err} | |
| sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err}) | |
| return nil | |
| } | |
| shodanResponse := &ShodanResponse{} | |
| if err := json.NewDecoder(resp.Body).Decode(shodanResponse); err != nil { | |
| results <- sources.Result{Source: agent.Name(), Error: err} | |
| sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err}) | |
| return nil | |
| func (agent *Agent) query(ctx context.Context, URL string, session *sources.Session, shodanRequest *ShodanRequest, results chan sources.Result) *ShodanResponse { | |
| resp, err := agent.queryURL(ctx, session, URL, shodanRequest) | |
| if err != nil { | |
| sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err}) | |
| return nil | |
| } | |
| defer func() { | |
| _ = resp.Body.Close() | |
| }() | |
| shodanResponse := &ShodanResponse{} | |
| if err := json.NewDecoder(resp.Body).Decode(shodanResponse); err != nil { | |
| sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err}) | |
| return nil |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@sources/agent/shodan/shodan.go` around lines 73 - 83, In the Agent.query
function you never close resp.Body from agent.queryURL which leaks connections
across paginated requests; immediately after getting resp and verifying err
(inside query, before decoding or any early returns) add a defer
resp.Body.Close() so every HTTP response body is closed on all code paths in
query (including error/early returns) to prevent connection leaks when calling
agent.queryURL repeatedly for pagination.
Comment on lines
+73
to
86
| resp, err := agent.queryURL(ctx, session, URL, &ShodanRequest{Query: ip}) | ||
| if err != nil { | ||
| results <- sources.Result{Source: agent.Name(), Error: err} | ||
| if !sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err}) { | ||
| return | ||
| } | ||
| continue | ||
| } | ||
|
|
||
| shodanResponse := &ShodanResponse{} | ||
| if err := json.NewDecoder(resp.Body).Decode(shodanResponse); err != nil { | ||
| results <- sources.Result{Source: agent.Name(), Error: err} | ||
| if !sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err}) { | ||
| return | ||
| } | ||
| continue |
Contributor
There was a problem hiding this comment.
Close each InternetDB response body.
This loop issues one request per IP, but the response body is never closed after decoding. On larger CIDRs that will leak connections/file descriptors and degrade the whole scan.
Suggested fix
resp, err := agent.queryURL(ctx, session, URL, &ShodanRequest{Query: ip})
if err != nil {
if !sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err}) {
return
}
continue
}
+ func() {
+ defer func() { _ = resp.Body.Close() }()
- shodanResponse := &ShodanResponse{}
- if err := json.NewDecoder(resp.Body).Decode(shodanResponse); err != nil {
- if !sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err}) {
- return
- }
- continue
- }
+ shodanResponse := &ShodanResponse{}
+ if err := json.NewDecoder(resp.Body).Decode(shodanResponse); err != nil {
+ if !sources.SendResult(ctx, results, sources.Result{Source: agent.Name(), Error: err}) {
+ return
+ }
+ return
+ }
- result := sources.Result{Source: agent.Name(), IP: shodanResponse.IP}
- result.Raw, _ = json.Marshal(shodanResponse)
- for _, port := range shodanResponse.Ports {
- result.Port = port
- if !sources.SendResult(ctx, results, result) {
- return
- }
- for _, hostname := range shodanResponse.Hostnames {
- result.Host = hostname
- if !sources.SendResult(ctx, results, result) {
- return
- }
- }
- }
+ result := sources.Result{Source: agent.Name(), IP: shodanResponse.IP}
+ result.Raw, _ = json.Marshal(shodanResponse)
+ for _, port := range shodanResponse.Ports {
+ result.Port = port
+ if !sources.SendResult(ctx, results, result) {
+ return
+ }
+ for _, hostname := range shodanResponse.Hostnames {
+ result.Host = hostname
+ if !sources.SendResult(ctx, results, result) {
+ return
+ }
+ }
+ }
+ }()🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@sources/agent/shodanidb/shodan.go` around lines 73 - 86, The loop that calls
agent.queryURL and decodes into ShodanResponse never closes resp.Body, leaking
connections; update the code around the resp variable returned by agent.queryURL
so resp.Body is always closed after use (do NOT use defer inside the loop
without scoping), e.g. call resp.Body.Close() in both the success path after
decoding (or use io.ReadAll then close) and in any early error/continue paths;
ensure this change touches the block handling resp from agent.queryURL and the
decoding into ShodanResponse so resp.Body is reliably closed on all execution
paths.
Comment on lines
89
to
+100
| result := sources.Result{Source: agent.Name(), IP: shodanResponse.IP} | ||
| result.Raw, _ = json.Marshal(shodanResponse) | ||
| for _, port := range shodanResponse.Ports { | ||
| result.Port = port | ||
| results <- result | ||
| if !sources.SendResult(ctx, results, result) { | ||
| return | ||
| } | ||
| for _, hostname := range shodanResponse.Hostnames { | ||
| result.Host = hostname | ||
| results <- result | ||
| if !sources.SendResult(ctx, results, result) { | ||
| return | ||
| } |
Contributor
There was a problem hiding this comment.
Reset Host before emitting the port-only result.
result is reused across iterations. After the first hostname loop, result.Host keeps the last hostname, so the next port emits a stale/duplicate host value before the hostname loop runs again.
Suggested fix
result := sources.Result{Source: agent.Name(), IP: shodanResponse.IP}
result.Raw, _ = json.Marshal(shodanResponse)
for _, port := range shodanResponse.Ports {
result.Port = port
+ result.Host = ""
if !sources.SendResult(ctx, results, result) {
return
}
for _, hostname := range shodanResponse.Hostnames {
- result.Host = hostname
- if !sources.SendResult(ctx, results, result) {
+ hostResult := result
+ hostResult.Host = hostname
+ if !sources.SendResult(ctx, results, hostResult) {
return
}
}
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| result := sources.Result{Source: agent.Name(), IP: shodanResponse.IP} | |
| result.Raw, _ = json.Marshal(shodanResponse) | |
| for _, port := range shodanResponse.Ports { | |
| result.Port = port | |
| results <- result | |
| if !sources.SendResult(ctx, results, result) { | |
| return | |
| } | |
| for _, hostname := range shodanResponse.Hostnames { | |
| result.Host = hostname | |
| results <- result | |
| if !sources.SendResult(ctx, results, result) { | |
| return | |
| } | |
| result := sources.Result{Source: agent.Name(), IP: shodanResponse.IP} | |
| result.Raw, _ = json.Marshal(shodanResponse) | |
| for _, port := range shodanResponse.Ports { | |
| result.Port = port | |
| result.Host = "" | |
| if !sources.SendResult(ctx, results, result) { | |
| return | |
| } | |
| for _, hostname := range shodanResponse.Hostnames { | |
| hostResult := result | |
| hostResult.Host = hostname | |
| if !sources.SendResult(ctx, results, hostResult) { | |
| return | |
| } | |
| } | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@sources/agent/shodanidb/shodan.go` around lines 89 - 100, The loop reuses a
single sources.Result (result) and does not clear result.Host before sending the
port-only result, causing stale host values to be emitted; inside the ports loop
in shodan.go (where shodanResponse.Ports and shodanResponse.Hostnames are
iterated) clear result.Host (e.g., set to empty string) immediately before
calling sources.SendResult for the port-only emission, then populate result.Host
for the hostname-specific sends so each SendResult uses the correct host state.
dogancanbakir
approved these changes
May 19, 2026
Contributor
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
sources/agent/nerdydata/nerdydata.go (1)
128-140:⚠️ Potential issue | 🟠 MajorThread context cancellation through the rate-limit call.
Line 136 calls
session.RateLimits.Take(agent.Name())without a context parameter. IfTakeblocks waiting for rate-limit quota, the goroutine remains stuck even afterctx.Done(), which undermines the context-aware execution this PR introduces. TheTakemethod currently accepts only a key string—context support needs to be added to make rate-limit waits cancellable.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@sources/agent/nerdydata/nerdydata.go` around lines 128 - 140, The call in Agent.queryURL to session.RateLimits.Take(agent.Name()) can block without honoring the request context; update the rate limiter API to accept a context (e.g. add TakeContext(ctx context.Context, key string) error or Change Take to Take(ctx context.Context, key string) error), implement cancellation-aware waiting in the rate limiter, and replace the call in queryURL with the context-aware variant (call it with the passed ctx and agent.Name()) so that the wait is cancelled when ctx.Done() is triggered.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@sources/agent/nerdydata/nerdydata.go`:
- Around line 128-140: The call in Agent.queryURL to
session.RateLimits.Take(agent.Name()) can block without honoring the request
context; update the rate limiter API to accept a context (e.g. add
TakeContext(ctx context.Context, key string) error or Change Take to Take(ctx
context.Context, key string) error), implement cancellation-aware waiting in the
rate limiter, and replace the call in queryURL with the context-aware variant
(call it with the passed ctx and agent.Name()) so that the wait is cancelled
when ctx.Done() is triggered.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: d582d64c-f4c9-45ed-8ded-5d418b8b6522
📒 Files selected for processing (6)
README.mdrunner/options.gosources/agent/nerdydata/nerdydata.gosources/keys.gosources/provider.gouncover.go
✅ Files skipped from review due to trivial changes (1)
- README.md
🚧 Files skipped from review as they are similar to previous changes (4)
- sources/keys.go
- sources/provider.go
- runner/options.go
- uncover.go
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary by CodeRabbit
New Features
Enhancements
Documentation
Tests