Skip to content

Add NetAPI threat intelligence provider - #956

Open
Slava Olesik (slavaolesik) wants to merge 3 commits into
microsoft:mainfrom
slavaolesik:netapi-ti-provider
Open

Slava Olesik (slavaolesik) wants to merge 3 commits into
microsoft:mainfrom
slavaolesik:netapi-ti-provider

Conversation

@slavaolesik

@slavaolesik Slava Olesik (slavaolesik) commented Oct 26, 2026 •

Copy link
Copy Markdown

What this adds

A NetAPI threat intelligence provider. NetAPI publishes a free feed of compromised domains and IP addresses, the hosts that public threat feeds (URLhaus, CERT Polska, blackbook and others) report for malware or phishing, and the NetAPI Top 1M, a daily popularity ranking. A lookup answers "is this host in the feed now, was it in the past, and since when".

Supported observables: dns, hostname, url, ipv4, ipv6. No file hashes.

The API needs no key (30 requests a minute and 1,000 a day per IP), so the config entry has no Args block, as for ScanMalware:

TIProviders:
  NetAPI:
    Primary: True
    Provider: "NetAPI"

Behaviour

  • The default query calls compromised-check. A current listing (seen within the last 24 hours) is a hit with severity high. A past listing only is a hit with severity information: most of those are incidents that are over, often a legitimate site that was compromised and cleaned up. A value that was never listed is a negative result. Details carries the listing state, the matched entry, first and last seen, the reporting feed and the registrar.
  • A hostname or URL is matched through its registered domain, and Matched Value shows which entry matched, so a listing of example.com is not presented as a listing of www.example.com itself.
  • The rank subtype (dns-rank, hostname-rank) calls domain-rank and returns the position of the registered domain in the NetAPI Top 1M, the same kind of context as the OpenPageRank provider.
  • Both queries are declarative _QUERIES entries, one request each, no override of the lookup flow. No auth header is sent. The provider sets its own User-Agent, netapi-msticpy/1.0.

Verification

  • pytest tests/context/test_tiproviders.py: 41 passed on unmodified main, 42 passed with this change.
  • pytest tests/context/test_netapi.py: 11 passed: current, past and never-listed results, the registered-domain match, URLs, both rank results, a 429, an empty answer, no auth header and unsupported hashes. Coverage of the new module is 100% (measured with coverage run and COVERAGE_CORE=sysmon, since pytest-cov hits the same numpy: cannot load module more than once per process error mentioned in Add ScanMalware threat intelligence provider #938).
  • ruff check and ruff format --check clean, pylint 10.00/10, pydocstyle --convention=numpy and mypy clean on the new module.
  • Verified end to end through TILookup against the live API:
www.000359.xyz   hostname        True   high          matched 000359.xyz, blackbook, registrar Alibaba Cloud
1.1.1.1          ipv4            True   information   listed until 2026-06-17, urlhaus.abuse.ch
github.com       dns             False  information   Not listed, now or in the past.
github.com       dns    rank     True   information   Rank 11, snapshot 2026-10-26

Disclosure

I run NetAPI. Happy to adjust anything here, including dropping the provider from the default config.

@slavaolesik

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree company="CodeRockers LTD"

@slavaolesik

Copy link
Copy Markdown
Author

NetAPI is discontinuing its compromised IP feed, so I have narrowed this PR to domains, hostnames and URLs. The ipv4 and ipv6 queries are gone, and the provider is now tested in its own module instead of the shared harness, because that harness looks up IP addresses in bulk. I have updated the description to match.

@slavaolesik

Copy link
Copy Markdown
Author

Update: NetAPI keeps its compromised IP feed after all, so I put the IP part of this PR back as it was first submitted. Sorry for the back and forth.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant