Repository navigation
feat: add data collection endpoint and rule modules AB#9312 - #463
Merged
Merged
Conversation
Both modules are needed to send custom logs through the Logs Ingestion API, which replaces the HTTP Data Collector API that Azure Monitor retires on 14 September 2026. The rule module takes a list of custom streams. Each stream gets a stream declaration describing the incoming payload and a data flow that transforms it into an existing custom table, so a caller can keep the column names of a table that was previously fed by the Data Collector API.
nbroers-hanab
approved these changes
Aug 19, 2026
nbroers-hanab
left a comment
There was a problem hiding this comment.
Looks good. Both modules follow the repo layout (main/variables/outputs, required_version ~> 1.12, azurerm ~> 3.117, backend + provider block so validate.sh can override them) and the resource labels match the naming used elsewhere in modules/azure. Lint and Validate are green, and I checked the deployed pair in the Telecom dev environment: dce-hti-int-platform-dev is Succeeded with both endpoints populated, and dcr-hti-int-platform-dev has 20 stream declarations plus 20 matching data flows into the logAnalytics destination, with no kind set, which is right for direct ingestion.
A few non-blocking notes for a follow-up:
- The constraints in the
streamsdescription (name must start withCustom-, output stream must point at an existing table) and the allowed column types are not enforced byvalidationblocks, so a typo surfaces as an Azure error at apply instead of at plan. An emptystreamslist runs into the provider's minimum of onedata_flowfor the same reason. data_flowis a list in the provider, and iterating a map gives lexicographic order, so adding a stream that sorts early reshuffles the indices in the plan output. Cosmetic only, the result is the same.transform_kqlandoutput_streamcould beoptional(string)(withtransform_kqldefaulting to"source") now that the modules require Terraform 1.12, which saves callers from passing explicit nulls.- Consumers still need a Monitoring Metrics Publisher assignment on the rule before they can send anything. Worth calling out in the consuming repo, since it is outside the scope of these modules.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds two modules under
modules/azure/:data_collection_endpointanddata_collection_rule.Azure Monitor retires the HTTP Data Collector API on 14 September 2026. Its replacement, the Logs Ingestion API, needs a data collection endpoint and a data collection rule, and neither exists in this library yet.
The rule module takes a list of custom streams. Each stream gets a stream declaration describing the incoming payload plus a data flow with a transform into an existing custom table. That way a caller can move a table off the Data Collector API while keeping its column names, so the workbooks and alert rules that query those columns keep working.
Both modules pass
terraform fmt -checkandvalidate/validate.shagainst azurerm 3.117.1, and they are already running on the VWT Telecom dev environment with 20 streams.