Merge Develop into main

[pallakartheekreddy]

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

Type of change

Please choose appropriate options.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes in the below checkboxes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

• Ran Test A
• Ran Test B

Test Configuration:

• Software versions:
• Hardware versions:

Checklist:

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• Any dependent changes have been merged and published in downstream modules

Summary by CodeRabbit

• New Features

  • Introduced JanusGraph CDC extension for capturing and processing transaction log events from the database
  • Added multiple message converter formats to support different output requirements
  • Enabled event logging to files with configurable rotation and retention

• Documentation

  • Added comprehensive setup and deployment guide with end-to-end configuration instructions, troubleshooting, and performance tuning recommendations

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: b5926ef7-0580-4823-98c3-2b7363fd6e04

📥 Commits

Reviewing files that changed from the base of the PR and between fc550bc and 67a923f.

📒 Files selected for processing (17)

• .gitignore
• janusgraph-cdc-extension/README.md
• janusgraph-cdc-extension/pom.xml
• janusgraph-cdc-extension/scripts/empty-sample.groovy
• janusgraph-cdc-extension/scripts/register-cdc.groovy
• janusgraph-cdc-extension/src/main/java/org/sunbird/janusgraph/cdc/EventSink.java
• janusgraph-cdc-extension/src/main/java/org/sunbird/janusgraph/cdc/GraphLogProcessor.java
• janusgraph-cdc-extension/src/main/java/org/sunbird/janusgraph/cdc/LogFileEventSink.java
• janusgraph-cdc-extension/src/main/java/org/sunbird/janusgraph/cdc/MessageConverter.java
• janusgraph-cdc-extension/src/main/java/org/sunbird/janusgraph/cdc/SimpleMessageConverter.java
• janusgraph-cdc-extension/src/main/java/org/sunbird/janusgraph/cdc/SunbirdLegacyMessageConverter.java
• janusgraph-cdc-extension/src/main/java/org/sunbird/janusgraph/cdc/TelemetryMessageConverter.java
• janusgraph-cdc-extension/src/main/resources/cdc-converter.conf
• janusgraph-cdc-extension/src/main/resources/cdc-log4j.properties
• janusgraph-cdc-extension/src/main/resources/log4j2-server.xml
• janusgraph-cdc-extension/src/test/java/org/sunbird/janusgraph/cdc/GraphLogProcessorTest.java
• pom.xml

---

📝 Walkthrough

Walkthrough

A new janusgraph-cdc-extension Maven module is introduced that attaches to the JanusGraph transaction log (learning_graph_events) and publishes vertex change events (CREATE/UPDATE/DELETE) to configurable sinks. Three MessageConverter implementations are provided (Simple, Telemetry, SunbirdLegacy), with Groovy bootstrap scripts for Gremlin Server, Log4j2 configuration, unit tests, and a full setup README.

Changes

JanusGraph CDC Extension

Layer / File(s)	Summary
Maven build scaffolding pom.xml, janusgraph-cdc-extension/pom.xml, .gitignore	Root pom.xml gains the new module entry; janusgraph-cdc-extension/pom.xml defines the Java 11 JAR artifact with provided JanusGraph/Jackson/SLF4J and test-scoped JUnit/Mockito dependencies; .gitignore excludes the extension's target/ directory.
Core CDC contracts src/main/java/org/sunbird/janusgraph/cdc/EventSink.java, MessageConverter.java, LogFileEventSink.java	EventSink interface defines send(key, message) and close(); MessageConverter interface defines convert(vertex, changeState, opType, txId); LogFileEventSink implements EventSink by forwarding events to SLF4J info.
GraphLogProcessor singleton src/main/java/org/sunbird/janusgraph/cdc/GraphLogProcessor.java	Singleton with start(graph, config) / shutdown() that reads config to select a MessageConverter and sink list, registers a JanusGraph ChangeProcessor on learning_graph_events, classifies vertices as CREATE/UPDATE/DELETE, delegates conversion, serializes to JSON, and dispatches to all sinks; includes timestamp parsing helpers (parseTimestamp, getLastUpdatedOn, parseLastUpdatedOn) used by shouldProcessEvent.
MessageConverter implementations src/main/java/org/sunbird/janusgraph/cdc/SimpleMessageConverter.java, TelemetryMessageConverter.java, SunbirdLegacyMessageConverter.java, src/main/resources/cdc-converter.conf	SimpleMessageConverter builds a basic event map with full property snapshot on CREATE; TelemetryMessageConverter wraps events in a Sunbird telemetry envelope with per-operation property diffs; SunbirdLegacyMessageConverter emits the full legacy format with {ov,nv} diffs, relation change tracking, filtering for internal/root vertices, and JSON string-field control via cdc-converter.conf.
Groovy Gremlin Server bootstrap scripts janusgraph-cdc-extension/scripts/register-cdc.groovy, scripts/empty-sample.groovy	register-cdc.groovy dynamically loads GraphLogProcessor via reflection and invokes start(JanusGraph, Map) with hardcoded defaults; empty-sample.groovy uses a LifeCycleHook in globals to bootstrap CDC and bind a traversal source g.
Logging config, tests, and documentation src/main/resources/log4j2-server.xml, cdc-log4j.properties, src/test/java/org/sunbird/janusgraph/cdc/GraphLogProcessorTest.java, janusgraph-cdc-extension/README.md	log4j2-server.xml and cdc-log4j.properties route LogFileEventSink output to a rolling CDC events file; GraphLogProcessorTest uses reflection to exercise private shouldProcessEvent for event ordering, ISO timestamp parsing, and nested/flat event structures; README.md provides an 8-step end-to-end setup guide with troubleshooting.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

• Sunbird-Knowlg/knowledge-platform-db-extensions#12: Introduces the same core classes (GraphLogProcessor, EventSink, MessageConverter implementations, and Groovy startup scripts), making it a direct predecessor or parallel implementation of this module.

Poem

🐰 A rabbit hops through JanusGraph's log,
Catching each vertex that slips through the fog!
CREATE, UPDATE, DELETE — all neatly observed,
CDC events dispatched as deserved.
Sinks receive the JSON with a hop and a cheer,
The graph's every change is now perfectly clear! 🎉

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch develop

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}