fix: match capture v1 rules for empty names, drop logs, retry identity and typed values

.changeset/android-v4-major.md

@@ -20,7 +20,7 @@
 - **Breaking:** rename `PostHogConfig.maxRetries` to `pushNotificationSubscriptionMaxRetries`, because it only limits push-notification subscription retries
 - **Breaking:** set `$lib` and `$lib_version` from the `PostHog-Sdk-Info` request header; a value set in properties, super properties or `beforeSend` is removed before sending
 - **Breaking:** send `$session_id` and `$window_id` as top-level `session_id` and `window_id` fields; a non-string value is dropped with a warning
-- **Breaking:** disable the SDK at setup when `projectToken` is a personal API key (`phx_...`), as for an empty token
+- **Breaking:** disable the SDK at setup when `projectToken` is a personal API key (`phx_...`), as for an empty token, and print a warning for both even when `debug` is off
 - **Breaking:** add an `options` argument to `capture`, `screen`, `identify`, `setPersonProperties`, `alias`, `group` and `captureException` for per-event capture options such as `cookieless_mode`, sent apart from the properties and editable in `beforeSend` through `PostHogEvent.options`; Java callers pass the trailing argument or `null`, and classes that implement `PostHogInterface` add the parameter
 - **Breaking:** move `identify(distinctId, userProperties, userPropertiesSetOnce)` from `PostHogCoreInterface` to `PostHogStatelessInterface`; `PostHogInterface` declares `identify` with the trailing `options` parameter
 - **Breaking:** registered super properties now override the properties the SDK sets itself, such as `$os_name`; super and event properties also override `$is_identified`, `$process_person_profile` and the session replay debug properties, which the SDK no longer re-applies after the caller's values
@@ -30,7 +30,9 @@
 - use the current distinct ID when the `distinctId` argument to `capture` is empty or whitespace-only, instead of dropping the event
 - **Breaking:** skip deep link query parameters whose name starts with `$` in the `Deep Link Opened` event, so a link can't set a PostHog property such as `$process_person_profile`
 - stop storing or matching the person-properties deduplication hash for a `$set` sent with `process_person_profile: false`
-- debug logs for capture requests name the event count and the response status instead of printing the request and response bodies, and name each event PostHog drops with the reason it gives
+- debug logs for capture requests name the event count and the response status instead of printing the request and response bodies, and count the events PostHog drops, the duplicate copies it drops and the events it asks to retry, in one line per batch, and log the status when PostHog rejects a whole batch
+- **Breaking:** drop an event whose name is empty before sending it, including one `beforeSend` renames to `""`
+- **Breaking:** let `userProperties`, `userPropertiesSetOnce` and `groups` win key by key over `$set`, `$set_once` and `$groups` maps passed in the same call's properties, instead of being replaced by them, and merge a `$groups` map in properties into the groups set with `group()`
 - add `captureAiCompression`, `captureAiMaxQueueSize`, `captureAiTimeout` and `captureAiMaxEventBytes` to `PostHogConfig` to tune AI requests separately from other events; the defaults are gzip, 100 queued events (and at most 50 MiB), 60 seconds and 8 MiB + 64 KiB per serialized event
 - cap a request at about 5 MiB of events, for analytics and AI events, and send as soon as that much is queued; an event larger than that is sent on its own
 - Add `onFeatureFlags()` to get notified after every feature flags load (on the main thread on Android), returning a `PostHogFeatureFlagsSubscription` you can `unsubscribe()`

docs/migration-3.x-to-4.0.md

@@ -54,6 +54,7 @@ If you send events through your own reverse proxy, update it before you upgrade:
 - Forward `/i/v1/analytics/events` and `/i/v1/ai/events` to PostHog.
 - Pass `Authorization` (the project token, sent as `Bearer <token>`), every `PostHog-*` header, `Content-Type`, `Content-Encoding` and `User-Agent` through unchanged. PostHog answers `401` without `Authorization` and `400` without the others, and the events are dropped.
 - Accept request bodies up to 20 MB. AI requests are often larger than 1 MB, the default limit in nginx and ingress-nginx. A `413` from the proxy makes the SDK halve the batch until it fits, and drops an event that doesn't fit on its own.
+- Pass PostHog's response body through unchanged. A 2xx response whose body isn't PostHog's `{"results": {...}}` drops the events instead of retrying them.
 
 PostHog's managed reverse proxy already does this. See the [proxy guides](https://posthog.com/docs/advanced/proxy).
 
@@ -69,16 +70,21 @@ If a proxy redirects capture requests to another host, point `host` at the final
 ## Event validation
 
 Capture v1 checks each event on its own and drops only the invalid ones, so the rest of the batch is delivered.
-The SDK doesn't check these before sending. PostHog drops the event, and the SDK removes it from the queue.
-With `debug` on, the SDK logs the name and uuid of each dropped event and the reason PostHog gives, such as `misrouted_event`, never its properties.
 
-- An event with an empty event name is dropped, for example `capture("")`.
+The SDK drops an event whose name is empty before sending it, for example `capture("")` or one `beforeSend` renames to `""`.
+With `debug` on, it logs this once.
+
+The SDK doesn't check the rest before sending. PostHog drops the event, and the SDK removes it from the queue.
+With `debug` on, the SDK logs one line per batch with how many events PostHog dropped, how many duplicate copies it dropped and how many it asked to retry, never the event names or properties.
+
 - An event with an empty or whitespace-only distinct ID is dropped, for example one `beforeSend` sets to `""`.
 - An event whose name is over 200 UTF-8 bytes is dropped. Legacy capture had no limit on event name length.
 - An event whose distinct ID is over 200 UTF-8 bytes is dropped. Legacy capture shortened the ID to 200 characters and kept the event.
 
 In 3.x, an empty event name or distinct ID made legacy capture reject the whole batch it was in.
 
+A `NaN` or infinite number in the properties is dropped from the event, and the event is sent without it.
+
 The distinct ID can come from `identify()`, the `distinctId` argument of `capture` or `beforeSend`.
 If you identify users with a long ID, such as a JWT or a long email address, shorten or hash it before you upgrade. Otherwise every event for that user is dropped.
 
@@ -95,7 +101,7 @@ An event's properties are merged in this order, and later values win:
 
 1. properties the SDK sets itself, such as `$os_name`, `$app_version`, `$screen_name`, `$is_identified`, `$session_id` and the session replay debug properties (`$recording_status`, `$sdk_debug_*`)
 2. registered super properties (`register`)
-3. the properties you pass to `capture` or another event-sending method; `userProperties`, `userPropertiesSetOnce` and `groups` are placed as `$set`, `$set_once` and `$groups` here
+3. the properties you pass to `capture` or another event-sending method; `userProperties`, `userPropertiesSetOnce` and `groups` are placed as `$set`, `$set_once` and `$groups` here, and when you also pass that key in properties they win key by key; a `$groups` map in properties adds to the groups set with `group()` instead of replacing them
 4. `beforeSend`
 
 In 3.x, the SDK's own values won over super properties, and `$is_identified`, `$process_person_profile` and the session replay debug properties also won over the properties you pass.

posthog/src/main/java/com/posthog/PostHog.kt

@@ -319,14 +319,14 @@ public class PostHog private constructor(
                     if (config.logger is PostHogNoOpLogger) PostHogPrintLogger(config) else config.logger
 
                 if (config.projectToken.isEmpty()) {
-                    config.logger.log(
+                    config.logger.logWarning(
                         "PostHog SDK is disabled because the project token is required and was empty after trimming whitespace.",
                     )
                     return
                 }
 
                 if (config.projectToken.startsWith("phx_")) {
-                    config.logger.log(
+                    config.logger.logWarning(
                         "PostHog SDK is disabled because the project token is a personal API key (phx_...), " +
                             "use your project token (phc_...).",
                     )
@@ -975,9 +975,20 @@ public class PostHog private constructor(
         }
 
         // Caller-supplied properties win over registered super properties and SDK-set values.
+        val currentGroups = props["\$groups"] as? Map<*, *>
         properties?.let {
             props.putAll(it)
         }
+        if (appendSharedProps) {
+            putTypedValuesOverCaller(
+                props,
+                properties,
+                userProperties,
+                userPropertiesSetOnce,
+                groups.takeIf { appendGroups },
+                currentGroups,
+            )
+        }
 
         // only Session replay needs distinct_id also in the props
         // remove after https://github.com/PostHog/posthog/pull/18954 gets merged
@@ -1416,6 +1427,9 @@ public class PostHog private constructor(
         aiLane: Boolean,
         onQueued: () -> Unit = {},
     ): UUID? {
+        if (isEmptyNameDropped(postHogEvent)) {
+            return null
+        }
         // Reevaluate if this is a snapshot event because the event might have been updated by the beforeSend hook
         val isSnapshotEvent = !aiLane && postHogEvent.event == PostHogEventName.SNAPSHOT.event
         // if this is a $snapshot event and $session_id is missing, don't process then event

posthog/src/main/java/com/posthog/PostHogStateless.kt

@@ -19,6 +19,7 @@ import java.util.Date
 import java.util.UUID
 import java.util.concurrent.ExecutorService
 import java.util.concurrent.Executors
+import java.util.concurrent.atomic.AtomicBoolean
 
 public open class PostHogStateless protected constructor(
     private val queueExecutor: ExecutorService =
@@ -35,6 +36,7 @@ public open class PostHogStateless protected constructor(
 
     protected val setupLock: Any = Any()
     protected val optOutLock: Any = Any()
+    private val warnedEmptyEventName = AtomicBoolean(false)
     private var featureFlagsCalled: PostHogFeatureFlagCalledCache? = null
 
     // Read on the capture path from any caller thread without setupLock.
@@ -60,14 +62,14 @@ public open class PostHogStateless protected constructor(
                     if (config.logger is PostHogNoOpLogger) PostHogPrintLogger(config) else config.logger
 
                 if (config.projectToken.isEmpty()) {
-                    config.logger.log(
+                    config.logger.logWarning(
                         "PostHog SDK is disabled because the project token is required and was empty after trimming whitespace.",
                     )
                     return
                 }
 
                 if (config.projectToken.startsWith("phx_")) {
-                    config.logger.log(
+                    config.logger.logWarning(
                         "PostHog SDK is disabled because the project token is a personal API key (phx_...), " +
                             "use your project token (phc_...).",
                     )
@@ -216,13 +218,39 @@ public open class PostHogStateless protected constructor(
             props.putAll(it)
         }
 
+        val currentGroups = props["\$groups"] as? Map<*, *>
         properties?.let {
             props.putAll(it)
         }
+        putTypedValuesOverCaller(props, properties, userProperties, userPropertiesSetOnce, groups.takeIf { appendGroups }, currentGroups)
 
         return props
     }
 
+    /**
+     * A typed per-call value wins key by key over the same map in the caller's [properties]:
+     * `userProperties` over `$set`, `userPropertiesSetOnce` over `$set_once` and `groups` over `$groups`.
+     * A caller `$groups` adds to [currentGroups] (the stored and per-call groups) instead of replacing them.
+     */
+    internal fun putTypedValuesOverCaller(
+        props: MutableMap<String, Any>,
+        properties: Map<String, Any>?,
+        userProperties: Map<String, Any>?,
+        userPropertiesSetOnce: Map<String, Any>?,
+        groups: Map<String, String>?,
+        currentGroups: Map<*, *>? = null,
+    ) {
+        for ((key, typed) in listOf("\$set" to userProperties, "\$set_once" to userPropertiesSetOnce, "\$groups" to groups)) {
+            val caller = properties?.get(key) as? Map<*, *> ?: continue
+            val merged = LinkedHashMap<Any?, Any?>()
+            if (key == "\$groups") currentGroups?.let { merged.putAll(it) }
+            if (typed == null && merged.isEmpty()) continue
+            merged.putAll(caller)
+            typed?.let { merged.putAll(it) }
+            props[key] = merged
+        }
+    }
+
     protected fun mergeGroups(givenGroups: Map<String, String>?): Map<String, String>? {
         val preferences = getPreferences()
 
@@ -294,6 +322,9 @@ public open class PostHogStateless protected constructor(
                 config?.logger?.log(message)
                 return
             }
+            if (isEmptyNameDropped(postHogEvent)) {
+                return
+            }
 
             queue?.add(postHogEvent)
         } catch (e: Throwable) {
@@ -366,6 +397,20 @@ public open class PostHogStateless protected constructor(
         return eventChecked
     }
 
+    /**
+     * Capture drops an event with an empty name, so the SDK drops it before queueing, after
+     * beforeSend too. Logs only the first drop, with no event name.
+     */
+    internal fun isEmptyNameDropped(event: PostHogEvent): Boolean {
+        if (event.event.isNotEmpty()) {
+            return false
+        }
+        if (warnedEmptyEventName.compareAndSet(false, true)) {
+            config?.logger?.log("Dropping an event with an empty name.")
+        }
+        return true
+    }
+
     public override fun optIn() {
         if (!isEnabled()) {
             return