Skip to content

Comments

chore(web): Add PostHog LLM analytics#882

Open
brendan-kellam wants to merge 1 commit intomainfrom
bkellam/posthog-ai
Open

chore(web): Add PostHog LLM analytics#882
brendan-kellam wants to merge 1 commit intomainfrom
bkellam/posthog-ai

Conversation

@brendan-kellam
Copy link
Contributor

@brendan-kellam brendan-kellam commented Feb 12, 2026

Summary by CodeRabbit

Chores

  • Updated chat feature infrastructure with enhanced tracing capabilities.
  • Refactored analytics utility exports for improved modularity and reusability.
  • Added new analytics dependency to support system improvements.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Feb 12, 2026

Walkthrough

A new PostHog AI dependency is added to the web package. PostHog utility functions are exported for distinct ID retrieval and client initialization. The chat model selection flow is refactored to initialize a PostHog client, retrieve a distinct ID, and apply tracing to the selected model with privacy settings.

Changes

Cohort / File(s) Summary
Dependency Addition
packages/web/package.json
Added @posthog/ai ^7.8.10 as a runtime dependency.
PostHog Utilities
packages/web/src/lib/posthog.ts
Exported tryGetPostHogDistinctId (renamed from internal tryGetDistinctId) and new createPostHogClient helper for PostHog client initialization.
Chat Model Selection
packages/web/src/features/chat/actions.ts
Refactored AISDK model selection flow with post-processing: initializes PostHog client, retrieves distinct ID, wraps model with tracing, and applies privacy settings based on telemetry configuration.

Sequence Diagram(s)

sequenceDiagram
    participant Client as Chat Client
    participant Action as Model Selection Action
    participant PostHog as PostHog Service
    participant AISDK as AISDK Model

    Client->>Action: Request model selection
    Action->>Action: Compute model & provider options
    Action->>PostHog: Initialize PostHog client
    Action->>PostHog: Get distinct ID (cookie/session/API key)
    Action->>AISDK: Wrap model with tracing
    AISDK-->>Action: Traced model instance
    Action->>Action: Apply privacy settings
    Action-->>Client: Return model + providerOptions
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~40 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The pull request title 'chore(web): Add PostHog LLM analytics' clearly and accurately summarizes the main change—integrating PostHog analytics for LLM tracking in the web package.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
  • 📝 Generate docstrings (stacked PR)
  • 📝 Generate docstrings (commit on current branch)
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch bkellam/posthog-ai

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@brendan-kellam brendan-kellam marked this pull request as ready for review February 23, 2026 21:25
@github-actions
Copy link
Contributor

@brendan-kellam your pull request is missing a changelog!

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
packages/web/src/lib/posthog.ts (1)

90-110: ⚠️ Potential issue | 🟡 Minor

distinctId may be undefined when passed to posthog.capture().

tryGetPostHogDistinctId() can return undefined, but posthog.capture() expects distinctId to be a string. When no distinct ID is resolvable (no cookie, no session, no API key), this will pass undefined to the PostHog client, which could cause silent data loss or a runtime error.

Consider guarding against this:

Suggested fix
     const distinctId = await tryGetPostHogDistinctId();
     const posthog = await createPostHogClient();
 
+    if (!distinctId) {
+        return;
+    }
+
     const headersList = await headers();
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/web/src/lib/posthog.ts` around lines 90 - 110, The call to
posthog.capture in captureEvent currently passes distinctId which can be
undefined (from tryGetPostHogDistinctId); ensure captureEvent always provides a
string by adding a fallback before calling posthog.capture (e.g., const
distinctIdSafe = distinctId ?? crypto.randomUUID() or a deterministic anon id
using env.SOURCEBOT_INSTALL_ID) and pass distinctIdSafe to posthog.capture
instead of distinctId; update references in captureEvent and keep
tryGetPostHogDistinctId usage but guard its result so posthog.capture never
receives undefined.
🧹 Nitpick comments (2)
packages/web/src/features/chat/actions.ts (1)

34-36: Consolidate duplicate imports from @/lib/posthog.

Lines 34 and 36 both import from @/lib/posthog. Merge them into a single import statement.

Suggested fix
-import { captureEvent } from "@/lib/posthog";
-import { withTracing } from "@posthog/ai";
-import { createPostHogClient, tryGetPostHogDistinctId } from "@/lib/posthog";
+import { captureEvent, createPostHogClient, tryGetPostHogDistinctId } from "@/lib/posthog";
+import { withTracing } from "@posthog/ai";
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/web/src/features/chat/actions.ts` around lines 34 - 36, There are
duplicate import lines from "@/lib/posthog"; consolidate them by replacing the
two separate imports with a single import that includes captureEvent,
createPostHogClient, and tryGetPostHogDistinctId together (so update the import
statements that reference captureEvent and the import that references
createPostHogClient/tryGetPostHogDistinctId into one unified import).
packages/web/package.json (1)

66-66: Consider updating @posthog/ai to the latest available version.

Version 7.8.10 is valid and exists on npm, but it is not current. The latest version is 7.9.1, with several newer patch releases also available (7.8.11, 7.8.12, 7.8.13). Since the dependency specifies ^7.8.10, it would automatically accept these compatible updates. Consider upgrading to 7.9.1 to benefit from the latest features and fixes.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/web/package.json` at line 66, Update the `@posthog/ai` dependency in
package.json by replacing the current version specifier "^7.8.10" with the newer
release (e.g. "7.9.1") so the project uses the latest patch/feature fixes;
locate the dependency entry for "@posthog/ai" and change the version string
accordingly, then run your package manager (npm/yarn/pnpm) to install and verify
no breakages.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@packages/web/src/lib/posthog.ts`:
- Around line 80-88: The code currently creates a new PostHog client on every
call of createPostHogClient (and callers like captureEvent) causing resource
leaks; change createPostHogClient to lazily initialize and return a module-level
singleton PostHog instance (e.g., keep a private let posthogInstance and
instantiate it once inside createPostHogClient if undefined), update callers
(captureEvent and any usages in actions.ts) to reuse createPostHogClient instead
of creating new clients per invocation, and add/export a shutdown function
(e.g., shutdownPostHog) that calls posthogInstance.shutdown() for graceful
teardown (also call it on process exit in initialization code or tests as
needed).

---

Outside diff comments:
In `@packages/web/src/lib/posthog.ts`:
- Around line 90-110: The call to posthog.capture in captureEvent currently
passes distinctId which can be undefined (from tryGetPostHogDistinctId); ensure
captureEvent always provides a string by adding a fallback before calling
posthog.capture (e.g., const distinctIdSafe = distinctId ?? crypto.randomUUID()
or a deterministic anon id using env.SOURCEBOT_INSTALL_ID) and pass
distinctIdSafe to posthog.capture instead of distinctId; update references in
captureEvent and keep tryGetPostHogDistinctId usage but guard its result so
posthog.capture never receives undefined.

---

Nitpick comments:
In `@packages/web/package.json`:
- Line 66: Update the `@posthog/ai` dependency in package.json by replacing the
current version specifier "^7.8.10" with the newer release (e.g. "7.9.1") so the
project uses the latest patch/feature fixes; locate the dependency entry for
"@posthog/ai" and change the version string accordingly, then run your package
manager (npm/yarn/pnpm) to install and verify no breakages.

In `@packages/web/src/features/chat/actions.ts`:
- Around line 34-36: There are duplicate import lines from "@/lib/posthog";
consolidate them by replacing the two separate imports with a single import that
includes captureEvent, createPostHogClient, and tryGetPostHogDistinctId together
(so update the import statements that reference captureEvent and the import that
references createPostHogClient/tryGetPostHogDistinctId into one unified import).

ℹ️ Review info

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 5be4667 and 3e5b31a.

⛔ Files ignored due to path filters (1)
  • yarn.lock is excluded by !**/yarn.lock, !**/*.lock
📒 Files selected for processing (3)
  • packages/web/package.json
  • packages/web/src/features/chat/actions.ts
  • packages/web/src/lib/posthog.ts

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant