Product Analytics Team

Q3 2024 objectives

1. Rock-solid analytics (Thomas + Julian + Sandy + Anirudh)

   1. Legacy Minus – removing legacy insights code so that we can move fast
      • FilterType gone from the frontend.
      • rm -rf posthog/queries/
      • Experiments ported to HogQL.
      • All the flags from HogQL/querying work.
   2. Tests Plus – shipping fewer bugs in the first place
      • Ensure we test with the feature flags that users actually experience, both in end-to-end and integration tests.
      • When shipping changes to queries, replay old vs. new version on thousands of real queries to check for regressions.
   3. Metrics Plus – catching issues before before users report
      • Analytics performance dashboard in Grafana (query duration, failures, etc.). Paging alerts on critical metrics, e.g. if the number of queries drops rapidly, or failures rise.
      • Analytics experience dashboard in PostHog (time till data available, result freshness across insights and subscriptions, refreshes initiated manually vs. automatically, etc.)
      • Alerts on major Product Analytics errors from Sentry, and us acting on every alert. (Bonus: checking up the Sentry routing rules for the #product-analytics team.)
      • Cohorts dashboard in Grafana (successful vs. failed calculations per day, recalculation backlog). Alerts here too.
   4. Performance Plus - eliminating UX pain via maximum query performance/reliability, based on Metrics Plus data
      • Partial calculation of multi-day time series results
      • …and more – work with Team Query Performance to find the lowest-hanging fruit, similarly to Tim's performance mega issue
   5. Support Plus – sparking joy for users when they’re led to report a bug
      • 1 hero + 1 sidekick
      • Goal: 90% of tickets fulfill the SLA

2. Answering more product questions, deeper (Thomas + Julian + Sandy + Anirudh)

   1. Growth Plus - increasing ease of onboarding, and subsequent retention
      • Identify growth opportunities working with Anna, our product manager – implement growth optimizations and track their impact whenever possible.
      • Work with Team Growth on optimizing the onboarding experience of Product Analytics.
   2. Analysis Plus - answering more product questions, more deeply
      • Analytics alerts are out to users (implemented with the contributor)
      • “Done for the first time” in Trends, to kill the janky First Time Event Plugin
      • Query in new insight URL for instant insight sharing
      • Optional funnel steps
      • ...and more, based on user feedback - see the most requested features in GitHub

3. ArtificialHog (Michael + Georgiy) – an LLM-based chat-like interface for answering product questions.

Who are we building for?

Personas

• Primary Personas:
  • Product engineer
    • These are the engineers building the product. Normally full-stack engineers skewing frontend or frontend engineers.
    • Product engineers have more limited time. Need to quickly get high-quality insights to inform what they are building and assess what they've shipped.
  • Product manager (ex-engineer type)
    • Supports the product teams (engineers, PMs, designers) to build the best products. They guide the product roadmap by speaking to customers and diving into the data.
    • Product managers are the power-users of analytics (further evidence in the data analysis of paying users). They have desire and the time to go significantly deeper into the data.
• Limited focus:
  • Growth engineer
• Not a focus but should be usable by:
  • Everyone in the product team (less technical PMs, designers)
  • Marketing
  • Leadership team

What types of companies?

The highest-performing product teams building the most loved products at high-growth startups. For more context on the company read about the ideal customer persona.

Jobs to be done

Product analytics is a wide tool which fulfills many job-to-be-done (non-exhaustive list):

• Monitoring KPIs - how are the specific KPIs (product/team/company) doing? Are there any big changes, is everything going roughly in the right direction?
• Insights into a new feature you've built - I've created a new feature and I want to make sure that it's being used successfully
• Watching for errors and debugging - something went wrong (error gets trigger, product regression, drop in conversion), getting told it went wrong, debugging it, shipping a solution and making sure that fixes it
• Conversion optimization - the growth team is monitoring how particular KPIs are doing, trying to come up with experiments, shipping features to try and improve these
• Answering product strategy questions - which customers should we focus on, what are our most used/valued features. e.g. should we increase the pricing from X to Y? Which customers should we focus on?

You can broadly group the job-to-be-done of Product Analytics in PostHog as:

• Creating: You have a specific query/dashboard in mind, you open PostHog to view it. E.g. creating a dashboard to Monitor KPIs, or creating the funnel for your onboarding flow
• Consuming: you or someone else has made something in Posthog that you refer back to. E.g. Checking the dashboard you made to Monitor KPIs
• Exploring: you're answering a broader open-ended question. E.g. If you're monitoring your KPIs and you see something not right - you then want to dive into understanding why

Roadmap

3 year goals

• You can explore data across all insights and dimensions
• You can trivially share any insight anywhere
• Onboarding is as easy as a video game
• Tight integration with developer workflows
• No more complex than it is today
• Using PostHog sparks joy
• We support trillion event querying

Feature ownership

You can find out more about the features we own here