Use Case - CF Worker to store and fetch visit history or user/org activity feed with KV inside a SaaS app

When you are logged into a SaaS app, like ours at Tallyfy - we know your identity, and you have a token/userID that’s unique and trusted.

We wanted to store your navigation history, not for marketing/creepy purposes, but to present in-app - so that you can see “Recently Visited” items.

This is useful because we can present a new in-app section where a specific user can see where they were before, and can get back to something they were doing before, in case they got lost. It’s kind of like the browsing history feature on your Google Chrome. This also lends itself well to CF Workers and KV cache because:

  1. It’s just a click on a URL that we can store into KV store from our actual client - i.e. the push of your navigation history as it relates to unique objects for a unique user.
  2. We fetch most recent visits straight from the edge via a nominated endpoint, by providing userID and some other secret.
  3. We auto-expire KV keys so that history older than say 30 days is auto-deleted.

Problems/thoughts:

  1. If you visit a specific URL/object 10 times, we don’t want 10 entries - we just want an existing entry updated with the latest timestamp of the visit. This ignores URL params - and focuses on interaction with unique objects.
  2. We want to store history securely but also cater for over-writes like (1).
  3. If a single user clicks rapidly e.g. 2 clicks per second on 2 separate objects, will we face a problem writing to the same key?
  4. We might want to aggregate browsing history to present internal security logs e.g. by a single tenant or organization i.e. show me all activity for all users for a given OrgID X.
  5. We might further want to segment history in terms of reads/writes based on the type of API verb used to one day go from just “browsing history” to a proper “activity feed”. We would need to fetch that data properly and render it just like every other activity feed (picture Facebook’s newsfeed) where it shows up WHO did WHAT, WHEN and (before/after).

It would be awesome to ultimately publish our work on CF’s gallery as a “log history” or “audit trail” type of app which anyone can use to help someone with their in-app experience. It’s especially useful for workflow apps like ours, since people go to a task, go somewhere else, and forget how to get back to it.

Can anyone help with these or similar issues we need to solve, to actually make this use case work to the ultimate end result - a proper activity feed that can also be aggregated at an organizational level?

Yeah I wouldn’t store that data in any KV store if you don’t really know how you’re going to want to query it. I’d just put every event in BigQuery and populate rollup tables in your main database periodically or based on some event. Postgres for the history table would be fine too if you keep the table size limited. Or whatever queryable database you prefer to use with CF workers…