Wondering why FINGERPRINTS_PREFIX SUBSCRIPTIONS_PREFIX keys don't have EXPIRE?

[gastonmorixe]

We only apply EXPIRE to CHANNEL_PREFIX and SUBSCRIPTION_PREFIX. Would like to understand the reasons why we can't apply it to the other keys.

graphql-anycable/lib/graphql/subscriptions/anycable_subscriptions.rb

Lines 147 to 158 in 736501c

	redis.multi do |pipeline|
	pipeline.sadd(CHANNEL_PREFIX + channel_uniq_id, subscription_id)
	pipeline.mapped_hmset(SUBSCRIPTION_PREFIX + subscription_id, data)
	events.each do |event|
	pipeline.zincrby(FINGERPRINTS_PREFIX + event.topic, 1, event.fingerprint)
	pipeline.sadd(SUBSCRIPTIONS_PREFIX + event.fingerprint, subscription_id)
	end
	next unless config.subscription_expiration_seconds
	pipeline.expire(CHANNEL_PREFIX + channel_uniq_id, config.subscription_expiration_seconds)
	pipeline.expire(SUBSCRIPTION_PREFIX + subscription_id, config.subscription_expiration_seconds)
	end
	end

Thank you

[Envek]

TL;DR: I just not sure that it is safe to apply expiration to other data structures, that's why I haven't added expiration on them. But I'm open to suggestions.

There are two opposite directions on that data structures are traversed:

1. On subscription or unsubscription, we create or delete subscription and channel for every client and register it for topics and fingerprints.
2. On event trigger, we search by topics and fingerprints queries that should be evaluated, evaluate them and broadcast results by channels.

Current expiration logic covers only first direction to clean up data left after clients that didn't disconnect cleanly for some reason, and doesn't care about opposite direction.

The question is whether it is safe to set expiration on topic-fingerprints and fingerprint-subscriptions sets (and not to forget to bump them on every client subscriprions) without probability to get them expired too early and left some clients without updates?

Right now, if you're executing cleaning rake task graphql:anycable:clean regularly, it will detect and remove absent entries from these sets here and Redis will automatically remove sets that became empty.

However, I haven't thought about it much, so if you have any thoughts on how to prove safety or non-safety, please share!

But in general I'm open to adding expiration.

P.S> Please tell more about your current problem, why you're asking?