Apex Code Development You need to sign in to do that
Don't have an account?
Nick Becker Design: Streaming API -- Way to retrieve last successfully processed replay id from a pushtopic subscription
Is there a way to retrieve last successfully processed replay id from a pushtopic subscription? The main issue here is that there doesn't appear to be a replay option when subscribing to a stream to say "continue where we last left off," with the closest options being "all events in the last 24 hours" and "continue from _this_ replay id"...
I'm using the streaming API to implement real-time data synchronization between SF and our backend data stores. If the backend service that subscribes to these pushtopics could be guaranteed to always run forever with no downtime, it might be ok, but reality is things happen, and even if they don't, updated versions of the service require that it be stopped temporarily. When it resumes, I'd really love a way to avoid having to have stored the last successful replay id to pass to SF, but am curious what that workaround might be. If I just say "give me everything in the last 24 hours", I'll need to implement something to de-dupe already processed messages, which is also not ideal, but, things seldom are.
Using the 'jsforce' module for a nodejs command line application on this one, using oauth username/password flow.
I'm using the streaming API to implement real-time data synchronization between SF and our backend data stores. If the backend service that subscribes to these pushtopics could be guaranteed to always run forever with no downtime, it might be ok, but reality is things happen, and even if they don't, updated versions of the service require that it be stopped temporarily. When it resumes, I'd really love a way to avoid having to have stored the last successful replay id to pass to SF, but am curious what that workaround might be. If I just say "give me everything in the last 24 hours", I'll need to implement something to de-dupe already processed messages, which is also not ideal, but, things seldom are.
Using the 'jsforce' module for a nodejs command line application on this one, using oauth username/password flow.
Have you ever tried to re-subscribe to the Channel passing the last event that your client processed sucessfuly?
Like described on: https://developer.salesforce.com/docs/atlas.en-us.api_streaming.meta/api_streaming/using_streaming_api_durability.htm
So, if you received 1000 during a whole day, but for whatever reason your client stopped at 600. You can re-subscribe passing 601, so you are going to receive 601, 602, 603 and so on until 1000 (ps.: since all these events are within the last 24 hours).
Kafka, for example, stores the processed messages offsets per consumer on Kafka side which eliminates the need to consumer to implement any "store my last processed message and re-invent the wheel 100000 times" things.
Does salesforce store this info somewhere?