Delays with replay?

Help & Questions
1

I’m testing some new rule sets so I wanted to replay existing messages (and have a temporary stop processing step at the end).

I noticed some delays yesterday when clicking on Replay but today its’ really bad. It’s been more than 5 minutes, almost 10, and the message isn’t replaying. I thought maybe I’d accidently hit cancel or something, so I did it again. Nothing. I also tried it on multiple messages, from both the main history and opening a specific message detail and replaying it there.

If the first history is collapsed, when you click Replay, it will expand it (slight little bug), but the new history entry for the replay is not appearing.

Here’s 3 messages I tried it for:
https://console.us1.mspintegrations.com/#/email/history/17889569645240040u1y
https://console.us1.mspintegrations.com/#/email/history/17889568279960040rcr
https://console.us1.mspintegrations.com/#/email/history/17889567918830040qvr

As I was writing this, the first one finally kicked off, then the others, and then the ones I had double submitted.

So it just appears to be a long delay. Since there is no feedback on the resubmit until it shows up in the processing logs, you just have to wait for a bit to see.

I like all of the status filtering you have on the main history page. Not sure if it’s worth a status for “waiting on replay” (or is that really just processing at that point).

Could all be a moot point if this is just a fleeting performance issue, but if there is more to it, such as decreased priority, or secondary systems or something, I wanted you to know.

2

Hi @DBachen

Thanks for reporting this. Yes, this was a temporary transient error caused by SentinelOne flooding the system with thousands upon thousands of notifications about successful process kills (for another customer, not you). This SentinelOne pattern is something new we have seen across a number of customers that use SentinelOne, and the pattern has emerged most acutely over the past month or so. We are still working on a mitigation for it, but the root cause of your delayed processing this morning was a flood of messages from SentinelOne for another customer.

That said, I am glad to hear of your experience because it gives me some ideas on how to make the feedback loop better when reprocessing. I’m going to look at this and see what we can do to confirm to you that the replay was accepted so that it does not appear as if nothing happened when you click replay, especially if there’s a delay in the system.