Handle Backlog Processor Startup Failures by nathan-miller23 · Pull Request #3002 · StackExchange/StackExchange.Redis

Behavior

Whenever our client host experiences transient memory contention, we note that a portion of our ConnectionMultiplexer clients get 'stuck' in state where backlog writer is in Activating state. Example error message below:

image

Further analysis showed that client returned IsConnected=true with PhysicalBridge in ConnectedEstablished state. ProcDump revealed that no StackExchange.Redis Backlog thread was present

Bug

When StartBacklogProcessor fails to start the backlog processor thread (say, when thread.Start fails with System.OutOFMemoryException) system gets stuck in state where _backlogProcessorIsRunning is true but no backlog thread is doing work.

Fix

try-catch-finally pattern to ensure we reset _backlogPRocessorIsRunning state when we fail to start thread