Flickering behavior due to redrawing

This is a fantastic plugin -- great idea, I like it pretty much.

One small thing that annoys me is that, whenever I scroll (e.g. press j many times) the buffer, all the buffer contents seem to be redrawn and because of this the vim screen flickers. Any solution to this?

I am using neovim 0.4.3 (linux), TERM=xterm-256color with termguicolors enabled.