When I am scrolling through a book that has dates in it, with the "Timeline events" visual filter turned on, each time I page down into a new section or article of that book, the first page of it displays initially without the timeline flags and then gets repainted with the timeline flags. This seems inefficient and slows down the perceived speed of scrolling/paging through the book, plus you can see a visual flicker when the page gets redrawn with some of the text in a slightly different position to make room for the flags.
I think you ought to wait to display anything until you've determined whether you're going to be displaying the visual filter stuff, so as not to have to draw it twice.