Re: [Tarantool-patches] [PATCH] vinyl: rotate mem during index build on demand

[Nikita Pettik <korablev@tarantool.org>]

On 04 Jun 23:23, Konstantin Osipov wrote:
> * Nikita Pettik <korablev@tarantool.org> [20/06/04 20:49]:
> > Meanwhile in 17f6af7dc the similar problem has been fixed, still it may
> > appear that in-memory level of secondary index being constructed has
> > lagging memory generation (in other words, it's values is less than
> > the value of current global generation). Such situation can be achieved
> > if yield which is required to check uniqueness of value being inserted
> > is too long. In this time gap other space may trigger dump process
> > bumping global memory generation counter, but dump itself is still not
> > yet scheduled.
> 
> It's hard for me to understand this comment, perhaps
>  you discussed the problem verbally, but I'm a bit out of context.
> 
> Could you write it using an event diagram, something like:
> 
> user1         user2            vy_scheduler
> 
> insert ...     
>              create_index()
> 
> 
> yield                            
>                                dump
> 
> ???
> 

Sure. In the first case vy_scheduler fiber is not waked up during yield in
vy_build_insert_tuple():

       user1                user2            scheduler         generation

s1:create_index              ---               idle               n

yield(*_insert_tuple)       s2:insert()        idle               n

yield                     *_trigger_dump()     idle              n+1

yield                      s2:rotate_mem()     idle              n+1

yield                         ---              idle              n+1

s1:*_insert_tuple continue     ---             idle              n+1

^
|

s1 still has mem generation n, meanwhile global and s2 generations is n+1.

Since we can't schedule fibers (and can't provide guarantee that
vy_scheduler fiber won't wake up), to reproduce the problem let's
assume that vy_task_dump_new fails for whatever reason:

       user1                user2            scheduler         generation

s1:create_index              ---               idle               n

yield(*_insert_tuple)       s2:insert()        idle               n

yield                     *_trigger_dump()     idle              n+1

yield                          ---         vy_task_dump_new      n+1

yield                          ---            throttled          n+1

s1:*_insert_tuple continue      ---           throttled          n+1

Again s1 has mem generation n, meanwhile global and s2 generations is n+1.


 
> -- 
> Konstantin Osipov, Moscow, Russia