[PATCH 1/4] vinyl: fix EQ check in run iterator

[Vladimir Davydov]

On Tue, May 15, 2018 at 10:05:12PM +0300, Konstantin Osipov wrote:
> * Vladimir Davydov <vdavydov.dev at gmail.com> [18/05/15 17:10]:
> 
> I am pushing this as is now, please a couple of comments below.
> 
> > vy_run_iterator_seek() is supposed to check that the resulting statement
> > matches the search key in case of ITER_EQ, but if the search key lies at
> > the beginning of the slice, it doesn't. As a result, vy_point_lookup()
> > may fail to find an existing tuple as demonstrated below.
> > 
> > Suppose we are looking for key {10} in the primary index which consists
> > of an empty mem and two runs:
> > 
> >     run 1: DELETE{15}
> >     run 2: INSERT{10}
> > 
> > vy_run_iterator_next() returns DELETE{15} for run 1 because of the
> > missing EQ check and vy_point_lookup() stops at run 1 (since the
> > terminal statement is found) and mistakenly returns NULL.
> > 
> > The issue manifests itself as crash in vinyl_iterator_secondary_next(),
> > when we fail to find the tuple in the primary index corresponding to a
> > statement found in a secondary index.
> 
> I believe this explanation belongs to the code, not only to the
> changeset comment.

Well, all those ITER_EQ checks are scattered throughout iterator code
and none of them has a comment as they all are pretty self-explaining.
Adding a comment to just one of them looks pointless IMO.

Actually, I was thinking about extracting all the EQ checks out of
source iterators and moving them to vy_read_iterator as this would allow
to reduce the number of EQ comparisons, but I didn't get my hands on it
as there were some problems with the cache iterator. May be, later.

> 
> Ideally, this special case should be covered in a unit test, not
> just in the code or in CS comment.

After the last patch in the series is applied, vinyl/select_consistency
functional test triggers this bug in 100% cases. Anyway, I'll think
about a unit test.

> 
> > Part of #3393
> > ---
> >  src/box/vy_run.c | 13 ++++++++++++-
> >  1 file changed, 12 insertions(+), 1 deletion(-)
> > 
> > diff --git a/src/box/vy_run.c b/src/box/vy_run.c
> > index 587cb002..8c922895 100644
> > --- a/src/box/vy_run.c
> > +++ b/src/box/vy_run.c
> > @@ -1316,6 +1316,7 @@ vy_run_iterator_seek(struct vy_run_iterator *itr,
> >  {
> >  	const struct key_def *cmp_def = itr->cmp_def;
> >  	struct vy_slice *slice = itr->slice;
> > +	const struct tuple *check_eq_key = NULL;
> >  	int cmp;
> >  
> >  	if (slice->begin != NULL &&
> > @@ -1340,6 +1341,8 @@ vy_run_iterator_seek(struct vy_run_iterator *itr,
> >  			return 0;
> >  		}
> >  		if (cmp < 0 || (cmp == 0 && iterator_type != ITER_GT)) {
> > +			if (iterator_type == ITER_EQ)
> > +				check_eq_key = key;
> >  			iterator_type = ITER_GE;
> >  			key = slice->begin;
> >  		}
> > @@ -1365,7 +1368,15 @@ vy_run_iterator_seek(struct vy_run_iterator *itr,
> >  		}
> >  	}
> >  
> > -	return vy_run_iterator_do_seek(itr, iterator_type, key, ret);
> > +	if (vy_run_iterator_do_seek(itr, iterator_type, key, ret) != 0)
> > +		return -1;
> > +
> > +	if (check_eq_key != NULL && *ret != NULL &&
> > +	    vy_stmt_compare(check_eq_key, *ret, cmp_def) != 0) {
> > +		vy_run_iterator_stop(itr);
> > +		*ret = NULL;
> > +	}
> 
> As far as I understand the code flow, this adds an extra check for
> cases when key != slice->begin. This is 99.9% of cases. Can we
> avoid an extra check if it is not needed?

We do this extra tuple comparison only if cmp == 0 and the original
iterator_type is ITER_EQ. The (cmp < 0) case is excluded, because we
stop the iteration in this case for ITER_EQ - see several lines above:

> 		cmp = vy_stmt_compare_with_key(key, slice->begin, cmp_def);
> 		if (cmp < 0 && iterator_type == ITER_EQ) {
> 			vy_run_iterator_stop(itr);
> 			return 0;
> 		}
> 		if (cmp < 0 || (cmp == 0 && iterator_type != ITER_GT)) {
> 			if (iterator_type == ITER_EQ)
> 				check_eq_key = key;

				^^^^^^^^^^^^^^^^^^
cmp can't be < 0 if this assignment takes place

> 			iterator_type = ITER_GE;
> 			key = slice->begin;
> 		}

Looks rather messy, but then again I think we should try to clean up
this mess by extracting ITER_EQ check out of source iterators.

Thanks.