it.unimi.dsi.mg4j.index.cluster
Class DocumentalMergedClusterDocumentIterator

java.lang.Object
  it.unimi.dsi.fastutil.ints.AbstractIntIterator
      it.unimi.dsi.mg4j.search.AbstractDocumentIterator
          it.unimi.dsi.mg4j.index.cluster.DocumentalMergedClusterDocumentIterator

All Implemented Interfaces:

IntIterator, DocumentIterator, Iterable<Interval>, Iterator<Integer>

Direct Known Subclasses:

DocumentalMergedClusterIndexIterator

public class DocumentalMergedClusterDocumentIterator
extends AbstractDocumentIterator
implements DocumentIterator

A document iterator merging iterators from local indices.

Author:

Sebastiano Vigna

Nested Class Summary

Nested classes/interfaces inherited from class it.unimi.dsi.mg4j.search.AbstractDocumentIterator

AbstractDocumentIterator.AbstractIntervalIterator

Field Summary

protected int	currentIterator The current iterator.
protected DocumentIterator[]	documentIterator The component document iterators.
protected boolean	exhausted Whether there are no more documents to be returned.
protected int[]	globalDocumentPointer The reference array for the queue (containing global document pointers).
protected ReferenceSet<Index>	indices The set of indices involved in this iterator.
protected int	n The number of component iterators.
protected IntHeapSemiIndirectPriorityQueue	queue The queue of document iterator indices (offsets into documentIterator and usedIndex).
protected DocumentalClusteringStrategy	strategy The cached strategy of the index we refer to.
protected int[]	usedIndex The indices corresponding to each underlying document iterator.

Fields inherited from class it.unimi.dsi.mg4j.search.AbstractDocumentIterator

last, next, weight

Constructor Summary

DocumentalMergedClusterDocumentIterator(DocumentalClusterIndexReader indexReader, DocumentIterator[] documentIterator, int[] usedIndex)
Creates a new document iterator for a documental cluster.

Method Summary

<T> T	accept(DocumentIteratorVisitor<T> visitor) Accepts a visitor.
<T> T	acceptOnTruePaths(DocumentIteratorVisitor<T> visitor) Accepts a visitor after a call to DocumentIterator.nextDocument(), limiting recursion to true paths.
void	dispose() Disposes this document iterator, releasing all resources.
ReferenceSet<Index>	indices() Returns the set of indices over which this iterator is built.
IntervalIterator	intervalIterator() Returns the interval iterator of this document iterator for single-index queries.
IntervalIterator	intervalIterator(Index index) Returns the interval iterator of this document iterator for the given index.
Reference2ReferenceMap<Index,IntervalIterator>	intervalIterators() Returns an unmodifiable map from indices to interval iterators.
int	nextDocument() Returns the next document provided by this document iterator, or -1 if no more documents are available.
int	skipTo(int p) Skips all documents smaller than n.

Methods inherited from class it.unimi.dsi.mg4j.search.AbstractDocumentIterator

document, hasNext, iterator, nextInt, weight, weight

Methods inherited from class it.unimi.dsi.fastutil.ints.AbstractIntIterator

next, remove, skip

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface it.unimi.dsi.mg4j.search.DocumentIterator

document, iterator, nextInt, weight, weight

Methods inherited from interface it.unimi.dsi.fastutil.ints.IntIterator

skip

Methods inherited from interface java.util.Iterator

hasNext, next, remove

Field Detail

documentIterator

protected final DocumentIterator[] documentIterator

The component document iterators.

n

protected final int n

The number of component iterators.

usedIndex

protected final int[] usedIndex

The indices corresponding to each underlying document iterator.

strategy

protected final DocumentalClusteringStrategy strategy

The cached strategy of the index we refer to.

queue

protected final IntHeapSemiIndirectPriorityQueue queue

The queue of document iterator indices (offsets into documentIterator and usedIndex).

globalDocumentPointer

protected final int[] globalDocumentPointer

The reference array for the queue (containing global document pointers).

indices

protected final ReferenceSet<Index> indices

The set of indices involved in this iterator.

currentIterator

protected int currentIterator

The current iterator.

exhausted

protected boolean exhausted

Whether there are no more documents to be returned.

Constructor Detail

DocumentalMergedClusterDocumentIterator

public DocumentalMergedClusterDocumentIterator(DocumentalClusterIndexReader indexReader,
                                               DocumentIterator[] documentIterator,
                                               int[] usedIndex)
                                        throws IOException

Creates a new document iterator for a documental cluster.

This constructor uses an array of document iterators that it is not required to be full. This is very useful with rare terms.

Parameters:

indexReader - the underlying index reader.

documentIterator - an array of document iterators.

usedIndex - an array parallel to documentIterator containing the ordinal numbers of the indices corresponding to the iterators.

Throws:

IOException

Method Detail

intervalIterator

public IntervalIterator intervalIterator()
                                  throws IOException

Description copied from interface: DocumentIterator

Returns the interval iterator of this document iterator for single-index queries.

This is a commodity method that can be used only for queries built over a single index.

Specified by:

intervalIterator in interface DocumentIterator

Returns:

an interval iterator.

Throws:

IOException

See Also:

DocumentIterator.intervalIterator(Index)

intervalIterator

public IntervalIterator intervalIterator(Index index)
                                  throws IOException

Description copied from interface: DocumentIterator

Returns the interval iterator of this document iterator for the given index.

After a call to DocumentIterator.nextDocument(), this iterator can be used to retrieve the intervals in the current document (the one returned by DocumentIterator.nextDocument()) for the index index.

Note that if all indices have positions, it is guaranteed that at least one index will return an interval. However, for disjunctive queries it cannot be guaranteed that all indices will return an interval.

Indices without positions always return IntervalIterators.TRUE. Thus, in presence of indices without positions it is possible that no intervals at all are available.

Specified by:

intervalIterator in interface DocumentIterator

Parameters:

index - an index (must be one over which the query was built).

Returns:

an interval iterator over the current document in index.

Throws:

IOException

intervalIterators

public Reference2ReferenceMap<Index,IntervalIterator> intervalIterators()
                                                                 throws IOException

Description copied from interface: DocumentIterator

Returns an unmodifiable map from indices to interval iterators.

After a call to DocumentIterator.nextDocument(), this map can be used to retrieve the intervals in the current document. An invocation of Map.get(java.lang.Object) on this map with argument index yields the same result as intervalIterator(index).

Specified by:

intervalIterators in interface DocumentIterator

Returns:

a map from indices to interval iterators over the current document.

Throws:

IOException

See Also:

DocumentIterator.intervalIterator(Index)

indices

public ReferenceSet<Index> indices()

Description copied from interface: DocumentIterator

Returns the set of indices over which this iterator is built.

Specified by:

indices in interface DocumentIterator

Returns:

the set of indices over which this iterator is built.

skipTo

public int skipTo(int p)
           throws IOException

Description copied from interface: DocumentIterator

Skips all documents smaller than n.

Define the current document k associated with this document iterator as follows:

• -1, if DocumentIterator.nextDocument() and this method have never been called;
• Integer.MAX_VALUE, if a call to this method returned Integer.MAX_VALUE;
• the last value returned by a call to DocumentIterator.nextDocument() or this method, otherwise.

If k is larger than or equal to n, then this method does nothing and returns k. Otherwise, a call to this method is equivalent to

 while( ( k = nextDocument() ) < n && k != -1 );
 return k == -1 ? Integer.MAX_VALUE : k;
 

Thus, when a result k ≠ Integer.MAX_VALUE is returned, the state of this iterator will be exactly the same as after a call to DocumentIterator.nextDocument() that returned k. In particular, the first document larger than or equal to n (when returned by this method) will not be returned by the next call to DocumentIterator.nextDocument().

Specified by:

skipTo in interface DocumentIterator

Parameters:

p - a document pointer.

Returns:

a document pointer larger than or equal to n if available, Integer.MAX_VALUE otherwise.

Throws:

IOException

nextDocument

public int nextDocument()
                 throws IOException

Description copied from interface: DocumentIterator

Returns the next document provided by this document iterator, or -1 if no more documents are available.

Warning: the specification of this method has significantly changed as of MG4J 1.2. The special return value -1 is used to mark the end of iteration (a NoSuchElementException would have been thrown before in that case, so ho harm should be caused by this change). The reason for this change is providing fully lazy iteration over documents. Fully lazy iteration does not provide an hasNext() method—you have to actually ask for the next element and check the return value. Fully lazy iteration is much lighter on method calls (half) and in most (if not all) MG4J classes leads to a much simpler logic. Moreover, DocumentIterator.nextDocument() can be specified as throwing an IOException, which avoids the pernicious proliferation of try/catch blocks in very short, low-level methods (it was having a detectable impact on performance).

Specified by:

nextDocument in interface DocumentIterator

Returns:

the next document, or -1 if no more documents are available.

Throws:

IOException

accept

public <T> T accept(DocumentIteratorVisitor<T> visitor)
         throws IOException

Description copied from interface: DocumentIterator

Accepts a visitor.

A document iterator is usually structured as composite, with operators as internal nodes and IndexIterators as leaves. This method implements the visitor pattern.

Specified by:

accept in interface DocumentIterator

Parameters:

visitor - the visitor.

Returns:

an object resulting from the visit, or null if the visit was interrupted.

Throws:

IOException

acceptOnTruePaths

public <T> T acceptOnTruePaths(DocumentIteratorVisitor<T> visitor)
                    throws IOException

Description copied from interface: DocumentIterator

Accepts a visitor after a call to DocumentIterator.nextDocument(), limiting recursion to true paths.

After a call to DocumentIterator.nextDocument(), a document iterator is positioned over a document. This call is equivalent to DocumentIterator.accept(DocumentIteratorVisitor), but visits only along true paths.

We define a true path as a path from the root of the composite that passes only through nodes whose associated subtree is positioned on the same document of the root. Note that OrDocumentIterators detach exhausted iterators from the composite tree, so true paths define the subtree that is causing the current document to satisfy the query represented by this document iterator.

For more elaboration, and the main application of this method, see CounterCollectionVisitor.

Specified by:

acceptOnTruePaths in interface DocumentIterator

Parameters:

visitor - the visitor.

Returns:

an object resulting from the visit, or null if the visit was interrupted.

Throws:

IOException

See Also:

DocumentIterator.accept(DocumentIteratorVisitor), CounterCollectionVisitor

dispose

public void dispose()
             throws IOException

Description copied from interface: DocumentIterator

Disposes this document iterator, releasing all resources.

This method should propagate down to the underlying index iterators, where it should release resources such as open files and network connections. If you're doing your own resource tracking and pooling, then you do not need to call this method.

Specified by:

dispose in interface DocumentIterator

Throws:

IOException