Package org.opensearch.index.rankeval

Class PrecisionAtK

java.lang.Object

org.opensearch.index.rankeval.PrecisionAtK

All Implemented Interfaces:

NamedWriteable, Writeable, ToXContent, ToXContentObject, EvaluationMetric

public class PrecisionAtK
extends Object
implements EvaluationMetric

Metric implementing Precision@K (https://en.wikipedia.org/wiki/Evaluation_measures_(information_retrieval)#Precision).
By default documents with a rating equal or bigger than 1 are considered to be "relevant" for this calculation. This value can be changed using the `relevant_rating_threshold` parameter.
The `ignore_unlabeled` parameter (default to false) controls if unrated documents should be ignored. The `k` parameter (defaults to 10) controls the search window size.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static final class	PrecisionAtK.Detail

Nested classes/interfaces inherited from interface org.opensearch.common.xcontent.ToXContent

ToXContent.DelegatingMapParams, ToXContent.MapParams, ToXContent.Params

Nested classes/interfaces inherited from interface org.opensearch.common.io.stream.Writeable

Writeable.Reader<V extends Object>, Writeable.Writer<V extends Object>

Field Summary

Fields

Modifier and Type	Field	Description
static final String	NAME

Fields inherited from interface org.opensearch.common.xcontent.ToXContent

EMPTY_PARAMS

Constructor Summary

Constructors

Constructor	Description
PrecisionAtK()
PrecisionAtK(boolean ignoreUnlabeled)
PrecisionAtK(int relevantRatingThreshold, boolean ignoreUnlabeled, int k)	Metric implementing Precision@K.

Method Summary

Modifier and Type	Method	Description
final boolean	equals(Object obj)
EvalQueryQuality	evaluate(String taskId, SearchHit[] hits, List<RatedDocument> ratedDocs)	Compute precision at k based on provided relevant document IDs.
OptionalInt	forcedSearchSize()	Metrics can define a size of the search hits windows they want to retrieve by overwriting this method.
static PrecisionAtK	fromXContent(XContentParser parser)
boolean	getIgnoreUnlabeled()	Gets the 'ignore_unlabeled' parameter.
int	getK()
int	getRelevantRatingThreshold()	Return the rating threshold above which ratings are considered to be "relevant" for this metric.
String	getWriteableName()
final int	hashCode()
XContentBuilder	toXContent(XContentBuilder builder, ToXContent.Params params)
void	writeTo(StreamOutput out)

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.opensearch.index.rankeval.EvaluationMetric

combine

Methods inherited from interface org.opensearch.common.xcontent.ToXContentObject

isFragment

Field Details

NAME

public static final String NAME

See Also:

• Constant Field Values

Constructor Details

PrecisionAtK

public PrecisionAtK(int relevantRatingThreshold,
 boolean ignoreUnlabeled,
 int k)

Metric implementing Precision@K.

Parameters:

relevantRatingThreshold - ratings equal or above this value will be considered relevant.

ignoreUnlabeled - Controls how unlabeled documents in the search hits are treated. Set to 'true', unlabeled documents are ignored and neither count as true or false positives. Set to 'false', they are treated as false positives.

k - controls the window size for the search results the metric takes into account

PrecisionAtK

public PrecisionAtK(boolean ignoreUnlabeled)

PrecisionAtK

public PrecisionAtK()

Method Details

fromXContent

public static PrecisionAtK fromXContent(XContentParser parser)

writeTo

public void writeTo(StreamOutput out)
             throws IOException

Specified by:

writeTo in interface Writeable

Throws:

IOException

toXContent

public XContentBuilder toXContent(XContentBuilder builder,
 ToXContent.Params params)
                           throws IOException

Specified by:

toXContent in interface ToXContent

Throws:

IOException

getWriteableName

public String getWriteableName()

Specified by:

getWriteableName in interface NamedWriteable

getRelevantRatingThreshold

public int getRelevantRatingThreshold()

Return the rating threshold above which ratings are considered to be "relevant" for this metric. Defaults to 1.

getIgnoreUnlabeled

public boolean getIgnoreUnlabeled()

Gets the 'ignore_unlabeled' parameter.

getK

public int getK()

forcedSearchSize

public OptionalInt forcedSearchSize()

Description copied from interface: EvaluationMetric

Metrics can define a size of the search hits windows they want to retrieve by overwriting this method. The default implementation returns an empty optional.

Specified by:

forcedSearchSize in interface EvaluationMetric

Returns:

the number of search hits this metrics requests

evaluate

public EvalQueryQuality evaluate(String taskId,
 SearchHit[] hits,
 List<RatedDocument> ratedDocs)

Compute precision at k based on provided relevant document IDs.

Specified by:

evaluate in interface EvaluationMetric

Parameters:

taskId - an identifier of the query for which the search ranking is evaluated

hits - the search result hits

ratedDocs - the documents that contain the document rating for this query case

Returns:

precision at k for above SearchResult list.

equals

public final boolean equals(Object obj)

Overrides:

equals in class Object

hashCode

public final int hashCode()

Overrides:

hashCode in class Object