com.bbn.bue.common.evaluation

Class BootstrapInspector<ObsT,SummaryT>

java.lang.Object

com.bbn.bue.common.evaluation.BootstrapInspector<ObsT,SummaryT>

All Implemented Interfaces:

Finishable, Inspector<ObsT>

@Beta
public final class BootstrapInspector<ObsT,SummaryT>
extends Object
implements Inspector<ObsT>

An inspector which can use a supplied strategy to do some sort of scoring or inspection of a corpus using bootstrapping. A strategy must supply two things: a way of mapping every observed item to some sort of summary (an BootstrapInspector.ObservationSummarizer and one or more ways of aggregating a collection of collections of summaries into a final output (a BootstrapInspector.SummaryAggregator. The bootstrap inspector will apply the ObservationSummarizer to each item it observes and remember the result. When it is finish()ed, it will then take a specified number of samples with replacement from the collection of summaries. Each of these samples will be presented to the SummaryAggregators. Finally, each SummaryAggregator's finish() method will be called to produce final output. To make an example concrete, your ObservationSummarizer could produce the true positive, false positive, and false negative counts for some classification task on a document. The aggregator could sum these of the corpus and create a corpus-wide F-measure and then output the mean and standard deviation of these F-measures. This would give you some idea how robust your F-measure is with respect to the corpus composition. If you don't know how many samples to use, we suggest 1000 as a reasonable default.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	BootstrapInspector.BootstrapStrategy<ObsT,SummaryT>
static class	BootstrapInspector.Builder<ObsT,SummaryT>
static interface	BootstrapInspector.ObservationSummarizer<ObsT,SummaryT> A strategy for reducing a pair observation to a some sort of summary we can later bootstrap sample over.
static interface	BootstrapInspector.SummaryAggregator<SummaryT> A strategy of aggregating bootstrap samples into some sort of final output.

Method Summary

Methods

Modifier and Type	Method and Description
void	finish()
static <ObsT,SummaryT> BootstrapInspector<ObsT,SummaryT>	forStrategy(BootstrapInspector.BootstrapStrategy<ObsT,SummaryT> strategy, int numSamples, Random rng)
static <ObsT,SummaryT> BootstrapInspector.Builder<ObsT,SummaryT>	forSummarizer(BootstrapInspector.ObservationSummarizer<? super ObsT,? extends SummaryT> observationSummarizer, int numSamples, Random rng)
void	inspect(ObsT item)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

inspect

public void inspect(ObsT item)

Specified by:

inspect in interface Inspector<ObsT>

finish

public void finish()
            throws IOException

Specified by:

finish in interface Finishable

Throws:

IOException

forSummarizer

public static <ObsT,SummaryT> BootstrapInspector.Builder<ObsT,SummaryT> forSummarizer(BootstrapInspector.ObservationSummarizer<? super ObsT,? extends SummaryT> observationSummarizer,
                                                                      int numSamples,
                                                                      Random rng)

forStrategy

public static <ObsT,SummaryT> BootstrapInspector<ObsT,SummaryT> forStrategy(BootstrapInspector.BootstrapStrategy<ObsT,SummaryT> strategy,
                                                            int numSamples,
                                                            Random rng)