com.articulate.sigma.nlp

Class TFIDF

java.lang.Object

com.articulate.sigma.nlp.TFIDF

public class TFIDF
extends Object

Field Summary

Fields

Modifier and Type	Field and Description
ArrayList<String>	lines

Constructor Summary

Constructors

Constructor and Description
TFIDF(List<String> documents, String stopwordsFilename)
TFIDF(List<String> documents, String stopwordsFilename, boolean resource)
TFIDF(String stopwordsFilename)
TFIDF(String filename, String stopwordsFilename, boolean resource)

Method Summary

Modifier and Type	Method and Description
void	addInput(String input) add a new document to the set
protected void	calcDFs()
static void	main(String[] args) Run a series of tests containing a filename, a query and an expected answer.
ArrayList<String>	matchBestInput(String input)
String	matchInput(String input)
List<String>	matchInput(String input, int n)
protected TreeMap<Float,ArrayList<Integer>>	matchInputFull(String input)
void	newLine(String line)
void	prepare(List<String> documents, String stopwordsFilename)
protected void	prepareLine(String line)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

lines

public ArrayList<String> lines

Constructor Detail

TFIDF

public TFIDF(String stopwordsFilename)
      throws IOException

Throws:

IOException

TFIDF

public TFIDF(List<String> documents,
             String stopwordsFilename,
             boolean resource)
      throws IOException

Throws:

IOException

TFIDF

public TFIDF(List<String> documents,
             String stopwordsFilename)
      throws IOException

Throws:

IOException

TFIDF

public TFIDF(String filename,
             String stopwordsFilename,
             boolean resource)

Method Detail

prepare

public void prepare(List<String> documents,
                    String stopwordsFilename)
             throws IOException

Throws:

IOException

newLine

public void newLine(String line)

prepareLine

protected void prepareLine(String line)

calcDFs

protected void calcDFs()

addInput

public void addInput(String input)

add a new document to the set

matchBestInput

public ArrayList<String> matchBestInput(String input)

Returns:

a list of matches ranked by relevance to the input. If there is a cluster of top matches, return all elements of the cluster. If no answer has a good match, return "I don't know". Iterate the number of clusters until the top cluster is no more than 3.

matchInput

public String matchInput(String input)

matchInputFull

protected TreeMap<Float,ArrayList<Integer>> matchInputFull(String input)

Returns:

a list of matches ranked by relevance to the input.

matchInput

public List<String> matchInput(String input,
                               int n)

Returns:

a list of strings which are the top n best guess matches to the input. If global variable alternating is set to true, the return the next line in the input file, which is therefore treated like a dialog in which the best response to a given input is the line after the line in the dialog that matches. If there's more than one reasonable response, pick a random one.

main

public static void main(String[] args)
                 throws IOException

Run a series of tests containing a filename, a query and an expected answer.

Throws:

IOException