org.supermind.crawl.util
Class MD5Persister

java.lang.Object
  extended by org.supermind.crawl.util.MapFilePersister<org.apache.nutch.io.MD5Hash,org.apache.nutch.io.NullWritable>
      extended by org.supermind.crawl.util.MD5Persister

public class MD5Persister
extends MapFilePersister<org.apache.nutch.io.MD5Hash,org.apache.nutch.io.NullWritable>

Saves MD5Hash values to a MapFile.

Field Summary
 
Fields inherited from class org.supermind.crawl.util.MapFilePersister
baseDir, buffer, bufferIdx, LOG, mapdir, mapdirFile, maxBufferSize, nfs, overwrite, reader
 
Constructor Summary
MD5Persister()
           
 
Method Summary
 void add(org.apache.nutch.io.MD5Hash hash)
           
 boolean contains(org.apache.nutch.io.MD5Hash hash)
           
protected  org.apache.nutch.io.WritableComparator getKeyComparator()
          Get comparator for MapFile key class.
 org.apache.nutch.io.WritableComparable getKeyInstance(org.apache.nutch.io.MD5Hash md5Hash, boolean throwaway)
          Return a new instance of the key.
 java.util.Comparator getTypeComparator()
          Get comparator for type.
 
Methods inherited from class org.supermind.crawl.util.MapFilePersister
add, close, flushToDisk, getValueInstance, init, setBaseDir, setMaxBufferSize, setNfs, setOverwrite
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

MD5Persister

public MD5Persister()
Method Detail

add

public void add(org.apache.nutch.io.MD5Hash hash)
         throws java.io.IOException
Throws:
java.io.IOException

contains

public boolean contains(org.apache.nutch.io.MD5Hash hash)
                 throws java.io.IOException
Throws:
java.io.IOException

getKeyComparator

protected org.apache.nutch.io.WritableComparator getKeyComparator()
Description copied from class: MapFilePersister
Get comparator for MapFile key class.

Specified by:
getKeyComparator in class MapFilePersister<org.apache.nutch.io.MD5Hash,org.apache.nutch.io.NullWritable>
Returns:

getKeyInstance

public org.apache.nutch.io.WritableComparable getKeyInstance(org.apache.nutch.io.MD5Hash md5Hash,
                                                             boolean throwaway)
Description copied from class: MapFilePersister
Return a new instance of the key.

Specified by:
getKeyInstance in class MapFilePersister<org.apache.nutch.io.MD5Hash,org.apache.nutch.io.NullWritable>
Returns:

getTypeComparator

public java.util.Comparator getTypeComparator()
Description copied from class: MapFilePersister
Get comparator for type.

Specified by:
getTypeComparator in class MapFilePersister<org.apache.nutch.io.MD5Hash,org.apache.nutch.io.NullWritable>
Returns: