org.supermind.crawl
Class NutchSegmentPostFetchProcessor

java.lang.Object
  extended by org.supermind.crawl.NutchSegmentPostFetchProcessor
All Implemented Interfaces:
PostFetchProcessor

public class NutchSegmentPostFetchProcessor
extends java.lang.Object
implements PostFetchProcessor

Persists a fetched page to a Nutch segment.


Field Summary
(package private) static java.util.logging.Logger LOG
           
 
Fields inherited from interface org.supermind.crawl.PostFetchProcessor
NO_OP
 
Constructor Summary
NutchSegmentPostFetchProcessor()
           
 
Method Summary
 void close()
          Cleanup.
 void process(FetcherOutput fo, org.apache.nutch.protocol.Content content, org.apache.nutch.parse.Parse parse)
          Processes a fetched page.
 void setDirectory(java.lang.String directory)
           
 void setNfs(org.apache.nutch.fs.NutchFileSystem nfs)
           
 void setOverwrite(boolean overwrite)
           
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

LOG

static java.util.logging.Logger LOG
Constructor Detail

NutchSegmentPostFetchProcessor

public NutchSegmentPostFetchProcessor()
Method Detail

close

public void close()
           throws java.io.IOException
Description copied from interface: PostFetchProcessor
Cleanup.

Specified by:
close in interface PostFetchProcessor
Throws:
java.io.IOException

process

public void process(FetcherOutput fo,
                    org.apache.nutch.protocol.Content content,
                    org.apache.nutch.parse.Parse parse)
             throws java.io.IOException
Description copied from interface: PostFetchProcessor
Processes a fetched page.

Specified by:
process in interface PostFetchProcessor
parse - Parse data, can be null if parse failed
Throws:
java.io.IOException

setDirectory

public void setDirectory(java.lang.String directory)

setNfs

public void setNfs(org.apache.nutch.fs.NutchFileSystem nfs)

setOverwrite

public void setOverwrite(boolean overwrite)