org.supermind.crawl
Class NutchSegmentPostFetchProcessor
java.lang.Object
org.supermind.crawl.NutchSegmentPostFetchProcessor
- All Implemented Interfaces:
- PostFetchProcessor
public class NutchSegmentPostFetchProcessor
- extends java.lang.Object
- implements PostFetchProcessor
Persists a fetched page to a Nutch segment.
Field Summary |
(package private) static java.util.logging.Logger |
LOG
|
Method Summary |
void |
close()
Cleanup. |
void |
process(FetcherOutput fo,
org.apache.nutch.protocol.Content content,
org.apache.nutch.parse.Parse parse)
Processes a fetched page. |
void |
setDirectory(java.lang.String directory)
|
void |
setNfs(org.apache.nutch.fs.NutchFileSystem nfs)
|
void |
setOverwrite(boolean overwrite)
|
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
LOG
static java.util.logging.Logger LOG
NutchSegmentPostFetchProcessor
public NutchSegmentPostFetchProcessor()
close
public void close()
throws java.io.IOException
- Description copied from interface:
PostFetchProcessor
- Cleanup.
- Specified by:
close
in interface PostFetchProcessor
- Throws:
java.io.IOException
process
public void process(FetcherOutput fo,
org.apache.nutch.protocol.Content content,
org.apache.nutch.parse.Parse parse)
throws java.io.IOException
- Description copied from interface:
PostFetchProcessor
- Processes a fetched page.
- Specified by:
process
in interface PostFetchProcessor
parse
- Parse data, can be null if parse failed
- Throws:
java.io.IOException
setDirectory
public void setDirectory(java.lang.String directory)
setNfs
public void setNfs(org.apache.nutch.fs.NutchFileSystem nfs)
setOverwrite
public void setOverwrite(boolean overwrite)