edu.harvard.hul.ois.jhove.module

Class WarcModule

java.lang.Object

edu.harvard.hul.ois.jhove.ModuleBase

edu.harvard.hul.ois.jhove.module.WarcModule

All Implemented Interfaces:

Module

public class WarcModule
extends ModuleBase

JHOVE module for identifying, validating and characterizing WARC files. Ported from the JHOVE2 WARC module and based on the JWAT-tool, both created by [email protected] (nclarkekb@git). This is a non-recursive validation. It only validates the WARC file format and WARC headers, not the actual payload of the WARC records.

Author:

[email protected]

Field Summary

Fields inherited from class edu.harvard.hul.ois.jhove.ModuleBase

_app, _bigEndian, _checksumFinished, _countStream, _coverage, _crc32, _date, _defaultParams, _features, _format, _init, _isRandomAccess, _je, _logger, _md5, _mimeType, _name, _nByte, _note, _param, _release, _repInfoNote, _rights, _sha1, _signature, _specification, _validityNote, _vendor, _verbosity, _wellFormedNote

Fields inherited from interface edu.harvard.hul.ois.jhove.Module

MAXIMUM_VERBOSITY, MINIMUM_VERBOSITY

Constructor Summary

Constructors

Constructor and Description
WarcModule() Constructor.

Method Summary

Modifier and Type	Method and Description
int	parse(InputStream stream, RepInfo info, int parseIndex) Parse the content of a stream digital object and store the results in RepInfo.
void	parse(RandomAccessFile file, RepInfo info) Parse the content of a random access digital object and store the results in RepInfo.
protected void	parseRecords(org.jwat.warc.WarcReader reader) Parse WARC records.
protected void	processRecord(org.jwat.warc.WarcRecord record) Process a WARC record.
void	resetParams() Reset parameter settings.
protected void	setReaderOptions(org.jwat.warc.WarcReader reader) Set digest options for WARC reader.

Methods inherited from class edu.harvard.hul.ois.jhove.ModuleBase

addIntegerProperty, addIntegerProperty, applyDefaultParams, calcRAChecksum, checkSignatures, checkSignatures, getApp, getBase, getBufferedDataStream, getCoverage, getCRC32, getDate, getDefaultParams, getFeatures, getFormat, getMimeType, getName, getNByte, getNote, getRelease, getRepInfoNote, getRights, getSignature, getSpecification, getValidityNote, getVendor, getWellFormedNote, hasFeature, init, initFeatures, initParse, isBigEndian, isRandomAccess, param, readByteBuf, readDouble, readDouble, readDouble, readFloat, readFloat, readSignedByte, readSignedByte, readSignedByte, readSignedInt, readSignedInt, readSignedInt, readSignedLong, readSignedRational, readSignedRational, readSignedShort, readSignedShort, readSignedShort, readUnsignedByte, readUnsignedByte, readUnsignedByte, readUnsignedInt, readUnsignedInt, readUnsignedInt, readUnsignedRational, readUnsignedRational, readUnsignedRational, readUnsignedShort, readUnsignedShort, readUnsignedShort, setApp, setBase, setChecksums, setCRC32, setDefaultParams, setMD5, setNByte, setSHA1, setValidityNote, setVerbosity, show, skipBytes, skipBytes, vectorToPropArray

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

WarcModule

public WarcModule()

Constructor.

Method Detail

resetParams

public void resetParams()
                 throws Exception

Reset parameter settings. Returns to a default state without any parameters.

Specified by:

resetParams in interface Module

Overrides:

resetParams in class ModuleBase

Throws:

Exception

parse

public void parse(RandomAccessFile file,
                  RepInfo info)
           throws IOException

Description copied from class: ModuleBase

Parse the content of a random access digital object and store the results in RepInfo. A given Module will normally override only one of the two parse methods; the default method does nothing.

Specified by:

parse in interface Module

Overrides:

parse in class ModuleBase

Parameters:

file - A RandomAccessFile, positioned at its beginning, which is generated from the object to be parsed

info - A fresh RepInfo object which will be modified to reflect the results of the parsing

Throws:

IOException

parse

public int parse(InputStream stream,
                 RepInfo info,
                 int parseIndex)
          throws IOException

Description copied from class: ModuleBase

Parse the content of a stream digital object and store the results in RepInfo. A given Module will normally override only one of the two parse methods; the default method does nothing.

Specified by:

parse in interface Module

Overrides:

parse in class ModuleBase

Parameters:

stream - An InputStream, positioned at its beginning, which is generated from the object to be parsed. If multiple calls to parse are made on the basis of a nonzero value being returned, a new InputStream must be provided each time.

info - A fresh (on the first call) RepInfo object which will be modified to reflect the results of the parsing If multiple calls to parse are made on the basis of a nonzero value being returned, the same RepInfo object should be passed with each call.

parseIndex - Must be 0 in first call to parse. If parse returns a nonzero value, it must be called again with parseIndex equal to that return value.

Throws:

IOException

setReaderOptions

protected void setReaderOptions(org.jwat.warc.WarcReader reader)
                         throws JhoveException

Set digest options for WARC reader.

Parameters:

reader - WARC reader instance

Throws:

JhoveException

parseRecords

protected void parseRecords(org.jwat.warc.WarcReader reader)
                     throws IOException,
                            JhoveException

Parse WARC records. Parsing should be straight forward with all records accessible through the same source.

Parameters:

reader - WARC reader used to parse records

Throws:

EOFException - if EOF occurs prematurely

IOException - if an IO error occurs while processing

JhoveException - if a serious problem needs to be reported

processRecord

protected void processRecord(org.jwat.warc.WarcRecord record)
                      throws IOException,
                             JhoveException

Process a WARC record. Does not characterize the record payload.

Parameters:

record - WARC record from WARC reader

Throws:

EOFException - if EOF occurs prematurely

IOException - if an IO error occurs while processing

JhoveException - if a serious problem needs to be reported