org.opendatafoundation.data.spss

Class SPSSFile

java.lang.Object

java.io.RandomAccessFile

org.opendatafoundation.data.spss.SPSSFile

All Implemented Interfaces:

Closeable, DataInput, DataOutput, AutoCloseable

public class SPSSFile
extends RandomAccessFile

A class to read SPSS files, produce DDI compatible metadata and export the data to ASCII.

Version:

2007.07

Author:

Pascal Heus (pheus@opendatafoundation.org)

Field Summary

Fields

Modifier and Type	Field and Description
String	categorySchemeIDSuffix
String	codeSchemeIDSuffix
String	dataRelationshipID
static String	DDI2_NAMESPACE
static String	DDI3_ARCHIVE_NAMESPACE
static String	DDI3_INSTANCE_NAMESPACE
static String	DDI3_LOGICAL_PRODUCT_NAMESPACE
static String	DDI3_PHYSICAL_INSTANCE_NAMESPACE
static String	DDI3_PHYSICAL_PRODUCT_NAMESPACE
static String	DDI3_PROPRIETARY_RECORD_NAMESPACE
static String	DDI3_REUSABLE_NAMESPACE
static String	DDI3_STUDY_UNIT_NAMESPACE
File	file
String	grossFileID
String	grossRecordStructureID
boolean	isDataLoaded
boolean	isMetadataLoaded
File	logFile
Boolean	logFlag
String	logicalProductIDSuffix
String	logicalRecordID
String	physicalDataProductIDSuffix
String	physicalInstanceFileID
String	physicalInstanceIDSuffix
String	physicalRecordSegmentID
String	physicalStructureID
String	physicalStructureSchemeSuffix
String	recordLayoutSchemeSuffix
String	variableCategoryPrefix
String	variableIDPrefix
String	variableSchemeIDSuffix
static int	VERSION_MAJOR
static int	VERSION_MINOR

Constructor Summary

Constructors

Constructor and Description
SPSSFile(File file) Constructor
SPSSFile(File file, Charset charset) Constructor
SPSSFile(File file, String mode) Constructor
SPSSFile(File file, String mode, Charset charset) Constructor
SPSSFile(String name) Constructor
SPSSFile(String name, Charset charset) Constructor
SPSSFile(String name, String mode) Constructor
SPSSFile(String name, String mode, Charset charset) Constructor

Method Summary

Modifier and Type	Method and Description
void	dumpData() Dumps the file data to the console (for debugging purposes)
void	dumpData(int nRecords) Dumps the file data to the console (for debugging purposes)
void	dumpData(int nRecords, FileFormatInfo dataFormat) Dumps the file data (for debugging purposes)
void	dumpDDI2() Dumps the DDI2 metadata (for debugging purposes).
void	dumpDDI3() Dumps the DDI3 metadata(for debugging purposes).
void	dumpMetadata() Dumps the file metadata to the console (for debugging purposes)
long	exportData(File file, FileFormatInfo dataFormat) Reads the data from the disk and exports a file based on the specified format
Document	getDDI2() Creates a DDI 2.0 XML Document based for the SPSS data format
Document	getDDI2(FileFormatInfo dataFormat) Creates a DDI 2.0 XML Document based for the specified data file format
String	getDDI3DefaultLogicalProductID() Returns a default Physical Data Product identifier based on the file unique identifier.
String	getDDI3DefaultPhysicalDataProductID(FileFormatInfo dataFormat) Returns a default Physical Data Product identifier based on the file unique identifier and the data format.
String	getDDI3DefaultPhysicalInstanceID(FileFormatInfo dataFormat) Returns a default Physical Instance identifier based on the file unique identifier and the data format.
String	getDDI3DefaultPhysicalStructureSchemeID(FileFormatInfo dataFormat) Returns a default Physical Structure Scheme identifier based on the file unique identifier and the data format.
String	getDDI3DefaultRecordLayoutSchemeID(FileFormatInfo dataFormat) Returns a default Record Layout Scheme identifier based on the file unique identifier and the data format.
String	getDDI3DefaultVariableSchemeID() Returns a default Variable Scheme identifier based on the file unique identifier.
Document	getDDI3LogicalProduct() Creates a DDI 3.0 Logical Product XML Module for this file.
Document	getDDI3LogicalProduct(String uniqueID) Creates a DDI 3.0 Logical Product XML Module for this file.
Document	getDDI3LogicalProduct(String uniqueID, String identifyingAgency) Creates a DDI 3.0 LogiclaProduct XML Document for this file.
Document	getDDI3PhysicalDataProduct(FileFormatInfo dataFormat) Creates a DDI 3.0 PhysicalDataProduct XML Document for this file based on the specified file format using the default logical product ID.
Document	getDDI3PhysicalDataProduct(FileFormatInfo dataFormat, String logicalProductID) Creates a DDI 3.0 PhysicalDataProduct XML Document for this file based on the specified file format.
Document	getDDI3PhysicalDataProduct(FileFormatInfo dataFormat, String logicalProductID, String uniqueID) Creates a DDI 3.0 PhysicalDataProduct XML Document for this file based on the specified file format.
Document	getDDI3PhysicalInstance(URI uri, FileFormatInfo dataFormat) Creates a DDI 3.0 PhysicalInstance XML Document for this file based on the specified file format using default indentifiers.
Document	getDDI3PhysicalInstance(URI uri, FileFormatInfo dataFormat, String recordLayoutSchemeID) Creates a DDI 3.0 PhysicalInstance XML Document for this file based on the specified file format.
Document	getDDI3PhysicalInstance(URI uri, FileFormatInfo dataFormat, String recordLayoutSchemeID, String uniqueID) Creates a DDI 3.0 PhysicalInstance XML Document for this file based on the specified file format.
String	getRecord(int obsNumber, FileFormatInfo dataFormat) Gets a data record from data in memory based on on the record number and specified format.
int	getRecordCount() Returns the total number of records (cases) in the file.
String	getRecordFromDisk(FileFormatInfo dataFormat, boolean rewind) Gets a data record in the specified format.
String	getUniqueID() Gets the unique identifier for this file.
SPSSVariable	getVariable(int index) Gets a SPSSVariable based on its 0-based file index position.
int	getVariableCount() Returns the total number of variables in the file.
boolean	isCompressed() Determines if the data section of the file is compressed
void	loadData() Load the data section of the file into the variables in memory.
void	loadMetadata() Loads the dictionary and other SPSS metadata from the file
void	log(String msg) Logs a message to the console.
double	readSPSSDouble() Reads a 8-byte IEEE 754 from SPSS file
int	readSPSSInt() Reads a 4-byte integer from the SPSS file
String	readSPSSString(int length) Reads a string from the SPSS file
void	setUniqueID(String str) Sets the unique identifier for this file.

Methods inherited from class java.io.RandomAccessFile

close, getChannel, getFD, getFilePointer, length, read, read, read, readBoolean, readByte, readChar, readDouble, readFloat, readFully, readFully, readInt, readLine, readLong, readShort, readUnsignedByte, readUnsignedShort, readUTF, seek, setLength, skipBytes, write, write, write, writeBoolean, writeByte, writeBytes, writeChar, writeChars, writeDouble, writeFloat, writeInt, writeLong, writeShort, writeUTF

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

VERSION_MAJOR

public static final int VERSION_MAJOR

See Also:

Constant Field Values

VERSION_MINOR

public static final int VERSION_MINOR

See Also:

Constant Field Values

DDI2_NAMESPACE

public static final String DDI2_NAMESPACE

See Also:

Constant Field Values

DDI3_ARCHIVE_NAMESPACE

public static final String DDI3_ARCHIVE_NAMESPACE

See Also:

Constant Field Values

DDI3_INSTANCE_NAMESPACE

public static final String DDI3_INSTANCE_NAMESPACE

See Also:

Constant Field Values

DDI3_LOGICAL_PRODUCT_NAMESPACE

public static final String DDI3_LOGICAL_PRODUCT_NAMESPACE

See Also:

Constant Field Values

DDI3_PHYSICAL_PRODUCT_NAMESPACE

public static final String DDI3_PHYSICAL_PRODUCT_NAMESPACE

See Also:

Constant Field Values

DDI3_PHYSICAL_INSTANCE_NAMESPACE

public static final String DDI3_PHYSICAL_INSTANCE_NAMESPACE

See Also:

Constant Field Values

DDI3_PROPRIETARY_RECORD_NAMESPACE

public static final String DDI3_PROPRIETARY_RECORD_NAMESPACE

See Also:

Constant Field Values

DDI3_REUSABLE_NAMESPACE

public static final String DDI3_REUSABLE_NAMESPACE

See Also:

Constant Field Values

DDI3_STUDY_UNIT_NAMESPACE

public static final String DDI3_STUDY_UNIT_NAMESPACE

See Also:

Constant Field Values

categorySchemeIDSuffix

public String categorySchemeIDSuffix

codeSchemeIDSuffix

public String codeSchemeIDSuffix

dataRelationshipID

public String dataRelationshipID

grossFileID

public String grossFileID

grossRecordStructureID

public String grossRecordStructureID

logicalRecordID

public String logicalRecordID

physicalRecordSegmentID

public String physicalRecordSegmentID

logicalProductIDSuffix

public String logicalProductIDSuffix

physicalDataProductIDSuffix

public String physicalDataProductIDSuffix

physicalStructureSchemeSuffix

public String physicalStructureSchemeSuffix

physicalStructureID

public String physicalStructureID

recordLayoutSchemeSuffix

public String recordLayoutSchemeSuffix

physicalInstanceIDSuffix

public String physicalInstanceIDSuffix

physicalInstanceFileID

public String physicalInstanceFileID

variableSchemeIDSuffix

public String variableSchemeIDSuffix

variableCategoryPrefix

public String variableCategoryPrefix

variableIDPrefix

public String variableIDPrefix

logFlag

public Boolean logFlag

logFile

public File logFile

file

public File file

isMetadataLoaded

public boolean isMetadataLoaded

isDataLoaded

public boolean isDataLoaded

Constructor Detail

SPSSFile

public SPSSFile(File file)
         throws FileNotFoundException

Constructor

Parameters:

file -

Throws:

FileNotFoundException

SPSSFile

public SPSSFile(File file,
                String mode)
         throws FileNotFoundException

Constructor

Parameters:

file -

mode -

Throws:

FileNotFoundException

SPSSFile

public SPSSFile(String name)
         throws FileNotFoundException

Constructor

Parameters:

name -

Throws:

FileNotFoundException

SPSSFile

public SPSSFile(String name,
                String mode)
         throws FileNotFoundException

Constructor

Parameters:

name -

mode -

Throws:

FileNotFoundException

SPSSFile

public SPSSFile(File file,
                Charset charset)
         throws FileNotFoundException

Constructor

Parameters:

file -

charset -

Throws:

FileNotFoundException

SPSSFile

public SPSSFile(File file,
                String mode,
                Charset charset)
         throws FileNotFoundException

Constructor

Parameters:

file -

mode -

charset -

Throws:

FileNotFoundException

SPSSFile

public SPSSFile(String name,
                Charset charset)
         throws FileNotFoundException

Constructor

Parameters:

name -

charset -

Throws:

FileNotFoundException

SPSSFile

public SPSSFile(String name,
                String mode,
                Charset charset)
         throws FileNotFoundException

Constructor

Parameters:

name -

mode -

charset -

Throws:

FileNotFoundException

Method Detail

dumpData

public void dumpData()
              throws SPSSFileException,
                     IOException

Dumps the file data to the console (for debugging purposes)

Throws:

IOException

SPSSFileException

dumpData

public void dumpData(int nRecords)
              throws SPSSFileException,
                     IOException

Dumps the file data to the console (for debugging purposes)

Parameters:

nRecords - the number of records to dump (0 for all)

Throws:

IOException

SPSSFileException

dumpData

public void dumpData(int nRecords,
                     FileFormatInfo dataFormat)
              throws SPSSFileException,
                     IOException

Dumps the file data (for debugging purposes)

Parameters:

nRecords - the number of records to dump (0 for all)

dataFormat -

Throws:

IOException

SPSSFileException

dumpDDI2

public void dumpDDI2()
              throws SPSSFileException,
                     TransformerException

Dumps the DDI2 metadata (for debugging purposes).

Throws:

SPSSFileException

TransformerException

dumpDDI3

public void dumpDDI3()
              throws SPSSFileException,
                     TransformerException

Dumps the DDI3 metadata(for debugging purposes).

Throws:

SPSSFileException

TransformerException

dumpMetadata

public void dumpMetadata()

Dumps the file metadata to the console (for debugging purposes)

exportData

public long exportData(File file,
                       FileFormatInfo dataFormat)
                throws IOException,
                       SPSSFileException

Reads the data from the disk and exports a file based on the specified format

Parameters:

file -

dataFormat -

Returns:

The number of milliseconds taken to export the file

Throws:

SPSSFileException

IOException

isCompressed

public boolean isCompressed()
                     throws SPSSFileException

Determines if the data section of the file is compressed

Throws:

SPSSFileException

getDDI2

public Document getDDI2()
                 throws SPSSFileException

Creates a DDI 2.0 XML Document based for the SPSS data format

Returns:

the generated document

Throws:

SPSSFileException

getDDI2

public Document getDDI2(FileFormatInfo dataFormat)
                 throws SPSSFileException

Creates a DDI 2.0 XML Document based for the specified data file format

Returns:

the generated document

Throws:

SPSSFileException

getDDI3DefaultLogicalProductID

public String getDDI3DefaultLogicalProductID()

Returns a default Physical Data Product identifier based on the file unique identifier.

Returns:

a String containing the r:ID

getDDI3DefaultPhysicalDataProductID

public String getDDI3DefaultPhysicalDataProductID(FileFormatInfo dataFormat)

Returns a default Physical Data Product identifier based on the file unique identifier and the data format.

Returns:

a String containing the r:ID

getDDI3DefaultPhysicalInstanceID

public String getDDI3DefaultPhysicalInstanceID(FileFormatInfo dataFormat)

Returns a default Physical Instance identifier based on the file unique identifier and the data format.

Parameters:

dataFormat -

Returns:

a String containing the r:ID

getDDI3DefaultPhysicalStructureSchemeID

public String getDDI3DefaultPhysicalStructureSchemeID(FileFormatInfo dataFormat)

Returns a default Physical Structure Scheme identifier based on the file unique identifier and the data format.

Returns:

a String containing the r:ID

getDDI3DefaultRecordLayoutSchemeID

public String getDDI3DefaultRecordLayoutSchemeID(FileFormatInfo dataFormat)

Returns a default Record Layout Scheme identifier based on the file unique identifier and the data format.

Returns:

a String containing the r:ID

getDDI3DefaultVariableSchemeID

public String getDDI3DefaultVariableSchemeID()

Returns a default Variable Scheme identifier based on the file unique identifier.

Returns:

a String containing the r:ID

getDDI3LogicalProduct

public Document getDDI3LogicalProduct()
                               throws SPSSFileException

Creates a DDI 3.0 Logical Product XML Module for this file.

Returns:

a Document containing the generated Logical Data Product

Throws:

SPSSFileException

getDDI3LogicalProduct

public Document getDDI3LogicalProduct(String uniqueID)
                               throws SPSSFileException

Creates a DDI 3.0 Logical Product XML Module for this file.

Parameters:

uniqueID - a String value for the r:ID element. If null, a unique ID will be generated automatically based on the java.util.UUID class

Returns:

a org.w3c.dom.Document containing the generated Logical Data Product

Throws:

SPSSFileException

getDDI3LogicalProduct

public Document getDDI3LogicalProduct(String uniqueID,
                                      String identifyingAgency)
                               throws SPSSFileException

Creates a DDI 3.0 LogiclaProduct XML Document for this file.

Parameters:

uniqueID - a String value for the r:ID element. If null, a unique ID will be generated automatically based on the java.util.UUID class

identifyingAgency -

Returns:

a org.w3c.dom.Document containing the generated Logical Data Product

Throws:

SPSSFileException

getDDI3PhysicalDataProduct

public Document getDDI3PhysicalDataProduct(FileFormatInfo dataFormat)
                                    throws SPSSFileException

Creates a DDI 3.0 PhysicalDataProduct XML Document for this file based on the specified file format using the default logical product ID.

Parameters:

dataFormat -

Returns:

a org.w3c.dom.Document containing the generated Physical Data Product

Throws:

SPSSFileException

getDDI3PhysicalDataProduct

public Document getDDI3PhysicalDataProduct(FileFormatInfo dataFormat,
                                           String logicalProductID)
                                    throws SPSSFileException

Creates a DDI 3.0 PhysicalDataProduct XML Document for this file based on the specified file format.

Parameters:

dataFormat -

logicalProductID - the r:ID of the Logical Product this is refering to

Returns:

a org.w3c.dom.Document containing the generated Physical Data Product

Throws:

SPSSFileException

getDDI3PhysicalDataProduct

public Document getDDI3PhysicalDataProduct(FileFormatInfo dataFormat,
                                           String logicalProductID,
                                           String uniqueID)
                                    throws SPSSFileException

Creates a DDI 3.0 PhysicalDataProduct XML Document for this file based on the specified file format.

Parameters:

dataFormat -

logicalProductID - the r:ID of the Logical Product this is referring to

uniqueID - a String value for the r:ID element. If null, a unique ID will be generated automatically based on the java.util.UUID class

Returns:

a org.w3c.dom.Document containing the generated Physical Data Product

Throws:

SPSSFileException

getDDI3PhysicalInstance

public Document getDDI3PhysicalInstance(URI uri,
                                        FileFormatInfo dataFormat)
                                 throws SPSSFileException

Creates a DDI 3.0 PhysicalInstance XML Document for this file based on the specified file format using default indentifiers.

Parameters:

uri -

dataFormat -

Returns:

a org.w3c.dom.Document containing the generated Physical Data Instance

Throws:

SPSSFileException

getDDI3PhysicalInstance

public Document getDDI3PhysicalInstance(URI uri,
                                        FileFormatInfo dataFormat,
                                        String recordLayoutSchemeID)
                                 throws SPSSFileException

Creates a DDI 3.0 PhysicalInstance XML Document for this file based on the specified file format.

Parameters:

uri -

dataFormat -

recordLayoutSchemeID - the Record Layout Scheme Data Product this instance refers to

Returns:

a org.w3c.dom.Document containing the generated Physical Data Instance

Throws:

SPSSFileException

getDDI3PhysicalInstance

public Document getDDI3PhysicalInstance(URI uri,
                                        FileFormatInfo dataFormat,
                                        String recordLayoutSchemeID,
                                        String uniqueID)
                                 throws SPSSFileException

Creates a DDI 3.0 PhysicalInstance XML Document for this file based on the specified file format.

Parameters:

uri -

dataFormat -

recordLayoutSchemeID - the Record Layout Scheme Data Product this instance refers to

uniqueID - a String value for the r:ID element. If null, a unique ID will be generated automatically based on the java.util.UUID class

Returns:

a org.w3c.dom.Document containing the generated Physical Data Instance

Throws:

SPSSFileException

getRecord

public String getRecord(int obsNumber,
                        FileFormatInfo dataFormat)
                 throws SPSSFileException

Gets a data record from data in memory based on on the record number and specified format.

Parameters:

obsNumber -

dataFormat -

Returns:

A string holding the record values

Throws:

SPSSFileException

getRecordFromDisk

public String getRecordFromDisk(FileFormatInfo dataFormat,
                                boolean rewind)
                         throws SPSSFileException,
                                IOException

Gets a data record in the specified format. If rewind is false, this assumes the file pointer is ta the correct location

Parameters:

dataFormat -

rewind - If true start reading from first record otherwise continue from current location until end of file is reached

Returns:

A string holding the record values

Throws:

SPSSFileException

IOException

getRecordCount

public int getRecordCount()

Returns the total number of records (cases) in the file.

Returns:

the number of records in the file

getUniqueID

public String getUniqueID()

Gets the unique identifier for this file. If this value is not set, a unique string will be generated using java.util.UUID.randomUUID()

Returns:

a String holding the identifier

getVariable

public SPSSVariable getVariable(int index)

Gets a SPSSVariable based on its 0-based file index position.

Returns:

SPSSVariable

Throws:

SPSSFileException

getVariableCount

public int getVariableCount()

Returns the total number of variables in the file.

Returns:

the number of records in the file

loadData

public void loadData()
              throws IOException,
                     SPSSFileException

Load the data section of the file into the variables in memory. This may be expensive on memory, use with care on large datasets

Throws:

SPSSFileException

IOException

loadMetadata

public void loadMetadata()
                  throws FileNotFoundException,
                         IOException,
                         SPSSFileException

Loads the dictionary and other SPSS metadata from the file

Throws:

FileNotFoundException

IOException

SPSSFileException

log

public void log(String msg)

Logs a message to the console.

Parameters:

msg -

Throws:

IOException

readSPSSDouble

public double readSPSSDouble()
                      throws IOException

Reads a 8-byte IEEE 754 from SPSS file

Returns:

the double value read from the file

Throws:

IOException

readSPSSInt

public int readSPSSInt()
                throws IOException

Reads a 4-byte integer from the SPSS file

Returns:

the integer value read from the file

Throws:

IOException

readSPSSString

public String readSPSSString(int length)
                      throws IOException

Reads a string from the SPSS file

Parameters:

length - Number of characters to read

Returns:

the String value read from the file

Throws:

IOException

setUniqueID

public void setUniqueID(String str)

Sets the unique identifier for this file.

Parameters:

str -