resistics.time.reader_phoenix module

class resistics.time.reader_phoenix.TimeReaderPhoenix(dataPath: str)

Bases: resistics.time.reader.TimeReader

Data reader for Phoenix data

The Phoenix data and recording format is different and does not nicely fit with the way resistics tries to model data.

There are three frequencies recorded concurrently (e.g. 2400Hz, 150Hz, 15Hz). The lowest sampling frequency is continuous whilst the others record data files at regular intervals. There is no issue with the continous sampling frequency.

However, as resistics separates out data into continuous recordings, the consistent gaps for the higher frequencies will lead to lots of small data folders if converted to internal data format.

This class returns the lowest frequency recording (the continuous one) when time series data is requested. However, higher frequencies can be converted to the internal data format using the methods available here.

Warning

The appropriate scaling for Phoenix data to return field units has not yet been verified.

It is not actually recommended to reformat the high frequency recordings as this will lead to potentially thousands of data folders. There is currently no straight-forward way to support the high-frequency Phoenix recordings.

Notes

Phoenix data is stored in 3 bytes two’s-complement format.

Attributes

recChannelsDict

Channels in each data file

dtypenp.float32

The data type

numHeaderFilesint

The number of header files

numDataFilesint

The number of data files

Methods

setParameters()	Set data reader parameters for Phoenix files
getSamplesRatesTS()	Get the sampling frequencies of the time series data
getNumberSamplesTS()	Get the number of samples for each time series file
getUnscaledSamples(**kwargs)	Get raw data from data file
getRecordsForSamples(startSample, endSample)	Get the records to read for a sample range
readTag(dataFile)	Read the tag from a data file
readRecord(dataFile, numChans, numScans)	Read numScans from a record
twosComplement(dataBytes)	Read the two’s complement data from the file
getPhysicalSamples(**kwargs)	Get data scaled to physical values
chanDefaults()	Get defaults for channel headers
readHeader()	Read header file
readTable()	Read table file
removeControl(inBytes)	Remove control characters from a byte string
headersFromTable(tableData)	Parse the information in the table file to get headers
getDates(tableData)	Get recording dates (start and end time)
checkSamples()	Check the number of samples for all the timeseries (ts) files
reformatHigh(path, **kwargs)	Write out high frequency time series in internal format
reformatContinuous(path)	Write out the continuous time series in internal format
reformat(path)	Write out all recorded time series to internal format
printDataFileList()	Information about the data files as a list of strings
printDataFileInfo()	Print a list of the data files
printTableFileList()	Information about the table file as a list of strings
printTableFileInfo()	Print table file info

chanDefaults(self)

Get defaults for channel headers

Returns

Dict[str, Any]

Dictionary of headers for channels and default values

checkSamples(self) → None

Check the number of samples for all the timeseries (ts) files

Recall, the format is 3 bytes two’s complement per sample

getDates(self, tableData) → Tuple[str, str, str, str]

Get recording dates (start and end time)

Parameters

tableDataOrderedDictDict

Ordered dictionary with table data

Returns

firstDatestr

Date of first sample as string

firstTimestr

Time of first sample as string

lastDatestr

Date of last sample as string

lastTimestr

Time of last sample as string

getNumberSamplesTS(self) → Dict

Get the number of samples for each time series file

Returns

Dict

Dictionary with the time series file number as keys and their number of samples as values

getPhysicalSamples(self, **kwargs) → resistics.time.data.TimeData

Get data scaled to physical values

Parameters

chansList[str]

List of channels to return if not all are required

startSampleint

First sample to return

endSampleint

Last sample to return

remaveragebool

Remove average from the data

remzerosbool

Remove zeroes from the data

remnans: bool

Remove NanNs from the data

Returns

TimeData

Time data object

getRecordsForSamples(self, startSample: int, endSample: int) → Tuple[List, List]

Get the records to read for a sample range

Parameters

startSampleint

The starting sample of the range

endSampleint

The ending sample of the range

Returns

recordsToReadList

The records to read from the time series data files

samplesToReadList

The samples to read from each record

getSamplesRatesTS(self) → Dict

Get the sampling frequencies of the time series data

Returns

Dict

Dictionary with the time series file number as keys and their sampling frequencies in Hz as values

getUnscaledSamples(self, **kwargs) → resistics.time.data.TimeData

Get raw data from data file

Only returns the continuous data. The continuous data is in 24 bit two’s complement (3 bytes) format and is read in using struct as this is not supported by numpy.

Parameters

chansList[str], optional

List of channels to return if not all are required

startSampleint, optional

First sample to return

endSampleint, optional

Last sample to return

Returns

TimeData

Time data object

headersFromTable(self, tableData: Dict) → Tuple[Dict, List]

Populate the headers from the table values

Parameters

tableDataOrderedDictDict

Ordered dictionary with table data

Returns

headersDict

Dictionary of general headers

chanHeadersDict

List of channel headers

printDataFileInfo(self)

Print a list of the data files

printDataFileList(self) → List[str]

Information about the data files as a list of strings

Returns

List[str]

List of information about the data files

printTableFileInfo(self)

Print table file info

printTableFileList(self) → List[str]

Information about the table file as a list of strings

Returns

List[str]

List of information about table file content

readHeader(self)

Read header file

For phoenix data, the header file is the table file and it is binary formatted.

readRecord(self, dataFile, numChans, numScans)

Read numScans from a record

Parameters

dataFilefile handle

File handle of the data file

numScansList

Number of scans in the tag

numChansList

Number of channels in the tag

Returns

datanp.ndarray(int)

Record data

readTable(self) → Dict

Read a header table

Returns

OrderedDict

An ordered dictionary of header table data

readTag(self, dataFile) → Tuple[str]

Read the tag from a data file

Parameters

dataFilefile handle

File handle of the data file

Returns

numScansList

Number of scans in the tag

numChansList

Number of channels in the tag

dateStringstr

The dataString of the tag

reformat(self, path)

Write out all recorded time series to internal format

Parameters

pathstr

Path to write out reformatted recordings

reformatContinuous(self, path: str)

Write out the continuous time series in internal format

Parameters

pathstr

Path to write out reformatted continuous recording

reformatHigh(self, path: str, **kwargs) → None

Write out high frequency time series in internal format

Parameters

pathstr

Directory to write out the reformatted time series

tsList[int], optional

A list of the high frequency ts files to reformat. By default, all of the higher frequency recordings are reformatted

removeControl(self, inBytes: bytes) → str

Remove control characters from byte strings

Parameters

inBytesbytes

Bytes from which to remove control

Returns

str :

Decodes bytes object with control character removed

setParameters(self) → None

Set data reader parameters for Phoenix files

Phoenix time series data is not contiguous in the file and is separated into records. There are multiple time series data files, one for the continuous recording and two others for the other frequencies. Therefore, there are a few other class variables defined here than in the parent DataReader class.

twosComplement(self, dataBytes)

Read the two’s complement data from the file

This parses two’s complement 24-bit integer, little endian, unsigned and signed. The method is to pad out 3 bytes out with a null byte and read as unsigned integer with little endian (<).

Parameters

dataByesbytes

The bytes to parse

Returns

datanp.ndarray(int)

Record data