resistics.time.reader_lemib423 module

class resistics.time.reader_lemib423.TimeReaderLemiB423(dataPath: str)[source]

Bases: resistics.time.reader_internal.TimeReaderInternal

Data reader for Lemi B423 data

Lemi B423 data has the following characteristics:

  • 1024 bytes of ASCII headers in the data file with basic scaling information

  • There is no separate header file for Lemi B423 data. No information for number of samples, sampling rate etc

  • Header files need to be constructed before Lemi B423 data can be read in by resistics. There are helper methods to do this

  • Lemi B423 raw measurement data is signed long integer format

  • Getting unscaled samples returns data with unit count for both the electric and magnetic fields.

  • Scalings specified in B423 files convert electric channels to uV (microvolt) and magnetic channels to millivolts with internal gain still applied

  • Getting physical samples converts the electric channels to mV/km by dividing the uV by 1000 and then dividing by the dipole length in km

  • For the magnetic channels, the gain is removed to give the magnetic measurements in mV. Calibrating these will give measurements in nT

In situations where a Lemi B423 dataset is recorded in multiple files, it is required that the recording is continuous.

Attributes
dtypenp.int_

The data type, a long integer

recordByteSizeint

The size of a record in bytes

numHeaderFilesint

The number of header files

numDataFilesint

The number of data files

Methods

setParameters()

Set parameters specific to a data format

getUnscaledSamples(**kwargs)

Get raw, unscaled data

getDataFilesForSamples(startSample, endSample)

Get the data files that contribute to the request data

readRecords(bts, numRecords)

Read the data records from a provided bytes object

getPhysicalSamples(**kwargs)

Get data in physical units

def readHeader()

Read the B423 measurement file headers

readMeasParams(sampleFreq)

Calculate out the a B423 parameters including start and end date and number of samples

getScalars(paramsDict)

Get the scalars for each channel as given in the data files

printDataFileList()

Get data file information as a list of strings

printDataFiles()

Print data file information to terminal
getDataFilesForSamples(self, startSample: int, endSample: int) → Tuple[List[str], List[List[int]], List[float]][source]

Get the data files that have to be read to cover the sample range

Parameters
startSampleint

Starting sample of the sample range

endSamplesint

Ending sample of the sample range

Returns
dataFilesToRead

Time data object

getPhysicalSamples(self, **kwargs)[source]

Get data scaled to physical values

resistics uses field units, meaning physical samples will return the following:

  • Electrical channels in mV/km

  • Magnetic channels in mV

  • To get magnetic fields in nT, calibration needs to be performed

Parameters
chansList[str]

List of channels to return if not all are required

startSampleint

First sample to return

endSampleint

Last sample to return

remaveragebool

Remove average from the data

remzerosbool

Remove zeroes from the data

remnans: bool

Remove NanNs from the data

Returns
TimeData

Time data object

Notes

Once Lemi B423 data is scaled (which optionally happens in getUnscaledSamples), the magnetic channels is in mV with gain applied and the electric channels is uV (microvolts). Therefore:

  • Electric channels need to divided by 1000 along with dipole length division in km (east-west spacing and north-south spacing) to return mV/km.

  • Magnetic channels need to be divided by the internal gain value which should be set in the headers

To get magnetic fields in nT, they have to be calibrated.

getScalars(self, paramsDict: Dict) → Dict[source]

Returns the scalars from a parameter dictionary

Parameters
paramsDictDict

The parameter dictionary for a data file usually read from the headers in the file

Returns
Dict

Dictionary with channels as keys and scalings as values

getUnscaledSamples(self, **kwargs) → resistics.time.data.TimeData[source]

Get raw data from data file, returned in mV

Lemi B423 data always has five channels, in order Hx, Hy, Hz, Ex, Ey. The raw data is integer counts. Therefore, getting unscaled samples returns raw counts for the measurement. There are additional scalings which can be applied using the scale optional argument. Lemi B423 is recorded in multiple files. It has not been verified whether it is possible for each individual file to have different scaling.

Without the scale option, the data is returned in:

  • Counts for both magnetic and electric channels (reading long integers)

With the scaling option, the data is returned in:

  • microvolts for the electric channels

  • millivolts for the magnetic with the gain applied

Applying the scaling does not appear to remove the internal gain of the Lemi. This will be removed when getting physical samples and the appropriate value must be set in the headers.

Parameters
chansList[str], optional

List of channels to return if not all are required

startSampleint, optional

First sample to return

endSampleint, optional

Last sample to return

scalebool, optional

Boolean flag for applying the gain scaling

Returns
TimeData

Time data object

printDataFileInfo(self) → None[source]

Print a list of the data files

printDataFileList(self) → List[str][source]

Information about the data files as a list of strings

Returns
List[str]

List of information about the data files

readHeader(self) → None[source]

Read the B423 measurement file headers

readMeasParams(self, sampleFreq: float = None) → None[source]

Get the recording parameters for each of the subfiles

Parameters
sampleFreqfloat

The sampling frequency in Hz

readRecords(self, bts: bytes, numRecords: int)[source]

Read a number of B423 records from bytes

Parameters
btsbytes

The bytes to be read

numRecordsint

The number of records to read. Size of bytes should be numRecords * recordByteSize

Returns
datanp.ndarray

Array holding the data. The data repeats in channel blocks e.g. [Hx, Hy, Hx, Ex, Ey][Hx, Hy, Hx, Ex, Ey]

setParameters(self) → None[source]

Set some data reader parameters for reading Lemi B423 data

resistics.time.reader_lemib423.folderB423Headers(folderpath: str, sampleFreq: float, hxSensor: int = 0, hySensor: int = 0, hzSensor: int = 0, hGain: int = 1, dx: float = 1, dy: float = 1, folders: List = []) → None[source]

Construct B423 headers for subfolders of a folder

Parameters
folderpathstr

The path to the folder

sampleFreqfloat

The sampling frequency of the data

hxSensorstr, optional

The x direction magnetic sensor, used for calibration

hySensorstr, optional

The y direction magnetic sensor, used for calibration

hzSensorstr, optional

The z direction magnetic sensor, used for calibration

hGainint

Any gain on the magnetic channels which will need to be removed

dxfloat, optional

Distance between x electrodes

dyfloat, optional

Distance between y electrodes

folderList, optional

An optional list of subfolders

resistics.time.reader_lemib423.measB423Headers(datapath: str, sampleFreq: float, hxSensor: int = 0, hySensor: int = 0, hzSensor: int = 0, hGain: int = 1, dx: float = 1, dy: float = 1) → None[source]

Read a single B423 measurement directory and construct headers

Parameters
datapathstr

The path to the measurement

sampleFreqfloat

The sampling frequency of the data

hxSensorstr, optional

The x direction magnetic sensor, used for calibration

hySensorstr, optional

The y direction magnetic sensor, used for calibration

hzSensorstr, optional

The z direction magnetic sensor, used for calibration

hGainint

Any gain on the magnetic channels which will need to be removed

dxfloat, optional

Distance between x electrodes

dyfloat, optional

Distance between y electrodes

resistics.time.reader_lemib423.readB423Header(hdrStr: str)[source]

Read B423 header for a single Lemi B423 file

Parameters
hdrStrstr

The headers as a string

Returns
dict

A dictionary of header values

resistics.time.reader_lemib423.readB423Params(dataFile: str, sampleFreq: float, dataByteOffset: int, recordByteSize: int)[source]

Get the parameters of the B423 data file

Parameters
dataFilestr

The data file as a string

sampleFreqfloat

The sampling frequency in Hz

dataByteOffsetint

The offset till the start of the data in bytes

recordByteSizeint

The size in bytes of a record

Returns
headersdict

The header values as a dictionary

firstDatetimedatetime

The time of the first sample in the data file

lastDatetimedatetime

The time of the last sample in the data file

numSamplesint

The number of samples in the data file given the sampling frequency