Log Processor

This problem is about Automated Telemetry Log Parser for Aircraft Stress Testing. During structural fatigue testing of an aircraft wing assembly, an array of digital sensors transmits raw telemetry batches of logs over a serial bus. As a data engineer on the flight test team, you must write a Python function(s) to parse these raw logs, identify critical stress warnings, filter out corrupted frames, and generate a summary report for the engineering team.

The log batch is composed of exactly 5 log entries and is received as a single str where each log ends by an end of line ('\n') character. A log entry is valid if and only if complies with the following format:

"STATION_<ID>#READING:<stress_value_MPa>#FLAGS:<flag1>,<flag2>,..."

Notice that:

  • It starts with the word STATION followed by _.

  • The field <ID> is the sensor id number, an integer between 1 and 10000 inclusive. It is never 0 or negative.

  • It contains the words contains the READING and FLAGS both preceded by the # and followed by the : single characters.

  • The field <stress_value_MPa> right after READING: is the stress_value reading, a float between 0.0001 and 10.0.

  • <flag1>, <flag2>, ... ,<flagk> right after FLAGS: are text messages regarding various features controlled by that sensor (it is a warning about something if it contains the word WARN).

See next examples of bolth, valid and corrupted logs:

"STATION_101#READING:210.50#FLAGS:CALIB_OK\n"
"CORRUPT_DATA_BUS_ERROR_0x88\n"
"STATION_102#READING:415.20#FLAGS:WARN_OVERLOAD,WARN_VIBE\n"
"STATION_103#READING:195.80#FLAGS:CALIB_OK,WARN_TEMP\n"
"BAD_HEADER#READING:0.00#FLAGS:NONE\n"

You are required to deliver the following Python functions in the module log_processor (file log_processor.py) according to the following specifications:

Firstly, the auxiliar function to parse and extract the info of a single log:

log_entry_parse(log_line)

such that

given log_line, a str with a single log as described above

returns four values:

  1. a bool that is True if the log is valid according to the above format

  2. an int with the sensor id

  3. a float with the sensor reading

  4. an int with the number of times the tag “WARN” appears within the flags segment

In the case of a corrupted log, the boolean retured must be False and the following values must be -1, -1.0, -1 respectively.

For example:


>>> log_line = "STATION_101#READING:210.50#FLAGS:CALIB_OK"
>>> ok, station, reading, warnings = log_entry_parse(log_line)
>>> ok, station, reading, warnings
(True, 101, 210.5, 0)

>>> log_line = "CORRUPT_DATA_BUS_ERROR_0x88"
>>> ok, station, reading, warnings = log_entry_parse(log_line)
>>> ok, station, reading, warnings
(False, -1, -1.0, -1)

>>> log_line = "STATION_102#READING:415.20#FLAGS:WARN_OVERLOAD,WARN_VIBE"
>>> ok, station, reading, warnings = log_entry_parse(log_line)
>>> ok, station, reading, warnings
(True, 102, 415.2, 2)

Doctests are available in the log_entry_parse.test file.


Secondly, the main function that processed a batch containing exactly five (5) single logs (each on can be valid or corrupted independently of the others):

log_batch_summary(log_batch)

such that

given log_batch, a str with a 5-entry log batch as described above.

returns the following 4 values:

  1. an int with the counting of valid logs.

  2. a str with the concatenation of the ids of the valid stations, separeted by ',' and with a coma , at the end aswell.

  3. a float with the mean stress reading across all valid logs, rounded to 2 decimal places.

  4. an int with the sum of all warning occurrences (warn_count) across all valid logs.

This function must perform the parsing and data extraction of each of the 5 logs in log_batch by calling parse_log_entry() above on each of them. The data extracted from the valid ones are the date to calculate the summary from.

For instance,


>>> log_batch = "STATION_101#READING:210.50#FLAGS:CALIB_OK\n"
>>> log_batch += "CORRUPT_DATA_BUS_ERROR_0x88\n"
>>> log_batch += "STATION_102#READING:415.20#FLAGS:WARN_OVERLOAD,WARN_VIBE\n"
>>> log_batch += "STATION_103#READING:195.80#FLAGS:CALIB_OK,WARN_TEMP\n"
>>> log_batch += "BAD_HEADER#READING:0.00#FLAGS:NONE\n"
>>> a, b, c, d = log_batch_summary(log_batch)
>>> a, b, c, d 
(3, '101,102,103,', 273.83, 3)

Warning

This function must be implemented by calling the previous function.

Doctests are available in the log_batch_summary.test file.