benchmarks package

Subpackages

• benchmarks.eval_rate_benchmark package
  • Submodules
  • benchmarks.eval_rate_benchmark.eval_rate_benchmark module
    • EvalRateBenchmark
      • EvalRateBenchmark.average_rate()
      • EvalRateBenchmark.process_eval_rate()
      • EvalRateBenchmark.store_eval_rate()
  • benchmarks.eval_rate_benchmark.eval_rate_plotter module
    • EvalRatePlotter
      • EvalRatePlotter.None
      • EvalRatePlotter.plot()
  • benchmarks.eval_rate_benchmark.eval_rate_processor module
    • EvalRateProcessor
      • EvalRateProcessor.average_eval_rates()
      • EvalRateProcessor.process_eval_rate()
  • Module contents

Submodules

benchmarks.call_audio_benchmark module

class benchmarks.call_audio_benchmark.CallAudioBenchmark(model: str = 'base')

Bases: object

A class for processing call audio and extracting call notes.

Parameters:

model (str, optional) – The name of the Whisper ASR model to use. Defaults to “base”.

current_transcript

The most recent transcript obtained from audio processing.

Type:

str

transcripts

A list of all transcripts processed.

Type:

list

model_transcripts

A dictionary mapping LLAVA models to their respective transcripts.

Type:

dict

call_notes

Extracted call notes from benchmark results.

Type:

str

model_call_notes

A dictionary mapping LLAVA models to their call notes.

Type:

dict

model

The Whisper ASR model instance.

Type:

whisper.WhisperModel

media_file_path(media_file

str) -> str: Returns the absolute path to the specified media file.

process_audio(audio_file

str) -> whisper.Audio: Loads and processes the audio file, returning the padded or trimmed audio.

transcribe_audio(call_audio

whisper.Audio) -> str: Transcribes the audio using the Whisper model and returns the transcript.

store_transcript(llava_model

str, fp16: bool = False) -> None: Stores the current transcript along with the LLAVA model it corresponds to.

store_call_notes(llava_model

str, benchmark_result: str) -> None: Extracts and stores call notes from benchmark results.

print_call_notes() → None

Prints formatted call notes.

Note

This class assumes the existence of the Whisper ASR model and audio data.

media_file_path(media_file: str)

Returns the absolute path to the specified call audio file.

Parameters:

media_file (str) – The name of the call audio file.

Returns:

The absolute call audio file path.

Return type:

str

print_call_notes()

Prints formatted call notes.

Note

The call notes are wrapped to a maximum width of 40 characters using textwrap.

process_audio(audio_file: str)

Loads and processes the audio file, returning the padded or trimmed audio.

Parameters:

audio_file (str) – The name of the audio file.

Returns:

Processed audio data.

Return type:

whisper.Audio

store_call_notes(llava_model: str, benchmark_result: str)

Extracts and stores call notes from benchmark results.

Parameters:

• llava_model (str) – The LLAVA model name.

• benchmark_result (str) – The benchmark result output.

store_transcript(llava_model: str, fp16: bool = False) → None

Stores the current transcript along with the LLAVA model it corresponds to.

Parameters:

• llava_model (str) – The LLAVA model name.

• fp16 (bool, optional) – Whether FP16 mode is enabled. Defaults to False.

transcribe_audio(call_audio: <module 'whisper.audio' from '/opt/hostedtoolcache/Python/3.12.4/x64/lib/python3.12/site-packages/whisper/audio.py'>)

Transcribes the audio using the Whisper model and returns the transcript.

Parameters:

call_audio (whisper.Audio) – Processed audio data.

Returns:

The transcribed text.

Return type:

str

benchmarks.license_plate_benchmark module

class benchmarks.license_plate_benchmark.LicensePlateBenchmark

Bases: object

A benchmark class for license plate numbers.

current_license_plate_number

The current license plate number being processed.

Type:

str

license_plate_numbers

List of extracted license plate numbers.

Type:

list

model_license_plate_numbers

Mapping of model names to license plate numbers.

Type:

dict

extract_license_plate_number(stdout) → str

Extract the license plate number from benchmark result output.

Parameters:

stdout (str) – Benchmark result output.

Returns:

Extracted license plate number or None if not found.

Return type:

str

static media_file_path(media_file: str) → str

Get the absolute path to a media file.

Parameters:

media_file (str) – The name of the media file.

Returns:

Absolute path to the media file.

Return type:

str

process_license_plate_number(benchmark_result: str) → None

Process the license plate number from benchmark result.

Parameters:

benchmark_result (str) – Result of the benchmark execution.

store_license_plate(model: str, benchmark_result: str) → None

Store the license plate number for a specific model.

Parameters:

• model (str) – Model name.

• benchmark_result (str) – Result of the benchmark execution.

Module contents