benchmarks.eval_rate_benchmark package

Submodules

benchmarks.eval_rate_benchmark.eval_rate_benchmark module

class benchmarks.eval_rate_benchmark.eval_rate_benchmark.EvalRateBenchmark

Bases: object

average_rate()

Calculates the average of the evaluation rates stored in the instance. Interface to EvalRateProcessor’s average_eval_rates method from the LlavaBenchmark class.

Returns:

The average evaluation rate.

Return type:

float

process_eval_rate(benchmark_result)

store_eval_rate(image_file_path: str)

benchmarks.eval_rate_benchmark.eval_rate_plotter module

class benchmarks.eval_rate_benchmark.eval_rate_plotter.EvalRatePlotter

Bases: object

This class is responsible for plotting evaluation rates.

None

plot(eval_rates)

Plots the evaluation rates as an ASCII line chart.

Parameters:

eval_rates (list) – A list of evaluation rates.

Returns:

None

benchmarks.eval_rate_benchmark.eval_rate_processor module

class benchmarks.eval_rate_benchmark.eval_rate_processor.EvalRateProcessor

Bases: object

static average_eval_rates(eval_rates)

static process_eval_rate(benchmark_result)

Extract and process the evaluation rate from a benchmark result.

Parameters:

benchmark_result – Result of the subprocess.run call.

Returns:

The extracted evaluation rate in a list.

Return type:

list

Module contents

This is the __init__.py file for the eval_rate_benchmark package, which is a part of the larger project.

The eval_rate_benchmark package is located in the benchmarks directory. It contains the EvalRateBenchmark class, which is used to evaluate the rate of a model as part of the benchmarking process.

The EvalRateBenchmark class is imported here to make it accessible when the eval_rate_benchmark package is imported elsewhere in the project. This allows for a cleaner import statement, as you can directly import the EvalRateBenchmark class from the eval_rate_benchmark package.

For example, instead of using:

from benchmarks.eval_rate_benchmark.eval_rate_benchmark import EvalRateBenchmark

you can use:

from benchmarks.eval_rate_benchmark import EvalRateBenchmark