Refactor benchmark script for LLM benchmark integration (#2897)

* Added unit test for benchmark-ab.py; refactor test plans into module

* Declutter benchmark script

* Move benchmark reporting into module; remove global variable

* refactored system under test in benchmark script

* Move benchmark script refactoring

* Remove unnecessary method

* Refactor benchmark into benchmark class

* Resort methods

* Move print

* split benchmark and systems under test into separate files

* Split benchmark and ts reporting

* Remove global metrics variable

* Refactor reporting

* Skip benchmark test if click is missing

* Fix mising import after precommit

* Added locust as a benchmark backend

* Fix spell check test

---------

Co-authored-by: Ankith Gunapal <[email protected]>

Author: mreso

benchmarks/README.md

@@ -2,13 +2,15 @@
 
 The benchmarks measure the performance of TorchServe on various models and benchmarks. It supports either a number of built-in models or a custom model passed in as a path or URL to the .mar file. It also runs various benchmarks using these models (see benchmarks section below). The benchmarks are executed in the user machine through a python3 script in case of jmeter and a shell script in case of apache benchmark. TorchServe is run on the same machine in a docker instance to avoid network latencies. The benchmark must be run from within `serve/benchmarks`
 
-We currently support benchmarking with JMeter & Apache Bench. One can also profile backend code with snakeviz.
+We currently support benchmarking with JMeter, Apache Bench and Locust. One can also profile backend code with snakeviz.
 
 * [Benchmarking with Apache Bench](#benchmarking-with-apache-bench)
 * [Auto Benchmarking with Apache Bench](#auto-benchmarking-with-apache-bench)
 * [Benchmarking and Profiling with JMeter](jmeter.md)
 
-# Benchmarking with Apache Bench
+⚠️**Please Note**⚠️ In future we will deprecate Apache Bench in favor of locust.
+
+# Benchmarking with Locust/Apache Bench
 
 ## Installation
 
@@ -43,6 +45,8 @@ Refer [parameters section](#benchmark-parameters) for more details on configurab
 
 `python benchmark-ab.py`
 
+To switch between Apache Bench and Locust you can set the --benchmark-backend/-bb parameter to either "ab" or "locust".
+
 ### Run benchmark using a config file
 The config parameters can be provided using cmd line args and a config json file as well.
 This command will use all the configuration parameters given in config.json file.