AutoSim
Content: AutoSim is developed to increase the work efficiency and make comparison of the result of simple scalar simulation, which is usually carried out on different host, possible. A specification file or command line arguments must provided when calling AutoSim. AutoSim itself has no functionality of compiling and simulation, all it does is to gather enough information and then iteratively calls the Makefile of different benchmarks. The Makefile will contains the rules to compile the code, the details of which simulator to run and etc. See more details about the scripts itself on developer's documents · Latest Simple Scalar been installed, for more details visit SS. · Perl 5.8 · GNU make installed (It only been tested on Linux/slackware version.) The user must create a simple directory structure for the AutoSim to run. With the structure the simulation would be easy to maintain, debug, and read. An example is shown below
AutoSim can create the file structure for you, the only thing you need to tell AutoSim is the point in the file system you want them to be created. The file system in the figure above is actually created by AutoSim by issuing "AutoSim.pl --initial=/auto" The name of the three subdirectories of /auto are default. You can choose the name of directory when you manually created it. The only important thing is that in your Makefile for each benchmark you have to make sure that all the files has been provided the correct path. The syntax of the command to run AutoSim is : Perl autosim.pl –spec spec_file [arguments] Command line arguments or specification file can be provided when running AutoSim, if both exist the command line arguments will take priority. The following is the table of attributes may appeared in the configure file:
You don't need to give value for the attributes, which have default value, unless you want to alter the it. The syntax of assigning value to an attribute is : attribute = value Each line in the configure file should only contains one attribute. '#' symbol can be used to indicate the start of comment. The benchmarking would be carried out in the order you specify the them. AutoSim does no more than just take what's you provided and issue a system call iteratively. All the functionality should be provided by Makefile and Simple Scalar tool sets. In your MakeFile you should specify how to cross compile the benchmark and use which simulator and etc. The command line arguments gets 2 additional attributes : -h and -v. The two extra arguments is for returning help information and the version of the AutoSim. Before you run AutoSim make sure that the PATH variable got enough information to find Simple Scalar simulator. At the end of the automated benchmarking, AutoSim will also produce a short summary about how many benchmarks were tried, and how many of them were failed to run successfully.
In this section I will use a real example to show you how to use the AutoSim Perl scripts. In this example I want to run 31 different WCETA (Worst Case Execution Time Analysis) benchmarks on 18 different configurations, and each benchmarking will have the result logged in a file for the analysis purpose. The areas of processor configurations I interested are:
According to this requirements, I have written 18 configuration files for the arm-sim-outorder simulator. The reason to choose this simulator is because it is the most comprehensive simulator and it is the only simulator capable of doing the simulation under all the requirements listed above. You can download all the benchmarks and corresponding configurations at here. Each benchmarks are put into a separate directory with the GNU Makefile, which is used to compile and run the simulation. Since AutoSim does no compiling and simulating, all the task are done through the Makefile. For an example, in this case I want the test to generate the log files I have to provide the options when issue command in Makefile. Before kick off the automated benchmarking, some directory structure must be created if not already been done. First you need to create a directory, under which you will put all your benchmarks, configuration files for the simple scalar toolsets. In this case I choose to create directories /auto, /auto/benchmarks (store all the benchmarks), /auto/configs (store all the configuration files). Or you can let the AutoSim create it for you by just issuing command : Perl AutoSim.pl --initial /home Actually you don't need to put the configure file for simple scalar into configs directory. But if you do, it will make it easier for others to understand your simulation test and it is also easier to maintain the configuration files. But no matter in which case you need to make sure in your makefile you choose to use the right configure file. Then I wrote a specification file (spec) just for this automated benchmarking. The content of the spec files is listed below (refer to the previous section for the details of the attributes): root = /auto .... Since the tests didn't be assigned a value, then all the benchmarks stored in the /auto/benchmarks will be tested. To kick off the test just simply issue command : AutoSim.pl --spec spec At the end of the test, you will see a summary about how many benchmarks been tested and how many failed to execute. If some benchmarks failed to execute(in this case "sqrt"), it only means the benchmarks failed to execute or system returned a non zero value. The results of the benchmarking can be find in the place you specified in the Makefile. For the failed benchmark I will left as an exercise for the reader to debug and fix it. Hint, manually fix the bug then kick off the AutoSim again.
Note: If ;you see similar error message like : "Make : *** error code xxx", it may not be an real error. Sometimes the benchmark returns none zero value or returns Null when it is finished, then GNU make will think there are something wrong will the program. Because in UNIX world, successful command will return 0, only returns none zero value when something goes wrong. You can modify you benchmarks to return 0 if everything works fine to eliminate this error message.
|
||||||||||||||||||||||||||||||||