Update!
Apparently, our optimizations were so good, the AMX Mod Dev Team decided to merge them into their own project. The data reflected on this page is before AMX Mod used AMX Mod X optimizations, but it is still kept here for posterity. If and when there are another round optimizations, check back for updates!

Each release, AMX Mod X takes a bigger step forward to becoming more and more attractive to end-users and developers. We do this by providing more functionality and by making it as functionally fast as possible. To the end user, developers can provide a larger set of plugins at the lowest possible CPU usage. To prove this, we've done some interesting benchmarks against our close rival, AMX Mod.

Sample view of our optimization improvements. Graphs time to test completion, so lower is better.

In the past, people have claimed that AMX Mod X was "slower" than the competition. So, we decided to get to the bottom of this myth. Our results showed that AMX Mod X was usually as fast the competition, however, improvements could be made. So, we spent an entire release optimizing everything we could. AMX Mod X 1.70 is the most optimized version of any AMX Mod to date! If you don't believe it yourself, check out our comprehensive benchmarks. We even provide full source code and raw data for you to benchmark your own servers with. So, start reading, or go straight to the pretty graphs!.

• The Setup
• The Process
• The Results - Float Math
• The Results - String Formatting
• The Results - Multilingual Translation
• The Results - By Processor

The Setup

We used three computers for the reference benchmarks. Their specs:

For software versions, we tracked down publicly posted binaries:

In order to achieve accurate and believable benchmarks, we made a few tools tool run through the tests. Below is a description of the tests we ran, as well as the tools involved. To download the complete toolset, click here.

• Bench Module - This module provides a single native - get_tick_count(). It returns an arbitrary number of milliseconds that have passed. You can use the difference between two get_tick_count() calls to find the amount of time passed in between.
• Bench Plugin - Provides a set of benchmarks that stress test the five major things plugins do. This plugin requires different code paths for the different APIs of AMX Mod and AMX Mod X, although the functionality is designed to be exactly the same.
  
  • Integer Math - Stress tests the basic integer math. This is basic integer division, multipliation, addition, subtraction, and branching. These were omitted from the graphs because the results were virtually identical for all versions. This makes sense, since the JIT math is virtually unchanged.
  • Floating Point Math - Stress tests basic decimal math. This is usually used in CPU intensive functions, such as entity manipulation or entity event hooking (like entity thinking or touching). This is a critical area for optimization. The benchmarks test floating point arithmetic (multiplication, division, addition, subtraction), rounding, unrounding, and branching.
  • String Formatting - Stress tests string packing and formatting. This is an extremely critical point of optimization. While the benchmark plugin only tests format() calls (direct string formatting), format() is actually the backend to many, many functions, such as client_print(), log*(), client_cmd(), et cetera -- anything which takes something like "Hello, %s! You have %d frags" and inserts the correct values. The benchmark tests formatting strings, float values, integer values, width alignment, and precision.
  • Multilingual Translation - Stress tests "ML", or the translating of one string into another language. As plugins begin to support this, optimizing it becomes more and more critical. The benchmark plugin attempts to translate six strings from a large, randomized dictionary. For both versions, it only attempts english translations, despite hardcoded advantages that might be possible (for example, AMX Mod uses the same string as both the translation key and the english result). Two keys are from the start of the dictionary file, two others from the middle, and two from the end. None have "extra" parameters.
  • Dictionary Load - A non-critical (in fact, almost meaningless) statistic. This tests the amount of time it takes to parse the randomized dictionary file. This is only done at load time, and would not affect anything like game time. It is not included in the graph results.
• Dictionary Generator - A simple perl script which generates a table of 500 random translations for German, English, and French. It outputs one in AMX Mod format and another in AMX Mod X format, both containing the same data and in the closest orders possible.

The Process

AMX Mod 0.9.9/2005rc1a

• Disable all plugins in amx/configs/plugins.ini, then add "benchmark_099.amx" or "benchmark_2k5.amx" depending on the version. Copy the .amx file into amx/plugins.
• Disable all modules in amx/configs/modules.ini, then add "bench.dll". If you're using Linux, choose either "bench_099_i386.so" or "bench_2k5_i386.so" depending on your version. Copy the related .dll/.so module file into amx/modules.
• Place "fakedict-amxmod.txt" into amx/lang.

AMX Mod X 1.60/1.70

• Disable all plugins in amxmodx/configs/plugins.ini, then add "benchmark.amxx". Copy the .amxx plugin to amxmodx/plugins.
• Disable all modules in amxmodx/configs/modules.ini, then add "bench_amxx". Copy the .dll/.so file to amxmodx/modules.
• Place "fakedict-amxmodx.txt" into amxmodx/data/lang.

We created four folders on each server - amx099, amx2k5, amxmodx160, amxmodx170. For each test, we renamed each folder to the current active version and started HLDS. The benchmark plugin was activated by simply typing "bench" into the HLDS GUI/Console (Linux). We tested each mod twice - each test was separated by a mapchange to de_dust. We rebooted the Windows machines in between each mod in order to get the most accurate results as possible.

We made sure that each mod version had the JIT enabled. The JIT is a huge performance boost to Pawn/Small. Normally, scripts are processed by normal functions written in C. The script is read and parsed as a binary file. The JIT converts the script to an executable in memory, in pure assembly. This makes Pawn/Small lightning fast.

The Results - Float Math

Floating point math is used everywhere in complex scripts. In AMX Mod, this means the VexD module, and in AMX Mod X, it means the Engine and Fakemeta class of modules. Floating point math (any arithmetic with decimals) is used for entity manipulation, line tracing, timing, and many other things.

Pawn/Small does not have true floating point support. Instead, floating point math is implemented by creating natives/functions, in order to make support for them easier. As one might be able to imagine, this process is slow and compiler dependent. Rather than processing floats directly in the JIT, Pawn by default uses compiler/architecture dependent functions for doing simple floating point arithmetic.

The initial increase in speed between AMX Mod and AMX Mod X is most likely due to choice in compiler. AMX Mod uses GCC (on mingw for Windows), while AMX Mod X uses MSVC. MSVC most likely generated better code for basic float operations. This is evident from the fact that this drop only occurs on Windows.

The large increase in speed in AMX Mod X 1.70 is due to the plugin optimizer. The plugin optimizer searches plugins for floating point operations at load time, then converts them to ultra high-speed pieces of code in the JIT, rather than natives - this means float math occurs directly in your processor's floating point unit, rather than adding the overhead of a native call.

The Results - String Formatting

String formatting, as said before, is used literally everywhere in AMX Mod and AMX Mod X. It is the heart of every single native which takes in a list of parameters and produces a pretty-looking string. Optimizing it is essential.

At first, we thought format() was impossible to optimize. The version written by OLO for AMX Mod 0.x was astoundingly simple and short. However, it contained buffer overflow vulnerabilities, and correcting this in AMX Mod X 1.60 caused us to lose some performance.

However, in AMX Mod X 1.70, we discovered the key to optimizing format(). The results show it! The routine is up to three times faster. We achieved this miraculous increase by analyzing exactly what causes the slowdown. OLO's format() spends most of its time copying back and forth in between data streams of different sizes. Copying between streams of different sizes requires "resizing" as each block is copied, and the processor loses the ability to cache/optimize the copying procedure. AMX Mod X solved this by rewriting format() from scratch. Now, it copies data much more efficiently, and the benefits are noticeable.

The Results - Multilingual Translation

Multilingual translation is a rather new feature to the AMX Mod family, and is still slowly catching on with developers. Because it is being embedded in string manipulation functions, it is becoming more and more of a point for optimization.

The first interesting difference is between the two AMX Mod versions. AMX Mod's ML lookup code is "switchable" at compile time to different algorithms. It is most likely that a faster algorithm was omitted from 0.9.9 until it was tested, and then turned on for 2005rc1a. This would explain the massive speed difference.

As for AMX Mod X, the ML system was largely rewritten after 1.60. We realized many deficiencies in not only the storage method, but the searching algorithm. As can be seen by the benchmarks, we have successfully brought the ML system down to nearly a small overhead on top of the normal format speed.

The Results - By Processor