Benchmarking AGI performance in C, PHP, and Perl

Home » General » Benchmarking AGI performance in C, PHP, and Perl
General, VoIP News No Comments

This is a very interesting post made by Steve Edwards on the Asterisk Users mailing list. Basically he was demonstrating the difference in speed when working with AGI scripts in different programming languages. It says as follows:

Many times, I’ve made the statement that you can execute hundreds of AGIs written in C in the time it takes to load an interpreter and parse a script written in PHP or Perl.

Recently, a Doubting Thomas asked me to substantiate my claim.

I suspect nobody has made the effort to implement an AGI of any reasonable size and function in multiple languages.

I’m guessing it may not really be all that important and the results would be too task specific to be relevant.

I suspect once an AGI is executing, the choice of source language is unimportant. I’ll go out on a limb and say executing:

select prompt_path from foo where bar;
stream file prompt_path “1234*”
stream file you_entered “”
say digits selected “”

will execute in effectively the same time regardless of source language. Waiting for Asterisk to play a file or for your database to return a row is beyond the scope of your AGI.

It’s what you do between your AGI and database calls that will determine how much your choice of source language will impact the total execution time.

Unless you’re doing a lot of stuff in between these API calls, the only place you can make an impact is getting your code into memory and ready to execute.

I ‘wrote’ 2 different AGIs in C, PHP, and Perl.

The first AGI, ‘null-agi’ reads the AGI environment variables from STDIN and exits. To me, this is the bare minimum a program can do and call itself an AGI. Each AGI was less than 10 lines.

The second AGI, ‘neutered-agi’ is an AGI of ‘production length’ (around 1,600 lines) and supporting access to a MySQL database. The AGI is of ‘production length’ but still exits after reading the AGI environment variables because we are measuring program startup time.

For both AGIs, the C implementation used an AGI library I developed way too many years ago. The PHP implementation used PHPAGI. The Perl implementation used Asterisk::AGI.

The C version of neutered-agi was based on a ‘voicemail-like’ AGI I wrote many years ago that stored the user credentials and messages in MySQL.

The PHP version of neutered-agi was based on dialparties.agi (nicked from PIAF). dialparties.agi is only about 800 lines long, so I ‘doubled it’ by copying and pasting it into the same source file. While dialparties.agi does not use MySQL, MySQL is available in PHP without including additional header or class files.

The Perl version of neutered-agi was based on agi-VDAD_ALL_outbound.agi (nicked from Vicidial).

I ran the tests in 3 different environments:

|- – – – – – – – – -+ – – – -+ – – – -+ – – – – -|
| CPU               | RAM    | CentOS | Asterisk |
| – – – – – – – – – + – – – -+ – – – -+ – – – – -|
| Geode 500MHz      | 256 MB |    4.9 |   1.2.37 |
| Atom D525 1.80GHz | 4 GB   |    5.6 |  1.8.4.1 |
| Xeon 3.40GHz      | 2 GB   |    4.8 |   1.2.40 |
| – – – – – – – – – + – – – -+ – – – -+ – – – – -|

No swapping occurred during the tests.

My dialplan executed each AGI 1,000 times to make the cumulative execution time more measurable. I wrote the dialplan in both ‘inline dialplan’ and an AEL ‘for’ loop. The initial execution times were the same so the test runs were made with the AEL version because it is more manageable. (5 lines versus 1,000 lines.)

Here’s the results for executing each AGI 1,000 times on each host in seconds:

Geode:

| – – – – -+- – – – – + – – – – – – -|
| language | null-agi | neutered-agi |
| – – – – -+ – – – – -+ – – – – – – -|
| C        |        6 |            6 |
| PHP      |      116 |          160 |
| Perl     |       99 |          639 |
| – – – – -+ – – – – -+ – – – – – – -|

Atom:

| – – – – -+ – – – – -+ – – – – – – -|
| language | null-agi | neutered-agi |
| – – – – -+ – – – – -+ – – – – – – -|
| C        |        6 |            6 |
| PHP      |       52 |           65 |
| Perl     |       38 |          197 |
| – – – – -+ – – – – -+ – – – – – – -|

Xeon:

| – – – – -+ – – – – -+ – – – – – – -|
| language | null-agi | neutered-agi |
| – – – – -+ – – – – -+ – – – – – – -|
| C        |        2 |            2 |
| PHP      |       40 |           47 |
| Perl     |       10 |          107 |
| – – – – -+ – – – – -+ – – – – – – -|

Summary:

Geode – Perl / C: 106
Atom – Perl / C: 33
Xeon – Perl / C: 54

The C null-agi AGI was statically linked. I didn’t have all the libraries needed to statically link neutered-agi on these boxes, but the dynamically linked versions of null-agi and neutered-agi took the same time to execute (16 seconds on the Geode) so I’m assuming statically linked versions of null-agi and neutered-agi would also take the same time to execute. It also helps support my original statement 🙂

I guessing the Perl version of neutered-agi took a big hit from having to load the database code as well as the AGI framework (use DBI; use Asterisk::AGI😉 while PHP only had to load the AGI framework (require_once “phpagi.php”;).

I’ll let you decide if the methodology is meaningful to your environment. I don’t consider myself to be a PHP or Perl expert, so if I’ve made some colossal blunder in my methodology, please let me know.

I’m guessing you’d have to resurrect a Soekris net4801 from the way-back-machine to substantiate my orignal claim of ‘hundreds.’ I’ll have to remind myself to say ‘dozens’ from now on.

(I don’t have a ‘thing’ against Soekris — I just bought 2 off Ebay to play with.)


Thanks in advance,
————————————————————————-
Steve Edwards       sedwards [at] sedwards [dot] com      Voice: +1-760-468-3867 PST
Newline                                              Fax: +1-760-731-3000