Need Input On Scalable System Design…

Home » Asterisk Users » Need Input On Scalable System Design…
Asterisk Users 4 Comments

Hey All,

Growing call center. Currently at about 200 call center staff, running about 1000 calls per hour. Gearing up to double that. Not too sure that a single server will support that growth. So, I’m trying to come up with ways to scale the system and still maintain a simplistic design. So I’d like to bounce some ideas around.

Currently I am running on a Dell 1950, dual quad core 2.33ghz xeons, with 16gb ram, and 2 tce400p cards. This server is managing the full load of the company. We are recording all calls, running ivr, queues, cdr, cel, and web for reporting. I currently have another 1950 of the exact same specifications as a cold spare.

Here’s where you can see drawings of my current connectivity and an optional connectivity I’m contemplating…

http://www.paydaysupportcenter.com/current.pdf
http://www.paydaysupportcenter.com/option.pdf

As you can see I currently have a separate sql server and a separate storage server for the call recordings. This is all working fine.

However, I’m thinking for scalability I should be looking to migrate to a configuration similar to the one in option.pdf. Where I have a VOIP
gateway server that simply relays traffic and possibly can do some load balancing or intellegent routing. But nothing more then that, and possibly a second one of these online as a hot failover.

Then have separate sql, storage, (i forgot it in the pic) web, and asterisk servers behind that on separate dedicated network. Here’s my dilemma though, how do I balance the load across multiple machines for scalability…

Since 95% of our calls come into queues, I need to be able to maintain queue stats and presence across all of the servers. Thus far, I’ve got everything except the extensions.conf file into the mysql database. I
thought about setting up 2 servers, 1 for sales, and 1 for customer service, then possibly break out each call queue to it’s own server as things grow. Just not sure if that’s the right way to go.

Then regarding extensions.conf, I’ve read that it too can be placed in the sql database and accessed via switch. however it’s resource intense, so now I’m thinking of maybe putting that file on the nfs server for all of the boxes to read from.

As for the design of that file, I was kind of thinking of a modular design within the file using various goto’s and gosubs. Our business model is based on affiliates and corporate marketing, so we have a ton of did’s that follow the same call flow with minor modifications in some variables, as well as variations in call flow, and hours of operation. Thus the modular design of the call flow. Then the primary inbound context would simply be a list of did’s pointing to a goto with a list of the variations and variables for the did.

Ok, now that I’ve melted your brains…. thoughts?

Thanks all in advance for the discussion… Greg

4 thoughts on - Need Input On Scalable System Design…

  • Thanks Laszlo, but if I’m reading this correct, this unit is load balancing based on traffic type, transcoding/voicemail/etc/… As all of my traffic would be the same, I don’t see that this would fit too well. Additionally, my problem is more, how do I maintain queue stats across the “work horses” so that the strategies maintain from one server to anotheras callload is balanced across the servers.

  • We have a similar server but a single quad core at 3Ghz. It easily handles 400 concurrent calls with a lot shorter average call duration than you have. It doesnt do as much call recording but does do a lot of AGI stuff.

    With regard to nfs thats fine for non real time stuff. Personally we have a test machine and multiple live machines and use subversion to commit any approved changes and then check them out on the live boxes. We dont need to worry about shared file space and we get version control of the configuration as an additional benefit. Its similar for call recordings. We have call recordings going to a ram disc and then when they are complete there is a background process to copy them to the nfs volume. If nfs is unavailable then they are moved to the internal disk temporarily until the nfs is back online. We have never used this functionality but it add a little redundancy.

    I would put opensips at the front end which looks at the incoming destination number and routes the call to the appropiate front end asterisk box depending on the queue it should go to. The other asterisk box(s) will be a backup so if one asterisk box fails then one of the others takes over running that queue automatically.

    For call recording are you using mixmonitor?
    I would consider using the normal monitor command and pass these recordings off to the nfs and have another machine process them by mixing both legs and perhaps converting to mp3 aswell.

  • Hi Greg, I am aware of a couple of solutions that come prepackaged and offer distributed queues for Asterisk. One of them, that seems to work well and reliably, is the one from Raynet. I am sure there are more. On the other side, I have seen a number of in-house solutions where you basically have a daemon polling queues statuses and redirecting calls based on the relative wait times. Rough but effective, and can be deployed easily.

    About recordings, my suggestion would be to use something to offload them right from the servers, like Oreka. Have a number of large clients using it and they are quite happy (plus, the guys supporting it are superb).

    Just my two cents, l.

    2013/8/27 Gregory Malsack