How Does Verbosity Work?
I’m trying to track down a CPU spike we are seeing in a system.
We have tracked down the spike to a single CPU and TID using that CPU. Indications are that it’s asterisk running this TID.
I’m trying to figure out what asterisk is doing on this thread around that time, but haven’t been able to match the tid to anything I’m seeing in asterisk debugging. Is there a good way to diagnose what asterisk is doing for each thread id?
We enabled the debugging in the logger.conf. Is there a verbose level that would be recommended to track the threading?
Also, does the verbose help if it may be confbridge related?
3 thoughts on - How Does Verbosity Work?
If you are able to get a backtrace during the time the issue is occurring, that could provide some insight. You can get a backtrace while Asterisk is running by following the steps on [1].
Is there a verbose level that would be recommended to track the threading?
Verbose is a very chatty debug that spits out information about what’s going on. You could try with verbose 3, and if that doesn’t provide the information you’re looking for, you can dynamically bump it up at runtime. If you are looking for something specific, you can always do a search in the confbridge files for the verbose message and set the verbose level to whatever number that message specifies.
VGhhbmsgeW91IEJlbg0KDQpGcm9tOiBhc3Rlcmlzay11c2VycyA8YXN0ZXJpc2stdXNlcnMtYm91
bmNlc0BsaXN0cy5kaWdpdW0uY29tPiBPbiBCZWhhbGYgT2YgQmVuIEZvcmQNClNlbnQ6IE1vbmRh eSwgU2VwdGVtYmVyIDE2LCAyMDE5IDk6NTggQU0NClRvOiBBc3RlcmlzayBVc2VycyBNYWlsaW5n IExpc3QgLSBOb24tQ29tbWVyY2lhbCBEaXNjdXNzaW9uIDxhc3Rlcmlzay11c2Vyc0BsaXN0cy5k aWdpdW0uY29tPg0KU3ViamVjdDogUmU6IFthc3Rlcmlzay11c2Vyc10gSG93IGRvZXMgdmVyYm9z aXR5IHdvcms/DQoNCklzIHRoZXJlIGEgZ29vZCB3YXkgdG8gZGlhZ25vc2Ugd2hhdCBhc3Rlcmlz ayBpcyBkb2luZyBmb3IgZWFjaCB0aHJlYWQgaWQ/DQoNCklmIHlvdSBhcmUgYWJsZSB0byBnZXQg YSBiYWNrdHJhY2UgZHVyaW5nIHRoZSB0aW1lIHRoZSBpc3N1ZSBpcyBvY2N1cnJpbmcsIHRoYXQg Y291bGQgcHJvdmlkZSBzb21lIGluc2lnaHQuIFlvdSBjYW4gZ2V0IGEgYmFja3RyYWNlIHdoaWxl IEFzdGVyaXNrIGlzIHJ1bm5pbmcgYnkgZm9sbG93aW5nIHRoZSBzdGVwcyBvbiBbMV0uDQoNCklz IHRoZXJlIGEgdmVyYm9zZSBsZXZlbCB0aGF0IHdvdWxkIGJlIHJlY29tbWVuZGVkIHRvIHRyYWNr IHRoZSB0aHJlYWRpbmc/DQpBbHNvLCBkb2VzIHRoZSB2ZXJib3NlIGhlbHAgaWYgaXQgbWF5IGJl IGNvbmZicmlkZ2UgcmVsYXRlZD8NCg0KVmVyYm9zZSBpcyBhIHZlcnkgY2hhdHR5IGRlYnVnIHRo YXQgc3BpdHMgb3V0IGluZm9ybWF0aW9uIGFib3V0IHdoYXQncyBnb2luZyBvbi4gWW91IGNvdWxk IHRyeSB3aXRoIHZlcmJvc2UgMywgYW5kIGlmIHRoYXQgZG9lc24ndCBwcm92aWRlIHRoZSBpbmZv cm1hdGlvbiB5b3UncmUgbG9va2luZyBmb3IsIHlvdSBjYW4gZHluYW1pY2FsbHkgYnVtcCBpdCB1
cCBhdCBydW50aW1lLiBJZiB5b3UgYXJlIGxvb2tpbmcgZm9yIHNvbWV0aGluZyBzcGVjaWZpYywg eW91IGNhbiBhbHdheXMgZG8gYSBzZWFyY2ggaW4gdGhlIGNvbmZicmlkZ2UgZmlsZXMgZm9yIHRo ZSB2ZXJib3NlIG1lc3NhZ2UgYW5kIHNldCB0aGUgdmVyYm9zZSBsZXZlbCB0byB3aGF0ZXZlciBu dW1iZXIgdGhhdCBtZXNzYWdlIHNwZWNpZmllcy4NCg0KT24gTW9uLCBTZXAgMTYsIDIwMTkgYXQg OTo0NCBBTSBEYW4gQ3JvcHAgPGRhbkBhbXRlbGNvLmNvbTxtYWlsdG86ZGFuQGFtdGVsY28uY29t Pj4gd3JvdGU6DQpJ4oCZbSB0cnlpbmcgdG8gdHJhY2sgZG93biBhIENQVSBzcGlrZSB3ZSBhcmUg c2VlaW5nIGluIGEgc3lzdGVtLg0KDQpXZSBoYXZlIHRyYWNrZWQgZG93biB0aGUgc3Bpa2UgdG8g YSBzaW5nbGUgQ1BVIGFuZCBUSUQgdXNpbmcgdGhhdCBDUFUuICBJbmRpY2F0aW9ucyBhcmUgdGhh dCBpdOKAmXMgYXN0ZXJpc2sgcnVubmluZyB0aGlzIFRJRC4NCg0KSeKAmW0gdHJ5aW5nIHRvIGZp Z3VyZSBvdXQgd2hhdCBhc3RlcmlzayBpcyBkb2luZyBvbiB0aGlzIHRocmVhZCBhcm91bmQgdGhh dCB0aW1lLCBidXQgaGF2ZW7igJl0IGJlZW4gYWJsZSB0byBtYXRjaCB0aGUgdGlkIHRvIGFueXRo aW5nIEnigJltIHNlZWluZyBpbiBhc3RlcmlzayBkZWJ1Z2dpbmcuDQpJcyB0aGVyZSBhIGdvb2Qg d2F5IHRvIGRpYWdub3NlIHdoYXQgYXN0ZXJpc2sgaXMgZG9pbmcgZm9yIGVhY2ggdGhyZWFkIGlk Pw0KDQpXZSBlbmFibGVkIHRoZSBkZWJ1Z2dpbmcgaW4gdGhlIGxvZ2dlci5jb25mLiAgSXMgdGhl cmUgYSB2ZXJib3NlIGxldmVsIHRoYXQgd291bGQgYmUgcmVjb21tZW5kZWQgdG8gdHJhY2sgdGhl IHRocmVhZGluZz8NCkFsc28sIGRvZXMgdGhlIHZlcmJvc2UgaGVscCBpZiBpdCBtYXkgYmUgY29u ZmJyaWRnZSByZWxhdGVkPw0KLS0NCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXw0KLS0gQmFuZHdpZHRoIGFuZCBDb2xv Y2F0aW9uIFByb3ZpZGVkIGJ5IGh0dHA6Ly93d3cuYXBpLWRpZ2l0YWwuY29tIC0tDQoNCkNoZWNr IG91dCB0aGUgbmV3IEFzdGVyaXNrIGNvbW11bml0eSBmb3J1bSBhdDogaHR0cHM6Ly9jb21tdW5p dHkuYXN0ZXJpc2sub3JnLw0KDQpOZXcgdG8gQXN0ZXJpc2s/IFN0YXJ0IGhlcmU6DQogICAgICBo dHRwczovL3dpa2kuYXN0ZXJpc2sub3JnL3dpa2kvZGlzcGxheS9BU1QvR2V0dGluZytTdGFydGVk DQoNCmFzdGVyaXNrLXVzZXJzIG1haWxpbmcgbGlzdA0KVG8gVU5TVUJTQ1JJQkUgb3IgdXBkYXRl IG9wdGlvbnMgdmlzaXQ6DQogICBodHRwOi8vbGlzdHMuZGlnaXVtLmNvbS9tYWlsbWFuL2xpc3Rp bmZvL2FzdGVyaXNrLXVzZXJzDQoNCg0KLS0NCkJlbmphbWluIEZvcmQNCkRpZ2l1bSAtIEEgU2Fu Z29tYSBDb21wYW55IHwgU29mdHdhcmUgRW5naW5lZXINCjQ0NSBKYW4gRGF2aXMgRHJpdmUgTlcg LSBIdW50c3ZpbGxlLCBBTCAzNTgwNiAtIFVTPGh0dHBzOi8vbWFwcy5nb29nbGUuY29tLz9xPTQ0
NStKYW4rRGF2aXMrRHJpdmUrTlcrLStIdW50c3ZpbGxlLCtBTCszNTgwNistK1VTJmVudHJ5PWdt YWlsJnNvdXJjZT1nPg0KQ2hlY2sgdXMgb3V0IGF0OiBodHRwczovL2RpZ2l1bS5jb208aHR0cHM6
Ly9kaWdpdW0uY29tLz4gwrcgaHR0cHM6Ly9zYW5nb21hLmNvbTxodHRwczovL3NhbmdvbWEuY29t Lz4NCg0KW2h0dHBzOi8vd3d3LmRpZ2l1bS5jb20vc2l0ZXMvYWxsL3RoZW1lcy9wb3VuZGtleS9z aGFyZWQvaW1hZ2VzL2RpZ2l1bS9sb2dvLnBuZ10NCg=
I just saw I forgot to link the wiki page:
https://wiki.asterisk.org/wiki/display/AST/Getting+a+Backtrace
Hope this helps.