Memory Leak?

Home » Asterisk Users » Memory Leak?
Asterisk Users 3 Comments

This is asterisk 16.30.1, after 1 hours, with 17 contacts, yet we need to kill it every two hours because it stops processing calls.

rasterisk -x “core show channels count”

43 active channels

29 active calls

7260 calls processed

time rasterisk -x “memory show summary” | sort -n | tail -20

720985 bytes in deferred free allocations

775206 bytes in 20242 allocations in file res_rtp_asterisk.c

1142456 bytes in 18 allocations in file chan_iax2.c

1190880 bytes in 4392 allocations in file stasis_bridges.c

1217799 bytes in 6141 allocations in file xmldoc.c

1508416 bytes in 8782 allocations in file cdr.c

1514774 bytes in 20797 allocations in file devicestate.c

1521524 bytes in 18237 allocations in file stasis.c

2012008 bytes in 18969 allocations in file pbx.c

3057408 bytes in 42464 allocations in file astobj2_hash.c

4230560 bytes in 40320 allocations in file rtp_engine.c

27781422 bytes in 577295 allocations in file stasis_cache.c

28497984 bytes in 1495 allocations in file
../src/pj/pool_policy_malloc.c

56621945 bytes in 1766936 allocations in file taskprocessor.c

160393938 bytes in 2227792 allocations in file stasis_message.c

440540801 bytes in 8647565 allocations in file json.c

3011728948 bytes in 2923317 allocations in file stasis_channels.c

3750730746 bytes allocated (69073 in caches) in 16358993 selected allocations

3750730746 bytes in all allocations

3751451731 bytes in all allocations and deferred free allocations

real 0m15.676s

user 0m0.006s

sys 0m0.005s

(one second per contact)

time rasterisk -x “pjsip show contacts”

Contact:

========================================================================================

Contact: 1123/sip:1123@X.X.X.X:42700;ob 82a79730c7 NonQual nan

Contact: 1124/sip:1124@X.X.X.X:12061;ob 056c670307 NonQual nan

Contact: 1130/sip:1130@X.X.X.X:14302;ob 2fccd6c498 NonQual nan

Contact: 1131/sip:1131@X.X.X.X:23729;ob 277dda2550 NonQual nan

Contact: 1132/sip:1132@X.X.X.X:12173;ob 71e2436f7e NonQual nan

Contact: 1136/sip:1136@X.X.X.X:10085;ob 93300d6b2a Unknown nan

Contact: 1138/sip:1138@X.X.X.X:14178;ob 9ac8eb538c NonQual nan

Contact: 1148/sip:1148@X.X.X.X:28858;ob 3f6f6eca2a NonQual nan

Contact: Reiner_Georgia/sip:X.X.X.X/24 5da85f3be0 NonQual nan

Contact: Reiner_texas/sip:X.X.X.X/24 5da85f3be0 NonQual nan

Contact: Reiner_texas_2/sip:X.X.X.X/24 5da85f3be0 NonQual nan

Contact: Whisl_Telecom/sip:X.X.X.135:5060 5d61346cde NonQual nan

Contact: fede_arg/sip:X.X.X.X a5993dc66d NonQual nan

Contact: iplink/sip:X.X.X.X 4fc2b9913d Avail
203.144

Contact: nueva_reiner/sip:X.X.X.X/24 5da85f3be0 NonQual nan

Contact: reiner colombia/sip:X.X.X.X/24 df94afc597 NonQual nan

Contact: reiner_colombia/sip:X.X.X.X/24 5da85f3be0 NonQual nan

Objects found: 17

real 0m14.942s

user 0m0.003s

sys 0m0.004s

3 thoughts on - Memory Leak?

  • Are you using the JSON_DECODE function by chance? There was a memory leak bug in it fixed a while back[1]. However, it was not cherry picked to the 16 branch as that was after 16 went out of regular support. So if you’re using this function, you have a buggy version that is likely responsible for your leak. You can fix this by upgrading to a supported version of Asterisk.

    [1] https://gerrit.asterisk.org/c/asterisk/+/19477

  • I think PJSIP never releases memory

    This Is without any open calls.

    Every command is very slow.

    rasterisk -x “memory show allocations” |sort -t” ” -n -k1 | tail -20

    25192 bytes allocated by sip_options_contact_statuses_alloc() line
    415 of res_pjsip/pjsip_options.c

    25216 bytes allocated by ast_res_pjsip_init_options_handling() line
    2946 of res_pjsip/pjsip_options.c

    26496 bytes allocated by load_module() line 155 of codec_resample.c

    32720 bytes allocated by acf_retrieve_docs() line 359 of pbx_functions.c

    65488 bytes allocated by acf_retrieve_docs() line 359 of pbx_functions.c

    65488 bytes allocated by ast_register_application2() line 157 of pbx_app.c

    68000 bytes allocated by default_block_alloc() line 46 of
    ../src/pj/pool_policy_malloc.c

    68000 bytes allocated by default_block_alloc() line 46 of
    ../src/pj/pool_policy_malloc.c

    524408 bytes allocated by load_objects() line 14840 of chan_iax2.c

    524408 bytes allocated by load_objects() line 14846 of chan_iax2.c

    524800 bytes allocated by default_block_alloc() line 46 of
    ../src/pj/pool_policy_malloc.c

    524800 bytes allocated by default_block_alloc() line 46 of
    ../src/pj/pool_policy_malloc.c

    703967 bytes in deferred free large allocations

    718236 bytes in deferred free allocations

    1052000 bytes allocated by default_block_alloc() line 46 of
    ../src/pj/pool_policy_malloc.c

    2100000 bytes allocated by default_block_alloc() line 46 of
    ../src/pj/pool_policy_malloc.c

    20972000 bytes allocated by default_block_alloc() line 46 of
    ../src/pj/pool_policy_malloc.c

    41054128 bytes allocated (9224 in caches) in 82929 selected allocations

    41054128 bytes in all allocations

    41772364 bytes in all allocations and deferred free allocations

    From: asterisk-users On Behalf Of Federico Sent: Thursday, July 13, 2023 8:10 PM
    To: ‘Asterisk Users Mailing List – Non-Commercial Discussion’

    Subject: [asterisk-users] memory leak?

    This is asterisk 16.30.1, after 1 hours, with 17 contacts, yet we need to kill it every two hours because it stops processing calls.

    rasterisk -x “core show channels count”

    43 active channels

    29 active calls

    7260 calls processed

    time rasterisk -x “memory show summary” | sort -n | tail -20

    720985 bytes in deferred free allocations

    775206 bytes in 20242 allocations in file res_rtp_asterisk.c

    1142456 bytes in 18 allocations in file chan_iax2.c

    1190880 bytes in 4392 allocations in file stasis_bridges.c

    1217799 bytes in 6141 allocations in file xmldoc.c

    1508416 bytes in 8782 allocations in file cdr.c

    1514774 bytes in 20797 allocations in file devicestate.c

    1521524 bytes in 18237 allocations in file stasis.c

    2012008 bytes in 18969 allocations in file pbx.c

    3057408 bytes in 42464 allocations in file astobj2_hash.c

    4230560 bytes in 40320 allocations in file rtp_engine.c

    27781422 bytes in 577295 allocations in file stasis_cache.c

    28497984 bytes in 1495 allocations in file
    ../src/pj/pool_policy_malloc.c

    56621945 bytes in 1766936 allocations in file taskprocessor.c

    160393938 bytes in 2227792 allocations in file stasis_message.c

    440540801 bytes in 8647565 allocations in file json.c

    3011728948 bytes in 2923317 allocations in file stasis_channels.c

    3750730746 bytes allocated (69073 in caches) in 16358993 selected allocations

    3750730746 bytes in all allocations

    3751451731 bytes in all allocations and deferred free allocations

    real 0m15.676s

    user 0m0.006s

    sys 0m0.005s

    (one second per contact)

    time rasterisk -x “pjsip show contacts”

    Contact:

    ========================================================================================

    Contact: 1123/sip:1123@X.X.X.X:42700;ob 82a79730c7 NonQual nan

    Contact: 1124/sip:1124@X.X.X.X:12061;ob 056c670307 NonQual nan

    Contact: 1130/sip:1130@X.X.X.X:14302;ob 2fccd6c498 NonQual nan

    Contact: 1131/sip:1131@X.X.X.X:23729;ob 277dda2550 NonQual nan

    Contact: 1132/sip:1132@X.X.X.X:12173;ob 71e2436f7e NonQual nan

    Contact: 1136/sip:1136@X.X.X.X:10085;ob 93300d6b2a Unknown nan

    Contact: 1138/sip:1138@X.X.X.X:14178;ob 9ac8eb538c NonQual nan

    Contact: 1148/sip:1148@X.X.X.X:28858;ob 3f6f6eca2a NonQual nan

    Contact: Reiner_Georgia/sip:X.X.X.X/24 5da85f3be0 NonQual nan

    Contact: Reiner_texas/sip:X.X.X.X/24 5da85f3be0 NonQual nan

    Contact: Reiner_texas_2/sip:X.X.X.X/24 5da85f3be0 NonQual nan

    Contact: Whisl_Telecom/sip:X.X.X.135:5060 5d61346cde NonQual nan

    Contact: fede_arg/sip:X.X.X.X a5993dc66d NonQual nan

    Contact: iplink/sip:X.X.X.X 4fc2b9913d Avail
    203.144

    Contact: nueva_reiner/sip:X.X.X.X/24 5da85f3be0 NonQual nan

    Contact: reiner colombia/sip:X.X.X.X/24 df94afc597 NonQual nan

    Contact: reiner_colombia/sip:X.X.X.X/24 5da85f3be0 NonQual nan

    Objects found: 17

    real 0m14.942s

    user 0m0.003s

    sys 0m0.004s

  • PJSIP will use some memory while it’s loaded, even if nothing is going on. The PJSIP allocations there don’t seem alarming. json.c is using an abnormally large amount of memory, though stasis uses JSON under the hood so that may be the ultimate culprit. You also didn’t answer the previous question. At this point, the output of “memory show allocations” will be more helpful; it’s very large so put it in a pastebin rather than pasting directly.