Toggle sidebar Toggle sidebar

Huge logs

Status
Not open for further replies.

lucasasdelli

Joined
Oct 28, 2007
Messages
7
Reaction score
0
Hi all,

since version 5.0.3790, I experience some system freezes due to huge log creation: the last night was something like 40GB log files.

Here the message, repeated near infinite times:
.\CM_Stack.cpp(163)|Critical0|SipStack|CallMgr::Stack::thread:[CM100002]: *!* Exception detected: bad allocation *!*<br>

It seems that shutting down 3Cx' Apache webserver stops the issue.

Cheers
Luca
 
I have the same issue... any solution?

The unique way to stop then are to reiniciate the entire server..., please... any help, is that a bug in the program?

Please... help....
 
I have had the same problem on a test system that was not actively used or monitored for a while, except that I can't tell what the problem log entries were, because they completely filled the disk drive, to the point where it was a struggle to log on, because there was not enough space to update my profile.

It also resulted in the corruption of the system database and, since it was a test system, I did not have a backup, so the system is off at the moment.

The obvious solution for me, assuming the log messages were legitimate, is to schedule scripts to delete old logs, but it would seem a better idea for 3CX to have a mechanism to automatically do this.
 
Hello,
Today I had the same problem. I have an 3CX running since several months ago, with no problems.
Today, the partition where there is the 3CX was full, and looking for the problem, I detected that at
C:/Program Files/3CX PhoneSystem/Data/logs/Backup/20080508
was full of files
3CXPhoneSystem.HHMMSS

And these files are full of the same message:
08:07:18.626|.\CM_Stack.cpp(163)|Critical0|SipStack|CallMgr::Stack::thread:[CM100002]: *!* Exception detected: bad allocation *!*<br>

I stopped and started the service, but the logs kept appearing.

Then, in the Web interface, in General Settings, I saw that the level of log was for debugging purposes, and I changed it to low level, so seems that this log is not generated again.

I have the last version of the 3CX (5.1.41280), and I want to know if someone knows the reason of this log, and if I can do something to repair the problem.

I have no VoIP providers. We use Linksys 3102 as Gateways, and each gateway has an analogic phone and a IP phone (3108 3Com).

Thanks and best regards.
 
Hello,
I detected in another PC with the 3CX the same problem at the same date, so it seems very weird.
In the second PC, there was the debugging level of logs, too.
I made the same steps, to remove the log files and change the debug level to low. Nevertheless, the log is no writing now, but the problem still ocurring.

Thanks and best regards.
 
I suspect you have installed some new hardware recently, or, probably had changed some settings. Exceptions in stack usually happens when there's badly formed SIP messages, for example with unreasonable long Contents-Length: 234234324234, or something like this. I would ask you to capture a SIP messages at one of the 3CX boxes at the moment when exceptions happens, if possible. Wireshark or Ethereal sniffers is just great for this task. Than I can try to find what is broken and make a fix in our SIP stack to eliminate exceptions.
 
BTW, good place to look: check display names of your devices. They shouldn't contain any "funny" symbols. Especially - star symbol '*'.
 
Thanks for the answer.
I don´t think any value was changed recently, the 3CX is working for several months, and until last week, we had no problems with it.

I am capturing the packets with Wireshark, but at this moment there are no huge logs, and I can´t force them, because I don´t know why they produce themselves.

When I get the huge logs, I will send the Wireshark capture.

About device names, I don´t know if you refer gateway names, extension names, etc... I only use letters, numbers, some spaces and _. And these name are the same since several months.

Thanks and best regards,
 
I can also report the very same error at practically the same time as rduran, and possibly I have also located what triggered it.

My system (5.1.4128.0 on an XP machine) started generating the sipstack error messages at 00.15.50 on 8 May (Swedish time = GMT+1). It kept going until it had filled up its partition with some 12 GB a few hours later.

But before this, it had spent the previous two days trying to speak to the 3CX STUN server at 64.69.76.23, generating a timeout error every three seconds. I don't know if the server was down or if there was some other problem, but it's clear from the logs that my system did not get a proper response anytime from 00.32.06 on 6 May, until it stopped trying and went into the sipstack error loop.

Would be interesting to know if rduran can see the same pattern on his two machines.

The logs look like this:

[6 May]

[last normal STUN request]
00:32:06.513|.\StunClient.cpp(210)|Log2|STUN|StunClient::onInitTests:[CM506001]: STUN request to resolve SIP external IP:port mapping is sent to STUN server 64.69.76.23:3478 over Transport 192.168.0.2:5060<br>

[timeout errors start]
00:52:03.783|.\StunClient.cpp(210)|Log2|STUN|StunClient::onInitTests:[CM506001]: STUN request to resolve SIP external IP:port mapping is sent to STUN server 64.69.76.23:3478 over Transport 192.168.0.2:5060<br>
00:52:06.837|.\StunClient.cpp(252)|Log2|STUN|StunClient::onTestTout:[CM506004]: STUN request to STUN server 64.69.76.23 has timed out; used Transport: 192.168.0.2:5060<br>
00:52:09.892|.\StunClient.cpp(252)|Log2|STUN|StunClient::onTestTout:[CM506004]: STUN request to STUN server 64.69.76.23 has timed out; used Transport: 192.168.0.2:5060<br>
00:52:12.963|.\StunClient.cpp(252)|Log2|STUN|StunClient::onTestTout:[CM506004]: STUN request to STUN server 64.69.76.23 has timed out; used Transport:

[continues every three seconds until]

[8 May]

00:15:50.207|.\StunClient.cpp(252)|Log2|STUN|StunClient::onTestTout:[CM506004]: STUN request to STUN server 64.69.76.23 has timed out; used Transport: 192.168.0.2:5060<br>
00:15:50.441|.\CM_Stack.cpp(163)|Critical0|SipStack|CallMgr::Stack::thread:[CM100002]: *!* Exception detected: bad allocation *!*<br>
00:15:50.441|.\CM_Stack.cpp(163)|Critical0|SipStack|CallMgr::Stack::thread:[CM100002]: *!* Exception detected: bad allocation *!*<br>
00:15:50.441|.\CM_Stack.cpp(163)|Critical0|SipStack|CallMgr::Stack::thread:[CM100002]: *!* Exception detected: bad allocation *!*<br>
 
Thank you for input, I will check for memory leaks in case of STUN request failed. Meanwhile I would suggest you to upgrade to latest V5.1 build which is 4510.
 
Hello,
My problem was the same, with the STUN Server error. Sorry because I didn´t mencioned it, but I didn´t relation the two issues.
I can show the logs in the second server,
The problems with the STUN server started on 6th,

13:30:42.640|.\StunClient.cpp(210)|Log2|STUN|StunClient::onInitTests:[CM506001]: STUN request to resolve SIP external IP:port mapping is sent to STUN server 64.69.76.23:3478 over Transport 192.168.99.5:5060<br>
13:30:42.843|.\StunClient.cpp(115)|Log2|STUN|StunClient::process:[CM506003]: Resolved SIP external IP:port has changed to (213.60.130.224:4247) on Transport 192.168.99.5:5060<br>
13:50:42.999|.\StunClient.cpp(210)|Log2|STUN|StunClient::onInitTests:[CM506001]: STUN request to resolve SIP external IP:port mapping is sent to STUN server 64.69.76.23:3478 over Transport 192.168.99.5:5060<br>
13:50:43.203|.\StunClient.cpp(115)|Log2|STUN|StunClient::process:[CM506003]: Resolved SIP external IP:port has changed to (213.60.130.224:5517) on Transport 192.168.99.5:5060<br>

Later, on same day, the logs changed to
00:55:25.266|.\StunClient.cpp(210)|Log2|STUN|StunClient::onInitTests:[CM506001]: STUN request to resolve SIP external IP:port mapping is sent to STUN server 64.69.76.23:3478 over Transport 192.168.99.5:5060<br>
00:55:28.325|.\StunClient.cpp(250)|Log2|STUN|StunClient::onTestTout:[CM506004]: STUN request to STUN server 64.69.76.23 has timed out; used Transport: 192.168.99.5:5060<br>
00:55:31.385|.\StunClient.cpp(250)|Log2|STUN|StunClient::onTestTout:[CM506004]: STUN request to STUN server 64.69.76.23 has timed out; used Transport: 192.168.99.5:5060<br>
00:55:34.445|.\StunClient.cpp(250)|Log2|STUN|StunClient::onTestTout:[CM506004]: STUN request to STUN server 64.69.76.23 has timed out; used Transport: 192.168.99.5:5060<br>
00:55:37.504|.\StunClient.cpp(250)|Log2|STUN|StunClient::onTestTout:[CM506004]: STUN request to STUN server 64.69.76.23 has timed out; used Transport: 192.168.99.5:5060<br>

And finally, on 8th in the morning, started the logs with the exception until the hard disk was full, and I rebooted the 3CX and changed the logs lo low level.

Yesterday, I tried to capture SIP packets in the error, but I wasn´t able to reproduce the issue.
I hope we can have a solution to resolve this.

Thanks
 
Hello,
This week the problem happened again, so I had to reboot the 3CX.
I don´t know if someone has detected the issue, and there is some solution.
I still have the same 3CX version: 5.1.4128.0. I don´t know if it is recommended to upgrade to solution this.

Thanks and best regards,
 
rduran said:
Hello,
This week the problem happened again, so I had to reboot the 3CX.
I don´t know if someone has detected the issue, and there is some solution.
I still have the same 3CX version: 5.1.4128.0. I don´t know if it is recommended to upgrade to solution this.

Thanks and best regards,
Click to expand...

Rduran

1) What firewall is being used here?
2) Is this a dynamic IP?
 
Hello Keith,

The 3CX Server is in a Windows 2003 Server in a subnet with static IP. The phones are in another subnet, but the router make connectivity at level 3.
There are some firewall rules: the only traffic allowed between 3CX Server and IP phones are: 5060 UDP, 5061 UDP and 9000 - 9009 UDP. The system is working ok during months, and suddenly this issue happens, so I don´t think this have relation with ports and firewall rules.

Thanks for your help
 
My home router periodically stops to forward requests and I can not browse internet during that time. It can work OK a month or so, though. I don't think it's a bug of Firefox or IE.
 
Hello Archie,

Sorry if I have no express myself correctly. I don´t use to speak English, so my message can seem angry. My problem is no very important, and the 3CX is working ok, so I have no complaint. I only want to know if someone know the source of this problem, to correct it.

I think there is no problem with the firewall, but I can be wrong. If you think I have to check the firewall policies, and add some port to permit a better comunication there is no problem.

Thanks for your help,
Rduran
 
No, I didn't take your message as an angry post. I'm just trying to tell that sometimes hardware also may fault. And it is much more difficult to diagnose such faults. As in example with my home router. I still don't know what happens when I lose access to internet. It just stops to work. Though I do not touch any forwarding or settings of router. After reboot it again works OK.
 
Ok.

I don´t give this issue priority, because restart the system once each several months, is no problem. The problem is that I have the 3CX in the same partition that the operating system, so the huge amount of logs use all the partition. I think I will move the 3CX to another partition to avoid this issue.

Thanks for your help,
Rduran
 
You can limit the amount of logs 3CX will write and keep. But this requires manual configuring of 3CXPhoneSystem.ini files. For the beginning try to set logs verbosity to minimum. May be it will be enough for you.
 
Yes, the first time I had the log level in verbose level, but then I changed it to minimum, so it has the minimum level now.
I opened the 3CXPhoneSystem.ini, and during the file, I find Severity=Log Error Critical

Thanks,
 
Status
Not open for further replies.

Latest Posts

Forum statistics

Threads
111,887
Messages
589,562
Members
164,753
Latest member
GemmaC
3CX Logo Footer Copyright © Sitemap Privacy
Top Bottom