Hello All,
We've had our system up for a while and noticed this issue occur where the 3cx app appears to stop being able to access the PBX. The symptom is the 3cx app does not have chat functionality, call history, or contacts. Below are the logs from the system. Also, in the dashboard, the call logs are stuck on the same calls and do not update according to the actual active calls. Working with support - but wanted to post here as well.
2023/02/24 12:08:31.165|22664|0388|Err|[GS] GatewayClient: Error occured in channel with remote PBX: Cannot send request: RequestProviderMapping: System.Threading.Tasks.TaskCanc
meout of 100 seconds elapsing.
---> System.TimeoutException: The operation was canceled.
---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
---> System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
---> System.Net.Sockets.SocketException (125): Operation canceled
--- End of inner exception stack trace ---
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)
at System.Net.Http.HttpConnection.FillAsync(Boolean async)
at System.Net.Http.HttpConnection.ReadNextResponseHeaderLineAsync(Boolean async, Boolean foldedHeadersAllowed)
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.SendAsyncCore(HttpRequestMessage request, HttpCompletionOption completionOption, Boolean async, Boolean emitTelemetryStartStop, CancellationToke
--- End of inner exception stack trace ---
--- End of inner exception stack trace ---
at System.Net.Http.HttpClient.SendAsyncCore(HttpRequestMessage request, HttpCompletionOption completionOption, Boolean async, Boolean emitTelemetryStartStop, CancellationToke
at MyPhoneClientChannelNet.Implementation.ClientChannelImpl.SendRequest(IExtensible request, Uri uri, HttpClient client)
at MyPhoneClientChannelNet.Implementation.ClientChannelImpl.DoRequestAsync(IExtensible request)
at MyPhoneClientChannelNet.Implementation.ClientChannelImpl.RequestAsync(IExtensible request)
2023/02/24 12:08:31.166|22664|0388|Exc|System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
---> System.TimeoutException: The operation was canceled.
---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
---> System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
---> System.Net.Sockets.SocketException (125): Operation canceled
--- End of inner exception stack trace ---
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)
at System.Net.Http.HttpConnection.FillAsync(Boolean async)
at System.Net.Http.HttpConnection.ReadNextResponseHeaderLineAsync(Boolean async, Boolean foldedHeadersAllowed)
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.SendAsyncCore(HttpRequestMessage request, HttpCompletionOption completionOption, Boolean async, Boolean emitTelemetryStartStop, CancellationToke
--- End of inner exception stack trace ---
--- End of inner exception stack trace ---
at System.Net.Http.HttpClient.SendAsyncCore(HttpRequestMessage request, HttpCompletionOption completionOption, Boolean async, Boolean emitTelemetryStartStop, CancellationToke
at MyPhoneClientChannelNet.Implementation.ClientChannelImpl.SendRequest(IExtensible request, Uri uri, HttpClient client)
at MyPhoneClientChannelNet.Implementation.ClientChannelImpl.DoRequestAsync(IExtensible request)
at MyPhoneClientChannelNet.Implementation.ClientChannelImpl.RequestAsync(IExtensible request)
at GatewayServiceImpl.MyPhoneClient.GatewayClient.RequestAsync(IExtensible request)
at GatewayServiceImpl.MainService.SendMapping()
2023/02/24 12:08:33.887|22664|0386|Err|MYPHONE: Skip login for 1214. Server is busy. (traceid=0HMO8LSFQ8LA3:00000002). Current requests: 145.
2023/02/24 12:08:35.865|22664|0374|Err|MYPHONE: Skip login for 1081. Server is busy. (traceid=0HMO8LSFQ8LA3:00000004). Current requests: 145.
2023/02/24 12:08:43.580|22664|0372|Err|MYPHONE: Skip login for 1078. Server is busy. (traceid=0HMO8LSFQ8LA5:00000003). Current requests: 147.
2023/02/24 12:08:53.419|22664|0297|Err|MYPHONE: Skip login for 1214. Server is busy. (traceid=0HMO8LSFQ8LA6:00000002). Current requests: 148.
2023/02/24 12:08:55.864|22664|0393|Err|MYPHONE: Skip login for 1081. Server is busy. (traceid=0HMO8LSFQ8LA6:00000004). Current requests: 148.
We've had our system up for a while and noticed this issue occur where the 3cx app appears to stop being able to access the PBX. The symptom is the 3cx app does not have chat functionality, call history, or contacts. Below are the logs from the system. Also, in the dashboard, the call logs are stuck on the same calls and do not update according to the actual active calls. Working with support - but wanted to post here as well.
2023/02/24 12:08:31.165|22664|0388|Err|[GS] GatewayClient: Error occured in channel with remote PBX: Cannot send request: RequestProviderMapping: System.Threading.Tasks.TaskCanc
meout of 100 seconds elapsing.
---> System.TimeoutException: The operation was canceled.
---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
---> System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
---> System.Net.Sockets.SocketException (125): Operation canceled
--- End of inner exception stack trace ---
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)
at System.Net.Http.HttpConnection.FillAsync(Boolean async)
at System.Net.Http.HttpConnection.ReadNextResponseHeaderLineAsync(Boolean async, Boolean foldedHeadersAllowed)
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.SendAsyncCore(HttpRequestMessage request, HttpCompletionOption completionOption, Boolean async, Boolean emitTelemetryStartStop, CancellationToke
--- End of inner exception stack trace ---
--- End of inner exception stack trace ---
at System.Net.Http.HttpClient.SendAsyncCore(HttpRequestMessage request, HttpCompletionOption completionOption, Boolean async, Boolean emitTelemetryStartStop, CancellationToke
at MyPhoneClientChannelNet.Implementation.ClientChannelImpl.SendRequest(IExtensible request, Uri uri, HttpClient client)
at MyPhoneClientChannelNet.Implementation.ClientChannelImpl.DoRequestAsync(IExtensible request)
at MyPhoneClientChannelNet.Implementation.ClientChannelImpl.RequestAsync(IExtensible request)
2023/02/24 12:08:31.166|22664|0388|Exc|System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
---> System.TimeoutException: The operation was canceled.
---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
---> System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
---> System.Net.Sockets.SocketException (125): Operation canceled
--- End of inner exception stack trace ---
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)
at System.Net.Http.HttpConnection.FillAsync(Boolean async)
at System.Net.Http.HttpConnection.ReadNextResponseHeaderLineAsync(Boolean async, Boolean foldedHeadersAllowed)
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.SendAsyncCore(HttpRequestMessage request, HttpCompletionOption completionOption, Boolean async, Boolean emitTelemetryStartStop, CancellationToke
--- End of inner exception stack trace ---
--- End of inner exception stack trace ---
at System.Net.Http.HttpClient.SendAsyncCore(HttpRequestMessage request, HttpCompletionOption completionOption, Boolean async, Boolean emitTelemetryStartStop, CancellationToke
at MyPhoneClientChannelNet.Implementation.ClientChannelImpl.SendRequest(IExtensible request, Uri uri, HttpClient client)
at MyPhoneClientChannelNet.Implementation.ClientChannelImpl.DoRequestAsync(IExtensible request)
at MyPhoneClientChannelNet.Implementation.ClientChannelImpl.RequestAsync(IExtensible request)
at GatewayServiceImpl.MyPhoneClient.GatewayClient.RequestAsync(IExtensible request)
at GatewayServiceImpl.MainService.SendMapping()
2023/02/24 12:08:33.887|22664|0386|Err|MYPHONE: Skip login for 1214. Server is busy. (traceid=0HMO8LSFQ8LA3:00000002). Current requests: 145.
2023/02/24 12:08:35.865|22664|0374|Err|MYPHONE: Skip login for 1081. Server is busy. (traceid=0HMO8LSFQ8LA3:00000004). Current requests: 145.
2023/02/24 12:08:43.580|22664|0372|Err|MYPHONE: Skip login for 1078. Server is busy. (traceid=0HMO8LSFQ8LA5:00000003). Current requests: 147.
2023/02/24 12:08:53.419|22664|0297|Err|MYPHONE: Skip login for 1214. Server is busy. (traceid=0HMO8LSFQ8LA6:00000002). Current requests: 148.
2023/02/24 12:08:55.864|22664|0393|Err|MYPHONE: Skip login for 1081. Server is busy. (traceid=0HMO8LSFQ8LA6:00000004). Current requests: 148.
