Avaya (Nortel) 1100-series SIP phone unregister issue.

[lowen]

I have 40 Nortel/Avaya 1120E phones that were donated to us (we're a 501(c)(3) nonprofit), and I have set up my TFTP provisioning server for those phones to deliver the appropriate SIP firmware and configuration files to set up for the connection to 3CX as the SIP proxy.

The phones register, and outgoing and incoming calls work ok. After 30 minutes, however, I see the following in my log (I'm including the register message as well):

27-Sep-2013 05:13:39.104   [CM504002]: Endpoint Extn:854: a contact is unregistered. Contact(s): []
27-Sep-2013 04:43:37.214   [CM504001]: Endpoint Extn:854: new contact is registered. Contact(s): [sip:[email protected]:5060 / 854]

After the phone is unregistered, I can still make outgoing calls on that extension, but incoming, as you may have figured, goes straight to voicemail. I can make the time shorter by shortening the Avaya's REG_REFRESH_INTERVAL parameter in the [DEVICE_CONFIG] file to below 1800 seconds, but making that parameter greater than 1800 seconds does not make the time longer; the default for REG_REFRESH_INTERVAL is 86400 seconds. Now the phones do re-register periodically, typically around 18-22 hours after unregistering, but the re-register doesn't appear to be correlated to much of anything else, like DHCP lease expiry or similar.

Now, I know the Avaya 1100-series is not officially supported, but I have seen posts here and there on various forums that say people are using these phones, and they're working, but no one seems to post how they got them working (and I've done a lot of searching!).

This is 3CX V11, SP4, build 28976. The Avaya firmware is the latest version, 4.3.18 (SP2, I believe). I've tried older SIP firmware on the phones; neither 2.2 nor 3.2 fix the issue.

Oh, and the phones do not do this if I set them to point to a FreePBX Distro (Asterisk-based) server that I set up specifically to test this issue; the phones Just Work (TM) like one would expect, using the same settings as with 3CX, same firmware, and using SIP, not UNIstim (UNIstim, the native protocol, is supported on Asterisk, but not 3CX). So the basic phone configuration sent to the phones through TFTP seems to be OK (and I can post that if needed).

I have six Cisco 7960's (SIP firmware 8.5.4) and four Cisco ATA-186's (latest 3.2 SIP firmware) working perfectly with the 3CX server, but these Avaya phones in theory should work. It seems like 3CX isn't handshaking with the phones the same way as FreePBX is; to the point that when registered to FreePBX the phones show the date and time, but not when registered to 3CX. Now, the 7960's registered to 3CX do show the date and time. Incoming and outgoing calls to and from the 7960's and the ATA-186's work perfectly, and DID's coming in from our RBS/CAS T1 via a Patton SN4960 are properly routed. So the basic system is working, except for the Avaya phones.

In any case, if anyone has made an Avaya 1100-series phone (1120E specifically, running the 4.3 SIP firmware) work properly with 3CX, please post how you did it. And any advice on how to get these phones working with 3CX would be appreciated (getting other phones is not an option, so please don't suggest that......).

Thanks in advance.

 

[leejor]

A few years ago I converted an Avaya/Nortel set to SIP and set it up. I believe it was the 1130e. I came across this site and it was a lot of help.

http://blog.michaelfmcnamara.com/2011/01/avaya-ip-1100-series-ip-phone-upgrade-to-sip/#comment-18496

I also ran into a problem with configuring the set to re-register on a timely basis. There was no option in the sets GUI. From what I had gathered it had to be dome via a configuration file uploaded to the set, which I never bothered to do as i had other sets to work with. Some day I may go back and "play" with it again.
If you go though that site is does have a lot of information on configuring the 1100 series, and Michael is very good at getting back to any questions that you have. Hopefully it will be of some help.

 

[lowen]

leejor,

Thanks for the pointer. I had found your thread, and was hoping that you had gotten the sets to work. My thread on his forum site:

http://forums.networkinfrastructure.info/nortel-ip-telephony/1100-series-with-3cx/msg14924/#msg14924

I've tried a number of things in the configuration files, but haven't found anything yet that makes the unregister go away. I guess my next step is to put a tap on the ethernet and see what FreePBX is doing that 3CX is not.

 

[leejor]

As I said, my assumtion is that there is no registration interval option in the GUI for the set, therefore is isn't doing it. I'm assuming that it is et , along with other features on the phone, in a configuration file that you must create and download.

I know that with many sets, upon first registration, a re-registration interval is "suggested" to the device. Some can choose to ignore it , or it can be overridden by a shorter interval, that is manually provisioned.

For me, this was a one-off phone that I had a chance to try out, so getting it working wasn't critical and i really didn't have a lot of time to spend on it.

A key to making these work may be a generic configuration file, that provides the basics (like re-registration interval time), and then modifying it as needed per set.

It sounds like you have the sets working with FreePBX, is that correct? If so it would be interesting to examine the differences in the SIP messaging that takes place during the initial registration. How often does the set re-register with FreePBX?

 

[lowen]

Yes, I have them working with FreePBX. I'll need to do a bit of replumbing, network-wise, and set up to do a packet trace to see what's different.

It seems to me that it's not just no re-registering with 3CX, but that it's actively unregistering, at least that's the log message in the 3CX activity log.

I'll know more, hopefully, after getting a packet trace of the phone's setup with 3CX and with FreePBX.

The configuration file I'm using is the same between 3CX and FreePBX, for what it's worth.

 

[leejor]

It seems to me that it's not just no re-registering with 3CX, but that it's actively unregistering, at least that's the log message in the 3CX activity log.

You suspect that it is actually actively UN-registering?

If all is left at default, 3CX will give a device 30 minutes (after every registration), to re-register before assuming it is no longer there.

 

[lowen]

You suspect that it is actually actively UN-registering?

Yes, I do suspect this, but I of course reserve the right to be wrong, too.

If I set up the 1120e's configuration file with REG_REFRESH_INTERVAL set to 360 seconds (this is documented in the Avaya SIP firmware docs) I get the following in the 3CX activity log:

01-Oct-2013 10:50:32.663   Endpoint Extn:823 has removed contact <sip:[email protected]:5060>
01-Oct-2013 10:50:32.663   [CM504002]: Endpoint Extn:823: a contact is unregistered. Contact(s): []
01-Oct-2013 10:44:17.811   Endpoint Extn:823 has added contact <sip:[email protected]:5060> for device Dev(66):[sip:[email protected]:5060 / 823]
01-Oct-2013 10:44:17.811   [CM504001]: Endpoint Extn:823: new contact is registered. Contact(s): [sip:[email protected]:5060 / 823]

So that makes it go unregistered after about 360 seconds instead of 1800. I've set REG_REFRESH_INTERVAL above 1800 and that doesn't make the unregister interval go past 1800 seconds, though.

However, while I was doing this test with this extension, I did notice, thanks to a typo in users.dat, the following:

01-Oct-2013 10:42:02.322   Endpoint Extn:823 has removed contact <sip:[email protected]:5060>
01-Oct-2013 10:42:02.322   [CM504002]: Endpoint Extn:823: a contact is unregistered. Contact(s): []
01-Oct-2013 10:25:01.497   Endpoint Extn:823 has removed contact <sip:[email protected]:5060>
01-Oct-2013 10:25:01.497   [CM504002]: Endpoint Extn:823: a contact is unregistered. Contact(s): [sip:[email protected]:5060 / 823]
01-Oct-2013 10:18:36.274   Endpoint Extn:823 has added contact <sip:[email protected]:5060> for device Dev(65):[sip:[email protected]:5060 / 823]
01-Oct-2013 10:18:36.274   [CM504001]: Endpoint Extn:823: new contact is registered. Contact(s): [sip:[email protected]:5060 / 823,sip:[email protected]:5060 / 823]
01-Oct-2013 10:11:33.380   Endpoint Extn:823 has added contact <sip:[email protected]:5060> for device Dev(64):[sip:[email protected]:5060 / 823]
01-Oct-2013 10:11:33.380   [CM504001]: Endpoint Extn:823: new contact is registered. Contact(s): [sip:[email protected]:5060 / 823]

The two IP addresses above are the same phone; not sure why it's getting a different IP address each boot from the Cisco switch, but, there it is. I see the first IP, 192.168.1.166, unregistering after .170, even though .170 registered later.

So does this mean that 3CX is taking the initiative to unregister? Why when I change the registration refresh interval timer on the phone to 360 would 3CX then unregister the phone after 360 seconds (+/- a few seconds)?

On another extension, however, I looked through last night's logs and found this:

30-Sep-2013 19:18:23.473   [CM504002]: Endpoint Extn:854: a contact is unregistered. Contact(s): []
30-Sep-2013 18:47:55.508   [CM504001]: Endpoint Extn:854: new contact is registered. Contact(s): [sip:[email protected]:5060 / 854]
30-Sep-2013 18:18:53.650   [IPBL] Packet from banned IP/range: ip = 10.4.250.27; Comment: PBX: blocked for too many failed authentications
30-Sep-2013 18:17:53.552   Blacklisted (Too many failed auth)
IP = 10.4.250.27; Failed auth: 0; unauth: 0; auth: 0; 407: 1000

Notice the blacklisted IP and what is said about it, and that it successfully re-registers later. Is it possible the format of the re-registration request isn't what 3CX wants to see? I need to set up a network tap......

In any case, FreePBX (Asterisk 11) and the 1120E get along fine.... further pointing me to getting a tap online to watch the actual traffic.....

 

[leejor]

What I would suggest , is to use a hub (not a switch, you won't see any data if you use a switch) between the set and the network then plug a PC into the hub that is running Wireshark. You will then be able to see all messaging between the set and 3CX. Anything iI could say about what or why this is happening would simply be a guess. The way one expects it to work, is that 3CX will wait 30 minutes (or until it attempts to send a call to the set and fails) before assuming that a set is no longer there and removes registration. Given that you are seeing an UN-registration in an interval less than that make me believe that the set is indeed sending something. You would have to confirm that. there may very well be something in the configuration file, or a setting in the phone, that is causing this. Don't forget, not all SIP devices follow SIP standards.

As far as that other IP that is being blacklisted...I assume that it is valid.

I have seen blacklisting happen on valid IP's, at times, usually from a remote location but I've also seen it happen on the local LAN. Sometimes tweaking the blacklist settings is enough to fix that. Or...you can whitelist the IP, or range of IP's (local LAN), if it becomes a real headache. Again, Wireshark may help in uncovering why the blacklisting is happening.

 

[lowen]

Ok, I think I may have a bit more clue as to what is going on, maybe.

I set up for a packet capture with a system running http://www.networksecuritytoolkit.org to do the capture. NST has a really nice web GUI, and is highly recommended for this sort of thing.

Since our backbone here is all on Cisco 7600-series 'routing switches' it was a snap to take the port into which the phone was plugged and do a SPAN monitoring session to a link to the NST box.

The capture session clearly shows that the phone is not unregistering itself.

The phone sends a REGISTER packet; the 3CX proxy sends back 407 Proxy Authentication Required; the phone sends another REGISTER packet, and I see the 3CX proxy send back a 200 Ok. The two REGISTER packets have Expires set to the value of REG_REFRESH_INTERVAL (for testing I used 360 seconds). 360 seconds later there is no forthcoming REGISTER packets.

What there is, however, about a half-second after the 200 reply to the second REGISTER packet, and after a NOTIFY packet from the 3CX proxy, is an exchange of SUBSCRIBE packets from the phone, with various replies from the 3CX proxy. All of these SUBSCRIBE packets have Expires set to 86400, not 360. The sequence goes: SUBSCRIBE, 407, SUBSCRIBE, 489 Event Package not Supported, SUBSCRIBE, 407, SUBSCRIBE, 500 Server Internal Error, SUBSCRIBE, 407, SUBSCRIBE, 200 Ok.

And then another NOTIFY, with a expiration set to 86400. But, as has been said, 3CX is supposed to expire after 1800 seconds, but I see clearly here a NOTIFY with Subscription-State: active;expires=86400

So, I've edited my custom parameters in 3CX to allow a MAXEXPIRES of greater than 86400, rebooted the 3CX box, and we'll see what happens. I have the NST box set for a 25 hour capture, and we'll see what we'll see.

Also, I set the 1120 to pull date and time from an NTP server, since 3CX is not providing that to the phone, whereas FreePBX does.

Afterward, I'll reset the capture and set up a phone to register to the FreePBX system and see what the difference really is.

 

[lowen]

Ok, I have a couple of additional items of information.

First, I despise patch Tuesday and Window's addiction to rebooting after updates, as the 3CX host's update-triggered reboot kept me from seeing the full story on the 86400 second re-register and if it worked or not.

Second, I got a trace of the SIP registration between the 1120 and FreePBX. It is quite a bit different from the 3CX exchange. The phone sends the REGISTER, and FreePBX replies with a 401 Unauthorized, to which the phone sends a second REGISTER with the auth info, and FreePBX replies with a 200 Ok and Expires set to 3600 (the REGISTER requests had Expires: set to 86400). The 3CX 200 Ok did not have the Expires set in reply to the registration. I've started another capture session to see if the phone actually re-registers to FreePBX after 3600 seconds.

During all of this an exchange of OPTIONS requests from FreePBX to the phone also occurs, and there is the same NOTIFY as with 3CX. So far the difference I can see is the Expires: being set by FreePBX in the 200 Ok response to the REGISTER, and its absence in the 200 Ok response to the REGISTER from 3CX.

The SUBSCRIBE chain and its replies is also a bit different, but I've not dug into all the details yet; in a nutshell, it goes SUBSCRIBE, 401, SUBSCRIBE, 489 Bad Event, SUBSCRIBE, 401, SUBSCRIBE, 489, SUBSCRIBE, 401, SUBSCRIBE, 200 Ok. The 489's appear to be related to the SUBSCRIBE options 'Event: network-redirection-reminder' and 'Event: ua-profile' requests from the phone. The 200 Ok from FreePBX after the last SUBSCRIBE has an Expires: of 3600, in contrast to 3CX's Expires: of 86400 there, before I set MAXEXPIRES to 86400.

If needed I can sanitize the captures for more experienced analysis, since this will be the first time I've analyzed SIP (not my first ethernet analysis, just first SIP analysis).

 

[leejor]

Since I have not had any experience with FreePBX, and only a bit with the Nortel/Avaya sets themselves, and given that they are not "officially" supported by 3CX, it is probably going to take someone more familiar with BOTH systems, to come up with an explanation of the differences.

Given that the Nortel/Avaya sets seem to be originally designed for a proprietary system first, and SIP is somewhat an afterthought, there may be some non-standard things going on.

 

[lowen]

Ok, after capturing for another couple of hours, I have more info yet again.

The 1120 is honoring the Expires: 3600 header in the 200 Ok reply from FreePBX, and is in fact re-registering at 3569 seconds, as confirmed by packet trace.

In my trace from last night, the very same phone with the same exact configuration (other than the SIP Proxy IP address), only sent a REGISTER packet once in 73,000 seconds. I stopped the capture at that point, since the 3CX box had rebooted for updates.

3CX is not sending an Expires: value back in the 200 Ok response to registering, and the 1120 is not attempting to re-register even if configured by REG_REFRESH_INTERVAL to do exactly that. So, IMHO at least, both 3CX and the 1120 are at fault.

Now, looking more carefully at the traces, 3CX is sending the requested expire time in the Contact: header, but it's not sending a separate Expires: header. The former is required by RFC3261; the latter is shown in the example registration session setup in the RFC. FreePBX is sending both the expires parameter and the Expires: header, and the 1120 is happy with that, but it is not happy without the Expires: header.

 

[SY]

...
Now, looking more carefully at the traces, 3CX is sending the requested expire time in the Contact: header, but it's not sending a separate Expires: header. The former is required by RFC3261; the latter is shown in the example registration session setup in the RFC. FreePBX is sending both the expires parameter and the Expires: header, and the 1120 is happy with that, but it is not happy without the Expires: header.

Hi lowen,

Regarding RFC 3261 and Expires header.

"Expires:" header is optional for REGISTER request/answer.
Formally, the samples which you are referring to, do not confirm even requirements which are specified by "RFC" itself

...
The registrar returns a 200 (OK) response. The response MUST
contain Contact header field values enumerating all current
bindings. Each Contact value MUST feature an "expires"
parameter indicating its expiration interval chosen by the
registrar.
...
Example:
...
Contact: <sip:[email protected]>
Expires: 7200
...

It is not a big problem, because the description of REGISTER method contains explanation - Expires header can be used to specify value for the contacts which do not have expires parameters:

...
The 200 (OK) response from the registrar contains a list of Contact
fields enumerating all current bindings. The UA compares each
contact address to see if it created the contact address, using
comparison rules in Section 19.1.4. If so, it updates the expiration
time interval according to the expires parameter or, if absent, the
Expires field value. The UA then issues a REGISTER request for each
of its bindings before the expiration interval has elapsed.

Regards

 

[lowen]

SY,

First, thanks much for taking the time to respond.

"Expires:" header is optional for REGISTER request/answer.

Yes, this is correct. For whatever reason, the Avaya 1120E SIP implementation is treating it as if it weren't optional. While that is a bug in the implementation, at the same time, as you noted, RFC3261's examples section contains it. Asterisk 11, the backend of the FreePBX distribution, is covering both bases with the expires parameter and the Expires: header.

If I used Facebook much, I'd ask for this as a feature request, just to aid in interoperability with 3CX, that the Expires: header be set to the same value as the expires parameter in the Contact: header in the responses to the REGISTER request. While I have no knowledge of how difficult that may or may not be, I wouldn't think it would be a difficult change, and I would be glad to field trial it for 3CX.

I do realize that 3CX's official support channels aren't supposed to assist with making it work with unsupported hardware, which is why I haven't gone that path with this. But having a 32 call license of 3CX with only 7 working phones (Cisco 7960) and four two-port ATA's (Cisco ATA 186's) is bit of a letdown. Having that plus 40 1120E phones that work fine with a different system is also a bit of a letdown, especially with the 32 call licensed 3CX system.

So I would be a happy camper to have a version of 3CX that would set Expires: as well as setting the expires parameter of the Contact: header.... 8)

Now, I am no expert at reading SIP traces, but this is my best analysis so far of what seems to be going on. I'll be glad to share sanitized packet traces if those would be of help.

 

[lowen]

UPDATE for Nortel/Avaya 1120E with SIP Firmware 4.4.

Avaya released newer firmware for these phones, and I've been trying again to get them to work. So far, for the first day, the same issue is found with the 4.4 firmware as with the previous SIP firmwares. However, at some point the phones decide to try again, typically sometime after 10 hours and before 20 hours (probably a random time within 24 hours....) and at that point the phones re-register and stay registered properly. I have not yet done a packet trace on the updated firmware, but will be doing so in the next few days.

 

[leejor]

However, at some point the phones decide to try again, typically sometime after 10 hours and before 20 hours (probably a random time within 24 hours....) and at that point the phones re-register and stay registered properly.

So...are you saying that after this random registration time, the set continues to re-register every 30 minutes?

 

[lowen]

leejor,

It appears that they do then start re-registering, although I have not determined the timing as yet. I set my 3CX to allow a max expires of 86400 instead of 1800.

I have just gotten the packet sniffer hooked back up, and have instrumented a packet capture set for 200,000 seconds to monitor a freshly booted 1120E. I set REG_REFRESH_INTERVAL to 28800 seconds (8 hours) so that if I get a complaint about calls not being received I can just tell the user in question (after verifying that the phone is actually unregistered) to power cycle their phone. That way it will stay up all day. This I consider to be a workaround until the firmware is actually completely fixed.

So far I haven't had any go unregistered after being up over a day; and I have twelve of them deployed at the moment, all successfully remaining registered after several days. We'll see how it goes the next time updates require a reboot of the server.

Once I get some results from the packet capture of this one 1120E I'll post them.

 

[lowen]

Ok, a bit more info.

All seconds listed are relative to the bootup of the phone.

I see the 1120E doing the initial registration, with Expires set to 28800, and the 200 OK comes back with Expires=28800. I see SIP_PINGs going out every 60 seconds, with a 407 Proxy Auth Required returned back. REG_REFRESH_INTERVAL is set to 28,800 seconds. After 16,000 seconds, I see a TFTP refresh of the configuration (I don't see a DHCP exchange at that time). I do not see a REGISTER after 28,800 seconds, but SIP_PINGS are continuing, with the 407 reply.

After 60,150 seconds, the 3CX box replies to the SIP_PINGS a 403; after this, the SIP_PINGS go unanswered until 60,300 seconds, at which point the phone tries to connect to the second proxy (which doesn't exist, and it actually is attempting, through SRV DNS requests, to find sip:domain.tld, but in my case this points to our e-mail server, which causes the phone to fail back, as there are no SIP services on the e-mail server. It tries to REGISTER with the second server for about a minute and a half, at which point it REGISTERS with the 3CX box again (but can't, because it's still blacklisted). It fails over to the domain.tld server again, which still can't register, and repeats this process another twelve times, finally reconnecting to the 3CX box at 61,950 seconds. At this point it re-registers, again with an Expires of 28800.

At 86400 seconds, it does the DHCP refresh and ACK.

At 90,770 seconds, the 1120E does indeed attempt a REGISTER, 30 seconds shy of the expiry of REG_REFRESH_INTERVAL, and it is successful.

At 111,450 or so the 3CX box again either blacklists the 1120E or the 3CX proxy has rebooted (I have it set to pull updates and restart early Sunday morning, and this would be the time in the capture, and I don't see a 403 Forbidden like I do the first time). It tries the failover again to the mail server, but only once, as on the second attempt to contact the 3CX box, it registers successfully, at 111,669 seconds, again with a 28800 Expires.

At 129,619 the DHCP request/ack handshake occurs again.

At 140,438 the 1120E again REGISTERs successfully with 3CX (again, about 30 seconds before the 28800 second Expires).

At 169,208 the 1120E again REGISTERs successfully with 3CX (28769 seconds after the previous).

DHCP is refreshed at 172,800, just as expected.

TFTP is refreshed at 188,958 seconds, the first time since 16,000.

And, finally, at 197,977 seconds (28769 seconds after doing so before) the 1120E REGISTERs once again to the 3CX proxy.

If I had set the REG_REFRESH_INTERVAL to 1800 seconds I think it would then reregister after 1770 seconds, after the first time of getting blacklisted and attempting to failover. I haven't tested that, but I do plan to perhaps this coming weekend.

I'll also look at the blacklist parameters in 3CX and see if tuning on those might trigger the failover sooner.

So, in a nushell:
1.) The 1120E does the initial REGISTER with an Expires set to REG_REFRESH_INTERVAL
2.) After REG_REFRESH_INTERVAL seconds, the phone does not re-REGISTER but continues to SIP_PING
3.) After the 3CX blacklist time is triggered by the SIP_PINGS after the phone goes unregistered in 3CX, the phone attempts a failover.
4.) After the 3CX blacklist times out (I assume) the phone will re-REGISTER with 3CX, again with an Expires set to REG_REFRESH_INTERVAL.
5.) The phone now properly refreshes the REGISTER every (REG_REFRESH_INTERVAL-30) seconds, give or take a second.

Whew.

(I'm posting this same info at Avaya's forums).

 

[lowen]

In checking my 3CX security settings, the "Failed Challenge Requests (407)" is set to 1,000; I haven't counted them, but that seems to be the right number and gives the right delay. The blacklist timeout is set to 1,800 seconds, and that is exactly the behavior I'm seeing.

Time to do some tuning.

 

[lowen]

Further update.

Tuning the "Failed Challenge Requests (407)" down to 500 and setting the blacklist timeout to 180 seconds did exactly what I thought it might. Here's an excerpt from the 3CX activity log (extension 851 is the one being sniffed):

07-Jan-2014 01:17:03.853   [CM504001]: Endpoint Extn:851: new contact is registered. Contact(s): [sip:[email protected]:5061 / 851]
07-Jan-2014 01:14:25.613   [IPBL] Packet from banned IP/range: ip = 192.168.1.148; Comment: PBX: blocked for too many failed authentications
07-Jan-2014 01:13:25.516   Blacklisted (Too many failed auth)
IP = 192.168.1.148; Failed auth: 0; unauth: 0; auth: 0; 407: 500
07-Jan-2014 00:51:43.659   [CM504002]: Endpoint Extn:851: a contact is unregistered. Contact(s): []
06-Jan-2014 22:53:08.056   [CM504002]: Endpoint Extn:851: a contact is unregistered. Contact(s): [sip:[email protected]:5060 / 851]
...
06-Jan-2014 16:51:37.802   [CM504001]: Endpoint Extn:851: new contact is registered. Contact(s): [sip:[email protected]:5061 / 851,sip:[email protected]:5060 / 851]

At the same time, the packet sniffer shows the same responses as detailed in my previous message, but with a correspondingly shorter timespan. After the last registration succeeded on 07-Jan-2014 at 01:17:03.853 the phone has re-registered properly every 28770 or so seconds, just like REG_REFRESH_INTERVAL is set to do.

So I may be able to tune the blacklisting such that I can have it trigger earlier and the phones can be set to a more reasonable REG_REFRESH_INTERVAL than 28800...... I'll probably need to ramp it down, setting REG_REFRESH_INTERVAL down first, letting it propagate to all the phones (as each phone does do a periodic TFTP refresh), lowering the blacklist time, then lowering REG_REFRESH_INTERVAL again, lather, rinse, and repeat..... until I get the REG_REFRESH_INTERVAL down to 1800 or so (with a corresponding 407 blacklist count of 40 or 50 and a timeout of 180 seconds).

With those parameters I should be able to boot an 1120E and it only go unregistered for about ten to twenty minutes; I can live with that until the firmware is actually fixed so that the initial registration (before doing the failover) refreshes after REG_REFRESH_INTERVAL instead of the current behavior. Another possible workaround would maybe be setting up a 'dummy' proxy in the configs for the 1120E, and have the 3CX proxy be the second or even the third in the list instead of the first. I'll have to try that at some point.....