I came across a situation for a multinational client who has a complex UCCE setup with four clusters worldwide with separate CUCM, ICM, CVP, PGs, etc. They also have a complex call flow where customers in North America for some queues are served by agents in the Asia Pacific who are logged in on the Asia Pacific cluster.
Problem: Customers in North America were dialing a Toll-free number (NA) which will end up in Indonesia and will hit the Indonesia VXML Gateway. The call will then go to CVP <> ICM and will get a Label and will end up on Agents Phone who is physically located in Indonesia. The call is connected fine but when the Agent puts the caller on hold the call will drop. At Finesse you will see wrap-up time as if the call is dropped and the caller will briefly hear ‘nothing’ and then the call will be dropped. The situation got worst as more and more Agents started having this issue with some angry customers complaining that the agents are hanging upon them!
Analysis: I got involved and my first question was how it was working before? I mean why was not it reported earlier and what changed. I mean it is not that agents just started putting callers on hold right? During my conversation with their IT Manager, I found that they have a Direct line number as well for the same queue (Indonesia based) and they have no problem whatsoever on that number and the problem is only on Toll-free number which they have recently provisioned but it was working fine up until last week!
I jumped in a conference call with a user in North America, an agent in Indonesia, and some other technical people from the customer side while I set up some logging on Gateway and CUCM. I found that the calls to the Direct line are actually ISDN calls end-to-end and there was no issue when the agent was putting the caller on hold. The toll-free call, however, was coming over SIP and was hitting a SONUS CUBE before reaching the local VXML Gateway! I went through the ‘debug ccsip messages’ logs collected for a failed call and found that when Agent was putting the caller on hold, the VXML Gateway was sending INVITES with a=inactive towards SONUS CUBE to get an ACK/OK. But, there was no response from the SONUS side. The Gateway then sends another INVITE with a=inactive so that SONUS can teardown RTP stream so Gateway can then initiate MOH but again no response from Sonus CUBE. Instead, after a few seconds, SONUS CUBE sends a BYE to Gateway and this is when the call drops. The issue clearly was on the SONUS side.
We engaged people from SONUS and they found an issue with the code running on the CUBE. The code was updated and the issue got resolved.
Conclusion: The important point to note here is the comparison I made between the ISDN call and toll-free call to learn about the call path. I then went through the detailed SIP messages which gave us a clue as to where the problem is.
Hope this helps some of you who may come across a similar issue.