Scenario#47 – Jabber users see ‘offline’ status after logging in

I came across an issue for a client where few of their users would login to Jabber (ver 10.5) and see their presence status as ‘offline’. They won’t be able to change it as well. Other Jabber users will also see them as ‘offline’ even though they are logged in and should be online. That was one issue. Few other users would give ‘false’ presence status like if they are ‘away’ they will be shown online and vice versa. They were on 10.5(2) IM&P which is a fairly new release so I was so sure it cannot be due to a bug (I have seen in past these kind of issues in older version of CUPS due to bugs).

I checked the configuration of users in CUCM and IM&P which was all good. Both nodes were in balanced mode and that looking ok as well. I checked the services and they were up and running all fine. I did a dbreplication runtimestate and dbreplication status and found as follows:

admin:utils dbreplication runtimestate

Server Time: Thu Aug  6 01:37:15 MST 2015

Cluster Replication State: Replication status command started at: 2015-08-06-01-27
Replication status command COMPLETED 305 tables checked out of 305
Last Completed Table: batfileinfojobmap
Errors or Mismatches Were Found!!!

Use ‘file view activelog cm/trace/dbl/sdi/ReplicationStatus.2015_08_06_01_27_31.out’ to see the details

DB Version: ccm10_5_2_21900_4
Repltimeout set to: 300s
PROCESS option set to: 1

Cluster Detailed View from lxus3tp04 (2 Servers):

PING      DB/RPC/   REPL.    Replication    REPLICATION SETUP
SERVER-NAME         IP ADDRESS        (msec)    DbMon?    QUEUE    Group ID       (RTMT) & Details
———–         ———-        ——    ——-   —–    ———–    ——————
lxus3tp04           192.168.10.4      0.022     Y/Y/Y     0        (g_5)          (2) Setup Completed
lxus3tp05           192.168.10.5      0.248     Y/Y/Y     0        (g_6)          (2) Setup Completed

.

.

.

Errors found from the ‘utils dbreplication status’ command.
Please review the below logs for details.

options: q=quit, n=next, p=prev, b=begin, e=end (lines 1 – 7 of 7) :
Thu Aug  6 01:26:43 2015 main()  DEBUG:  –>
Thu Aug  6 01:26:48 2015 main()  DEBUG:  Replication cluster summary:
SERVER                 ID STATE    STATUS     QUEUE  CONNECTION CHANGED
———————————————————————–
g_5_ccm10_5_2_21900_4    5 Active   Local           0
g_6_ccm10_5_2_21900_4    6 Active   Connected       0 Jul 20 06:32:24
Thu Aug  6 01:26:56 2015 main()  DEBUG:  <–

———- Suspect Replication Summary ———-

For table: ccmdbtemplate_g_5_ccm10_5_2_21900_4_1_150_peuritoiuid
replication is suspect for node(s):
g_6_ccm10_5_2_21900_4

————————————————-

I noticed replication is showing errors for peuritoiuid table. I was a but curious so I jumped on the bug toolkit and searched for all bugs related to Presence server for Jabber offline status. After a bit of digging I came across a new bug for this kind of behavior where Jabber users will show offline status (on random) and the dbreplication will show a ‘bad’ peuritoiuid table. It is documented under Bug id: CSCuu65533.

To fix this I repaired the peuritoiuid table from IM&P at first node and after that replication was good and the issue related to ‘offline’ and ‘false’ presence status was resolved.

.

admin:utils dbreplication repairtable peuritoiuid
Repairing of replication for table peuritoiuid is in progress.
repairing replicate
replicatename: ccmdbtemplate_g_5_ccm10_5_2_21900_4_1_150_peuritoiuid
Output is in file cm/trace/dbl/sdi/ReplicationTblRepair.2015_08_06_01_54_02.out

Please use “file view activelog cm/trace/dbl/sdi/ReplicationTblRepair.2015_08_06_01_54_02.out” command to see the output
admin:file view activelog cm/trace/dbl/sdi/ReplicationTblRepair.2015_08_06_01_54_02.out

utils dbreplication repairtable tablename [nodename]|all output

To determine if replication is suspect, look for the following:
(1) Number of rows in a table do not match on all nodes.
(2) Non-zero values occur in any of the other output columns for a table

First, a cdr list of the replication status of the servers

SERVER                 ID STATE    STATUS     QUEUE  CONNECTION CHANGED
———————————————————————–
g_5_ccm10_5_2_21900_4    5 Active   Local           0
g_6_ccm10_5_2_21900_4    6 Active   Connected       0 Jul 20 06:32:24

Thu Aug  6 01:54:07 2015 dbllib.getReplServerName  DEBUG:  –>
Thu Aug  6 01:54:12 2015 dbllib.getReplServerName  DEBUG:  replservername: g_5_ccm10_5_2_21900_4
Thu Aug  6 01:54:12 2015 dbllib.getReplServerName  DEBUG:  <–

Aug 06 2015 01:54:13 ——   Table scan for ccmdbtemplate_g_5_ccm10_5_2_21900_4_1_150_peuritoiuid start  ——–

options: q=quit, n=next, p=prev, b=begin, e=end (lines 1 – 20 of 44) :
Node                  Rows     Extra   Missing  Mismatch Processed
—————- ——— ——— ——— ——— ———
g_5_ccm10_5_2_21900_4      3270         0         0         0       905
g_6_ccm10_5_2_21900_4      3240         0        30         0         0

The repair operation completed. Validating the repaired rows …
Validation completed successfully.
Aug 06 2015 01:54:15 ——   Table scan for ccmdbtemplate_g_5_ccm10_5_2_21900_4_1_150_peuritoiuid end   ———

Running a cdr check after repair to insure table is in sync

Aug 06 2015 01:54:16 ——   Table scan for ccmdbtemplate_g_5_ccm10_5_2_21900_4_1_150_peuritoiuid start  ——–

Node                  Rows     Extra   Missing  Mismatch Processed
—————- ——— ——— ——— ——— ———
g_5_ccm10_5_2_21900_4      3270         0         0         0         0

options: q=quit, n=next, p=prev, b=begin, e=end (lines 21 – 40 of 44) :
g_6_ccm10_5_2_21900_4      3270         0         0         0         0

Aug 06 2015 01:54:16 ——   Table scan for ccmdbtemplate_g_5_ccm10_5_2_21900_4_1_150_peuritoiuid end   ———

end of the file reached

Advertisements

One thought on “Scenario#47 – Jabber users see ‘offline’ status after logging in

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s