Problem description:-
Recently for a customer we had performed a DR Drill . The customer had OEM agent installed in all oracle database server . Now after the drill was successful there was a requirement to restore one of the server from VM snapshot backup which was taken before the drill. Now after restoration of that vm , the OEM agent for that target was showing status as ‘Diagnose for Agent Blocked (Bounce Counter Mismatch)’
Solution:-
To resolve this problem , we did the following.
Logged into target DB server where agent status was showing as follows
| E:\Agent13c\agent_inst\agent_13.5.0.0.0\bin>emctl status agent Oracle Enterprise Manager Cloud Control 13c Release 5 Copyright (c) 1996, 2021 Oracle Corporation. All rights reserved. ————————————————————— Agent Version : 13.5.0.0.0 OMS Version : (unknown) Protocol Version : 12.1.0.1.0 Agent Home : E:/Agent13c/agent_inst/agent_inst Agent Log Directory : E:/Agent13c/agent_inst/agent_inst\sysman\log Agent Binaries : E:\Agent13c\agent_inst\agent_13.5.0.0.0 Core JAR Location : E:\Agent13c\agent_inst\agent_13.5.0.0.0\jlib Agent Process ID : 6428 Parent Process ID : 5376 Agent URL : https://WMDR.com:3872/emd/main/ Local Agent URL in NAT : https://WMDR.com:3872/emd/main/ Repository URL : https://OEM-DBMGR.com:1159/empbs/upload Started at : 2024-05-13 03:30:26 Started by user : WMDR16$ Operating System : Windows version 10.0 (amd64) Number of Targets : 6 Last Reload : (none) Last successful upload : (none) Last attempted upload : (none) Total Megabytes of XML files uploaded so far : 0 Number of XML files pending upload : 672 Size of XML files pending upload(MB) : 0.91 Available disk space on upload filesystem : 44.17% Collection Status : Collections enabled Heartbeat Status : Agent is blocked Blocked Reason : Agent is out-of-sync with repository. This most likely means that the agent was reinstalled or recovered. Please contact an EM administrator to unblock the agent by performing an agent resync from the console. Last attempted heartbeat to OMS : 2024-05-13 04:55:01 Last successful heartbeat to OMS : (none) Next scheduled heartbeat to OMS : 2024-05-13 04:58:01 ————————————————————— Agent is Running and Ready |
As we can see the Hearbeat status is showing “Agent is Blocked” and reason behind block is agent is out of sync with its repository.
To resolve this problem, We logged into OMS Server and executed following
| E:\>cd E:\u01\app\middleware\bin E:\u01\app\middleware\bin>set OMS_HOME=E:\u01\app\middleware E:\u01\app\middleware\bin>SET JAVA_HOME=E:\u01\app\agent\agent_13.5.0.0.0\oracle_common\jdk E:\u01\app\middleware\bin>SET ORACLE_HOME=E:\u01\app\middleware E:\u01\app\middleware\bin>emcli login -username=sysman Enter password : Login successful E:\u01\app\middleware\bin>emcli sync Synchronized successfully E:\u01\app\middleware\bin>emcli resyncAgent -agent=WMDR.com:3872 Resync job RESYNC_20240513083609008 successfully submitted |
After a while, when we checked the oem hearbeat status was showing OK.
| E:\Agent13c\agent_inst\agent_13.5.0.0.0\bin>emctl status agent Oracle Enterprise Manager Cloud Control 13c Release 5 Copyright (c) 1996, 2021 Oracle Corporation. All rights reserved. ————————————————————— Agent Version : 13.5.0.0.0 OMS Version : 13.5.0.0.0 Protocol Version : 12.1.0.1.0 Agent Home : E:/Agent13c/agent_inst/agent_inst Agent Log Directory : E:/Agent13c/agent_inst/agent_inst\sysman\log Agent Binaries : E:\Agent13c\agent_inst\agent_13.5.0.0.0 Core JAR Location : E:\Agent13c\agent_inst\agent_13.5.0.0.0\jlib Agent Process ID : 2660 Parent Process ID : 7172 Agent URL : https://WMDR.com:3872/emd/main/ Local Agent URL in NAT : https://WMDR16.com:3872/emd/main/ Repository URL : https://OEM-DBMGR.com:1159/empbs/upload Started at : 2024-05-13 05:33:22 Started by user : WMDR16$ Operating System : Windows version 10.0 (amd64) Number of Targets : 6 Last Reload : (none) Last successful upload : 2024-05-13 05:37:19 Last attempted upload : 2024-05-13 05:37:19 Total Megabytes of XML files uploaded so far : 0.08 Number of XML files pending upload : 3 Size of XML files pending upload(MB) : 0 Available disk space on upload filesystem : 44.15% Collection Status : Collections enabled Heartbeat Status : Ok Last attempted heartbeat to OMS : 2024-05-13 05:36:42 Last successful heartbeat to OMS : 2024-05-13 05:36:42 Next scheduled heartbeat to OMS : 2024-05-13 05:37:43 ————————————————————— Agent is Running and Ready |
Leave a Reply