[JVM/APM] 應用診斷工具之jcmd

0 序言

  • 工欲善其事,必先利其器。最近遇到一個生產環境的極難排查的問題,已經消耗了近一段時間的較多心力了,故今日穩固和新學習一下JVM相關的診斷工具,以便於磨刀霍霍向"BUG"

1 jcmd 概述

1.1 jcmd 簡述

  • jcmd 是在 JDK1.7 以後,新增了一個命令行工具。
    • 即 支持的最低版本: JDK1.7
  • jcmd 是一個多功能的工具,相比 jstat 功能更爲全面的工具,可用於獲取目標 Java 進程性能統計、JFR、內存使用、垃圾收集、線程堆棧、JVM 運行時間,也可以手動執行 GC導出線程信息導出堆信息等信息。

1.2 版本兼容問題

  • 在 JDK 1.8.0_73中不支持 GC.heap_info 參數

以 JDK 1.8.0_261 示範:

C:\Users\{USER}\Desktop>jcmd 17124 GC.heap_info
17124:
 PSYoungGen      total 139264K, used 12359K [0x000000076c000000, 0x0000000776780000, 0x00000007c0000000)
  eden space 127488K, 0% used [0x000000076c000000,0x000000076c0c9cf8,0x0000000773c80000)
  from space 11776K, 98% used [0x0000000773e00000,0x0000000774948190,0x0000000774980000)
  to   space 22016K, 0% used [0x0000000775200000,0x0000000775200000,0x0000000776780000)
 ParOldGen       total 394752K, used 100223K [0x00000006c4000000, 0x00000006dc180000, 0x000000076c000000)
  object space 394752K, 25% used [0x00000006c4000000,0x00000006ca1dfe90,0x00000006dc180000)
 Metaspace       used 110321K, capacity 115630K, committed 115840K, reserved 1150976K
  class space    used 13288K, capacity 14196K, committed 14208K, reserved 1048576K

2 jcmd 用法

2.1 幫助手冊/help

  • jcmd help
C:\Users\{USER}\Desktop>jcmd help
Error parsing arguments: No command specified

Usage: jcmd <pid | main class> <command ...|PerfCounter.print|-f file>
   or: jcmd -l
   or: jcmd -h

  command must be a valid jcmd command for the selected jvm.
  Use the command "help" to see which commands are available.
  If the pid is 0, commands will be sent to all Java processes.
  The main class argument will be used to match (either partially
  or fully) the class used to start Java.
  If no options are given, lists Java processes (same as -p).

  PerfCounter.print display the counters exposed by this process
  -f  read and execute commands from the file
  -l  list JVM processes on the local machine
  -h  this help
  • jcmd {PID} help

命令:jcmd <pid> help
作用:列出當前運行的 java 進程可以執行的操作
說明:不同的 jvm 進程 支持的操作可能是不一樣的

# jcmd 17124 help
17124:
The following commands are available:
JFR.stop
JFR.start
JFR.dump
JFR.check
VM.native_memory
VM.check_commercial_features
VM.unlock_commercial_features
ManagementAgent.stop
ManagementAgent.start_local
ManagementAgent.start
VM.classloader_stats
GC.rotate_log
Thread.print
GC.class_stats
GC.class_histogram
GC.heap_dump
GC.finalizer_info
GC.heap_info
GC.run_finalization
GC.run
VM.uptime
VM.dynlibs
VM.flags
VM.system_properties
VM.command_line
VM.version
help

For more information about a specific command use 'help <command>'.

參數描述

  • pid:[常用] 接收診斷命令請求的進程ID。

  • main class :**[不常用] **接收診斷命令請求的進程的main類。匹配進程時,main類名稱中包含指定子字符串的任何進程均是匹配的。

    如果多個正在運行的Java進程共享同一個main類,診斷命令請求將會發送到所有的這些進程中。

  • command:[常用] 支持的子命令,示例:GC.class_stats、GC.class_histogram。

    注意: 如果任何參數含有空格,你必須使用英文的單引號或雙引號將其包圍起來。 此外,你必須使用轉義字符來轉移參數中的單引號或雙引號,以阻止操作系統shell處理這些引用標記。當然,你也可以在參數兩側加上單引號,然後在參數內使用雙引號(或者,在參數兩側加上雙引號,在參數中使用單引號)。

  • Perfcounter.print:[不常用] 打印目標Java進程上可用的性能計數器。性能計數器的列表可能會隨着Java進程的不同而產生變化。

  • -f file:[不常用] 從文件file中讀取命令,然後在目標Java進程上調用這些命令。在file中,每個命令必須寫在單獨的一行。以"#"開頭的行會被忽略。當所有行的命令被調用完畢後,或者讀取到含有stop關鍵字的命令,將會終止對file的處理。

  • -l:[常用] 查看所有的進程列表信息。

  • -h:[常用] 查看幫助信息。(同 -help)

2.2 查看具體命令的參數/help

命令:jcmd <pid> help command

作用:列出當前運行的 java 進程可以執行的操作

說明:不同的 jvm 進程 支持的操作可能是不一樣的

示例:

# jcmd 20399 help GC.class_stats
20399:
GC.class_stats
Provide statistics about Java class meta data. Requires -XX:+UnlockDiagnosticVMOptions.

Impact: High: Depends on Java heap size and content.

Syntax : GC.class_stats [options] [<columns>]

Arguments:
        columns : [optional] Comma-separated list of all the columns to show. If not specified, the following columns are shown: InstBytes,KlassBytes,CpAll,annotations,MethodCount,Bytecodes,MethodAll,ROAll,RWAll,Total (STRING, no default value)

Options: (options must be specified using the <key> or <key>=<value> syntax)
        -all : [optional] Show all columns (BOOLEAN, false)
        -csv : [optional] Print in CSV (comma-separated values) format for spreadsheets (BOOLEAN, false)
        -help : [optional] Show meaning of all the columns (BOOLEAN, false)

2.3 查看進程/-l

命令: jcmd -l

作用:查看 當前機器上所有的 jvm 進程信息。

同類作用的命令:jcmd、 jcmd -l 、 jps -lm 這三個命令作用相同

示例:

C:\Users\{USER}\Desktop>jps -lm
10036 org.jetbrains.idea.maven.server.RemoteMavenServer36
17124 cn.johnnyzen.bd.dataservice.biz.DataServiceBizApplication -Xlint:unchecked
21556
26260 org/netbeans/Main --branding visualvm --cachedir C:\Users\{USER}\AppData\Local\VisualVM\Cache/2.1.5
15768 org.jetbrains.jps.cmdline.Launcher D:/Program/IDEA/IDEA_COMMUNITY_2023.2/plugins/java/lib/jps-builders.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/plugins/java/lib/jps-builders-6.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/plugins/java/lib/jps-javac-extension.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/lib/util-8.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/lib/util_rt.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/lib/platform-loader.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/lib/annotations.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/lib/protobuf.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/lib/jps-model.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/plugins/java/lib/javac2.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/lib/forms_rt.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/plugins/java/lib/aether-dependency-resolver.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/plugins/java/lib/maven-resolver-connector-basic.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/plugins/java/lib/maven-resolver-transport-file.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/pl
25528 org.jetbrains.idea.maven.server.RemoteMavenServer36
3128 sun.tools.jps.Jps -lm
9948 org.jetbrains.jps.cmdline.Launcher D:/Program/IDEA/IDEA_COMMUNITY_2023.2/plugins/java/lib/jps-builders.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/plugins/java/lib/jps-builders-6.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/plugins/java/lib/jps-javac-extension.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/lib/util-8.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/lib/util_rt.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/lib/platform-loader.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/lib/annotations.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/lib/protobuf.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/lib/jps-model.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/plugins/java/lib/javac2.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/lib/forms_rt.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/plugins/java/lib/aether-dependency-resolver.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/plugins/java/lib/maven-resolver-connector-basic.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/plugins/java/lib/maven-resolver-transport-file.jar;D:/Program/IDEA/IDEA_COMMUNITY_2023.2/pl

2.4 查看內存信息/GC.heap_info

命令:jcmd <pid> GC.heap_info
作用:查看JVM內存信息,雖然名稱爲heap_info,但是除了堆內存信息,也會有堆外內存之一的Metaspace的信息。相比jstat命令結果會更直觀一些。

jcmd 17124 GC.heap_info
17124:
 PSYoungGen      total 133120K, used 92923K [0x000000076c000000, 0x0000000775f80000, 0x00000007c0000000)
  eden space 121344K, 67% used [0x000000076c000000,0x0000000770fb2c18,0x0000000773680000)
  from space 11776K, 96% used [0x0000000773800000,0x000000077430c1a0,0x0000000774380000)
  to   space 20992K, 0% used [0x0000000774b00000,0x0000000774b00000,0x0000000775f80000)
 ParOldGen       total 394752K, used 101212K [0x00000006c4000000, 0x00000006dc180000, 0x000000076c000000)
  object space 394752K, 25% used [0x00000006c4000000,0x00000006ca2d7120,0x00000006dc180000)
 Metaspace       used 110438K, capacity 115822K, committed 115840K, reserved 1150976K
  class space    used 13288K, capacity 14196K, committed 14208K, reserved 1048576K

2.5 查看性能統計/PerfCounter.print

命令:jcmd <pid> PerfCounter.print
作用:查看指定進程的性能統計信息。

示例:

# jcmd 17124 PerfCounter.print
jcmd 17124 PerfCounter.print
17124:
java.ci.totalTime=521327257
java.cls.loadedClasses=20009
java.cls.sharedLoadedClasses=0
java.cls.sharedUnloadedClasses=0
java.cls.unloadedClasses=167
java.property.java.class.path="C:\Users\{USER}\AppData\Local\Temp\classpath852656560.jar;D:\Program\IDEA\IDEA_COMMUNITY_2023.2\lib\idea_rt.jar"
java.property.java.endorsed.dirs="D:\Program\Java\jdk1.8.0_261\jre\lib\endorsed"
java.property.java.ext.dirs="D:\Program\Java\jdk1.8.0_261\jre\lib\ext;C:\Windows\Sun\Java\lib\ext"
java.property.java.home="D:\Program\Java\jdk1.8.0_261\jre"
java.property.java.library.path="D:\Program\Java\jdk1.8.0_261\bin;C:\Windows\Sun\Java\bin;C:\Windows\system32;C:\Windows;c:\Users\{USER}\AppData\Local\Programs\Cursor\resources\app\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\D:\Program\Java\jdk1.8.0_261\bin;D:\Program\Java\jdk1.8.0_261\jre\bin;D:\Program\git\cmd;D:\Program\IDEA\IDEA 2021.3.1\IntelliJ IDEA 2021.3.1\bin;;D:\Program\maven\apache-maven-3.8.1\bin;D:\Program\gradle\gradle-6.8\bin"
java.property.java.version="1.8.0_261"
java.property.java.vm.info="mixed mode"
java.property.java.vm.name="Java HotSpot(TM) 64-Bit Server VM"
java.property.java.vm.specification.name="Java Virtual Machine Specification"
java.property.java.vm.specification.vendor="Oracle Corporation"
java.property.java.vm.specification.version="1.8"
java.property.java.vm.vendor="Oracle Corporation"
java.property.java.vm.version="25.261-b12"
java.rt.vmArgs="-javaagent:D:\Program\IDEA\IDEA_COMMUNITY_2023.2\lib\idea_rt.jar=61512:D:\Program\IDEA\IDEA_COMMUNITY_2023.2\bin -Dfile.encoding=UTF-8"
java.rt.vmFlags=""
java.threads.daemon=92
java.threads.live=97
java.threads.livePeak=124
java.threads.started=9094
sun.ci.compilerThread.0.compiles=1068
sun.ci.compilerThread.0.method=""
sun.ci.compilerThread.0.time=3687
sun.ci.compilerThread.0.type=1
sun.ci.compilerThread.1.compiles=1018
sun.ci.compilerThread.1.method=""
sun.ci.compilerThread.1.time=3163
sun.ci.compilerThread.1.type=1
sun.ci.compilerThread.2.compiles=1020
sun.ci.compilerThread.2.method=""
sun.ci.compilerThread.2.time=3265
sun.ci.compilerThread.2.type=1
sun.ci.compilerThread.3.compiles=16541
sun.ci.compilerThread.3.method=""
sun.ci.compilerThread.3.time=24828
sun.ci.compilerThread.3.type=1
sun.ci.lastFailedMethod="com/google/common/collect/AbstractIterator hasNext"
sun.ci.lastFailedType=1
sun.ci.lastInvalidatedMethod=""
sun.ci.lastInvalidatedType=0
sun.ci.lastMethod="sun/rmi/server/MarshalInputStream done"
sun.ci.lastSize=160
sun.ci.lastType=1
sun.ci.nmethodCodeSize=37567648
sun.ci.nmethodSize=67456872
sun.ci.osrBytes=66982
sun.ci.osrCompiles=98
sun.ci.osrTime=11880286
sun.ci.standardBytes=3590593
sun.ci.standardCompiles=19547
sun.ci.standardTime=509446971
sun.ci.threads=4
sun.ci.totalBailouts=2
sun.ci.totalCompiles=19645
sun.ci.totalInvalidates=0
sun.classloader.findClassTime=12868824700
sun.classloader.findClasses=13619
sun.classloader.parentDelegationTime=366088100
sun.cls.appClassBytes=61762900
sun.cls.appClassLoadCount=16207
sun.cls.appClassLoadTime=121446248
sun.cls.appClassLoadTime.self=84483172
sun.cls.classInitTime=57661686
sun.cls.classInitTime.self=25100737
sun.cls.classLinkedTime=45888862
sun.cls.classLinkedTime.self=6373789
sun.cls.classVerifyTime=39505373
sun.cls.classVerifyTime.self=8706264
sun.cls.defineAppClassTime=33636674
sun.cls.defineAppClassTime.self=1188147
sun.cls.defineAppClasses=14312
sun.cls.initializedClasses=14428
sun.cls.isUnsyncloadClassSet=0
sun.cls.jniDefineClassNoLockCalls=189
sun.cls.jvmDefineClassNoLockCalls=14244
sun.cls.jvmFindLoadedClassNoLockCalls=33235
sun.cls.linkedClasses=17048
sun.cls.loadInstanceClassFailRate=0
sun.cls.loadedBytes=37403400
sun.cls.lookupSysClassTime=1249792
sun.cls.methodBytes=16981693
sun.cls.nonSystemLoaderLockContentionRate=13
sun.cls.parseClassTime=35935806
sun.cls.parseClassTime.self=24288804
sun.cls.sharedClassLoadTime=6571
sun.cls.sharedLoadedBytes=0
sun.cls.sharedUnloadedBytes=0
sun.cls.sysClassBytes=14153775
sun.cls.sysClassLoadTime=4822439
sun.cls.systemLoaderLockContentionRate=0
sun.cls.time=166118829
sun.cls.unloadedBytes=158560
sun.cls.unsafeDefineClassCalls=2138
sun.cls.verifiedClasses=16250
sun.gc.cause="No GC"
sun.gc.collector.0.invocations=72
sun.gc.collector.0.lastEntryTime=110680393836
sun.gc.collector.0.lastExitTime=110680464382
sun.gc.collector.0.name="PSScavenge"
sun.gc.collector.0.time=7267000
sun.gc.collector.1.invocations=4
sun.gc.collector.1.lastEntryTime=3181108754
sun.gc.collector.1.lastExitTime=3182109948
sun.gc.collector.1.name="PSParallelCompact"
sun.gc.collector.1.time=3125050
sun.gc.compressedclassspace.capacity=14548992
sun.gc.compressedclassspace.maxCapacity=1073741824
sun.gc.compressedclassspace.minCapacity=0
sun.gc.compressedclassspace.used=13607704
sun.gc.generation.0.capacity=165675008
sun.gc.generation.0.maxCapacity=1409286144
sun.gc.generation.0.minCapacity=88080384
sun.gc.generation.0.name="new"
sun.gc.generation.0.space.0.capacity=122683392
sun.gc.generation.0.space.0.initCapacity=0
sun.gc.generation.0.space.0.maxCapacity=1408237568
sun.gc.generation.0.space.0.name="eden"
sun.gc.generation.0.space.0.used=85759736
sun.gc.generation.0.space.1.capacity=19922944
sun.gc.generation.0.space.1.initCapacity=0
sun.gc.generation.0.space.1.maxCapacity=469762048
sun.gc.generation.0.space.1.name="s0"
sun.gc.generation.0.space.1.used=16528880
sun.gc.generation.0.space.2.capacity=21495808
sun.gc.generation.0.space.2.initCapacity=0
sun.gc.generation.0.space.2.maxCapacity=469762048
sun.gc.generation.0.space.2.name="s1"
sun.gc.generation.0.space.2.used=0
sun.gc.generation.0.spaces=3
sun.gc.generation.1.capacity=404226048
sun.gc.generation.1.maxCapacity=2818572288
sun.gc.generation.1.minCapacity=176160768
sun.gc.generation.1.name="old"
sun.gc.generation.1.space.0.capacity=404226048
sun.gc.generation.1.space.0.initCapacity=176160768
sun.gc.generation.1.space.0.maxCapacity=2818572288
sun.gc.generation.1.space.0.name="old"
sun.gc.generation.1.space.0.used=103788976
sun.gc.generation.1.spaces=1
sun.gc.lastCause="Allocation Failure"
sun.gc.metaspace.capacity=118620160
sun.gc.metaspace.maxCapacity=1178599424
sun.gc.metaspace.minCapacity=0
sun.gc.metaspace.used=113115576
sun.gc.policy.avgBaseFootprint=268435456
sun.gc.policy.avgMajorIntervalTime=79533
sun.gc.policy.avgMajorPauseTime=77
sun.gc.policy.avgMinorIntervalTime=145939
sun.gc.policy.avgMinorPauseTime=7
sun.gc.policy.avgOldLive=27434336
sun.gc.policy.avgPretenuredPaddedAvg=0
sun.gc.policy.avgPromotedAvg=262202
sun.gc.policy.avgPromotedDev=338687
sun.gc.policy.avgPromotedPaddedAvg=1278264
sun.gc.policy.avgSurvivedAvg=13827576
sun.gc.policy.avgSurvivedDev=2526529
sun.gc.policy.avgSurvivedPaddedAvg=21407164
sun.gc.policy.avgYoungLive=13809281
sun.gc.policy.boundaryMoved=0
sun.gc.policy.changeOldGenForMajPauses=0
sun.gc.policy.changeOldGenForMinPauses=0
sun.gc.policy.changeYoungGenForMajPauses=0
sun.gc.policy.changeYoungGenForMinPauses=0
sun.gc.policy.collectors=2
sun.gc.policy.decideAtFullGc=0
sun.gc.policy.decreaseForFootprint=6
sun.gc.policy.decrementTenuringThresholdForGcCost=0
sun.gc.policy.decrementTenuringThresholdForSurvivorLimit=0
sun.gc.policy.desiredSurvivorSize=21495808
sun.gc.policy.edenSize=122683392
sun.gc.policy.freeSpace=448790528
sun.gc.policy.fullFollowsScavenge=0
sun.gc.policy.gcTimeLimitExceeded=0
sun.gc.policy.generations=3
sun.gc.policy.increaseOldGenForThroughput=0
sun.gc.policy.increaseYoungGenForThroughput=0
sun.gc.policy.incrementTenuringThresholdForGcCost=1
sun.gc.policy.liveAtLastFullGc=63134592
sun.gc.policy.liveSpace=309679072
sun.gc.policy.majorCollectionSlope=0
sun.gc.policy.majorGcCost=1
sun.gc.policy.majorPauseOldSlope=561
sun.gc.policy.majorPauseYoungSlope=678
sun.gc.policy.maxTenuringThreshold=15
sun.gc.policy.minorCollectionSlope=0
sun.gc.policy.minorGcCost=0
sun.gc.policy.minorPauseOldSlope=22
sun.gc.policy.minorPauseTime=6
sun.gc.policy.minorPauseYoungSlope=45
sun.gc.policy.mutatorCost=98
sun.gc.policy.name="ParScav:MSC"
sun.gc.policy.oldCapacity=404226048
sun.gc.policy.oldEdenSize=124256256
sun.gc.policy.oldPromoSize=326107136
sun.gc.policy.promoSize=326107136
sun.gc.policy.promoted=147600
sun.gc.policy.scavengeSkipped=0
sun.gc.policy.survived=16528880
sun.gc.policy.survivorOverflowed=0
sun.gc.policy.tenuringThreshold=15
sun.gc.policy.youngCapacity=145752064
sun.gc.tlab.alloc=14890957
sun.gc.tlab.allocThreads=71
sun.gc.tlab.fastWaste=135
sun.gc.tlab.fills=928
sun.gc.tlab.gcWaste=456634
sun.gc.tlab.maxFastWaste=53
sun.gc.tlab.maxFills=73
sun.gc.tlab.maxGcWaste=104842
sun.gc.tlab.maxSlowAlloc=25
sun.gc.tlab.maxSlowWaste=1864
sun.gc.tlab.slowAlloc=107
sun.gc.tlab.slowWaste=5379
sun.management.JMXConnectorServer.address="service:jmx:rmi://127.0.0.1/stub/rO0ABXN9AAAAAQAkamF2YXgubWFuYWdlbWVudC5yZW1vdGUucm1pLlJNSVNlcnZlcnhyABdqYXZhLmxhbmcucmVmbGVjdC5Qcm94eeEn2iDMEEPLAgABTAABaHQAJUxqYXZhL2xhbmcvcmVmbGVjdC9JbnZvY2F0aW9uSGFtZGxlcjt4cHNyAC1qYXZhLnJtaS5zZXJ2ZXIuUmVtb3RlT2JqZWN0SW52b2NhdGlvbkhhbmRsZXIAAAAAAAAAAgIAAHhyABxqYXZhLnJtaS5zZXJ2ZXIuUmVtb3RlT2JqZWN002G0kQxhMx4DAAB4cHc3AAtVbmljYXN0UmVmMgAADDEwLjMzLjM3LjIyMAAA9E/vzmcbxrNih8TAAMMAAAGMPmJZpIABAHg="
sun.os.hrt.frequency=10000000
sun.os.hrt.ticks=111202048992
sun.perfdata.majorVersion=2
sun.perfdata.minorVersion=0
sun.perfdata.overflow=0
sun.perfdata.size=65536
sun.perfdata.timestamp=84490715009
sun.perfdata.used=19312
sun.property.sun.boot.class.path="D:\Program\Java\jdk1.8.0_261\jre\lib\resources.jar;D:\Program\Java\jdk1.8.0_261\jre\lib\rt.jar;D:\Program\Java\jdk1.8.0_261\jre\lib\sunrsasign.jar;D:\Program\Java\jdk1.8.0_261\jre\lib\jsse.jar;D:\Program\Java\jdk1.8.0_261\jre\lib\jce.jar;D:\Program\Java\jdk1.8.0_261\jre\lib\charsets.jar;D:\Program\Java\jdk1.8.0_261\jre\lib\jfr.jar;D:\Program\Java\jdk1.8.0_261\jre\classes"
sun.property.sun.boot.library.path="D:\Program\Java\jdk1.8.0_261\jre\bin"
sun.rt._sync_ContendedLockAttempts=4985
sun.rt._sync_Deflations=5529
sun.rt._sync_EmptyNotifications=0
sun.rt._sync_FailedSpins=0
sun.rt._sync_FutileWakeups=171
sun.rt._sync_Inflations=5449
sun.rt._sync_MonExtant=2304
sun.rt._sync_MonInCirculation=0
sun.rt._sync_MonScavenged=0
sun.rt._sync_Notifications=16931
sun.rt._sync_Parks=19756
sun.rt._sync_PrivateA=0
sun.rt._sync_PrivateB=0
sun.rt._sync_SlowEnter=0
sun.rt._sync_SlowExit=0
sun.rt._sync_SlowNotify=0
sun.rt._sync_SlowNotifyAll=0
sun.rt._sync_SuccessfulSpins=0
sun.rt.applicationTime=111152257757
sun.rt.createVmBeginTime=1701845232912
sun.rt.createVmEndTime=1701845233497
sun.rt.internalVersion="Java HotSpot(TM) 64-Bit Server VM (25.261-b12) for windows-amd64 JRE (1.8.0_261-b12), built on Jun 18 2020 06:56:32 by "" with unknown MS VC++:1916"
sun.rt.interruptedBeforeIO=0
sun.rt.interruptedDuringIO=0
sun.rt.javaCommand="cn.johnnyzen.bd.dataservice.biz.DataServiceBizApplication -Xlint:unchecked"
sun.rt.jvmCapabilities="1100000000000000000000000000000000000000000000000000000000000000"
sun.rt.jvmVersion=419497228
sun.rt.safepointSyncTime=11526218
sun.rt.safepointTime=39011973
sun.rt.safepoints=9127
sun.rt.threadInterruptSignaled=0
sun.rt.vmInitDoneTime=1701845232994
sun.threads.vmOperationTime=15162630
sun.urlClassLoader.readClassBytesTime=1222495000
sun.zip.zipFile.openTime=409313600
sun.zip.zipFiles=1126

2.6 查看已啓動時長/VM.uptime

命令:jcmd <pid> VM.uptime
作用:查看 JVM 的已啓動時長。

示例:

# jcmd 17124 VM.uptime
17124:
16096.667 s

2.7 查看系統中【類】的統計信息/GC.class_histogram

命令:jcmd <pid> GC.class_histogram
作用:查看系統中類統計信息。

同類作用的命令:和jmap -histo pid的效果是一樣的,可以查看每個類的實例數量和佔用空間大小。

示例:

# jcmd 17124 GC.class_histogram > class_histogram.txt
17124:

 num     #instances         #bytes  class name
----------------------------------------------
   1:        128192       14107064  [C
   2:          9404        6427296  [B
   3:         25119        5243544  [Ljava.lang.Object;
   4:        103014        3296448  java.util.concurrent.ConcurrentHashMap$Node
   5:        127042        3049008  java.lang.String
   6:         20991        2324280  java.lang.Class
   7:         23881        2101528  java.lang.reflect.Method
   8:          2106        1381536  com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.internal.shaded.org.jctools.queues.MpscArrayQueue
   9:         11330        1177072  [Ljava.util.HashMap$Node;
  10:         27061        1082440  java.util.LinkedHashMap$Entry
  11:          1984        1035680  [Ljava.util.concurrent.ConcurrentHashMap$Node;
  12:         34136         819264  org.springframework.core.MethodClassKey
  13:         13969         782264  java.util.LinkedHashMap
  14:         10491         752320  [I
  15:         21977         703264  java.util.HashMap$Node
  16:         34733         555728  java.lang.Object
  17:         18802         424088  [Ljava.lang.Class;
  18:          1330         378080  [Lcom.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.Recycler$DefaultHandle;
  19:             8         262272  [Ljava.util.concurrent.ForkJoinTask;
  20:          8084         258688  java.util.Hashtable$Entry
  21:          9897         237528  java.util.ArrayList
  22:          5894         235760  java.lang.ref.SoftReference
  23:          2732         218560  java.lang.reflect.Constructor
  24:          4297         206256  java.util.HashMap
  25:          4880         204632  [Ljava.lang.String;
  26:          5528         176896  java.lang.ref.WeakReference
  27:          3624         173952  org.springframework.core.ResolvableType
  28:          2605         166720  java.util.concurrent.ConcurrentHashMap
  29:            80         136960  [Lcom.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.buffer.PoolSubpage;
  30:          2205         123480  java.lang.invoke.MemberName
  31:          2723         108920  java.util.TreeMap$Entry
  32:          3359         107488  java.util.concurrent.locks.ReentrantLock$NonfairSync
  33:           668         105336  [Ljava.util.Hashtable$Entry;
  34:          2552         102080  java.lang.ref.Finalizer
  35:           508          97536  org.springframework.context.annotation.ConfigurationClassBeanDefinitionReader$ConfigurationClassBeanDefinition
  36:          1425          91200  com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.buffer.PoolSubpage
  37:          1536          86016  java.security.Provider$Service
  38:           852          84192  [Ljava.util.WeakHashMap$Entry;
  39:          1163          83736  java.lang.reflect.Field
  40:          2059          82360  java.lang.invoke.MethodType
  41:          3370          80880  sun.reflect.generics.tree.SimpleClassTypeSignature
  42:          1443          80808  java.lang.Package
  43:          1439          80584  org.springframework.core.annotation.TypeMappedAnnotation
  44:          3330          79920  sun.reflect.annotation.AnnotationInvocationHandler
  45:          2469          79008  java.lang.ref.ReferenceQueue
  46:          3213          77112  java.util.LinkedList$Node
  47:           428          75328  org.springframework.beans.factory.support.RootBeanDefinition
  48:          2266          72512  java.util.LinkedList
  49:          3016          72384  java.security.Provider$ServiceKey
  50:          1239          69384  org.codehaus.groovy.reflection.GeneratedMetaMethod$Proxy
  51:            26          66112  [Ljava.nio.channels.SelectionKey;
  52:          2065          66080  java.lang.invoke.MethodType$ConcurrentWeakInternSet$WeakEntry
  53:           510          64048  [Z
  54:          3370          63376  [Lsun.reflect.generics.tree.TypeArgument;
  55:          1944          62208  com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.buffer.PoolThreadCache$SubPageMemoryRegionCache
  56:          1546          61840  java.util.WeakHashMap$Entry
  57:           942          60288  java.net.URL
  58:           256          59392  com.clickhouse.client.ClickHouseConfig
  59:          1162          55776  org.springframework.util.ConcurrentReferenceHashMap$SoftEntryReference
  60:          2316          55584  java.util.concurrent.ConcurrentLinkedDeque$Node
  61:           825          52800  org.codehaus.groovy.reflection.CachedMethod
  62:          3299          52784  java.util.LinkedHashSet
  63:          3256          52096  sun.reflect.generics.tree.ClassTypeSignature
  64:            48          51072  [Ljava.nio.ByteBuffer;
  65:           512          49152  sun.util.calendar.Gregorian$Date
  66:          2030          48720  java.util.concurrent.atomic.AtomicLong
  67:           867          48552  java.beans.MethodDescriptor
  68:          2034          48160  [Ljava.lang.reflect.Type;
  69:           843          47208  org.codehaus.groovy.runtime.metaclass.MetaMethodIndex$Entry
  70:          1463          46816  java.lang.invoke.DirectMethodHandle
  71:           321          46104  [Ljava.lang.ClassValue$Entry;
  72:          1915          45960  java.lang.Class$AnnotationData
  73:          1329          42528  sun.security.util.ObjectIdentifier
  74:           976          41224  [Lorg.springframework.util.ConcurrentReferenceHashMap$Reference;
  75:          1671          40104  java.util.concurrent.CopyOnWriteArrayList
  76:          2471          39536  java.lang.ref.ReferenceQueue$Lock
  77:           976          39040  org.springframework.util.ConcurrentReferenceHashMap$Segment
  78:          1656          37712  [Lorg.springframework.core.ResolvableType;
  79:           771          37008  org.codehaus.groovy.util.ReferenceType$SoftRef
  80:          1110          35520  com.mysql.cj.conf.BooleanProperty
  81:          2182          34912  java.util.concurrent.atomic.AtomicInteger
  82:           484          34848  sun.reflect.DelegatingClassLoader
  83:          2172          34752  java.lang.Integer
  84:          1047          33504  com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.Recycler$WeakOrderQueue$Link
  85:          2086          33376  java.util.concurrent.locks.ReentrantLock
  86:           817          32680  java.math.BigInteger
  87:          1355          32520  java.beans.MethodRef
  88:          1016          32512  java.util.Vector
  89:           409          31104  [J
  90:          1274          30576  org.springframework.core.convert.converter.GenericConverter$ConvertiblePair
  91:           314          30144  java.util.jar.JarFile$JarFileEntry
  92:           622          29856  java.util.concurrent.ConcurrentHashMap$TreeNode
  93:          1242          29808  org.springframework.beans.factory.annotation.InjectionMetadata
  94:          1235          29640  org.springframework.beans.BeanMetadataAttribute

....

8029:             1             16  sun.security.util.ByteArrayLexOrder
8030:             1             16  sun.security.util.ByteArrayTagOrder
8031:             1             16  sun.text.normalizer.NormalizerBase$Mode
8032:             1             16  sun.text.normalizer.NormalizerBase$NFCMode
8033:             1             16  sun.text.normalizer.NormalizerBase$NFDMode
8034:             1             16  sun.text.normalizer.NormalizerBase$NFKCMode
8035:             1             16  sun.text.normalizer.NormalizerBase$NFKDMode
8036:             1             16  sun.text.normalizer.NormalizerImpl
8037:             1             16  sun.text.normalizer.NormalizerImpl$AuxTrieImpl
8038:             1             16  sun.text.normalizer.NormalizerImpl$FCDTrieImpl
8039:             1             16  sun.text.normalizer.NormalizerImpl$NormTrieImpl
8040:             1             16  sun.util.calendar.Gregorian
8041:             1             16  sun.util.locale.provider.AuxLocaleProviderAdapter$NullProvider
8042:             1             16  sun.util.locale.provider.CalendarDataUtility$CalendarWeekParameterGetter
8043:             1             16  sun.util.locale.provider.SPILocaleProviderAdapter
8044:             1             16  sun.util.locale.provider.TimeZoneNameUtility$TimeZoneNameGetter
8045:             1             16  sun.util.resources.LocaleData
8046:             1             16  sun.util.resources.LocaleData$LocaleDataResourceBundleControl
Total        927801       56802976

2.8 查看線程堆棧信息/Thread.print

命令:jcmd <pid> Thread.print
作用:查看線程堆棧信息。

同類作用的命令:和jstack -l作用一樣

jcmd 17124 Thread.print
17124:
2023-12-06 19:21:08
Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.261-b12 mixed mode):

"nacos-grpc-client-executor-3264" #13332 daemon prio=5 os_prio=0 tid=0x000001ebb9644800 nid=0x3574 waiting on condition [0x000000ea2abff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c542eae0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"nacos-grpc-client-executor-3267" #13331 daemon prio=5 os_prio=0 tid=0x000001ebb9649000 nid=0x60dc waiting on condition [0x000000ea2b0ff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c4bd2c40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"nacos-grpc-client-executor-3241" #13330 daemon prio=5 os_prio=0 tid=0x000001ebb9642800 nid=0x43e8 waiting on condition [0x000000ea2a2ff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5d40660> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"nacos-grpc-client-executor-3231" #13329 daemon prio=5 os_prio=0 tid=0x000001ebb9648800 nid=0x49b8 waiting on condition [0x000000ea289ff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5d82830> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"nacos-grpc-client-executor-3263" #13328 daemon prio=5 os_prio=0 tid=0x000001ebb9641800 nid=0x6068 waiting on condition [0x000000ea2b4ff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c542eae0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"nacos-grpc-client-executor-3266" #13327 daemon prio=5 os_prio=0 tid=0x000001ebb9641000 nid=0x1f90 waiting on condition [0x000000ea2b3ff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c4bd2c40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"nacos-grpc-client-executor-3240" #13326 daemon prio=5 os_prio=0 tid=0x000001ebb9647800 nid=0x2c8 waiting on condition [0x000000ea2b1ff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5d40660> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"nacos-grpc-client-executor-3230" #13325 daemon prio=5 os_prio=0 tid=0x000001ebb9646000 nid=0x2d9c waiting on condition [0x000000ea28aff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5d82830> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"RMI TCP Connection(26)-10.33.37.220" #13303 daemon prio=5 os_prio=0 tid=0x000001ebb9647000 nid=0x4c38 runnable [0x000000ea28bfd000]
   java.lang.Thread.State: RUNNABLE
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
        at java.net.SocketInputStream.read(SocketInputStream.java:171)
        at java.net.SocketInputStream.read(SocketInputStream.java:141)
        at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
        - locked <0x000000076fba8a70> (a java.io.BufferedInputStream)
        at java.io.FilterInputStream.read(FilterInputStream.java:83)
        at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:555)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:834)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:688)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$$Lambda$1306/1016102937.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:687)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"RMI TCP Connection(idle)" #12760 daemon prio=5 os_prio=0 tid=0x000001ebb964a800 nid=0x33e4 waiting on condition [0x000000ea2cbfe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c7181628> (a java.util.concurrent.SynchronousQueue$TransferStack)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"JMX server connection timeout 6922" #6922 daemon prio=5 os_prio=0 tid=0x000001ebb610e800 nid=0x5b68 in Object.wait() [0x000000ea2deff000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at com.sun.jmx.remote.internal.ServerCommunicatorAdmin$Timeout.run(ServerCommunicatorAdmin.java:168)
        - locked <0x00000006c7181c38> (a [I)
        at java.lang.Thread.run(Thread.java:748)

"RMI Scheduler(0)" #6921 daemon prio=5 os_prio=0 tid=0x000001ebc2c18800 nid=0x2854 waiting on condition [0x000000ea2ddff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c71856f0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"RMI TCP Accept-0" #6919 daemon prio=5 os_prio=0 tid=0x000001ebc2c1f000 nid=0x6810 runnable [0x000000ea2aefe000]
   java.lang.Thread.State: RUNNABLE
        at java.net.DualStackPlainSocketImpl.accept0(Native Method)
        at java.net.DualStackPlainSocketImpl.socketAccept(DualStackPlainSocketImpl.java:127)
        at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:535)
        at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:189)
        - locked <0x00000006c7181820> (a java.net.SocksSocketImpl)
        at java.net.ServerSocket.implAccept(ServerSocket.java:545)
        at java.net.ServerSocket.accept(ServerSocket.java:513)
        at sun.management.jmxremote.LocalRMIServerSocketFactory$1.accept(LocalRMIServerSocketFactory.java:52)
        at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:405)
        at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:377)
        at java.lang.Thread.run(Thread.java:748)

"grpc-nio-worker-ELG-1-16" #4585 daemon prio=5 os_prio=0 tid=0x000001ebba96f800 nid=0x4df0 runnable [0x000000ea2feff000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000006c4c3aee0> (a com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySet)
        - locked <0x00000006c4c3b008> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000006c4c3aef8> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:806)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

"grpc-nio-worker-ELG-1-15" #4584 daemon prio=5 os_prio=0 tid=0x000001ebba96f000 nid=0x167c runnable [0x000000ea2fdfe000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000006c4c20bb0> (a com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySet)
        - locked <0x00000006c4c20cd8> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000006c4c20bc8> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:806)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

"grpc-nio-worker-ELG-1-14" #4580 daemon prio=5 os_prio=0 tid=0x000001ebba96e000 nid=0x4200 runnable [0x000000ea2f9ff000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000006c4c129d0> (a com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySet)
        - locked <0x00000006c4c1efa8> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000006c4c1ee98> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:806)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

"grpc-nio-worker-ELG-1-13" #4579 daemon prio=5 os_prio=0 tid=0x000001ebba966800 nid=0x6814 runnable [0x000000ea2f8fe000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000006c4c51e98> (a com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySet)
        - locked <0x00000006c4c51fc0> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000006c4c51eb0> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:806)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

"grpc-nio-worker-ELG-1-12" #4578 daemon prio=5 os_prio=0 tid=0x000001ebba966000 nid=0x40d4 runnable [0x000000ea2f7ff000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000006c4c7a490> (a com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySet)
        - locked <0x00000006c4c7a5b8> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000006c4c7a4a8> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:806)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

"grpc-nio-worker-ELG-1-11" #4577 daemon prio=5 os_prio=0 tid=0x000001ebba835800 nid=0x60e4 runnable [0x000000ea2f6fe000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000006c4c9c0b0> (a com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySet)
        - locked <0x00000006c4c9c1d8> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000006c4c9c0c8> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:806)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

"grpc-nio-worker-ELG-1-10" #4575 daemon prio=5 os_prio=0 tid=0x000001ebba834000 nid=0x1f1c runnable [0x000000ea2edff000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000006c4cb5110> (a com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySet)
        - locked <0x00000006c4cb5238> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000006c4cb5128> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:806)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

"grpc-nio-worker-ELG-1-9" #4574 daemon prio=5 os_prio=0 tid=0x000001ebba83a000 nid=0x523c runnable [0x000000ea2ecfe000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000006c4c9cdb8> (a com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySet)
        - locked <0x00000006c4cb3518> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000006c4cb3408> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:806)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.naming.updater.3" #247 daemon prio=5 os_prio=0 tid=0x000001ebc2c20000 nid=0x5c60 waiting on condition [0x000000ea2beff000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c57310f8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.naming.updater.2" #198 daemon prio=5 os_prio=0 tid=0x000001ebc2c15800 nid=0xadc waiting on condition [0x000000ea2ccfe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c57310f8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #168 daemon prio=5 os_prio=0 tid=0x000001ebba839000 nid=0x6a20 waiting on condition [0x000000ea2cdff000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5d24e38> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #166 daemon prio=5 os_prio=0 tid=0x000001ebba838800 nid=0x68d4 waiting on condition [0x000000ea2bcfe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5ca7628> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #162 daemon prio=5 os_prio=0 tid=0x000001ebba83a800 nid=0x672c waiting on condition [0x000000ea2e9fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5d24e38> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #160 daemon prio=5 os_prio=0 tid=0x000001ebba82d000 nid=0x5598 waiting on condition [0x000000ea2dfff000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5ca7628> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #156 daemon prio=5 os_prio=0 tid=0x000001ebba833000 nid=0x587c waiting on condition [0x000000ea2ceff000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5d24e38> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #153 daemon prio=5 os_prio=0 tid=0x000001ebba830000 nid=0xc7c waiting on condition [0x000000ea2affe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5ca7628> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #149 daemon prio=5 os_prio=0 tid=0x000001ebba837800 nid=0x2ac4 waiting on condition [0x000000ea2c2ff000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5d24e38> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #147 daemon prio=5 os_prio=0 tid=0x000001ebba83b800 nid=0x6718 waiting on condition [0x000000ea2b2fe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5ca7628> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #145 daemon prio=5 os_prio=0 tid=0x000001ebba82f800 nid=0x386c waiting on condition [0x000000ea30bff000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5d24e38> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #144 daemon prio=5 os_prio=0 tid=0x000001ebba82e800 nid=0x3990 waiting on condition [0x000000ea30aff000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5ca7628> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"DestroyJavaVM" #142 prio=5 os_prio=0 tid=0x000001ebba834800 nid=0x2e10 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"grpc-nio-worker-ELG-1-8" #120 daemon prio=5 os_prio=0 tid=0x000001ebc2c17800 nid=0x54bc runnable [0x000000ea2f4fe000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000006c4cca7b8> (a com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySet)
        - locked <0x00000006c4cca8e0> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000006c4cca7d0> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:806)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

"grpc-nio-worker-ELG-1-7" #119 daemon prio=5 os_prio=0 tid=0x000001ebc2c1c000 nid=0x67f0 runnable [0x000000ea2f3fe000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000006c4ce8150> (a com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySet)
        - locked <0x00000006c4ce8278> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000006c4ce8168> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:806)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.remote.worker" #118 daemon prio=5 os_prio=0 tid=0x000001ebc2c14800 nid=0x6bec waiting on condition [0x000000ea2f2fe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5ef0608> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:418)
        at com.alibaba.nacos.common.remote.client.RpcClient$3.run(RpcClient.java:317)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.remote.worker" #117 daemon prio=5 os_prio=0 tid=0x000001ebc2c1b800 nid=0x2010 waiting on condition [0x000000ea2f1fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5e6b490> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at com.alibaba.nacos.common.remote.client.RpcClient$2.run(RpcClient.java:295)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #116 daemon prio=5 os_prio=0 tid=0x000001ebc2c1a800 nid=0x738 waiting on condition [0x000000ea2f0fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5d24e38> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #115 daemon prio=5 os_prio=0 tid=0x000001ebc2c14000 nid=0x6cf8 waiting on condition [0x000000ea2efff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5d24e38> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #114 daemon prio=5 os_prio=0 tid=0x000001ebc2c13000 nid=0x36a8 waiting on condition [0x000000ea2eefe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5e4b6a8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:418)
        at com.alibaba.nacos.client.config.impl.ClientWorker$ConfigRpcTransportClient$4.run(ClientWorker.java:744)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #109 daemon prio=5 os_prio=0 tid=0x000001ebb4a0c000 nid=0x656c waiting on condition [0x000000ea2e8ff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c464ea40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"grpc-nio-worker-ELG-1-6" #108 daemon prio=5 os_prio=0 tid=0x000001ebb4a0a000 nid=0x7258 runnable [0x000000ea2e7fe000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000006c4d09a48> (a com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySet)
        - locked <0x00000006c4d09b70> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000006c4d09a60> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:806)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

"grpc-nio-worker-ELG-1-5" #107 daemon prio=5 os_prio=0 tid=0x000001ebb4a08800 nid=0x56fc runnable [0x000000ea2e6ff000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000006c4d202f8> (a com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySet)
        - locked <0x00000006c4d20420> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000006c4d20310> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:806)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.remote.worker" #106 daemon prio=5 os_prio=0 tid=0x000001ebb4a09000 nid=0x4898 waiting on condition [0x000000ea2e5fe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5d5eae8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:418)
        at com.alibaba.nacos.common.remote.client.RpcClient$3.run(RpcClient.java:317)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.remote.worker" #105 daemon prio=5 os_prio=0 tid=0x000001ebb4a0b800 nid=0x5610 waiting on condition [0x000000ea2e4ff000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5ca8d68> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at com.alibaba.nacos.common.remote.client.RpcClient$2.run(RpcClient.java:295)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #104 daemon prio=5 os_prio=0 tid=0x000001ebb4a07800 nid=0x54e8 waiting on condition [0x000000ea2e3fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5ca7628> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #103 daemon prio=5 os_prio=0 tid=0x000001ebb4a0a800 nid=0x1818 waiting on condition [0x000000ea2e2fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5ca7628> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #102 daemon prio=5 os_prio=0 tid=0x000001ebb4a07000 nid=0x588c waiting on condition [0x000000ea2e1ff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5ca86b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:418)
        at com.alibaba.nacos.client.config.impl.ClientWorker$ConfigRpcTransportClient$4.run(ClientWorker.java:744)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.naming.updater.1" #101 daemon prio=5 os_prio=0 tid=0x000001ebb49ff800 nid=0x5d5c waiting on condition [0x000000ea2e0fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c57310f8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"http-nio-9527-Acceptor" #97 daemon prio=5 os_prio=0 tid=0x000001ebb49fd000 nid=0x4ce0 runnable [0x000000ea2dcfe000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:424)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:252)
        - locked <0x00000006c628af98> (a java.lang.Object)
        at org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:574)
        at org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:80)
        at org.apache.tomcat.util.net.Acceptor.run(Acceptor.java:106)
        at java.lang.Thread.run(Thread.java:748)

"http-nio-9527-ClientPoller" #96 daemon prio=5 os_prio=0 tid=0x000001ebb4a04000 nid=0xc40 runnable [0x000000ea2dbfe000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000006c62dc740> (a sun.nio.ch.Util$3)
        - locked <0x00000006c62dc730> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000006c62dc6b0> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:816)
        at java.lang.Thread.run(Thread.java:748)

"http-nio-9527-exec-10" #95 daemon prio=5 os_prio=0 tid=0x000001ebb4a03000 nid=0x3a10 waiting on condition [0x000000ea2dafe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c62be420> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:108)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

"http-nio-9527-exec-9" #94 daemon prio=5 os_prio=0 tid=0x000001ebb49fe800 nid=0x62d8 waiting on condition [0x000000ea2d9fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c62be420> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:108)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

"http-nio-9527-exec-8" #93 daemon prio=5 os_prio=0 tid=0x000001ebb4a01800 nid=0x37ec waiting on condition [0x000000ea2d8fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c62be420> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:108)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

"http-nio-9527-exec-7" #92 daemon prio=5 os_prio=0 tid=0x000001ebb4a02800 nid=0x5328 waiting on condition [0x000000ea2d7fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c62be420> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:108)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

"http-nio-9527-exec-6" #91 daemon prio=5 os_prio=0 tid=0x000001ebb4a01000 nid=0x36a0 waiting on condition [0x000000ea2d6ff000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c62be420> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:108)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

"http-nio-9527-exec-5" #90 daemon prio=5 os_prio=0 tid=0x000001ebb4a00000 nid=0x3684 waiting on condition [0x000000ea2d5fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c62be420> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:108)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

"http-nio-9527-exec-4" #89 daemon prio=5 os_prio=0 tid=0x000001ebba972000 nid=0x5f10 waiting on condition [0x000000ea2d4fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c62be420> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:108)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

"http-nio-9527-exec-3" #88 daemon prio=5 os_prio=0 tid=0x000001ebb953b800 nid=0x590 waiting on condition [0x000000ea2d3fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c62be420> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:108)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

"http-nio-9527-exec-2" #87 daemon prio=5 os_prio=0 tid=0x000001ebbf778800 nid=0x5ca0 waiting on condition [0x000000ea2d2fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c62be420> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:108)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

"http-nio-9527-exec-1" #86 daemon prio=5 os_prio=0 tid=0x000001ebbf777000 nid=0x52d0 waiting on condition [0x000000ea2d1fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c62be420> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:108)
        at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

"http-nio-9527-BlockPoller" #85 daemon prio=5 os_prio=0 tid=0x000001ebbf777800 nid=0x3ef8 runnable [0x000000ea2d0ff000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000006c63718e8> (a sun.nio.ch.Util$3)
        - locked <0x00000006c63718d8> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000006c6371778> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at org.apache.tomcat.util.net.NioBlockingSelector$BlockPoller.run(NioBlockingSelector.java:331)

"Nacos-Watch-Task-Scheduler-1" #84 prio=5 os_prio=0 tid=0x000001ebbf776000 nid=0x4620 waiting on condition [0x000000ea2cfff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5d7ac40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.naming.updater.0" #81 daemon prio=5 os_prio=0 tid=0x000001ebbf76d000 nid=0x69ec waiting on condition [0x000000ea2c0fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c57310f8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"grpc-nio-worker-ELG-1-4" #77 daemon prio=5 os_prio=0 tid=0x000001ebbf772800 nid=0x7070 runnable [0x000000ea2cafe000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000006c4d21000> (a com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySet)
        - locked <0x00000006c4d308b0> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000006c4d307a0> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:806)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

"grpc-nio-worker-ELG-1-3" #76 daemon prio=5 os_prio=0 tid=0x000001ebbf771000 nid=0x4c50 runnable [0x000000ea2c9fe000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000006c4d324a8> (a com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySet)
        - locked <0x00000006c4d325d0> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000006c4d324c0> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:806)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.remote.worker" #75 daemon prio=5 os_prio=0 tid=0x000001ebbf771800 nid=0x1f60 waiting on condition [0x000000ea2c8ff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5410fe8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:418)
        at com.alibaba.nacos.common.remote.client.RpcClient$3.run(RpcClient.java:317)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.remote.worker" #74 daemon prio=5 os_prio=0 tid=0x000001ebbf769800 nid=0xb6c waiting on condition [0x000000ea2c7fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c54109a8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at com.alibaba.nacos.common.remote.client.RpcClient$2.run(RpcClient.java:295)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.naming.grpc.redo.0" #73 daemon prio=5 os_prio=0 tid=0x000001ebbf76b000 nid=0x5028 waiting on condition [0x000000ea2c6fe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5541040> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.naming.push.receiver" #72 daemon prio=5 os_prio=0 tid=0x000001ebbf76a000 nid=0x210c runnable [0x000000ea2c5fe000]
   java.lang.Thread.State: RUNNABLE
        at java.net.DualStackPlainDatagramSocketImpl.socketReceiveOrPeekData(Native Method)
        at java.net.DualStackPlainDatagramSocketImpl.receive0(DualStackPlainDatagramSocketImpl.java:124)
        - locked <0x00000006c565eb48> (a java.net.DualStackPlainDatagramSocketImpl)
        at java.net.AbstractPlainDatagramSocketImpl.receive(AbstractPlainDatagramSocketImpl.java:149)
        - locked <0x00000006c565eb48> (a java.net.DualStackPlainDatagramSocketImpl)
        at java.net.DatagramSocket.receive(DatagramSocket.java:812)
        - locked <0x00000006c588d000> (a java.net.DatagramPacket)
        - locked <0x00000006c565eb08> (a java.net.DatagramSocket)
        at com.alibaba.nacos.client.naming.core.PushReceiver.run(PushReceiver.java:102)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.naming.security" #71 daemon prio=5 os_prio=0 tid=0x000001ebbf76e000 nid=0x6c14 waiting on condition [0x000000ea2c4fe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c575e630> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.naming.failover" #66 daemon prio=5 os_prio=0 tid=0x000001ebba968000 nid=0x4da0 waiting on condition [0x000000ea2c1ff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5547b68> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"nacos.publisher-com.alibaba.nacos.client.naming.event.InstancesChangeEvent" #65 daemon prio=5 os_prio=0 tid=0x000001ebb9539000 nid=0x4804 waiting on condition [0x000000ea2b7ff000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c59332d8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:403)
        at com.alibaba.nacos.common.notify.DefaultPublisher.openEventHandler(DefaultPublisher.java:117)
        at com.alibaba.nacos.common.notify.DefaultPublisher.run(DefaultPublisher.java:95)

"spring.cloud.inetutils" #64 daemon prio=5 os_prio=0 tid=0x000001ebb9538800 nid=0x32ec waiting on condition [0x000000ea2b6fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5731440> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #63 daemon prio=5 os_prio=0 tid=0x000001ebb953d000 nid=0x3220 waiting on condition [0x000000ea2b5fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c464ea40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #55 daemon prio=5 os_prio=0 tid=0x000001ebba96d800 nid=0x2d14 waiting on condition [0x000000ea2adfe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c464ea40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"HikariPool-1 housekeeper" #53 daemon prio=5 os_prio=0 tid=0x000001ebba971000 nid=0x3d24 waiting on condition [0x000000ea2bfff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c5541a58> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #51 daemon prio=5 os_prio=0 tid=0x000001ebba970800 nid=0x6728 waiting on condition [0x000000ea2bdfe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c464ea40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"mysql-cj-abandoned-connection-cleanup" #49 daemon prio=5 os_prio=0 tid=0x000001ebba96c800 nid=0xee4 in Object.wait() [0x000000ea2bbff000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144)
        - locked <0x00000006c5856018> (a java.lang.ref.ReferenceQueue$Lock)
        at com.mysql.cj.jdbc.AbandonedConnectionCleanupThread.run(AbandonedConnectionCleanupThread.java:91)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"container-0" #47 prio=5 os_prio=0 tid=0x000001ebba96c000 nid=0x24f4 waiting on condition [0x000000ea2bafe000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
        at java.lang.Thread.sleep(Native Method)
        at org.apache.catalina.core.StandardServer.await(StandardServer.java:570)
        at org.springframework.boot.web.embedded.tomcat.TomcatWebServer$1.run(TomcatWebServer.java:197)

"Catalina-utility-2" #46 prio=1 os_prio=-2 tid=0x000001ebba964800 nid=0x6fcc waiting on condition [0x000000ea2b9ff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c560ba68> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

"Catalina-utility-1" #45 prio=1 os_prio=-2 tid=0x000001ebba965000 nid=0x575c waiting on condition [0x000000ea2a3ff000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c560ba68> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #44 daemon prio=5 os_prio=0 tid=0x000001ebba96b000 nid=0xcb8 waiting on condition [0x000000ea2b8fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c464ea40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"grpc-nio-worker-ELG-1-2" #30 daemon prio=5 os_prio=0 tid=0x000001ebb953b000 nid=0x3ab8 runnable [0x000000ea2acfe000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000006c4c8e848> (a com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySet)
        - locked <0x00000006c4c8e860> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000006c4c8e7c8> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:806)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

"grpc-nio-worker-ELG-1-1" #28 daemon prio=5 os_prio=0 tid=0x000001ebb8a02000 nid=0xb80 runnable [0x000000ea2aafe000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x00000006c4cbd928> (a com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySet)
        - locked <0x00000006c4cbd980> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000006c4cbd858> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:806)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at com.alibaba.nacos.shaded.io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.remote.worker" #27 daemon prio=5 os_prio=0 tid=0x000001ebb6eb3000 nid=0x2a88 waiting on condition [0x000000ea2a9fe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c4ce3b70> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:418)
        at com.alibaba.nacos.common.remote.client.RpcClient$3.run(RpcClient.java:317)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.remote.worker" #26 daemon prio=5 os_prio=0 tid=0x000001ebb5a0d800 nid=0x58dc waiting on condition [0x000000ea2a8fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c4ce34e0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at com.alibaba.nacos.common.remote.client.RpcClient$2.run(RpcClient.java:295)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"nacos.publisher-com.alibaba.nacos.common.notify.SlowEvent" #24 daemon prio=5 os_prio=0 tid=0x000001ebba958800 nid=0x44fc waiting on condition [0x000000ea2a7ff000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c4d04cf0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:403)
        at com.alibaba.nacos.common.notify.DefaultPublisher.openEventHandler(DefaultPublisher.java:117)
        at com.alibaba.nacos.common.notify.DefaultPublisher.run(DefaultPublisher.java:95)

"com.alibaba.nacos.client.Worker" #22 daemon prio=5 os_prio=0 tid=0x000001ebbaa06800 nid=0x46ec waiting on condition [0x000000ea2a6ff000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c464ea40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #21 daemon prio=5 os_prio=0 tid=0x000001ebb7246000 nid=0x5d74 waiting on condition [0x000000ea2a5fe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c464eed0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:418)
        at com.alibaba.nacos.client.config.impl.ClientWorker$ConfigRpcTransportClient$4.run(ClientWorker.java:744)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.Worker" #20 daemon prio=5 os_prio=0 tid=0x000001ebb7245800 nid=0x4c58 waiting on condition [0x000000ea2a4fe000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c464ea40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"com.alibaba.nacos.client.identify.watcher.0" #17 daemon prio=5 os_prio=0 tid=0x000001ebba827800 nid=0x2694 waiting on condition [0x000000ea2a1ff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c464e068> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

"Service Thread" #11 daemon prio=9 os_prio=0 tid=0x000001ebb586a000 nid=0x6748 runnable [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C1 CompilerThread3" #10 daemon prio=9 os_prio=2 tid=0x000001ebb57cb800 nid=0x5f04 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread2" #9 daemon prio=9 os_prio=2 tid=0x000001ebb57ca000 nid=0x5dfc waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread1" #8 daemon prio=9 os_prio=2 tid=0x000001ebb57c4800 nid=0x27ec waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" #7 daemon prio=9 os_prio=2 tid=0x000001ebb57c3800 nid=0x4fa4 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Monitor Ctrl-Break" #6 daemon prio=5 os_prio=0 tid=0x000001ebb57c0000 nid=0x44e4 runnable [0x000000ea29aff000]
   java.lang.Thread.State: RUNNABLE
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
        at java.net.SocketInputStream.read(SocketInputStream.java:171)
        at java.net.SocketInputStream.read(SocketInputStream.java:141)
        at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
        at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
        at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
        - locked <0x00000006c402f1e0> (a java.io.InputStreamReader)
        at java.io.InputStreamReader.read(InputStreamReader.java:184)
        at java.io.BufferedReader.fill(BufferedReader.java:161)
        at java.io.BufferedReader.readLine(BufferedReader.java:324)
        - locked <0x00000006c402f1e0> (a java.io.InputStreamReader)
        at java.io.BufferedReader.readLine(BufferedReader.java:389)
        at com.intellij.rt.execution.application.AppMainV2$1.run(AppMainV2.java:53)

"Attach Listener" #5 daemon prio=5 os_prio=2 tid=0x000001ebb2bda800 nid=0x1c3c waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" #4 daemon prio=9 os_prio=2 tid=0x000001ebb2bc4000 nid=0x1b64 runnable [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Finalizer" #3 daemon prio=8 os_prio=1 tid=0x000001ebb2bad000 nid=0x71ac in Object.wait() [0x000000ea297ff000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144)
        - locked <0x00000006c403f3c8> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165)
        at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:216)

"Reference Handler" #2 daemon prio=10 os_prio=2 tid=0x000001ebb2ba5800 nid=0x1264 in Object.wait() [0x000000ea28dff000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:502)
        at java.lang.ref.Reference.tryHandlePending(Reference.java:191)
        - locked <0x00000006c40308f0> (a java.lang.ref.Reference$Lock)
        at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153)

"VM Thread" os_prio=2 tid=0x000001ebb2b78800 nid=0x718 runnable

"GC task thread#0 (ParallelGC)" os_prio=0 tid=0x000001eb97e2b000 nid=0x2f74 runnable

"GC task thread#1 (ParallelGC)" os_prio=0 tid=0x000001eb97e2e000 nid=0x6660 runnable

"GC task thread#2 (ParallelGC)" os_prio=0 tid=0x000001eb97e31800 nid=0x780 runnable

"GC task thread#3 (ParallelGC)" os_prio=0 tid=0x000001eb97e34000 nid=0x2fb4 runnable

"GC task thread#4 (ParallelGC)" os_prio=0 tid=0x000001eb97e35000 nid=0x4c24 runnable

"GC task thread#5 (ParallelGC)" os_prio=0 tid=0x000001eb97e3d000 nid=0x67c8 runnable

"GC task thread#6 (ParallelGC)" os_prio=0 tid=0x000001eb97e3b800 nid=0x3218 runnable

"GC task thread#7 (ParallelGC)" os_prio=0 tid=0x000001eb97e39000 nid=0xf78 runnable

"VM Periodic Task Thread" os_prio=2 tid=0x000001ebb5938000 nid=0x21e8 waiting on condition

JNI global references: 2220

2.9 導出內存Dump文件/GC.heap_dump

命令:jcmd <pid> GC.heap_dump FILE_NAME
作用:查看 JVM 的Heap Dump。導出的 dump 文件,可以使用MAT 或者 Visual VM 等工具進行分析(如果只指定文件名,默認會生成在啓動 JVM 的目錄裏)

同類作用的命令:和 jmap -dump:format=b,file=heapdump.phrof pid 作用一樣

示例:

# jcmd 17124 GC.heap_dump ./dump.hprof 
17124: Heap dump file created

2.10 查看JVM系統屬性/VM.system_properties

命令:jcmd <pid> VM.system_properties
作用:查看 JVM 的屬性信息。

同類作用的命令:和jinfo -sysprops pid作用一樣。jinfo pid會輸出 VM.system_properties 和 VM.flags,命令短,更好記住,所以推薦直接使用jinfo pid

示例:

# jcmd 17124 VM.system_properties
17124:
#Sun Nov 14 12:39:36 CST 2021
java.runtime.name=OpenJDK Runtime Environment
java.protocol.handler.pkgs=org.springframework.boot.loader
sun.boot.library.path=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.282.b08-1.el7_9.x86_64/jre/lib/amd64
java.vm.version=25.282-b08
java.vm.vendor=Red Hat, Inc.
java.vendor.url=https\://www.redhat.com/
.......省略.......
sun.io.unicode.encoding=UnicodeLittle
sun.cpu.endian=little
sun.cpu.isalist=

2.11 查看JVM啓動參數/VM.flags

命令:jcmd <pid> VM.flags
作用:查看 JVM 的啓動參數。

同類作用的命令:和jinfo -flags {pid}作用一樣。jinfo pid會輸出 VM.system_properties 和 VM.flags。

示例:

# jcmd 17124 VM.flags
17124:
-XX:CICompilerCount=4 -XX:InitialHeapSize=264241152 -XX:MaxHeapSize=4227858432 -XX:MaxNewSize=1409286144 -XX:MinHeapDeltaBytes=524288 -XX:NewSize=88080384 -XX:OldSize=176160768 -XX:+UnlockCommercialFeatures -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseFastUnorderedTimeStamps -XX:-UseLargePagesIndividualAllocation -XX:+UseParallelGC

2.12 查看JVM啓動命令行/VM.command_line

命令:jcmd <pid> VM.command_line
作用:查看 JVM 的啓動命令行。

示例:

# jcmd 17124 VM.command_line
17124:
VM Arguments:
jvm_args: -Xmx512m 
java_command: /etc/java-app/springtest-0.0.1-SNAPSHOT.jar
java_class_path (initial): /etc/java-app/springtest-0.0.1-SNAPSHOT.jar
Launcher Type: SUN_STANDARD

2.13 執行finalization操作/GC.run_finalization

命令:jcmd <pid> GC.run_finalization
作用: 對 JVM 執行 java.lang.System.runFinalization()。即 :執行一次 finalization 操作
調用已經失去引用的對象的finalize方法,但JVM可以選擇執行或者不執行。

示例:

# jcmd 17124 GC.run_finalization
17124:
Command executed successfully

2.14 執行垃圾收集/GC.run

命令:jcmd <pid> GC.run
作用:對 JVM 執行 java.lang.System.gc()。同 GC.run_finalization 告訴垃圾收集器打算進行垃圾收集,但是JVM可以選擇執行或者不執行。

$ jcmd 17124 GC.run
17124:
Command executed successfully

2.15 查看JVM進程版本信息/VM.version

命令:jcmd <pid> VM.version
作用:查看目標jvm進程的版本信息。

示例:

# jcmd 6339 VM.version
6339:
OpenJDK 64-Bit Server VM version 25.282-b08
JDK 8.0_282

2.16 VM.native_memory

命令:jcmd <pid> VM.native_memory
作用:查看目標jvm進程的Native Memory Tracking (NMT)信息,用於追蹤JVM的內部內存使用。
注意:

  • 打開NMT會帶來5%-10%的性能損耗。
  • -XX:NativeMemoryTracking=xx參數必須放在 -jar 前面
    • 示例:java -XX:NativeMemoryTracking=detail -jar -Xmn512m user.jar
  • 使用 -XX:NativeMemoryTracking=summary 可以用於開啓NMT,其中該值默認爲off。
    • 可以設置summary、detail來開啓;開啓的話,大概會增加5%-10%的性能消耗;
    • 使用-XX:+UnlockDiagnosticVMOptions -XX:+PrintNMTStatistics。 可以在jvm shutdown的時候輸出整體的native memory統計;
    • 其他的可以使用jcmd pid VM.native_memory相關命令進行查看、diff、shutdown等
  • 整個memory主要包含了Java Heap、Class、Thread、Code、GC、Compiler、Internal、Other、Symbol、Native Memory Tracking、Arena Chunk這幾部分;
    • 其中 reserved 表示應用可用的內存大小,committed 表示應用正在使用的內存大小
jcmd <pid> VM.native_memory [summary | detail | baseline | summary.diff | detail.diff | shutdown] [scale= KB | MB | GB]
 
# summary: 分類內存使用情況.
# detail: 詳細內存使用情況,除了summary信息之外還包含了虛擬內存使用情況。
# baseline: 創建內存使用快照,方便和後面做對比
# summary.diff: 和上一次baseline的summary對比
# detail.diff: 和上一次baseline的detail對比
# shutdown: 關閉NMT

示例:

# jcmd 17124 VM.native_memory
17124:
Native memory tracking is not enabled
  • 查看是否已開啓NMT
# jinfo -flag NativeMemoryTracking 17124
-XX:NativeMemoryTracking=off

# jinfo -flag UnlockDiagnosticVMOptions 17124

# jinfo -flag PrintNMTStatistics 17124
  • 實時設置開啓NMT
# jinfo -flag NativeMemoryTracking=summary/detail 17124 
	[X] 不支持運行時修改 Exception in thread "main" com.sun.tools.attach.AttachOperationFailedException: flag 'NativeMemoryTracking' cannot be changed
	
# jinfo -flag +UnlockDiagnosticVMOptions 17124
	[X] 不支持運行時修改
	
# jinfo -flag +PrintNMTStatistics 17124
	[X] 不支持運行時修改
  • 查看 NMT JVM 內存統計結果
# jcmd 24568 VM.native_memory
24568:

Native Memory Tracking:

Total: reserved=6091612KB, committed=1355164KB
-                 Java Heap (reserved=4128768KB, committed=661504KB)
                            (mmap: reserved=4128768KB, committed=661504KB)

-                     Class (reserved=1141748KB, committed=104052KB)
                            (classes #16686)
                            (malloc=11252KB #23205)
                            (mmap: reserved=1130496KB, committed=92800KB)

-                    Thread (reserved=101832KB, committed=101832KB)
                            (thread #100)
                            (stack: reserved=101376KB, committed=101376KB)
                            (malloc=307KB #506)
                            (arena=149KB #195)

-                      Code (reserved=255298KB, committed=35598KB)
                            (malloc=5698KB #8972)
                            (mmap: reserved=249600KB, committed=29900KB)

-                        GC (reserved=161302KB, committed=149514KB)
                            (malloc=10390KB #325)
                            (mmap: reserved=150912KB, committed=139124KB)

-                  Compiler (reserved=173KB, committed=173KB)
                            (malloc=42KB #802)
                            (arena=131KB #7)

-                  Internal (reserved=274448KB, committed=274448KB)
                            (malloc=274384KB #22420)
                            (mmap: reserved=64KB, committed=64KB)

-                    Symbol (reserved=23692KB, committed=23692KB)
                            (malloc=20327KB #195671)
                            (arena=3364KB #1)

-    Native Memory Tracking (reserved=3961KB, committed=3961KB)
                            (malloc=17KB #196)
                            (tracking overhead=3944KB)

-               Arena Chunk (reserved=391KB, committed=391KB)
                            (malloc=391KB)


# jcmd 24568 VM.native_memory summary scale=MB
24568:

Native Memory Tracking:

Total: reserved=5950MB, committed=1325MB
-                 Java Heap (reserved=4032MB, committed=646MB)
                            (mmap: reserved=4032MB, committed=646MB)

-                     Class (reserved=1115MB, committed=102MB)
                            (classes #16686)
                            (malloc=11MB #23336)
                            (mmap: reserved=1104MB, committed=91MB)

-                    Thread (reserved=100MB, committed=100MB)
                            (thread #101)
                            (stack: reserved=100MB, committed=100MB)

-                      Code (reserved=249MB, committed=35MB)
                            (malloc=6MB #9278)
                            (mmap: reserved=244MB, committed=29MB)

-                        GC (reserved=158MB, committed=146MB)
                            (malloc=10MB #325)
                            (mmap: reserved=147MB, committed=136MB)

-                  Internal (reserved=268MB, committed=268MB)
                            (malloc=268MB #22425)

-                    Symbol (reserved=23MB, committed=23MB)
                            (malloc=20MB #195671)
                            (arena=3MB #1)

-    Native Memory Tracking (reserved=4MB, committed=4MB)
                            (tracking overhead=4MB)

X 參考文獻

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章