nacos集羣配置服務註冊不上來

 

 

現象：

1、雲主機本機服務以docker bridge網絡運行，本機服務能註冊到nacos。但是不利於多服務器集羣。

2、雲主機本機服務直接運行，或者以docker host網絡運行，本機服務不能註冊到本機nacos。

3、如果nacos運行在服務器A，其他服務運行在服務器B，那麼註冊服務都正常。

4、機房服務器，無論怎麼折騰，都正常。

5、雲主機和機房服務器，都試過centos7，centos8。現象一致未改變。

報錯：

啓動日誌：

09:28:50.459 [main] ERROR c.a.n.c.r.c.g.GrpcClient - [printIfErrorEnabled,99] - Server check fail, please check server 10.100.0.19 ,port 9848 is available , error ={}
java.util.concurrent.TimeoutException: Waited 3000 milliseconds (plus 610940 nanoseconds delay) for com.alibaba.nacos.shaded.io.grpc.stub.ClientCalls$GrpcFuture@4d49af10[status=PENDING, info=[GrpcFuture{clientCall={delegate={delegate=ClientCallImpl{method=MethodDescriptor{fullMethodName=Request/request, type=UNARY, idempotent=false, safe=false, sampledToLocalTracing=true, requestMarshaller=com.alibaba.nacos.shaded.io.grpc.protobuf.lite.ProtoLiteUtils$MessageMarshaller@279ad2e3, responseMarshaller=com.alibaba.nacos.shaded.io.grpc.protobuf.lite.ProtoLiteUtils$MessageMarshaller@58134517, schemaDescriptor=com.alibaba.nacos.api.grpc.auto.RequestGrpc$RequestMethodDescriptorSupplier@4450d156}}}}}]]
        at com.alibaba.nacos.shaded.com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:508)
        at com.alibaba.nacos.common.remote.client.grpc.GrpcClient.serverCheck(GrpcClient.java:146)
        at com.alibaba.nacos.common.remote.client.grpc.GrpcClient.connectToServer(GrpcClient.java:268)
        at com.alibaba.nacos.common.remote.client.RpcClient.start(RpcClient.java:394)
        at com.alibaba.nacos.client.config.impl.ClientWorker$ConfigRpcTransportClient.ensureRpcClient(ClientWorker.java:941)
        at com.alibaba.nacos.client.config.impl.ClientWorker$ConfigRpcTransportClient.getOneRunningClient(ClientWorker.java:1104)
        at com.alibaba.nacos.client.config.impl.ClientWorker$ConfigRpcTransportClient.queryConfig(ClientWorker.java:996)
        at com.alibaba.nacos.client.config.impl.ClientWorker.getServerConfig(ClientWorker.java:407)
        at com.alibaba.nacos.client.config.NacosConfigService.getConfigInner(NacosConfigService.java:166)
        at com.alibaba.nacos.client.config.NacosConfigService.getConfig(NacosConfigService.java:94)
        at com.alibaba.cloud.nacos.client.NacosPropertySourceBuilder.loadNacosData(NacosPropertySourceBuilder.java:85)
        at com.alibaba.cloud.nacos.client.NacosPropertySourceBuilder.build(NacosPropertySourceBuilder.java:73)
        at com.alibaba.cloud.nacos.client.NacosPropertySourceLocator.loadNacosPropertySource(NacosPropertySourceLocator.java:199)
        at com.alibaba.cloud.nacos.client.NacosPropertySourceLocator.loadNacosDataIfPresent(NacosPropertySourceLocator.java:186)
        at com.alibaba.cloud.nacos.client.NacosPropertySourceLocator.loadNacosConfiguration(NacosPropertySourceLocator.java:158)
        at com.alibaba.cloud.nacos.client.NacosPropertySourceLocator.loadSharedConfiguration(NacosPropertySourceLocator.java:116)
        at com.alibaba.cloud.nacos.client.NacosPropertySourceLocator.locate(NacosPropertySourceLocator.java:101)
        at org.springframework.cloud.bootstrap.config.PropertySourceLocator.locateCollection(PropertySourceLocator.java:51)
        at org.springframework.cloud.bootstrap.config.PropertySourceLocator.locateCollection(PropertySourceLocator.java:47)
        at org.springframework.cloud.bootstrap.config.PropertySourceBootstrapConfiguration.initialize(PropertySourceBootstrapConfiguration.java:95)
        at org.springframework.boot.SpringApplication.applyInitializers(SpringApplication.java:638)
        at org.springframework.boot.SpringApplication.prepareContext(SpringApplication.java:402)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:336)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1336)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1325)
        at com.spt.edsp.basedata.BasedataApplication.main(BasedataApplication.java:19)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:108)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
        at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88)

問題很明顯，不是端口問題，而是localhost的網絡問題。

解決過程：

1、臨時關閉防火牆，未解決。

[root@10-100-0-19 ~]# setenforce 0
setenforce: SELinux is disabled

2、徹底關閉防火牆，未解決。

[root@10-100-0-19 ~]# systemctl disable firewalld
[root@10-100-0-19 ~]# systemctl stop firewalld

[root@10-100-0-19 ~]# systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
   Active: inactive (dead) since 四 2021-12-16 14:49:37 CST; 3 months 24 days ago
     Docs: man:firewalld(1)
 Main PID: 1197 (code=exited, status=0/SUCCESS)

12月 15 17:55:08 localhost.localdomain systemd[1]: Starting firewalld - dynamic firewall daemon...
12月 15 17:55:09 localhost.localdomain systemd[1]: Started firewalld - dynamic firewall daemon.
12月 15 17:55:09 localhost.localdomain firewalld[1197]: WARNING: AllowZoneDrifting is enabled. This is considered an insecure configuration option. It will be remo...g it now.
12月 16 11:31:27 10-100-0-19 firewalld[1197]: WARNING: AllowZoneDrifting is enabled. This is considered an insecure configuration option. It will be removed in a ...ng it now.
12月 16 11:53:51 10-100-0-19 firewalld[1197]: WARNING: AllowZoneDrifting is enabled. This is considered an insecure configuration option. It will be removed in a ...ng it now.
12月 16 14:49:36 10-100-0-19 systemd[1]: Stopping firewalld - dynamic firewall daemon...
12月 16 14:49:37 10-100-0-19 systemd[1]: Stopped firewalld - dynamic firewall daemon.
Hint: Some lines were ellipsized, use -l to show in full.

3、更改hosts，解決。

[root@10-100-0-19 ~]# vim /etc/hosts
127.0.0.1  localhost  10-100-0-19  # 添加這一行
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

在文件/etc/hosts，添加一行，localhost和主機名對應起來。

4、分析原因。

註冊服務異常的機器，hostname -i 沒有對應到 127.0.0.1。主機名沒有和127.0.0.1對應起來。

[root@10-100-0-18 file]# hostname -i
fe80::b9a:d281:e84b:1f6a%ens12 fe80::1018:1b6e:6103:6d95%ens13 10.216.115.211 10.100.0.18
[root@10-100-0-18 file]# hostname
10-100-0-18

註冊服務正常的機器：

[root@10-100-0-21 ~]# hostname
10-100-0-21
[root@10-100-0-21 ~]# hostname -i
127.0.0.1

接着查看

[root@10-100-0-19 ~]# hostnamectl
   Static hostname: localhost.localdomain
Transient hostname: 10-100-0-19
         Icon name: computer-vm
           Chassis: vm
        Machine ID: 46a19cb5f7064fbeb634f9668f22d158
           Boot ID: 07aadce6349c4d8eb55a855f8b31c58e
    Virtualization: kvm
  Operating System: CentOS Linux 7 (Core)
       CPE OS Name: cpe:/o:centos:centos:7
            Kernel: Linux 3.10.0-1160.49.1.el7.x86_64
      Architecture: x86-64

10-100-0-19爲主機名，Transient hostname 臨時主機名。而原本hosts文件已經有了localhost對應Static hostname: localhost.localdomain。看來nacos註冊服務，需要的是Transient hostname來替代localhost。

從nacos的配置文件，也有點類似的參數，不知道是否針對此問題。

[root@10-100-0-19 ~]# vim /opt/edsp/nacos/conf/application.properties
.....................................................................
#*************** Spring Boot Related Configurations ***************#
### Default web context path:
server.servlet.contextPath=/nacos
### Default web server port:

#*************** Network Related Configurations ***************#
### If prefer hostname over ip for Nacos server addresses in cluster.conf:
# nacos.inetutils.prefer-hostname-over-ip=false

### Specify local server's IP:
# nacos.inetutils.ip-address=
.........................................................................

類似 https://github.com/alibaba/nacos/issues/404