springboot Too many open files錯誤 打開文件數一直增加,已解決

問題描述:
前端框架VUE,請求插件axios,後端框架springboot。
操作:
在頁面中請求springboot的controller接口,在centos7中使用lsof -p 1605 | wc -l命令查詢打開文件數

每請求一次,查詢打開文件數的結果就會多出48個文件,一直到4000多的時候,就會報tomcat Too many open files異常,而且打開文件數只會上升不會下降,本人覺得原因應該是socket沒關閉吧,但是代碼看着沒有任何問題,不知道有沒有人遇到過這類的問題

解決方法:
**

https://blog.csdn.net/yangleiGJ/article/details/103121278

**

場景:
本地從kafka獲取數據,模擬設備,通過808協議給第三方平臺發送數據,一個設備一個連接。
問題:
使用netty做性能測試時,併發過大造成Too Many open files問題
每個設備對象,內部就一個netty對象請求,結果1000多個設備,使用服務器句柄達到了17w+,

解決如下: 最後通過修改bootstrap.group(new NioEventLoopGroup(1));然後句柄數據正常了。之前使用默認的new NioEventLoopGroup()

查看進程對應的句柄數

[root@xxxxx]# lsof -n|awk ‘{print $2}’|sort|uniq -c|sort -nr|more|grep ‘36576’
6619 36576

查看對應的連接

[root@xxxxx]# netstat -lnaop|grep 7008|wc -l
17

查看進程對應線程數
[root@xxxxx]# cat /proc/36576/status|grep ‘Threads’
Threads: 4323

這還不是最終的解決方法,解決方法是用連接池代替netty的connect放回的future.
PooledConnection

package com.siblings.iredis.client.netty;

import io.netty.channel.Channel;
import io.netty.channel.ChannelFuture;
import lombok.Data;

@Data
public class PooledConnection {
    private ChannelFuture future;
    private boolean isBusy;

    public PooledConnection() {
    }

    public PooledConnection(ChannelFuture future) {
        this.future = future;
        this.isBusy = true;
    }

    public Channel getChannel(){
        return future.channel();
    }
}

Connection

package com.siblings.iredis.client.netty;

import io.netty.bootstrap.Bootstrap;
import io.netty.channel.Channel;
import io.netty.channel.ChannelFuture;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.EventLoopGroup;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.nio.NioSocketChannel;
import io.netty.handler.codec.LineBasedFrameDecoder;
import io.netty.handler.codec.string.StringDecoder;
import io.netty.handler.codec.string.StringEncoder;

import java.util.ArrayList;
import java.util.List;

public class Connection {

    private static List<PooledConnection> list = new ArrayList<>();

    public static PooledConnection connect(String ip, int port) {
        PooledConnection free = match();
        if (free == null) {
            ChannelFuture future = getCon(ip, port);
            if (future != null) {
                free = new PooledConnection(future);
                list.add(free);
            }
        }
        return free;
    }

    public static PooledConnection match() {
        for (PooledConnection pool : list) {
            if (!pool.isBusy()) {
                pool.setBusy(true);
                return pool;
            }
        }
        return null;
    }
    public static ChannelFuture getCon(String ip, int port) {
        EventLoopGroup group = new NioEventLoopGroup(1);
        final Bootstrap bootstrap = new Bootstrap()
                .channel(NioSocketChannel.class)
                .handler(new ChannelInitializer<Channel>() {
                    @Override
                    protected void initChannel(Channel channel) throws Exception {
                        channel.pipeline()
                            .addLast(new LineBasedFrameDecoder(10240))
                            .addLast(new StringDecoder())
                            .addLast(new StringEncoder());
                    }
                })
                .group(group);

        try {
            ChannelFuture f = bootstrap.connect(ip, port).sync();
            return f;
        } catch (InterruptedException e) {
            e.printStackTrace();
        } catch (Exception e) {
            e.printStackTrace();
        }
        return null;
    }
}

netty中如果無限制的創建客戶端,netty客戶端的channel調用close方法後nio線程的打開文件不會釋放,原因還在查找中。

原因找到了,跟工作線程有關係,由於每次創建netty客戶端,我都在代碼中創建新的EventLoopGroup,導致線程的打開文件數不斷上升。
錯誤代碼:

public static ChannelFuture getCon(String ip, int port) {
		EventLoopGroup group = new NioEventLoopGroup(1);
        final Bootstrap bootstrap = new Bootstrap()
                .channel(NioSocketChannel.class)
                .handler(new ChannelInitializer<Channel>() {
                    @Override
                    protected void initChannel(Channel channel) throws Exception {
                        channel.pipeline()
                            .addLast(new LineBasedFrameDecoder(10240))
                            .addLast(new StringDecoder())
                            .addLast(new StringEncoder());
                    }
                })
                .group(group);

        try {
            ChannelFuture f = bootstrap.connect(ip, port).sync();
            return f;
        } catch (InterruptedException e) {
            e.printStackTrace();
        } catch (Exception e) {
            e.printStackTrace();
        }
        return null;
    }

正確代碼:

private static EventLoopGroup group = new NioEventLoopGroup(1);
public static ChannelFuture getCon(String ip, int port) {
        final Bootstrap bootstrap = new Bootstrap()
                .channel(NioSocketChannel.class)
                .handler(new ChannelInitializer<Channel>() {
                    @Override
                    protected void initChannel(Channel channel) throws Exception {
                        channel.pipeline()
                            .addLast(new LineBasedFrameDecoder(10240))
                            .addLast(new StringDecoder())
                            .addLast(new StringEncoder());
                    }
                })
                .group(group);

        try {
            ChannelFuture f = bootstrap.connect(ip, port).sync();
            return f;
        } catch (InterruptedException e) {
            e.printStackTrace();
        } catch (Exception e) {
            e.printStackTrace();
        }
        return null;
    }
發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章