springboot Too many open files错误 打开文件数一直增加,已解决

问题描述:
前端框架VUE,请求插件axios,后端框架springboot。
操作:
在页面中请求springboot的controller接口,在centos7中使用lsof -p 1605 | wc -l命令查询打开文件数

每请求一次,查询打开文件数的结果就会多出48个文件,一直到4000多的时候,就会报tomcat Too many open files异常,而且打开文件数只会上升不会下降,本人觉得原因应该是socket没关闭吧,但是代码看着没有任何问题,不知道有没有人遇到过这类的问题

解决方法:
**

https://blog.csdn.net/yangleiGJ/article/details/103121278

**

场景:
本地从kafka获取数据,模拟设备,通过808协议给第三方平台发送数据,一个设备一个连接。
问题:
使用netty做性能测试时,并发过大造成Too Many open files问题
每个设备对象,内部就一个netty对象请求,结果1000多个设备,使用服务器句柄达到了17w+,

解决如下: 最后通过修改bootstrap.group(new NioEventLoopGroup(1));然后句柄数据正常了。之前使用默认的new NioEventLoopGroup()

查看进程对应的句柄数

[root@xxxxx]# lsof -n|awk ‘{print $2}’|sort|uniq -c|sort -nr|more|grep ‘36576’
6619 36576

查看对应的连接

[root@xxxxx]# netstat -lnaop|grep 7008|wc -l
17

查看进程对应线程数
[root@xxxxx]# cat /proc/36576/status|grep ‘Threads’
Threads: 4323

这还不是最终的解决方法,解决方法是用连接池代替netty的connect放回的future.
PooledConnection

package com.siblings.iredis.client.netty;

import io.netty.channel.Channel;
import io.netty.channel.ChannelFuture;
import lombok.Data;

@Data
public class PooledConnection {
    private ChannelFuture future;
    private boolean isBusy;

    public PooledConnection() {
    }

    public PooledConnection(ChannelFuture future) {
        this.future = future;
        this.isBusy = true;
    }

    public Channel getChannel(){
        return future.channel();
    }
}

Connection

package com.siblings.iredis.client.netty;

import io.netty.bootstrap.Bootstrap;
import io.netty.channel.Channel;
import io.netty.channel.ChannelFuture;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.EventLoopGroup;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.nio.NioSocketChannel;
import io.netty.handler.codec.LineBasedFrameDecoder;
import io.netty.handler.codec.string.StringDecoder;
import io.netty.handler.codec.string.StringEncoder;

import java.util.ArrayList;
import java.util.List;

public class Connection {

    private static List<PooledConnection> list = new ArrayList<>();

    public static PooledConnection connect(String ip, int port) {
        PooledConnection free = match();
        if (free == null) {
            ChannelFuture future = getCon(ip, port);
            if (future != null) {
                free = new PooledConnection(future);
                list.add(free);
            }
        }
        return free;
    }

    public static PooledConnection match() {
        for (PooledConnection pool : list) {
            if (!pool.isBusy()) {
                pool.setBusy(true);
                return pool;
            }
        }
        return null;
    }
    public static ChannelFuture getCon(String ip, int port) {
        EventLoopGroup group = new NioEventLoopGroup(1);
        final Bootstrap bootstrap = new Bootstrap()
                .channel(NioSocketChannel.class)
                .handler(new ChannelInitializer<Channel>() {
                    @Override
                    protected void initChannel(Channel channel) throws Exception {
                        channel.pipeline()
                            .addLast(new LineBasedFrameDecoder(10240))
                            .addLast(new StringDecoder())
                            .addLast(new StringEncoder());
                    }
                })
                .group(group);

        try {
            ChannelFuture f = bootstrap.connect(ip, port).sync();
            return f;
        } catch (InterruptedException e) {
            e.printStackTrace();
        } catch (Exception e) {
            e.printStackTrace();
        }
        return null;
    }
}

netty中如果无限制的创建客户端,netty客户端的channel调用close方法后nio线程的打开文件不会释放,原因还在查找中。

原因找到了,跟工作线程有关系,由于每次创建netty客户端,我都在代码中创建新的EventLoopGroup,导致线程的打开文件数不断上升。
错误代码:

public static ChannelFuture getCon(String ip, int port) {
		EventLoopGroup group = new NioEventLoopGroup(1);
        final Bootstrap bootstrap = new Bootstrap()
                .channel(NioSocketChannel.class)
                .handler(new ChannelInitializer<Channel>() {
                    @Override
                    protected void initChannel(Channel channel) throws Exception {
                        channel.pipeline()
                            .addLast(new LineBasedFrameDecoder(10240))
                            .addLast(new StringDecoder())
                            .addLast(new StringEncoder());
                    }
                })
                .group(group);

        try {
            ChannelFuture f = bootstrap.connect(ip, port).sync();
            return f;
        } catch (InterruptedException e) {
            e.printStackTrace();
        } catch (Exception e) {
            e.printStackTrace();
        }
        return null;
    }

正确代码:

private static EventLoopGroup group = new NioEventLoopGroup(1);
public static ChannelFuture getCon(String ip, int port) {
        final Bootstrap bootstrap = new Bootstrap()
                .channel(NioSocketChannel.class)
                .handler(new ChannelInitializer<Channel>() {
                    @Override
                    protected void initChannel(Channel channel) throws Exception {
                        channel.pipeline()
                            .addLast(new LineBasedFrameDecoder(10240))
                            .addLast(new StringDecoder())
                            .addLast(new StringEncoder());
                    }
                })
                .group(group);

        try {
            ChannelFuture f = bootstrap.connect(ip, port).sync();
            return f;
        } catch (InterruptedException e) {
            e.printStackTrace();
        } catch (Exception e) {
            e.printStackTrace();
        }
        return null;
    }
發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章