Netty之OutOfDirectMemoryError

    最近使用Netty时，由于JVM参数设置错误，有OutOfDirectMemoryError出现，记录下：

    OutOfDirectMemoryError是在分配堆外内存时发生的，由于堆外内存不够，导致抛出异常，如下List-1，PlatformDependent中的

    List-1

private static void incrementMemoryCounter(int capacity) {
    if (DIRECT_MEMORY_COUNTER != null) {
        for (;;) {
            long usedMemory = DIRECT_MEMORY_COUNTER.get();
            long newUsedMemory = usedMemory + capacity;
            if (newUsedMemory > DIRECT_MEMORY_LIMIT) {
                throw new OutOfDirectMemoryError("failed to allocate " + capacity
                        + " byte(s) of direct memory (used: " + usedMemory + ", max: " + DIRECT_MEMORY_LIMIT + ')');
            }
            if (DIRECT_MEMORY_COUNTER.compareAndSet(usedMemory, newUsedMemory)) {
                break;
            }
        }
    }
}

    使用一个AtomicLong类型的变量DIRECT_MEMORY_COUNTER来存储目前分配了多少堆外内存，每次申请堆外内存，先判断下目前已经申请的+要申请的是否大于Direct memeory大小限制，如果超过，那么就抛出异常；对应的，释放堆外内存时，从DIRECT_MEMORY_COUNTER中减去对应的大小。

    那么DIRECT_MEMORY_LIMIT的值是怎么获取的？

    List-2

static {
    if (logger.isDebugEnabled()) {
        logger.debug("-Dio.netty.noPreferDirect: {}", !DIRECT_BUFFER_PREFERRED);
    }

    if (!hasUnsafe() && !isAndroid() && !IS_EXPLICIT_NO_UNSAFE) {
        logger.info(
                "Your platform does not provide complete low-level API for accessing direct buffers reliably. " +
                "Unless explicitly requested, heap buffer will always be preferred to avoid potential system " +
                "unstability.");
    }

    // Here is how the system property is used:
    //
    // * <  0  - Don't use cleaner, and inherit max direct memory from java. In this case the
    //           "practical max direct memory" would be 2 * max memory as defined by the JDK.
    // * == 0  - Use cleaner, Netty will not enforce max memory, and instead will defer to JDK.
    // * >  0  - Don't use cleaner. This will limit Netty's total direct memory
    //           (note: that JDK's direct memory limit is independent of this).
    long maxDirectMemory = SystemPropertyUtil.getLong("io.netty.maxDirectMemory", -1);

    if (maxDirectMemory == 0 || !hasUnsafe() || !PlatformDependent0.hasDirectBufferNoCleanerConstructor()) {
        USE_DIRECT_BUFFER_NO_CLEANER = false;
        DIRECT_MEMORY_COUNTER = null;
    } else {
        USE_DIRECT_BUFFER_NO_CLEANER = true;
        if (maxDirectMemory < 0) {
            maxDirectMemory = maxDirectMemory0();
            if (maxDirectMemory <= 0) {
                DIRECT_MEMORY_COUNTER = null;
            } else {
                DIRECT_MEMORY_COUNTER = new AtomicLong();
            }
        } else {
            DIRECT_MEMORY_COUNTER = new AtomicLong();
        }
    }
    DIRECT_MEMORY_LIMIT = maxDirectMemory;
    logger.debug("io.netty.maxDirectMemory: {} bytes", maxDirectMemory);
}

    如上List-2，

    首先从io.netty.maxDirectMemory设置的SystemProperty中获取，如果这个值大于0，说明设置了最大堆外内存，直接使用。

    如果没有设置io.netty.maxDirectMemory，那么获取maxDirectMemory0()方法来获取：

•     先获取sun包得到VM类获取最大内存，如果值大于0，那么返还使用
•     如果获取不到，则获取JVM参数MaxDirectMemorySize的值，即我们可以通过jvm参数指定使用最大多少堆外内存
•     如果还是没有获取得到，则使用Runtime.getRuntime().maxMemory()的值

    List-3

private static long maxDirectMemory0() {
    long maxDirectMemory = 0;
    try {
        // Try to get from sun.misc.VM.maxDirectMemory() which should be most accurate.
        Class<?> vmClass = Class.forName("sun.misc.VM", true, getSystemClassLoader());
        Method m = vmClass.getDeclaredMethod("maxDirectMemory");
        maxDirectMemory = ((Number) m.invoke(null)).longValue();
    } catch (Throwable ignored) {
        // Ignore
    }

    if (maxDirectMemory > 0) {
        return maxDirectMemory;
    }

    try {
        // Now try to get the JVM option (-XX:MaxDirectMemorySize) and parse it.
        // Note that we are using reflection because Android doesn't have these classes.
        Class<?> mgmtFactoryClass = Class.forName(
                "java.lang.management.ManagementFactory", true, getSystemClassLoader());
        Class<?> runtimeClass = Class.forName(
                "java.lang.management.RuntimeMXBean", true, getSystemClassLoader());

        Object runtime = mgmtFactoryClass.getDeclaredMethod("getRuntimeMXBean").invoke(null);

        @SuppressWarnings("unchecked")
        List<String> vmArgs = (List<String>) runtimeClass.getDeclaredMethod("getInputArguments").invoke(runtime);
        for (int i = vmArgs.size() - 1; i >= 0; i --) {
            Matcher m = MAX_DIRECT_MEMORY_SIZE_ARG_PATTERN.matcher(vmArgs.get(i));
            if (!m.matches()) {
                continue;
            }

            maxDirectMemory = Long.parseLong(m.group(1));
            switch (m.group(2).charAt(0)) {
                case 'k': case 'K':
                    maxDirectMemory *= 1024;
                    break;
                case 'm': case 'M':
                    maxDirectMemory *= 1024 * 1024;
                    break;
                case 'g': case 'G':
                    maxDirectMemory *= 1024 * 1024 * 1024;
                    break;
            }
            break;
        }
    } catch (Throwable ignored) {
        // Ignore
    }

    if (maxDirectMemory <= 0) {
        maxDirectMemory = Runtime.getRuntime().maxMemory();
        logger.debug("maxDirectMemory: {} bytes (maybe)", maxDirectMemory);
    } else {
        logger.debug("maxDirectMemory: {} bytes", maxDirectMemory);
    }

    return maxDirectMemory;
}

    List-3里，如果我们的机器有8G运行内存，那么sun的VM类返还maxDirectMemory的值会是8G吗，代码实际执行的结果来看，是不会的，是小于8G的