分析和解决JAVA 内存泄露的实战例子

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"这几天，一直在为Java的“内存泄露”问题纠结。Java应用程序占用的内存在不断的、有规律的上涨，最终超过了监控阈值。福尔摩 斯不得不出手了！"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"分析内存泄露的一般步骤"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果发现Java应用程序占用的内存出现了泄露的迹象，那么我们一般采用下面的步骤分析："}]},{"type":"numberedlist","attrs":{"start":null,"normalizeStart":1},"content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"把Java应用程序使用的heap dump下来"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"使用Java heap分析工具，找出内存占用超出预期（一般是因为数量太多）的嫌疑对象"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","text":"必要时，需要分析嫌疑对象和其他对象的引用关系。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":4,"align":null,"origin":null},"content":[{"type":"text","text":"查看程序的源代码，找出嫌疑对象数量过多的原因。"}]}]}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"dump heap"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果Java应用程序出现了内存泄露，千万别着急着把应用杀掉，而是要保存现场。如果是互联网应用，可以把流量切到其他服务器。保存现场的目的就是为了把 运行中JVM的heap dump下来。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"JDK自带的jmap工具，可以做这件事情。它的执行方法是："}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"jmap -dump:format=b,file=heap.bin \n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"format=b的含义是，dump出来的文件时二进制格式。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"file-heap.bin的含义是，dump出来的文件名是heap.bin。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"就是JVM的进程号。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"（在linux下）先执行ps aux | grep java，找到JVM的pid；然后再执行jmap -dump:format=b,file=heap.bin ，得到heap dump文件。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"analyze heap"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"将二进制的heap dump文件解析成human-readable的信息，自然是需要专业工具的帮助，这里推荐Memory Analyzer 。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Memory Analyzer，简称MAT，是Eclipse基金会的开源项目，由SAP和IBM捐助。巨头公司出品的软件还是很中用的，MAT可以分析包含数亿级对 象的heap、快速计算每个对象占用的内存大小、对象之间的引用关系、自动检测内存泄露的嫌疑对象，功能强大，而且界面友好易用。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"MAT的界面基于Eclipse开发，以两种形式发布：Eclipse插件和Eclipe RCP。MAT的分析结果以图片和报表的形式提供，一目了然。总之个人还是非常喜欢这个工具的。下面先贴两张官方的screenshots："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/e4/e44168001bda03170a543df5de032cd9.png","alt":"image.png","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/27/271c94a98fc85c28ebc2f0b4d5172396.png","alt":"image.png","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"言归正传，我用MAT打开了heap.bin，很容易看出，char[]的数量出其意料的多，占用90%以上的内存 。一般来说，char[]在JVM确实会占用很多内存，数量也非常多，因为String对象以char[]作为内部存储。但是这次的char[]太贪婪 了，仔细一观察，发现有数万计的char[]，每个都占用数百K的内存 。这个现象说明，Java程序保存了数以万计的大String对象 。结合程序的逻辑，这个是不应该的，肯定在某个地方出了问题。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"顺藤摸瓜"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在可疑的char[]中，任意挑了一个，使用Path To GC Root功能，找到该char[]的引用路径，发现String对象是被一个HashMap中引用的 。这个也是意料中的事情，Java的内存泄露多半是因为对象被遗留在全局的HashMap中得不到释放。不过，该HashMap被用作一个缓存，设置了缓 存条目的阈值，导达到阈值后会自动淘汰。从这个逻辑分析，应该不会出现内存泄露的。虽然缓存中的String对象已经达到数万计，但仍然没有达到预先设置 的阈值（阈值设置地比较大，因为当时预估String对象都比较小）。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"但是，另一个问题引起了我的注意：为什么缓存的String对象如此巨大？内部char[]的长度达数百K。虽然缓存中的 String对象数量还没有达到阈值，但是String对象大小远远超出了我们的预期，最终导致内存被大量消耗，形成内存泄露的迹象（准确说应该是内存消 耗过多） 。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"就这个问题进一步顺藤摸瓜，看看String大对象是如何被放到HashMap中的。通过查看程序的源代码，我发现，确实有String大对象，不 过并没有把String大对象放到HashMap中，而是把String大对象进行split（调用String.split方法），然后将split出 来的String小对象放到HashMap中 了。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"这就奇怪了，放到HashMap中明明是split之后的String小对象，怎么会占用那么大空间呢？难道是String类的split方法有问题？"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"查看代码"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"带着上述疑问，我查阅了Sun JDK6中String类的代码，主要是是split方法的实现："}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"public \nString[] split(String regex, int limit) { \n return Pattern.compile(regex).split(this, limit); \n} \n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"可以看出，Stirng.split方法调用了Pattern.split方法。继续看Pattern.split方法的代码："}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"public \nString[] split(CharSequence input, int limit) { \n int index = 0; \n boolean matchLimited = limit > 0; \n ArrayList matchList = new \nArrayList(); \n Matcher m = matcher(input); \n // Add segments before each match found \n while(m.find()) { \n if (!matchLimited || matchList.size() < limit - 1) { \n String match = input.subSequence(index, \nm.start()).toString(); \n matchList.add(match); \n index = m.end(); \n } else if (matchList.size() == limit - 1) { // last one \n String match = input.subSequence(index, \n \ninput.length()).toString(); \n matchList.add(match); \n index = m.end(); \n } \n } \n // If no match was found, return this \n if (index == 0) \n return new String[] {input.toString()}; \n // Add remaining segment \n if (!matchLimited || matchList.size() < limit) \n matchList.add(input.subSequence(index, \ninput.length()).toString()); \n // Construct result \n int resultSize = matchList.size(); \n if (limit == 0) \n while (resultSize > 0 && \nmatchList.get(resultSize-1).equals(\"\")) \n resultSize--; \n String[] result = new String[resultSize]; \n return matchList.subList(0, resultSize).toArray(result); \n } \n 注意看第9行：Stirng match = input.subSequence(intdex, m.start()).toString();\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"这里的match就是split出来的String小对象，它其实是String大对象subSequence的结果。继续看 String.subSequence的代码："}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"public \nCharSequence subSequence(int beginIndex, int endIndex) { \n return this.substring(beginIndex, endIndex); \n} \n String.subSequence有调用了String.subString，继续看：\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"public String \nsubstring(int beginIndex, int endIndex) { \n if (beginIndex < 0) { \n throw new StringIndexOutOfBoundsException(beginIndex); \n } \n if (endIndex > count) { \n throw new StringIndexOutOfBoundsException(endIndex); \n } \n if (beginIndex > endIndex) { \n throw new StringIndexOutOfBoundsException(endIndex - beginIndex); \n } \n return ((beginIndex == 0) && (endIndex == count)) ? this : \n new String(offset + beginIndex, endIndex - beginIndex, value); \n } \n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"看第11、12行，我们终于看出眉目，如果subString的内容就是完整的原字符串，那么返回原String对象；否则，就会创建一个新的 String对象，但是这个String对象貌似使用了原String对象的char[]。我们通过String的构造函数确认这一点："}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"// Package \nprivate constructor which shares value array for speed. \n String(int offset, int count, char value[]) { \n this.value = value; \n this.offset = offset; \n this.count = count; \n } \n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"为了避免内存拷贝、加快速度，Sun JDK直接复用了原String对象的char[]，偏移量和长度来标识不同的字符串内容。也就是说，subString出的来String小对象 仍然会指向原String大对象的char[]，split也是同样的情况 。这就解释了，为什么HashMap中String对象的char[]都那么大。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"原因解释"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"其实上一节已经分析出了原因，这一节再整理一下："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"程序从每个请求中得到一个String大对象，该对象内部char[]的长度达数百K。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"程序对String大对象做split，将split得到的String小对象放到HashMap中，用作缓存。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Sun JDK6对String.split方法做了优化，split出来的Stirng对象直接使用原String对象的char[]"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"HashMap中的每个String对象其实都指向了一个巨大的char[]"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"HashMap的上限是万级的，因此被缓存的Sting对象的总大小=万*百K=G级。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"G级的内存被缓存占用了，大量的内存被浪费，造成内存泄露的迹象。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"解决方案"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"原因找到了，解决方案也就有了。split是要用的，但是我们不要把split出来的String对象直接放到HashMap中，而是调用一下 String的拷贝构造函数String(String original)，这个构造函数是安全的，具体可以看代码："}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":" /** \n * Initializes a newly created {@code String} object so that it \nrepresents \n * the same sequence of characters as the argument; in other words, \nthe \n * newly created string is a copy of the argument string. Unless an \n * explicit copy of {@code original} is needed, use of this \nconstructor is \n * unnecessary since Strings are immutable. \n * \n * @param original \n * A {@code String} \n */ \n public String(String original) { \n int size = original.count; \n char[] originalValue = original.value; \n char[] v; \n if (originalValue.length > size) { \n // The array representing the String is bigger than the new \n // String itself. Perhaps this constructor is being called \n // in order to trim the baggage, so make a copy of the array. \n int off = original.offset; \n v = Arrays.copyOfRange(originalValue, off, off+size); \n } else { \n // The array representing the String is the same \n // size as the String, so no point in making a copy. \n v = originalValue; \n } \n this.offset = 0; \n this.count = size; \n this.value = v; \n } \n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"只是，new String(string)的代码很怪异，囧。或许，subString和split应该提供一个选项，让程序员控制是否复用String对象的 char[]。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"是否Bug"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"虽然，subString和split的实现造成了现在的问题，但是这能否算String类的bug呢？个人觉得不好说。因为这样的优化是比较合理 的，subString和spit的结果肯定是原字符串的连续子序列。只能说，String不仅仅是一个核心类，它对于JVM来说是与原始类型同等重要的 类型。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"JDK实现对String做各种可能的优化都是可以理解的。但是优化带来了忧患，我们程序员足够了解他们，才能用好他们。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"一些补充"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"有个地方我没有说清楚。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我的程序是一个Web程序，每次接受请求，就会创建一个大的String对象，然后对该String对象进行split，最后split之后的String对象放到全局缓存中。如果接收了5W个请求，那么就会有5W个大String对象。这5W个大String对象都被存储在全局缓存中，因此会造成内存泄漏。我原以为缓存的是5W个小String，结果都是大String。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"有同学后续建议用\"java.io.StreamTokenizer\"来解决本文的问题。确实是终极解决方案，比我上面提到的“new String()”，要好很多很多。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"看完三件事❤️"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果你觉得这篇内容对你还蛮有帮助，我想邀请你帮我三个小忙："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"numberedlist","attrs":{"start":null,"normalizeStart":1},"content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"点赞，转发，有你们的 『点赞和评论』，才是我创造的动力。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"关注公众号 『 "},{"type":"text","marks":[{"type":"strong"}],"text":"java烂猪皮"},{"type":"text","text":" 』，不定期分享原创知识。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","text":"同时可以期待后续文章ing🚀"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/85/8518f1f13ab07e46122c84420bbf39a8.png","alt":null,"title":"","style":[{"key":"width","value":"50%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"出处：https://club.perfma.com/article/1815828"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}