Websphere產生大量javacore與heapdump文件的分析

項目中碰到的情況，居然產生了20多G，把硬盤佔滿、服務Down掉。原因可能：出現過多內存泄漏,或者分配過多大內存等。解決方法：

 

1、進入was管理控制檯，選擇 應用程序服務器 > server1 > 進程定義 > Java 虛擬機，將"最大堆大小"改爲768或1024以上（跟機器內存相關，你的機器最好有較大內存）。保存。
2、優化你的程序，減少要求分配較大內存的設計，優化數據連接池。
3、給was打補丁。ibm網站上有相關補丁下載，不過最好升級到同系列的最新版本
4、修改啓動文件，使之不產生這些文件，設置如下：

   export IBM_HEAP_DUMP=false
   export IBM_HEAPDUMP=false
   export IBM_HEAPDUMP_OUTOFMEMORY=false
   export IBM_JAVACORE_OUTOFMEMORY=false

 

分析以上4中方法，只有方法2纔是根本解決之道。

 

針對4，IBM網站上有詳細闡述，特附如下：

http://www-1.ibm.com/support/docview.wss?uid=swg21140641

Out of Memory (Java heap) specific MustGather information
Set up for enabling heap dump file generation:

• For releases of V6.0:
  1. To set the Environment Entries from the administrative console, select the following:
     
     Servers > Application Servers > server_name > Java and Process Management > Process Definition > Environment Entries > New
     
     
  2. Add the following Name and Value pairs:
     
     

     Name	Value
     IBM_HEAPDUMP	true
     IBM_HEAP_DUMP	true
     IBM_HEAPDUMPDIR	your_directory
     IBM_HEAPDUMP_OUTOFMEMORY	true
     IBM_JAVADUMP_OUTOFMEMORY	true
     IBM_JAVA_HEAPDUMP_TEXT	true

  
  
  3. Make sure that you save changes to the master configuration.

• For release V5.1.1:
  1. To set the Environment Entries from the administrative console, select the following:
     
     Servers > Application Servers > server_name > Process Definition > Environment Entries > New
     
     
  2. Add the following Name and Value pairs:
     

     Name	Value
     IBM_HEAPDUMP	true
     IBM_HEAP_DUMP	true
     IBM_HEAPDUMPDIR	your_directory
     IBM_HEAPDUMP_OUTOFMEMORY	true
     IBM_JAVADUMP_OUTOFMEMORY	true
     IBM_JAVA_HEAPDUMP_TEXT	true

  3. Make sure that you save changes to the master configuration.
  
  For releases of V5.0 and V5.1.0:
  1. To set the Environment Entries from the administrative console, select the following:
     
     Servers > Application Servers > server_name > Process Definition > Environment Entries > New
     
     
  2. Add the following Name and Value pairs:
     

     Name	Value
     IBM_HEAPDUMP	true
     IBM_HEAP_DUMP	true
     IBM_HEAPDUMPDIR	your_directory
     IBM_HEAPDUMP_OUTOFMEMORY	true
     IBM_JAVADUMP_OUTOFMEMORY	true

  3. Make sure that you save changes to the master configuration.
     
• For releases of V4.0:
  1. To enable the heap dump, add the following lines to the top of the adminserver.bat file in the install_root/bin directory, after the call to setupCmdLine.bat:
     
     

     call "%~dp0setupCmdLine.bat"
     set IBM_HEAPDUMP=true
     set IBM_HEAP_DUMP=true
     set IBM_HEAPDUMP_OUTOFMEMORY=true
     set IBM_JAVADUMP_OUTOFMEMORY=true
     set IBM_HEAPDUMPDIR=directory_path

  2. When you start the administrative server, use the modified adminserver.bat file from a command prompt instead of using the Services panel.
     

• Notes:
  • The heap dump is written to the current directory, install_root/bin, or the directory specified by the IBM_HEAPDUMPDIR environment variable.
    
    
  • The file that is generated is called heapdumppid.time in internal format txt. This file can be very large; allow for at least the maximum heap size. If the maximum heap is set to 512 MB, allow for .5 GB for the dump.
    
    
  • Any version of WebSphere Application Server that is using the IBM JDK 1.3.1 SR5 or higher, and 1.4.X, will automatically generate heapdump and javacore files on an OutOfMemory condition. Adding the additional parameters will give you the option to manually generate heapdumps and also specify the directory in which to write the heapdump files. To manually generate heapdumps you will need to issue the WSADMIN command to generate javacore*.txt file. This will also produce a heapdump*.txt file.

When failure occurs:

1. This mechanism uses the same signal handler as javacore processing, so do not haveDISABLE_JAVADUMP set as an environment variable.
   
   
2. Follow instructions on enabling verbosegc in WebSphere Application Server.
   
   
3. Delete all Application Server and FFDC log files, if possible. Back them up, if needed.
   
   
4. Restart the Application Server.
   
   
5. Wait for the OutOfMemoryError condition to occur. This should cause the generation of a heapdump.
   
   
6. If you want to analyze the heapdump yourself, follow instructions to use IBM HeapAnalyzer toolfor analyzing JVM heap usage. If you do not want to analyze this yourself, proceed to step 8.
   
   
7. Collect the following files:
   • For releases of V6.0:
     • The server.xml file located in the following directory:
       
       

       install_root\profiles\profile_name\config\cells\
       cell_name\nodes\node_name\servers\server_name

     • Everything in the following directory:
       
       

       install_root\profiles\profile_name\logs\server_name

       
       Specifically, the following logs are needed:

       • systemErr
       • systemOut
       • native_stderr
       • native_stdout
       • heapdump*.txt or heapdump*.phd
       • javacore*.txt
     • Everything in the following directory:
       
       

       install_root\profiles\profile_name\logs\ffdc

     • All documents requested For all releases below.
       
       
   • For releases of V5.0 and V5.1:
     • The server.xml file located in the following directory:
       
       

       install_root\config\cells\nodes\node_name\servers\server_name

       
       Specifically, the following logs are needed:

       • systemErr
       • systemOut
       • native_stderr
       • native_stdout
       • heapdump*.txt or heapdump*.phd
       • javacore*.txt
     • Everything from the following directory:
       
       

       install_root\logs\server_name

     • Everything from the following directory:
       
       

       install_root\logs\ffdc

     • All documents requested For all releases below.
       
       
   • For releases of V4.0 and V3.5:
     • A XMLConfig full export.
       
       
     • Everything from the following directory:
       
       

       install_root\logs

       
       Specifically, the following log files are needed:

       • stderr
       • stdout
       • tracefile
       • heapdump*.txt
       • javacore*.txt
     • All documents requested For all releases below.
       
       
   • For all releases
     • All javacore*.txt files that were created when OutOfMemory conditions occurred. Ifjavacore*.txt files were created, you can find them in the install_root,install_root\bin or in the configured working directory.
       
       
     • Include the Application Server systemErr, systemOut, native_stderr, and native_stdout logs, if they are located in a different directory.
       
       
     • List of all other software and version information on the system. For example, any databases, WebSphere MQ, and so forth.
       
       
8. Follow instructions to send diagnostic information to IBM support.