近期weblogic 11g永久代內(nèi)存溢出，分析JVM dump文件是沒(méi)有用處的，因?yàn)槟侵皇嵌褍?nèi)存，永久代不在里面。目前永久代設(shè)置是1G,遙想當(dāng)年，只有400M，這么多年來(lái)一直在漲，現(xiàn)在一次full gc需要10多秒。可以增大到1.5G，但從GC日志中可以看到永久代在不斷的增長(zhǎng)，如果二周不重啟weblogic，內(nèi)存又不夠用了，不是長(zhǎng)久之計(jì)。java.lang.OutOfMemoryError: PermGen spaceat java.lang.Class.getDeclaredMethods0(Native Method) ~[na:1.6.0_30]at java.lang.Class.PRivateGetDeclaredMethods(Class.java:2427) ~[na:1.6.0_30]at java.lang.Class.getDeclaredMethod(Class.java:1935) ~[na:1.6.0_30]at java.io.ObjectStreamClass.getPrivateMethod(ObjectStreamClass.java:1382) ~[na:1.6.0_30]at java.io.ObjectStreamClass.access$1700(ObjectStreamClass.java:52) ~[na:1.6.0_30]at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:438) ~[na:1.6.0_30]at java.security.AccessController.doPrivileged(Native Method) ~[na:1.6.0_30]可以看到是方法區(qū)有問(wèn)題，創(chuàng)建了大量的類(lèi)，從GC日志上看，啟動(dòng)之后永久代有700M，隨著時(shí)間的推移，會(huì)緩慢增長(zhǎng)，這就是內(nèi)存泄露，堆內(nèi)存泄露也是這樣。如何定位哪些類(lèi)一直在漲呢，此時(shí)就用到兩個(gè)重要的JVM參數(shù)，一個(gè)顯示每個(gè)加載的類(lèi)，一個(gè)顯示每個(gè)卸載的類(lèi)：-XX:+TraceClassLoading -XX:+TraceClassUnloadingweblogic里面會(huì)出現(xiàn)這種日志，需要自己寫(xiě)程序解析最后還有哪些類(lèi)存活著。[Loaded MM.statistics.mmbillstatistics.exception.MmBillStatisticalTypeException from file:/data/Domain/mm_Domain/servers/mmServer1/stage/EAR/EAR/APP-INF/classes/mm/statistics/mmbillstatistics/exception/MmBillStatisticalTypeException.class][Loaded MM.statistics.mmbillstatistics.appservice.impl.MmBillStatisticalTypeSes_1k61gv_IMmBillStatisticalTypeServiceImpl_1035_WLStub from file:/data/wls1035/modules/com.bea.core.utils.wrapper_1.4.0.0.jar][Loaded MM.statistics.amfmHuiXuFinance.appservice.AmfmHuiXuFinanceBizService_f879n8_IAmfmHuiXuFinanceBizServiceRIntf from file:/data/Domain/mm_Domain/servers/mmServer1/cache/EJBCompilerCache/1nhs7towub2hs/mm/statistics/amfmHuiXuFinance/appservice/AmfmHuiXuFinanceBizService_f879n8_IAmfmHuiXuFinanceBizServiceRIntf.class][Loaded MM.statistics.amfmHuiXuFinance.exception.AmfmHuiXuFinanceException from file:/data/Domain/mm_Domain/servers/mmServer1/stage/EAR/EAR/APP-INF/classes/mm/statistics/amfmHuiXuFinance/exception/AmfmHuiXuFinanceException.class][Unloading class mm.purchase.purchaSEOrder.model.PurchaseItemVO][Unloading class mm.inventory.reservereplenishplan.appservice.impl.ReserveReplenishPlanBizService][Unloading class mm.inventory.reservequota.dao.ReserveQuotaBatchDAO]解析程序開(kāi)始：drop table load_class purge;truncate table load_class;create table load_class( nu number, action varchar2(20), class_name varchar2(1000), class_file varchar2(1000));import java.io.BufferedReader;import java.io.File;import java.io.FileInputStream;import java.io.InputStreamReader;import java.sql.Connection;import java.sql.DriverManager;import java.sql.PreparedStatement;/** * [Loaded java.lang.ClassNotFoundException from /usr/local/jdk1.6/jre/lib/rt.jar] * [Unloading class com.MM.externalinterface.fmis.impl.FmisBizService] * */public class PermAla { static final String driver_class = "Oracle.jdbc.driver.OracleDriver"; static final String connectionURL = "jdbc:oracle:thin:@10.10.15.164:1521:orcl"; static final String userID = "SPROC"; static final String userPassWord = "SPROC"; public static void readTxtFile(String filePath){ Connection con = null; String s_sql = "insert into load_class values(?,?,?,?)"; PreparedStatement pstmt = null; int i=0; try { Class.forName (driver_class).newInstance(); con = DriverManager.getConnection(connectionURL, userID, userPassword); pstmt = con.prepareStatement(s_sql); con.setAutoCommit(false); String encoding="GBK"; File file=new File(filePath); InputStreamReader read = new InputStreamReader( new FileInputStream(file),encoding);//考慮到編碼格式 BufferedReader bufferedReader = new BufferedReader(read); String lineTxt = null; String[] lineTxtArray= null; while((lineTxt = bufferedReader.readLine()) != null){ if(lineTxt.indexOf("[Loaded")==0 ) { lineTxtArray = lineTxt.split(" "); pstmt.setInt(1, i); pstmt.setString(2, lineTxtArray[0]); pstmt.setString(3, lineTxtArray[1]); if(lineTxtArray.length ==4) { pstmt.setString(4, lineTxtArray[3]); }else{ System.out.println(lineTxt); } pstmt.addBatch(); i++; } else if(lineTxt.indexOf("[Unloading")==0){ lineTxtArray = lineTxt.split(" "); pstmt.setInt(1, i); pstmt.setString(2, lineTxtArray[0]); pstmt.setString(3, lineTxtArray[2]); pstmt.addBatch(); i++; } if(i % 10000 == 0){ pstmt.executeBatch(); con.commit(); } } con.commit(); read.close(); } catch (Exception e) { System.out.println("讀取文件內(nèi)容出錯(cuò),行數(shù)："+i); e.printStackTrace(); }finally{ if(pstmt != null){ try { pstmt.close(); } catch (Exception e) { e.printStackTrace(); }finally{ pstmt = null; } } if(con != null){ try { con.close(); } catch (Exception e) { e.printStackTrace(); }finally{ con = null; } } } } public static void main(String argv[]){ String filePath = "E://永久代內(nèi)存溢出//20161213//Server1.log"; readTxtFile(filePath); }}--執(zhí)行java代碼導(dǎo)入之后，處理一些特殊的格式update load_class set action = replace(action, '[', ''), class_file = replace(class_file, ']', ''), class_name = replace(class_name, ']', '');commit;update load_class set class_name = replace(class_name, 'file:', ''), class_file = replace(class_file, 'file:', '');commit;SQL> select * from load_class where rownum <10; NU ACTION CLASS_NAME CLASS_FILE--- ------- --------------------------------------- -------------------------------- 96 Loaded sun.misc.SharedSecrets /data/jdk1.6.0_45/jre/lib/rt.jar 97 Loaded sun.misc.Unsafe /data/jdk1.6.0_45/jre/lib/rt.jar 98 Loaded java.lang.IncompatibleClassChangeError /data/jdk1.6.0_45/jre/lib/rt.jar 99 Loaded java.lang.NoSuchMethodError /data/jdk1.6.0_45/jre/lib/rt.jar100 Loaded sun.reflect.Reflection /data/jdk1.6.0_45/jre/lib/rt.jar101 Loaded java.util.Collections /data/jdk1.6.0_45/jre/lib/rt.jar102 Loaded java.lang.Iterable /data/jdk1.6.0_45/jre/lib/rt.jar103 Loaded java.util.Collection /data/jdk1.6.0_45/jre/lib/rt.jar104 Loaded java.util.Set /data/jdk1.6.0_45/jre/lib/rt.jar --顯示加載且沒(méi)有卸載的類(lèi) drop table purge;create table aswith res as(select count(*) over(partition by action,class_name order by nu asc) rn, t.* from load_class t),res1 as (select rn,class_name from res a where a.action='Loaded' minus select rn,class_name from res b where b.action='Unloading')select b.* from res1 a, res b where a.rn= b.rn and a.class_name= b.class_name;查看數(shù)據(jù)，對(duì)數(shù)據(jù)摸底 select count(1) from where class_file='__JVM_DefineClass__';select count(1) from where class_file like '/data/Domain/MMDomain/servers/MMServer1/stage/EAR/EAR/APP-INF/lib%';select count(1) from where class_file like '/data/Domain/MMDomain/servers/MMServer1/stage/EAR/EAR/APP-INF/classes%' or class_file like '/data/Domain/MMDomain/servers/MMServer1/stage/EAR/EAR/ejb%';select count(1) from where class_file like '/data/wls1035/%'; select count(1) from where class_file like '/data/Domain/MMDomain/servers/MMServer1/cache/EJBCompilerCache/%'; select count(1) from where class_file like '/data/jdk1.6.0_45/jre%'; select count(1) from where class_file in ('weblogic.utils.classloaders.GenericClassLoader', 'sun.misc.Launcher$AppClassLoader', 'weblogic.utils.classloaders.ChangeAwareClassLoader','instance');select count(1) from where class_file like '/data/Domain/MMDomain/jsp_temp%'; 可以看到反射的類(lèi)很多，重點(diǎn)研究__JVM_DefineClass__和sun.reflect.GeneratedMethodAccessor，發(fā)現(xiàn)是反射產(chǎn)生的一個(gè)問(wèn)題。參考http://stackoverflow.com/questions/16130292/java-lang-outofmemoryerror-permgen-space-java-reflection
When using Java reflection, the JVM has two methods of accessing the information on the class being reflected. It can use a JNI accessor, or a Java bytecode accessor.
If it uses a Java bytecode accessor, then it needs to have its own Java class and classloader (sun/reflect/GeneratedMethodAccessor class and sun/reflect/DelegatingClassLoader). Theses classes and classloaders use native memory.
The accessor bytecode can also get JIT compiled, which will increase the native memory use even more.
If Java reflection is used frequently, this can add up to a significant amount of native memory use. The JVM will use the JNI accessor first, then after some number of accesses on the same class, will change to use the Java bytecode accessor. This is called inflation, when the JVM changes from the JNI accessor to the bytecode accessor. Fortunately, we can control this with a Java property.
The sun.reflect.inflationThreshold property tells the JVM what number of times to use the JNI accessor. If it is set to 0, then the JNI accessors are always used. Since the bytecode accessors use more native memory than the JNI ones, if we are seeing a lot of Java reflection, we will want to use the JNI accessors. To do this, we just need to set the inflationThreshold property to zero.
如果節(jié)點(diǎn)使用sun公司jdk，配置-Dsun.reflect.inflationThreshold=2147483647 如果是IBM的jdk，配置-Dsun.reflect.inflationThreshold=0
以下是配置的前后類(lèi)數(shù)量對(duì)比，反射類(lèi)有明顯的下降，從gc日志上看也有回收的跡象：
| 加載類(lèi)的出處 | Server1 | server2 |
| 加參數(shù)前 | 加參數(shù)后 | 加參數(shù)前 | 加參數(shù)后 |
| __JVM_DefineClass__(反射產(chǎn)生的類(lèi)) | 42372 | 3632 | 19961 | 2822 |
| EAR/APP-INF/lib | 25552 | 23162 | 23613 | 23579 |
| EAR/APP-INF/classes | 21531 | 15684 | 15167 | 15176 |
| wls1035 | 19017 | 19420 | 19661 | 19168 |
| MMServer1/cache/EJBCompilerCache | 5996 | 5996 | 5996 | 5996 |
| /data/jdk1.6.0_45/jre | 3494 | 3493 | 3175 | 3458 |
| weblogic.utils.classloaders.GenericClassLoadersun.misc.Launcher$AppClassLoaderweblogic.utils.classloaders.ChangeAwareClassLoader | 2550 | 2574 | 2508 | 2516 |
| jsp_temp | 843 | 1395 | 843 | 849 |
當(dāng)然我遇到的情況是反射類(lèi)引起的問(wèn)題，如果你的永久代設(shè)置太小，不用糾結(jié)，直接把永久代內(nèi)存加大；如果是類(lèi)太多導(dǎo)致（不是反射類(lèi)），那就想辦法瘦身，想辦法 改造去掉一些依賴(lài)的jar包，可能是一個(gè)大工程。
