內(nèi)存泄漏問(wèn)題調(diào)查

1、內(nèi)存泄漏問(wèn)題調(diào)查XXX版本內(nèi)存泄漏XXX版本內(nèi)存泄漏長(zhǎng)期掛機(jī)結(jié)果Mem-info:DMA per-cpu:CPU 0: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0Active:3802 inactive:0 dirty:0 writeback:0 unstable:0 free:127 slab:1684 mapped:1 pagetables:124 bounce:0DMA free:508kB min:720kB low:900kB high:1080kB active:15208kB inactive:0kB presen

2、t:32512kB pages_scanned:3286541 all_unreclaimable? yeslowmem_reserve: 0 0DMA: 1*4kB 1*8kB 1*16kB 1*32kB 1*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 508kBFree swap: 0kB8192 pages of RAM0 pages of HIGHMEM1094 reserved pages30 pages shared0 pages swap cachedprintk: 12189 messages suppre

3、ssed.phone_voice invoked oom-killer: gfp_mask=0 x201d2, order=0, oomkilladj=0Call Trace: dump_stack+0 x8/0 x34 out_of_memory+0 x1f4/0 x214 _alloc_pages+0 x32c/0 x334 _do_page_cache_readahead+0 x1f4/0 x294 filemap_nopage+0 x2a4/0 x4d4 _handle_mm_fault+0 x3c8/0 x924 do_page_fault+0 x104/0 x3d0 ret_fro

4、m_exception+0 x0/0 x18表現(xiàn)現(xiàn)象：系統(tǒng)crash，不會(huì)自動(dòng)重啟大量進(jìn)程被killWeb和telnet無(wú)響應(yīng)可以ping通看門(mén)狗進(jìn)程還保持運(yùn)行用戶(hù)態(tài)進(jìn)程內(nèi)存 32位操作系統(tǒng)中，每個(gè)進(jìn)程擁有4G的虛擬內(nèi)存空間，其中03GB是每個(gè)進(jìn)程的私有用戶(hù)空間，這個(gè)空間對(duì)系統(tǒng)中其他進(jìn)程是不可見(jiàn)的。34GB是linux內(nèi)核空間，由系統(tǒng)所有的進(jìn)程以及內(nèi)核所共享的。 進(jìn)程占用內(nèi)存主要有如下幾個(gè)區(qū)域： 1、當(dāng)前執(zhí)行文件的代碼段，該代碼段稱(chēng)為text段。 2、執(zhí)行文件的數(shù)據(jù)段，主要存儲(chǔ)執(zhí)行文件用到的全局變量，靜態(tài)變量。 3、存儲(chǔ)全局變量和動(dòng)態(tài)產(chǎn)生的數(shù)據(jù)的堆。 4、用于保存局部變量和實(shí)現(xiàn)函數(shù)調(diào)用的棧。

5、5、采用mmap方式映射到虛擬地址空間中的內(nèi)存段查看內(nèi)存的幾個(gè)基本方法 Top 命令 /proc/meminfo /proc/pid/maps /proc/pid/statusTOP命令介紹 PID : 進(jìn)程ID PPID : 父進(jìn)程ID，1表示init進(jìn)程, USER :當(dāng)前用戶(hù) STAT : 進(jìn)程當(dāng)前運(yùn)行狀態(tài) VSZ ：此進(jìn)程一共占用了多大物理內(nèi)存。 %MEM : 此進(jìn)程占總內(nèi)存消耗的百分比 %CPU : 此進(jìn)程占總CPU消耗的百分比 COMMAND : 進(jìn)程啟動(dòng)時(shí)的命令行格式/PROC/MEMINFO $cat /proc/meminfoMemTotal: 2052440 kB /總內(nèi)存

6、MemFree: 50004 kB /空閑內(nèi)存Buffers: 19976 kB /給文件的緩沖大小Cached: 436412 kB /高速緩沖存儲(chǔ)器使用的大小SwapCached:19864 kB /被高速緩沖存儲(chǔ)用的交換空間大小Active: 1144512 kB /活躍使用中的高速緩沖存儲(chǔ)器頁(yè)面文件大小Inactive: 732788 kB /不經(jīng)常使用的高速緩沖存儲(chǔ)器頁(yè)面文件大小 可用的物理內(nèi)存=memfree+buffers+cached， 當(dāng)memfree不夠時(shí)，內(nèi)核會(huì)通過(guò)回寫(xiě)機(jī)制(pdflush線(xiàn)程)把cached和buffered內(nèi)存回寫(xiě)到后備存儲(chǔ)器，從而釋放相關(guān)內(nèi)存供進(jìn)程使

7、用，或者通過(guò)手動(dòng)方式顯式釋放cache內(nèi)存/PROC/PID/MAPS# cat maps00400000-00414000 r-xp 00000000 1f:05 193 /bin/wanManager 代碼段00453000-00454000 rw-p 00013000 1f:05 193 /bin/wanManager 數(shù)據(jù)段00454000-0045d000 rwxp 00454000 00:00 0 heap 堆2aaa8000-2aaae000 r-xp 00000000 1f:05 366 /lib/ld-uClibc-0.9.28.so 程序連接的共享庫(kù)的內(nèi)存地址2aaae00

8、0-2aaaf000 rw-p 2aaae000 00:00 0 2aaed000-2aaee000 r-p 00005000 1f:05 366 /lib/ld-uClibc-0.9.28.so2aaee000-2aaef000 rw-p 00006000 1f:05 366 /lib/ld-uClibc-0.9.28.so2aaef000-2ab4d000 r-xp 00000000 1f:05 319 /lib/libuClibc-0.9.28.so2ab4d000-2ab8c000 -p 2ab4d000 00:00 0 2ab8c000-2ab8d000 r-p 0005d000 1

9、f:05 319 /lib/libuClibc-0.9.28.so2ab8d000-2ab8e000 rw-p 0005e000 1f:05 319 /lib/libuClibc-0.9.28.so2ab8e000-2ab93000 rw-p 2ab8e000 00:00 0 2ab93000-2ab96000 r-xp 00000000 1f:05 313 /lib/libnvram-0.9.28.so2ab96000-2abd6000 -p 2ab96000 00:00 0 2abd6000-2abe4000 rw-p 00003000 1f:05 313 /lib/libnvram-0.

10、9.28.so2abe4000-2abe8000 r-xp 00000000 1f:05 365 /lib/libcfg.so2abe8000-2ac28000 -p 2abe8000 00:00 0 2ac28000-2ac29000 rw-p 00004000 1f:05 365 /lib/libcfg.so2ac29000-2ac38000 r-xp 00000000 1f:05 310 /lib/libpthread-0.9.28.so2ac38000-2ac77000 -p 2ac38000 00:00 0 2ac77000-2ac7c000 rw-p 0000e000 1f:05

11、310 /lib/libpthread-0.9.28.so2ac7c000-2ac7e000 rw-p 2ac7c000 00:00 0 線(xiàn)程棧7ff41000-7ff56000 rwxp 7ff41000 00:00 0 stack 進(jìn)程棧1）代表內(nèi)存段的虛擬地址2）代表執(zhí)行權(quán)限，r，w，x，p=私有 s=共享3) 代表在進(jìn)程地址里的偏移量4) 映射文件的主設(shè)備號(hào)和次設(shè)備號(hào)5) 映像文件的節(jié)點(diǎn)號(hào)，即inode6) 映像文件的路徑進(jìn)程內(nèi)存布局/PROC/PID/STATUS# cat statusName: wanManagerState: S (sleeping)SleepAVG: 97%T

12、gid: 2625Pid: 2625PPid: 1TracerPid: 0Uid: 0 0 0 0Gid: 0 0 0 0FDSize: 32Groups:VmPeak: 1836 kBVmSize: 1836 kBVmLck: 0 kBVmHWM: 492 kBVmRSS: 492 kBVmData: 1084 kBVmStk: 84 kBVmExe: 80 kBVmLib: 488 kBVmPTE: 16 kBThreads: 1SigQ: 1/256SigPnd: 00000000000000000000000000000000ShdPnd: 0000000000000000000000

13、0000000000SigBlk: 00000000000000000000000080000000SigIgn: 00000000000000000000000000001006SigCgt: 0000000000000000000000038001a000CapInh: 0000000000000000CapPrm: 00000000fffffeffCapEff: 00000000fffffeff VmSize:整個(gè)進(jìn)程使用虛擬內(nèi)存大小，是VmLib,VmExe,VmData,和VmStk的總和。 VmLck:虛擬內(nèi)存鎖。進(jìn)程當(dāng)前使用的并且加鎖的虛擬內(nèi)存總數(shù) VmHWM: 表示進(jìn)程所占用物

14、理內(nèi)存的峰值 VmRSS:虛擬內(nèi)存駐留集合大小。這是駐留在物理內(nèi)存的一部分。它沒(méi)有交換到硬盤(pán)。它包括代碼，數(shù)據(jù)和棧。 VmData:虛擬內(nèi)存數(shù)據(jù)。堆堆使用的虛擬內(nèi)存。 VmStk:虛擬內(nèi)存棧 棧棧使用的虛擬內(nèi)存 VmExe:可執(zhí)行的虛擬內(nèi)存, 可執(zhí)行的和靜態(tài)鏈接庫(kù)所使用的虛擬內(nèi)存 VmLib:虛擬內(nèi)存庫(kù)動(dòng)態(tài)鏈接庫(kù)所使用的虛擬內(nèi)存XX001版本內(nèi)存監(jiān)控實(shí)驗(yàn)內(nèi)存觀察：初始狀態(tài)# cat statusName: wanManagerState: S (sleeping)SleepAVG: 97%Tgid: 2625Pid: 2625PPid: 1TracerPid: 0Uid: 0 0 0 0Gid

15、: 0 0 0 0FDSize: 32Groups:VmPeak: 1836 kBVmSize: 1836 kBVmLck: 0 kBVmHWM: 492 kBVmRSS: 492 kBVmData: 1084 kBVmStk: 84 kBVmExe: 80 kBVmLib: 488 kBVmPTE: 16 kBThreads: 1SigQ: 1/256SigPnd: 00000000000000000000000000000000ShdPnd: 00000000000000000000000000000000SigBlk: 00000000000000000000000080000000Si

16、gIgn: 00000000000000000000000000001006SigCgt: 0000000000000000000000038001a000CapInh: 0000000000000000CapPrm: 00000000fffffeffCapEff: 00000000fffffeff# cat maps00400000-00414000 r-xp 00000000 1f:05 193 /bin/wanManager00453000-00454000 rw-p 00013000 1f:05 193 /bin/wanManager00454000-0045d000 rwxp 004

17、54000 00:00 0 heap2aaa8000-2aaae000 r-xp 00000000 1f:05 366 /lib/ld-uClibc-0.9.28.so2aaae000-2aaaf000 rw-p 2aaae000 00:00 0 2aaed000-2aaee000 r-p 00005000 1f:05 366 /lib/ld-uClibc-0.9.28.so2aaee000-2aaef000 rw-p 00006000 1f:05 366 /lib/ld-uClibc-0.9.28.so2aaef000-2ab4d000 r-xp 00000000 1f:05 319 /li

18、b/libuClibc-0.9.28.so2ab4d000-2ab8c000 -p 2ab4d000 00:00 0 2ab8c000-2ab8d000 r-p 0005d000 1f:05 319 /lib/libuClibc-0.9.28.so2ab8d000-2ab8e000 rw-p 0005e000 1f:05 319 /lib/libuClibc-0.9.28.so2ab8e000-2ab93000 rw-p 2ab8e000 00:00 0 2ab93000-2ab96000 r-xp 00000000 1f:05 313 /lib/libnvram-0.9.28.so2ab96

19、000-2abd6000 -p 2ab96000 00:00 0 2abd6000-2abe4000 rw-p 00003000 1f:05 313 /lib/libnvram-0.9.28.so2abe4000-2abe8000 r-xp 00000000 1f:05 365 /lib/libcfg.so2abe8000-2ac28000 -p 2abe8000 00:00 0 2ac28000-2ac29000 rw-p 00004000 1f:05 365 /lib/libcfg.so2ac29000-2ac38000 r-xp 00000000 1f:05 310 /lib/libpt

20、hread-0.9.28.so2ac38000-2ac77000 -p 2ac38000 00:00 0 2ac77000-2ac7c000 rw-p 0000e000 1f:05 310 /lib/libpthread-0.9.28.so2ac7c000-2ac7e000 rw-p 2ac7c000 00:00 0 7ff41000-7ff56000 rwxp 7ff41000 00:00 0 stack內(nèi)存觀察：半小時(shí)后# cat statusName: wanManagerState: S (sleeping)SleepAVG: 97%Tgid: 2625Pid: 2625PPid: 1

21、TracerPid: 0Uid: 0 0 0 0Gid: 0 0 0 0FDSize: 32Groups:VmPeak: 1864 kBVmSize: 1864 kBVmLck: 0 kBVmHWM: 520 kBVmRSS: 520 kBVmData: 1112 kBVmStk: 84 kBVmExe: 80 kBVmLib: 488 kBVmPTE: 16 kBThreads: 1SigQ: 0/256SigPnd: 00000000000000000000000000000000ShdPnd: 00000000000000000000000000000000SigBlk: 0000000

22、0000000000000000080000000SigIgn: 00000000000000000000000000001006SigCgt: 0000000000000000000000038001a000CapInh: 0000000000000000CapPrm: 00000000fffffeffCapEff: 00000000fffffeff# cat maps00400000-00414000 r-xp 00000000 1f:05 193 /bin/wanManager00453000-00454000 rw-p 00013000 1f:05 193 /bin/wanManage

23、r00454000-00463000 rwxp 00454000 00:00 0 heap2aaa8000-2aaae000 r-xp 00000000 1f:05 366 /lib/ld-uClibc-0.9.28.so2aaae000-2aaaf000 rw-p 2aaae000 00:00 0 2aaed000-2aaee000 r-p 00005000 1f:05 366 /lib/ld-uClibc-0.9.28.so2aaee000-2aaef000 rw-p 00006000 1f:05 366 /lib/ld-uClibc-0.9.28.so2aaef000-2ab4d000

24、r-xp 00000000 1f:05 319 /lib/libuClibc-0.9.28.so2ab4d000-2ab8c000 -p 2ab4d000 00:00 0 2ab8c000-2ab8d000 r-p 0005d000 1f:05 319 /lib/libuClibc-0.9.28.so2ab8d000-2ab8e000 rw-p 0005e000 1f:05 319 /lib/libuClibc-0.9.28.so2ab8e000-2ab93000 rw-p 2ab8e000 00:00 0 2ab93000-2ab96000 r-xp 00000000 1f:05 313 /

25、lib/libnvram-0.9.28.so2ab96000-2abd6000 -p 2ab96000 00:00 0 2abd6000-2abe4000 rw-p 00003000 1f:05 313 /lib/libnvram-0.9.28.so2abe4000-2abe8000 r-xp 00000000 1f:05 365 /lib/libcfg.so2abe8000-2ac28000 -p 2abe8000 00:00 0 2ac28000-2ac29000 rw-p 00004000 1f:05 365 /lib/libcfg.so2ac29000-2ac38000 r-xp 00

26、000000 1f:05 310 /lib/libpthread-0.9.28.so2ac38000-2ac77000 -p 2ac38000 00:00 0 2ac77000-2ac7c000 rw-p 0000e000 1f:05 310 /lib/libpthread-0.9.28.so2ac7c000-2ac7e000 rw-p 2ac7c000 00:00 0 7ff41000-7ff56000 rwxp 7ff41000 00:00 0 stack結(jié)果 wanManager 的堆使用可能有問(wèn)題，一直在增長(zhǎng) 檢查wanManager代碼中任何關(guān)于對(duì)堆內(nèi)存的使用 malloc allo

27、c realloc strdup（內(nèi)部會(huì)調(diào)用malloc）西電版本錯(cuò)誤（FIND_PID_BY_NAME函數(shù)） /* Buffer should contain a string like Name: binary_name */ sscanf(buffer, %*s %s, name); if (strcmp(name, pidName) = 0) /printf(name:%sn,name); pidList=realloc( pidList, sizeof(pid_t) * (i+2); pidListi+=strtol(next-d_name, NULL, 0); break; clo

28、sedir(dir); if (pidList) pidListi=0; /printf(pid:%ldn,pidList0); return pidList0;函數(shù)直接返回堆指針，如果調(diào)用者沒(méi)有釋放，就會(huì)造成內(nèi)存泄漏。XXX002版本內(nèi)存監(jiān)控實(shí)驗(yàn)/PROC/PID/FD fd目錄包含了所有該進(jìn)程使用的文件描述符# ls -l fdlr-x- 1 0 0 64 6 - /tmp/session_vbugkhffltibixfzlr-x- 1 0 0 64 5 - /tmp/session_vbugkhffltibixfzlrwx- 1 0 0 64 2 - /dev/consolelrwx-

29、1 0 0 64 1 - /dev/consolelr-x- 1 0 0 64 0 - /dev/null10分鐘后# ls -l /proc/1016/fdlr-x- 1 0 0 64 14 - /tmp/session_vbugkhffltibixfzlr-x- 1 0 0 64 13 - /tmp/session_vbugkhffltibixfzlr-x- 1 0 0 64 12 - /tmp/session_vbugkhffltibixfzlr-x- 1 0 0 64 11 - /tmp/session_vbugkhffltibixfzlr-x- 1 0 0 64 10 - /tmp/session_vbugkhffltibixfzlr-x- 1 0 0 64 9 - /tmp/session_vbugkhffltibixfzlr-x- 1 0 0 64 8 - /tmp/session_vbugkhffltibixfzlr-x- 1 0 0 64 7 - /tmp/session_vbugkhffltibixfzlr-x- 1 0 0 64 6 - /tmp/session_vbugkhffltibixfzlr-x- 1 0 0 64 5 - /tmp/session_vbugkhffltibixfzlrwx- 1 0 0 64 2 - /dev/consol