Debian(Squeeze)でヤツ「oom-killer」出たので対応する

最近は順調に動いていたDebianサーバだけど、またヤツ「oom-killer」が召喚された。

WordPressをアップグレードした際に、メモリ食いそうなプラグインを入れたので
メモリリークしている可能性が考えられると妄想し以下の対応を取る。
 
今回試してみるのはApacheの設定で「MaxRequestsPerChild」の値を設定してみる。
「MaxRequestsPerChild」はプロセス自体の生存回数を決める設定で、
デフォルト値「0」=無制限から「300」=300回でプロセスをクリーンアップするように変更する。
これでしばらく様子を見てみる。。
 
そもそもデフォルト値「0」にしていたのが誤りだつたと思う。

 

 

syslogの内容

Dec 23 23:24:54 brokendish kernel: [345048.525520] apache2 invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0
Dec 23 23:24:55 brokendish kernel: [345048.525526] apache2 cpuset=/ mems_allowed=0
Dec 23 23:24:55 brokendish kernel: [345048.525530] Pid: 24695, comm: apache2 Not tainted 2.6.32-5-amd64 #1
Dec 23 23:24:55 brokendish kernel: [345048.525532] Call Trace:
Dec 23 23:24:55 brokendish kernel: [345048.525541] [<ffffffff810b6324>] ? oom_kill_process+0x7f/0x23f
Dec 23 23:24:55 brokendish kernel: [345048.525545] [<ffffffff810b6848>] ? __out_of_memory+0x12a/0x141
Dec 23 23:24:55 brokendish kernel: [345048.525549] [<ffffffff810b699f>] ? out_of_memory+0x140/0x172
Dec 23 23:24:55 brokendish kernel: [345048.525553] [<ffffffff810ba704>] ? __alloc_pages_nodemask+0x4ec/0x5fc
Dec 23 23:24:55 brokendish kernel: [345048.525557] [<ffffffff810bc6bd>] ? ____pagevec_lru_add+0x160/0x176
Dec 23 23:24:55 brokendish kernel: [345048.525562] [<ffffffff810ccbf8>] ? handle_mm_fault+0x27a/0x80f
Dec 23 23:24:55 brokendish kernel: [345048.525568] [<ffffffff810482ed>] ? finish_task_switch+0x3a/0xaf
Dec 23 23:24:55 brokendish kernel: [345048.525574] [<ffffffff812feb96>] ? do_page_fault+0x2e0/0x2fc
Dec 23 23:24:55 brokendish kernel: [345048.525577] [<ffffffff812fca35>] ? page_fault+0x25/0x30
Dec 23 23:24:55 brokendish kernel: [345048.525579] Mem-Info:
Dec 23 23:24:55 brokendish kernel: [345048.525581] Node 0 DMA per-cpu:
Dec 23 23:24:55 brokendish kernel: [345048.525584] CPU 0: hi: 0, btch: 1 usd: 0
Dec 23 23:24:55 brokendish kernel: [345048.525586] CPU 1: hi: 0, btch: 1 usd: 0
Dec 23 23:24:55 brokendish kernel: [345048.525588] Node 0 DMA32 per-cpu:
Dec 23 23:24:55 brokendish kernel: [345048.525590] CPU 0: hi: 186, btch: 31 usd: 52
Dec 23 23:24:55 brokendish kernel: [345048.525592] CPU 1: hi: 186, btch: 31 usd: 52
Dec 23 23:24:55 brokendish kernel: [345048.525597] active_anon:309032 inactive_anon:59372 isolated_anon:0
Dec 23 23:24:55 brokendish kernel: [345048.525599] active_file:254 inactive_file:312 isolated_file:0
Dec 23 23:24:55 brokendish kernel: [345048.525600] unevictable:0 dirty:1 writeback:1 unstable:0
Dec 23 23:24:55 brokendish kernel: [345048.525601] free:2795 slab_reclaimable:1233 slab_unreclaimable:2534
Dec 23 23:24:55 brokendish kernel: [345048.525602] mapped:282 shmem:84 pagetables:6414 bounce:0
Dec 23 23:24:55 brokendish kernel: [345048.525604] Node 0 DMA free:6032kB min:48kB low:60kB high:72kB active_anon:5136kB inactive_anon:4648kB active_file:4kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15368kB mlocked:0kB dirty:0kB writeback:0kB mapped:4kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:8kB kernel_stack:8kB pagetables:68kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Dec 23 23:24:55 brokendish kernel: [345048.525622] lowmem_reserve[]: 0 1499 1499 1499

oomが発生した時間帯のApacheのLog(抜粋)

xxx.xxx.xxx.xxx - - [23/Dec/2011:23:24:29 +0900] "GET /xxx/xxx1/xxx2/images/sinputnbackground.png HTTP/1.1" 200 437 "http://brokendish.org/ubuntu/1346/" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.63 Safari/535.7"
xxx.xxx.xxx.xxx - - [23/Dec/2011:23:24:30 +0900] "GET /xxx/xxx1/xxx2/images/search.gif HTTP/1.1" 200 1476 "http://brokendish.org/ubuntu/1346/" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.63 Safari/535.7"
:
xxx.xxx.xxx.xxx - - [23/Dec/2011:23:24:41 +0900] "GET /blog/1464/ HTTP/1.1" 200 14709 "-" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6 (FlipboardProxy/0.0.5; +http://flipboard.com/browserproxy)"
xxx.xxx.xxx.xxx - - [23/Dec/2011:23:24:41 +0900] "GET /ubuntu/1311/ HTTP/1.1" 200 563 "-" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6 (FlipboardProxy/1.1; +http://flipboard.com/browserproxy)"
xxx.xxx.xxx.xxx - - [23/Dec/2011:23:24:46 +0900] "POST /wp-cron.php?doing_wp_cron=1324650283 HTTP/1.0" 200 422 "-" "WordPress/3.3; http://brokendish.org"
:
xxx.xxx.xxx.xxx - - [23/Dec/2011:23:24:44 +0900] "GET /debian/1256/ HTTP/1.1" 200 563 "-" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6 (FlipboardProxy/1.1; +http://flipboard.com/browserproxy)"
xxx.xxx.xxx.xxx - - [23/Dec/2011:23:24:49 +0900] "POST /wp-cron.php?doing_wp_cron=1324650284 HTTP/1.0" 200 422 "-" "WordPress/3.3; http://brokendish.org"
:
xxx.xxx.xxx.xxx - - [23/Dec/2011:23:24:52 +0900] "GET /debian/587/ HTTP/1.1" 200 562 "-" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6 (FlipboardProxy/1.1; +http://flipboard.com/browserproxy)"
xxx.xxx.xxx.xxx - - [23/Dec/2011:23:24:51 +0900] "GET /debian/627/ HTTP/1.1" 200 562 "-" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6 (FlipboardProxy/1.1; +http://flipboard.com/browserproxy)"
xxx.xxx.xxx.xxx - - [23/Dec/2011:23:24:51 +0900] "GET /debian/609/ HTTP/1.1" 200 15042 "-" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6 (FlipboardProxy/1.1; +http://flipboard.com/browserproxy)"
xxx.xxx.xxx.xxx - - [23/Dec/2011:23:24:52 +0900] "GET /debian/465/ HTTP/1.1" 200 14917 "-" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-

 

対応内容

vi apache2.conf
:
:
# prefork MPM
# StartServers: number of server processes to start
# MinSpareServers: minimum number of server processes which are kept spare
# MaxSpareServers: maximum number of server processes which are kept spare
# MaxClients: maximum number of server processes allowed to start
# MaxRequestsPerChild: maximum number of requests a server process serves
<IfModule mpm_prefork_module>
StartServers 5
MinSpareServers 5
MaxSpareServers 10
MaxClients 50
# MaxRequestsPerChild 0
MaxRequestsPerChild 300
</IfModule>
:
:
 

コメントを残す

メールアドレスが公開されることはありません。 * が付いている欄は必須項目です